I agree with both @discombobulated and @sean-king .

I also think indeed one of the strongest reasons people don't downvote is fear.
Let this be a message to all those who downvote to pay back or get revenge and let this be an encouragement to those who need it to do the right thing and promote value on Steemit:

We will downwote the crap out of you too! We are a community and will act as a community if you attack one of us!

PS. Revenge activity can be reported on the #steemitabuse channel on https://steemit.chat