This is excellent. I think people should have a small downvote allowance so it doesn't actually cost them votes to downvote.
That aside, regarding the fairness of the rewards system, I think the biggest problem is people "sport voting" - betting on favourites, guaranteed winners, more than outsiders, regardless of content. This is very polarising. The solution? I don't know but there needs to be some real incentive to route out and Upvote content from "unproven" minnows.