I think there is more to it than this. What happens if you have received rewards for a payout, then remove your vote and vote for something again? you could earn rewards over and over.
A decaying reward pool might be a valid option though, but would need to ask some one with more technical knowledge than myself. I wonder if @ned has ever addressed this?