I can not stop you, so I won't try.
Yet I don't see why you would do that.
These comments do not bring me any money, therefore you can not make me lose anything by downvoting them.
I might lose a bit of reputation, or maybe a lot, but I will earn it back sooner or later.

If you think you can achieve something - go ahead.