Post Promoter JavaScript Voting Bot - New Blacklist Options & Fix for Missed Transactions
![logo_small.png](https://steemitimages.com/DQmasuh2QK3RqVvxG3FPS3CwsE2sguTucUhvKtRXGS9GM1G/logo_small.png)
New logo by the amazing @nateaguila!
Hello everyone! I’ve been really busy with some larger, long-term Steem projects but I’ve still made some time to make updates to the Post Promoter software here and there and thought it would be a good time to make a post about it - both to let the community know about the changes and to support Utopian while they recover from the recent attack.
The GitHub repo for the project can be found here: https://github.com/MattyIce/postpromoter
New Blacklist & Whitelist Options
In the never-ending war against spam/plagiarism/junk content on the Steem blockchain, it’s of vital importance that operators of content promotion services have the tools they need to prevent that type of content from being promoted without limiting the options of legitimate publishers.
With that in mind, I have beefed up the blacklisting options offered by the Post Promoter software and also added a whitelist option at the request of some members of the community.
All of the existing and new settings have been grouped together and moved into the “blacklist_settings” object in the config.json file as shown below. This change is backwards compatible so you don’t have to worry about breaking anything if you update your software before your config.
"blacklist_settings": {
"flag_signal_accounts": ["spaminator", "cheetah", "steemcleaners", "mack-bot"],
"blacklist_location": "blacklist.txt",
"shared_blacklist_location": "http://sharedlist.com/list.txt",
"whitelist_location": "whitelist.txt",
"whitelist_only": false,
"refund_blacklist": false,
"blacklist_donation_account": "steemcleaners",
"blacklisted_tags": ["nsfw"]
}
You may notice there are now a few new settings there. In addition to the existing “blacklist_location” setting, there is now a “shared_blacklist_location” setting. This allows you to both use a shared blacklist which can either be a local file or a URL and a bot-specific blacklist if you want to blacklist any accounts not included in the shared list.
There is also now a “whitelist_location” setting which allows you to specify a whitelist for your bot. The whitelist can serve a few different purposes. If you set the “whitelist_only” setting to “true”, then only accounts on the whitelist will be able to use the service.
If “whitelist_only” is set to “false” then the whitelist can be used to override accounts that are on the blacklist, if, for example, you’re using a shared blacklist maintained by a third party and want to still allow certain accounts on the list to use your service.
Separate max bid setting for whitelisted accounts
Now that there is a whitelist option, a new max bid setting has been added for whitelisted accounts called “max_bid_whitelist”. This allows you to set a smaller max_bid for the majority of users but allow certain pre-qualified users to purchase larger amounts of promotion.
"max_bid": 50,
"max_bid_whitelist": 200,
In the above example regular users may only bid up to as much as 50 SBD or STEEM (if accepted) but users on the whitelist may bid up to 200. Any bids that exceed the maximum allowed will be automatically refunded.
Fix for Missed Transactions
This is a pretty major change to how the software determines which transactions it has processed vs which are new. It was made to fix a bug that has been puzzling me for quite some time where certain transactions were just completely missed by the software. There were no errors, or log entries or anything, it’s as if the software never saw them at all.
To get the transactions, the software polls the account history for the bot account every 10 seconds using the getAccountHistory() function call. Each transaction returned by this call includes an account-specific index which starts at 1 for the first transaction ever on the account and increases sequentially with each new transaction.
So to identify which transactions the software has already processed vs which are new, it used to just save the index of the last transaction it saw and then on subsequent calls check for any transactions with an index greater than that.
I knew that since transactions were getting missed, something was wrong with that logic, and I was finally able to catch the issue as it happened. It turns out that, for whatever reason, sometimes transaction indexes can change between calls to getAccountHistory() as shown below:
[ 299611,
{ trx_id: 'db53d503c3a2804802a14c7aa7ea49e575e603e4',
block: 22223537,
trx_in_block: 20,
op_in_trx: 0,
virtual_op: 0,
timestamp: '2018-05-07T14:20:15',
op: [ 'transfer', [Object] ] } ]
[ 299616,
{ trx_id: 'db53d503c3a2804802a14c7aa7ea49e575e603e4',
block: 22223537,
trx_in_block: 20,
op_in_trx: 0,
virtual_op: 0,
timestamp: '2018-05-07T14:20:15',
op: [ 'transfer', [Object] ] } ]
What you are seeing in the snippet above is the same transaction record from two separate calls to the getAccountHistory() method. In the first call that transaction had index 299,611 and in the second call the same transaction had index 299,616.
In this case it would actually cause that transaction to be processed twice, which was also happening along with some transactions being missed.
So it was finally clear what was happening, and that I could no longer use the transaction index on the account as a bookmark for which transactions the software already processed and which were new. Instead I have updated the software to keep track of the “trx_id” for the last 50 or so transactions it has processed, and that way whenever it sees another transaction it can check if it has already processed it or if it’s new - regardless of the order in which they get returned by the getAccountHistory() call.
You can see the transaction IDs getting saved also in the state.json file so they can be restored if the software is ever stopped and restarted:
"transactions": [
"cba86bcfbe1fa38bc9601026cc15b432187d5a03",
"7f3e4d51b8396b049ccfb660c273d19b8f9a30b1",
"ce94dfdf7ac1160aa6ffdb58b05460bdaf57c133",
"e3dab712c97f3212a47e21a364299ded2706b092",
"7be4b4800e0ab154423cb062e07cf13ce164dceb",
"eaa915a273e1078bde886d83d07ceb99e37d6e4c",
"5cd059662a18ffe148992df7cc6a04821ac239bd",
…
While it was somewhat rare, this issue seemed to happen in spurts and caused a lot of missed bids. I’m sorry it took so long but I’m happy to say that I believe it should be fixed for good now and hopefully we are closer to the goal of not missing a single transaction.
Thank you for your support!
As always I want to thank everyone who has helped and supported the development of this software, and to everyone who has submitted ideas, suggestions, bug reports, and contributions to the project!
Links to relevant commits:
Thanks for the contribution!
Some cool new features, and I really liked the way you explained everything! Keep it up!
Need help? Write a ticket on https://support.utopian.io/.
Chat with us on Discord.
[utopian-moderator]
Thank you @amosbastian! Great to see you guys keeping everything alive and well with Utopian while the issues are resolved!
Hey yapbapmatt,
As usual thanks up front for your continued efforts! The logo change is a lot more striking and certainly makes PP stand out more on the steambottracker.
I noticed the 50SBD limit recently and thought it was a great idea to prevent someone with a lot of SBD just consuming the entire vote.
Is PP currently running the whitelist model? If so do you have published criteria for how one would get access to it? I have found recently PP has been my go-to bid bot as I have a lot of experience with it and it just...well it just works, consistently.
Yes the @postpromoter account is using the whitelist model now. For the moment people can just contact me if they wish to be added to the whitelist and I will review.
There is no published criteria yet but it's just simply based on whether or not you are promoting good, original content that no one would reasonably have an issue with seeing on the trending page.
Okay great, nice and informal! Currently I don’t have more than 50 SBD to drop on a post so it’s not affecting me atm, I’ll drop you a message further down the line!
Great update. Only bot owners can fight against promoting spam on Steemit. This is good for everybody... Good job!
You got a 25.00% Upvote and Resteem from @ebargains, as well as upvotes from our curation trail followers!
If you are looking to earn a passive no hassle return on your Steem Power, delegate your SP to @ebargains by clicking on one of the ready to delegate links:
50SP | 100SP | 250SP | 500SP | 1000SP | 5000SP | Custom Amount
You will earn 80% of the voting service's earnings based on your delegated SP's prorated share of the service's SP pool daily! That is up to 38.5% APR! You can also undelegate at anytime.
We are also a very profitable curation trail leader on https://steemauto.com/. Follow @ebargains today and earn more on curation rewards!
@yabapmatt please review my profile for the whitelist category thanks i will be waiting for your response dear friend
Works like a charm. What other steem projects are you working on? can you you let us know?
wow good update. you are working very well
This comment has received a 0.15 % upvote from @speedvoter thanks to: @everything-4you.
glad you have fixed all the bugs and the new logo looks pretty cool...
In English version We can discuss how we give SteemPower to the BOT.
En la versión en inglés podemos discutir cómo le damos SteemPower al BOT.
Como sabeis fair.zombie es un bot de apoyo a una comunidad que promociona la actividad constante, lo que es importante para Steemit y STEEM. Las pruebas han salido bien, siempre vota a todos, por orden de publicación y con el 100% de voto, no hay favoritismos ni arbitariedad, es por orden riguroso.
Necesitamos ser unos 20 miembros, de esta manera publicando cada 2 días obtenemos el voto equivalente de los 20 miembros. También hay que ver la manera de darle poder de voto (STEEMPOWER) delelgándole SteemPower o de alguna otra manera que se proponga. El bot puede alquilar SteemPower si tiene fondos.
Necesitamos más stemians. Si conoceis a alguien que quiera entrar en la comunidad, que lo diga y lo incluimos. Lo importante es que publique con frecuencia y con cierta calidad.
Se pueden leer las bases provisionales aquí
saludos
@rafestrape
Thank you my friend for your great initiative, the supports aren’t possible if the primary idea from you isn’t there. Thanks all the way man. Thanks for the detailed update too