Post Promoter JavaScript Voting Bot - Update 6 - Automatic Failover

in #utopian-io7 years ago (edited)

As you may or may not know, a couple of days ago the memory requirements for a full Steem node crossed the 256 GB threshold which caused issues on a number of nodes and on the many sites and services which use them.

I'm happy to report that thanks to the last round of updates done on the post promoter software it was able to handle these issues MUCH better than the first time around. I was able to quickly switch the bots that I run to a new, working node and I believe that no votes were missed!

Nevertheless, it still required my manual intervention to switch nodes, so there was still work to do!

Automatic Fail-Over

I have spent a good deal of time over the last couple of days implementing an automatic RPC node failover system into the post promoter software so that the next time something like this happens it should pick up the errors and automatically switch to a new node until it finds one that's working.

Any type of automatic fail-over system is tricky to implement because you have to be careful not to trigger it accidentally. The current implementation works as follows (and this may be subject to change in the future after more thorough testing).

In the config.json file you can now specify a new "rpc_nodes" property. I purposely named this differently than the existing "rpc_node" property (no "s") so that the change would be backwards-compatible:

  "rpc_nodes": [
    "https://api.steemit.com",
    "https://rpc.buildteam.io",
    "https://steemd.minnowsupportproject.org",
    "https://steemd.privex.io",
    "https://gtg.steem.house:8090"
  ],

The software will connect to the first node in the list when it starts and will show in the logs which node it is connecting to. It will then automatically switch to the next node in the list if any of the following two conditions occur:

  1. An important transaction (such as a vote or refund) fails twice in a row
  2. There are 10 or more API call failures within a 1-minute period

In the first scenario, after an important transaction fails two times it will then try the transaction once more after it has switched to the new node. A very clear message will be shown in the logs when an automatic fail-over happens.

I have tested these changes locally using the hosts file to simulate a node going down and it appears to work as intended, but please note that these changes are still undergoing testing in live, production environments.

Moved Blacklist and Comment Content to Separate Files

I have also made a couple of other changes that have been requested a number of times in the past. The first change was to move the blacklist out of the config.json file and to its own file which is currently just called "blacklist" (with no extension). The format of this new file is just one Steem account name per line. The change is backwards-compatible so if you still have the "blacklist" setting in your config.json file that will continue to work just fine.

The second change was to move the comment content that the software will use to comment on posts that it upvotes to a separate file as well. In this case there is a new setting in config.json to specify the file location of the comment content:

"comment_location": "comment.md"

The content in the file should use the markdown format. This change is also backwards-compatible so if you are still using the previous "promotion_content" config setting that will continue to work.

Future Updates to Help Fight Spam and Plagiarism

In the near future there will be a much larger focus on blacklisting and spam / plagiarism prevention with the following new features. Many thanks to @themarkymark for putting together this list and working with @patrice to combat this huge problem!

  1. Add a config setting for the blacklist file location which can also include a URL to a centralized list that all bots can use
  2. Add a config setting for whether or not blacklisted users should receive a refund for their bids
  3. Add an option to respond to blacklisted users with a 0.001 SBD transaction letting them know they are blacklisted (with a configurable memo)
  4. Add an option to automatically transfer any bids sent from blacklisted users to an external account to help fight spam (e.g. steemcleaners, cheetah, etc)
  5. Add an option to automatically blacklist posts which are flagged by a spam / plagiarism service like steemcleaners, cheetah, mackbot, etc

Thanks for your support!

As always I want to thank everyone who has helped and supported me in creating this software. I love that it has been able to help so many people get their content promoted and to help so many investors earn a return on their STEEM investment. Please stay tuned for more updates in the coming weeks!

Links to relevant commits:



Posted on Utopian.io - Rewarding Open Source Contributors

Sort:  

Thank you for the contribution. It has been approved.

Great description, great contribution. I like how you go in details. Your posts are definitely great examples for other to be inspired by.

Keep up the great work.

You can contact us on Discord.
[utopian-moderator]

Thanks @helo! I actually find it really satisfying after spending many hours coding to be able to go and write about what I've done and get comments and feedback.

Utopian has really made working on open source projects much more interesting and engaging for me - and not because of the bot's upvote (i was posting on Utopian even before that was a thing), although that is a nice bonus!

Hi yabapmatt, I noticed you moved the steem bot tracker to it's own domain. Could you also list @sleeplesswhale there? It was previously listed on s3, but it's no longer listed on https://steembottracker.com/

Thanks and sorry for hijacking this message...

I removed @sleeplesswhale from the site because I received a large number of complaints about missed votes that were not refunded.

Sorry for inconvenience i was thinking to report you about this small problem.

minimum bid for that bot is 0.5 SBD and suggested bid is 0.36
Its not a big problem, but accuracy is always good.
Thank you.

check the post , i made on accurate use of "steem bot tracker".

Thanks for your reply. I'm aware only about one time when the bot was down and it voted after more than 9 hours. I had to change the hosting(I even tried postpromoter script) and I manually repaid 90% of the received funds.

I was unaware about other incidents, but to be frank I was not checking it very often. I will try to create a script to check all history to automatically repay the uncasted votes.

I perfectly understand your decision to unlist it. I will try to fix it and once done I'll contact you for a second chance. Hopefully I'll get it.

I will be happy to re-list it if you are committed to making things right if and when there is a missed vote.

Great work as always @yabapmatt!

As an aside, I tried out the power delegator, both adding and removing - all seems to work like a charm. I didn't try "update" - I assume that edits the amount you have delegated. Do you know if it brings it all back and sends the new amount or if it brings back a part of it? I think there would be a difference between the two with the delay or returning SP.

In terms of UI, if there's a way to have the green ticks come up once you finish typing the user name (say after a second or two delay) rather than having to click elsewhere, that would be useful. I was kind of unsure if it was waiting for me to do something or not.

And then I would just add a couple of boxes / pop-up explanation comments:

  • One on the "Delegate" button just to make it clear that this is the button that activates the delegation and will take you through to steemconnect.
  • And one on the "Delegate Max" to warn people that if you delegate the full amount then you won't have any bandwidth to make any actions on the blockchain - but it's a very small add.

But overall I think it's great. It encouraged me to make a delegation today to help someone out with bandwidth that I may not otherwise have got around to doing due to the complexity of the process. So a little bit of good for the world!

Thanks, this is great feedback! I'll try to answer what I can here but feel free to reach out if you have any more questions...

I didn't try "update" - I assume that edits the amount you have delegated.

Clicking "update" will just fill in the amount you have already delegated in the "Add/Update Delegation" section so you can then change the amount and update your delegation.

Do you know if it brings it all back and sends the new amount or if it brings back a part of it? I think there would be a difference between the two with the delay or returning SP.

This is one of the most confusing things about delegations. You can only have one delegation to a specific account, so if you have already delegated some and you want to delegate more you have to update that delegation to be the total amount.

So for example, if i have delegated 1,000 SP to utopian-io and i want to delegate 500 more, then i will need to make a delegation of 1,500 SP. This updates my original 1,000 delegation to 1,500.

And one on the "Delegate Max" to warn people that if you delegate the full amount then you won't have any bandwidth to make any actions on the blockchain - but it's a very small add.

The "delegate max" button will delegate the max available minus 6 SP to ensure that your account can still transact. I agree that should probably be better messaged!

Hi pal. Good work on the vote bot tracker. I just dropped you a witness vote.

I would like to know if it's possible to get more ROI historical data from the last bidding rounds of the bots.

Thanks.
@fortified

That's not something that's available on the steem bot tracker website, or any other site that I'm aware of. It's possible to obtain that information by going through the account history for the bots but it would be a project.

Hey. Thanks for the reply.
Do you think it's something that would be worth implementing on your bot tracker site? Maybe include the last 5 rounds. I rarely use voting bots, but I can see how useful your bot tracker is for people so I feel having as much data as possible would hopefully help people to make a more informed decision when choosing a bot.

Anyway, it was just an idea I wanted to share with you :)

Keep up the good work.
@fortified

Thanks @yabapmatt for providing us with the facility of #Steem-Bot-tracker .

It was really very hard for me to promote my post and to see to whom i should send the bid. When there was no #Steem-Bot-tracker , i used to visit the wallet of the bot and calculate the total bids to calculate my profit. But #Steem-Bot-tracker had made this process so easy.

I have made a post on accurate use of #Steem-Bot-tracker .

https://steemit.com/steemit/@aonraza/accurate-use-of-steem-bot-tracker-how-to-use-steem-bot-tracker-efficiently-to-promote-your-steemit-post-and-get-the-profitable

hello @yabapmatt for what I see you are a programmer I really liked your information and as you explain I would like to know more about what you do, when I talk about some programming in one of my post I will mention it nc if there is no problem with that. because I have plans to talk about network programming and other things if you are interested, you can write me happy day

this is something new to me, thanks for this information..

thanks for your contribution.steam really are doing great things

I like posting you

highly beneficial information of friends @yabapmatt I hope your day is fun

Coin Marketplace

STEEM 0.25
TRX 0.19
JST 0.035
BTC 92221.99
ETH 3313.28
USDT 1.00
SBD 3.85