Adding Retry Logics and Failover in Voting Services

in #steem7 months ago

I haven't revisited the code of voting services for a long long time since most of the time - it works. Today, there are some problems of the nodes, and that is why I decided to take some time to improve/fix it.

The voting services consist of several components but as you can imagine, it is decoupled, it works as something like this:

  1. there are several processes (components) that monitor the posts of delegators and good authors. Once criteras are met, the posts are pushed to database.
  2. the reader process checks periodically the database for new posts to vote, and then vote them one by one.

The problem is that sometimes the node API call failed, like this:

image.png

The error is Internal Error which doesn't give enough information of what went wrong here.
** getDiscussionsByAuthorBeforeDate Error [RPCError]: Internal Error **

Since this is scheduled to run every 10 minutes (as a crontab job), usually a next retry would work. But if a node is down for some time, then it will cause the problem.

Thus, i have added the retry - and also failover logics, so it will choose next RPC node and then try again 3 times before skipping.

for (let tc = 1; tc <= 3; tc ++) {
    console.log(`Try process counter ${tc}...`);
    if (await processForUser(user, totalDelegated)) {
        break;
     }
     failover();
     await sleep(3000);
}

It is not perfect, but it is better, and it works for now.

Steem to the Moon🚀!

Sort:  

sure, voted you here !thumbup

Hello @dorian-lee! You are Magnificent!


command: !thumbup is powered by witness @justyy and his contributions are: https://steemyy.com
More commands are coming!

Thank you for the job and for working and improving the code when needed :)

Steem to the moon

I have just delegate 5,000+ SP to @justyy

Yeah, I noticed that some problems with the nodes, I am still investigating, but voted you already on the post. Thanks !thumbup

Hello @puncakbukit! You are So Nice!


command: !thumbup is powered by witness @justyy and his contributions are: https://steemyy.com
More commands are coming!

Awesome that you updated the code, but doesn't seem to be working properly just yet, at least you haven't made any upvotes for several hours, and my most recent post didn't get one either...

Coin Marketplace

STEEM 0.21
TRX 0.20
JST 0.034
BTC 90641.97
ETH 3108.44
USDT 1.00
SBD 2.99