Steem Bot Tracker - Major Code Refactor
The Steem Bot Tracker website was the first Steem-related development project I've ever done. I built it in a couple days back in early October as an experiment and learning experience using the Steem APIs and to see if I could make something to help me better utilize the voting bots to promote my posts.
At the time I never imagined it would grow to be what it is today, and I also had a lot of learning to do on the best and most efficient ways of utilizing the Steem APIs. Over the past 5 months I added loads of new features on top of a messy and inefficient code base and it was finally time to go back and fix it.
What could have been done better
Previously, the entire site ran in the browser. It was just an HTML page with some JavaScript and there was no server-side component at all. This meant that all of the Steem API calls were made from each client that was viewing the site separately.
When the site first launched not many people used it and there were about 5 or 6 bid-based voting bots in total, so having each client viewing the page make its own API requests wasn't really a big deal. But now approximately 2,500 people use the site each day, with around 30 on at any given time, and there are almost 100 bid-based voting bots (which is kind of ridiculous, but we'll save that for another post).
The site reloads the voting bot info every 30 seconds, so that means that all 30 people with the site open at one time were making API calls to get information for up to 100 voting bots every 30 seconds!
While this is still probably inconsequential relative to the total traffic to the api.steemit.com node, it needed to be fixed. Additionally, when the api.steemit.com node has issues - like it did last Friday - the site wasn't able to load data which caused a number of problems for bot users and owners and there was no easy way for me to fix that other than to wait for the node to be fixed.
The fix
I have been working for the past couple weeks on updating the site to pull data from a central server instead of directly connecting to a full Steem node. This creates a number of advantages. Now the site can get nearly all of the data it needs about all of the bots from one API call to the server, as compared to potentially hundreds of calls before.
Additionally should the api.steemit.com node have issues again the server can easily switch over to another node that is working and the site will be unaffected.
On top of that, the site should now work better for the users. The list of bid bots should load nearly instantly and also update nearly instantly as you adjust the filters since what was previously a large number of Steem API calls has been reduced to one call to get data pre-loaded on a central server.
The code has been generally re-factored and cleaned up as well which should lead to fewer bugs and faster turnaround time for new features as well as making it much easier for others to contribute to the development of the project.
For bot owners
For bid-bot owners, one thing to note is that from now on the site will require that bots implement the bids API in order to display detailed bid information. Previously for bots that did not provide an API the site would try to figure out the bid details from the bot's account history, but that was never very accurate and caused too many problems by displaying incorrect data.
The following are the list of bid-bots which currently do not provide an API:
@boomerang, @minnowhelper, @discordia, @mrswhale, @hellowhale, @moneymatchgaming, @levitation, @yourwhale, @sleeplesswhale, @voterunner, @anadolu, @chronocrypto, @puppybot, @zapzap, @payforplay, @lightningbolt, @thebot
If the owners of any of those bots need help setting up the API please get in touch with me.
For bot users
This was a mostly behind the scenes change to make the site run better and faster, so for those of you who were hoping for some new cool features - sit tight, they will come soon! These changes were necessary to make it easier to develop new features and keep the site working when the SteemIt node has issues.
That being said, hopefully you notice the site running better and faster now! If you have any new features you would like to see on the site please let me know in the comments.
Thanks for your support!
As always I want to thank everyone who has helped and supported me in creating this tool. I love that it has been able to help so many people learn about and use vote selling services to promote their content and grow their following and earnings on the Steem platform. Once again if you have a suggestion or feature request I encourage you to submit an issue on the GitHub repo for the project. Please stay tuned for more updates in the coming weeks!
For those of you who are interested in looking at the code, there are too many commits to list here, but the updated code is in the "big_changes" branch which you can see here: https://github.com/MattyIce/bottracker/tree/big_changes
Posted on Utopian.io - Rewarding Open Source Contributors
Thank you for the contribution. It has been approved.
You can contact us on Discord.
[utopian-moderator]
Hey @ms10398, I just gave you a tip for your hard work on moderation. Upvote this comment to support the utopian moderators and increase your future rewards!
Hey, @yabapmatt, so nice to have you on our side!
Don't you think about storing data in the database for bidbot analytics?
Yeah.. Right.
Thanks for the good job @yabapmatt :)
I've got one question, why those voting bots allows negative ROI? The main purpose of the bot is to help user to get traction and in the same time make a few buck or at least not loose their capital... That's how most bot owners are advertising their bots and most of the time it's completely wrong:
yesterday I saw a -72% ROI ... An other exemple here:
My suggestion is to ad one more button on the top of the site to show only non negative ROI bots, then some people can votre for them first and more bot owner might be interested to setup the non negative ROI setup on their bot.
Hey! I am the owner of the bidbot, @fishbaitbot . The problem with having a fixed non negative ROI is that people will then start using bid bots for earning instead of getting earning through good quality posts. The only aim of BidBots is to provide exposure. Paid profits on steemit is detrimental to steemit!
Hello @omstavan thank you very much for your reply. I just think that voting bots should have at least a negative ROI limitation. That's my personal opinion, just to get a system more fair :)
And i agree, this is kind of advertising, so there should be an upper limit. There shouldn't be a very high cost for exposure for platforms like steemit. A negative ROI limit would ensure that it doesn't cost too much!
Many voting boys implement that. My bot has a -10% limit on ROI. Some others also have similarly set values. These values are mentioned in the Steembottracker. You can select which bout to use from there!
This from someone who sells his vote to the highest bidder. This is comedy.
His site does show when the bots are showing a positive ROI. In the final ten minutes they turn green. Also popups come up for profitable bots. But I don't believe there is a filter to filter out neg ROI bots. You can however see which ones are neg as max suggested bid is above 0.
Trouble with having only positive ones show is that pretty much all the bots that are more than say 20 mins from voting show a positive ROI and then people come in toward bid time and reduce your profitability.
In the actual bot he created he put in a min ROI setting where the bot owner could stop taking bids below a certain ROI.
Some say the bots are to gain visibility. Ie upvotes = more chance of being seen. Not to make a profit. Ultimately the culture of STEEM is about creating quality content over simply going for cash.
In any case the bots face the issue of if their vote pool doesn't fill they ... don't lose money per se but they get less than their vote value
Hello peter thanks for your reply. as @therealwolf mention there is a column where you can see negative ROI limit and some boots have a 0%, I guess if you bid you are certain to receive a fair vote back.
If the bots are fair with their user they will be used more, if it's the opposite people will stop using them. You can see on the Steem bot tracker page, most voting bot have no negative RIO limitation.
Thank you again @yabapmatt. I've been on your site pretty much daily as I try to understand voting and bots.
Interestingly I did 3 tests just over a week ago where I bought votes. Over the week the estimated payout according to steemit (which I assume is in SBD) dropped by more than I would expect inflation to be. I have not worked out why. Perhaps there is an acceleration in the amount of content being produced. Certainly the new posts page is showing that.
https://steemit.com/test/@petervroom/test-1-bot-vote
https://steemit.com/test/@petervroom/test-2-bot-vote
https://steemit.com/test/@petervroom/test-3-bot-vote
The payout shown on steemit.com is based on the market price of STEEM so it will go down as the price of steem goes down and vice versa.
Thank you for your generous vote @yabapmatt!
I'm so confused with the currencies! So on steemit the vote value is in STEEM not SBD? In any case the payment is in SBD/SP/STEEM and they did not change much in relationship to each other so shouldn't the the amount listed on steemit stay the same pretty much?
If it were in US$ that would be different.
Confused!
The payout value shown on steemit.com and other front end sites shows the current USD value of the Steem Power you will get from the post if you have chosen to receive 100% of the rewards as Steem Power.
So if the upvotes on a particular post would cause it to receive 1 Steem Power, and the market price of STEEM is currently at $2, then the payout for the post would show $2. If the price of STEEM goes up then the payout value shown goes up and vice-versa.
If you choose to take 50% of your post payout in SBD (which you always should right now) then you get half of the payout in SBD where the system assumes that 1 SBD is worth 1 USD. The fact that the actual market value of SBD is significantly different than what the Steem blockchain assumes it to be is the cause for all of this confusion.
Interesting, thank you. I will suggest they post this to the faq.
@yabapmatt are you sure steemit shows the payout in USD?
Have a look at these posts where I recorded what your site says will be paid out in SBD (ie selected SBD in the dropdown) and then what steemit said the payout would be. I think the USD to SBD/STEEM just over a week ago was something like USD $2 per Steem. Using the 30 day average it would be even higher. Yet the expected steemit payout and your calculated SBD payout were roughly the same.
https://steemit.com/test/@petervroom/test-1-bot-vote
https://steemit.com/test/@petervroom/test-2-bot-vote
https://steemit.com/test/@petervroom/test-3-bot-vote
Best
No, I think you're misunderstanding. What is shown on steemit.com is what I call "POST REWARDS" on the bot tracker site. If you select that from the drop down then it should match what is shown on steemit.com.
Steemit.com assumes 1 SBD is worth 1 USD, the bot tracker site values SBD at the actual market value, so that's why the numbers will be different.
You should talk to the owner of the bot to increase the percentage of the bot since lately they are giving a 150% refund and that brings an incredible loss. Invert 5 sdl and return 7 or $ 7.5 in voting .. for that reason stop using bot ..
previously they were super profitable, they returned 200% up to 245%, but that changed.
We as bot owners and authors are extremely grateful for your service here Matt in helping us find the bots available and promote our posts!
Resteeming soon to help more of us using https://steembottracker.com/ to get to know you!
Jerry - I think we all appreciate the contribution you are doing to this platform as well.
Nice to see so many involved in making Steemit, the best social platform out there.
There was a recent change on the gui, for the "action" part, I prefer the old way with one button for bid and another one for details.
I am lazy, I prefer to do only one click :)
Can you add the column with the current ROI ?
Yes, you are correct - this was necessary to make room for new information that has or will soon be added to the table which is getting quite large. I will consider adding the current ROI if I can fit it. Stay tuned for more new updates coming soon!
so i just need to run a bot, to clone this repo to any server and fit the config?
If it requires Javascript and overloads the CPU in a machine that lacks a GPU, it probably mines some Proof Of Work currency.
I still believe so.
I might be wrong.
Do API calls consume so much CPU capacity?
The Javascript could be spared by loading the newest image/frame/state every time the HTML page is reloaded.
In your defense, loading a Steemit page, and scrolling in it beyond a few pages overloads the CPU too.
steemit.com is shit, and in the past filling the memo part in the wallet felt like POW too.
It didn’t overload the CPU and it’s certainly not mining cryptocurrency, it was just inefficient and needed a refactor.
It did.
Not as much as steem.supply and some other websites unrelated to STEEM that I know, but it did.
We may regard the word overload in different ways.
I did not mean that it clogged it completely like some of those websites.
I saw your acknowledgement of the inefficiency in your initial post, and added another suggestion for improvement in my comment after I edited it.
JS is evil.
I try to avoid JS demanding websites as much as possible.
I will try your website again, but compulsory JS is always bad.
My suggestion in my previous comment applies to other real time needs as well.
This comment has received a 24.88 % upvote from @steemdiffuser thanks to: @stimialiti. Steem on my friend!
Above average bids may get additional upvotes from our trail members!
Get Upvotes, Join Our Trail, or Delegate Some SP
You got a 7.25% upvote from @adriatik courtesy of @stimialiti!
This comment has received a 50.00 % upvote from @steemdiffuser thanks to: @stimialiti. Steem on my friend!
Above average bids may get additional upvotes from our trail members!
Get Upvotes, Join Our Trail, or Delegate Some SP
You got a 50.00% upvote from @greengrowth thanks to @stimialiti! You too can use @GreenGrowth by sending your post URL in the memo field to the bot. Minimum bid is 0.01.
If you feel this post is spammy or not worthy of @Greengrowth you can contact a moderator in our Discord Channel https://discord.gg/6DhnVTQ.
You got a 6.94% upvote from @adriatik courtesy of @stimialiti!
you are on the right track my friend. getting bots on your side is what will keep you around. your hard work is paying of as you can see your post is on trending. quite a big deal. i wish you the very best of luck
Thanks Matt, can I ask a question..
How come the Bid History doesnt seem to work anymore?
Does it work for anyone else?