SteemSQL Wrapper — analyze any Steemit data right in your browser with one click
You see all these analytical posts, want to start creating something similar, but have no idea where to start? Now you can!
Problem
Steemit has really a great infrastructure — many various developers are working on great projects, some of them extending features of Steemit (like steemstats.com or Vessel by @jesta), some creating something absolutely new (like utopian.io by @elear), some helping users and giving answers to their questions (like steemwhales.com by @heimindanger).
And it also has many great projects for developers (e.g. SteemSQL by @arcange, Piston by @xeroc, or SteemJS by @fabien). The problem is that all these projects are not tolerant of non-tech people. For example, what if you just want to get info about the most upvoted posts in the past month? While I was writing my first Steemit post, I looked at all available tools and now can allocate several options:
Writing a crawler
It is very inefficient. And what if you want to get info about the most upvoted posts in January 2017? I just want to note that in this case, you will need to crawl more than 3.5M posts (more precisely, 3,561,711).
Piston/SteemJS/steem-python
It is great if you want to do a complex analysis of absolutely all Steemit data. But again, it is very inefficient in other cases — you will need to get data block-by-block, meaning that in order to create a list of the most upvoted posts in January 2017, you will need to get all blocks from today.
SteemData
MySQL right now is not available, API is not very flexible (but I think that in several cases it will be the best solution). MongoDB is NoSQL, meaning that Mongo has its own system of queries and does not support SQL. I personally do not have any experience with Mongo or any other NoSQL databases, so in my case, this is not the best solution. I believe that for the most non-tech users it is also not the best solution because SQL has many various guides and tutorials (I guess it is about dozens of thousands) while Mongo has significantly less popularity (this is logical because SQL is the standard and it is used by many databases). But if you have any experience with Mongo, then it will be a great solution.
SteemSQL
I personally believe that this is the best solution for the most analysis. At least in theory. In practice, it also can raise some problems. For example, it requires drivers for SQL Server, but installing these drivers from Linux or Mac is a real adventure; Microsoft even does not have any notes that they have these drivers for Ubuntu 14.04 and 17.04 (moreover, all answers on askubuntu, serverfault, stackoverflow, and on any other similar projects say that Microsoft has drivers only for Ubuntu 15/16).
But the biggest problem for non-tech users is that SteemSQL has only two options for use: programming languages and Excel. You cannot just read a quick tutorial about SQL, start experimenting while you are learning, and get the first results in less than an hour. Also, Steemit has many mobile-only users who cannot use any of these tools even in theory.
Solution
So I decided to create a solution — SteemSQL Wrapper
It allows use SteemSQL right in your browser without any dependencies. It even allows using the several most popular queries with one click! It means that non-tech users no longer need to install any drivers/Excel and learn programming languages. They can just go to SteemSQL Wrapper and write any SQL queries to get results right in their browser. It also has the link to the SQL Tutorial for users who do not know SQL.
But it is not everything. The important part of the project is the possibility to get results even without knowledge of SQL — I prepared the several most popular queries and anyone can use them with one click. These queries include:
- Тоp 30 posts by number of upvotes
- Top 30 posts by number of comments
- Top 30 posts by pending payout
- Daily number of posts
- Daily number of new accounts
- Daily number of votes
- Top 50 utopian contributors by pending payout
I will extend this list in the near future and if you want to add any specific query then just write me.
Future features
In fact, they depend on your feedback. But right now I have plans for the next features:
- Easy sharing of queries between users
- API that will allow developers to send a simple GET request with their SQL and get JSON with all data
- A simple interface which will show all tables/columns from the database
- Autocomplete
- History of your queries
- Analysis of received data (sort and calculate averages/medians/sums of columns)
- Ability to create custom graphs from data
For Moderators
My commits:
Posted on Utopian.io - Rewarding Open Source Contributors
Very cool project!, this will be very useful and interesting to follow!
I would love to see the future features!
Thank you for your submission.
Your contribution cannot be approved yet, because it is in the wrong category. See the Utopian Rules.
This fits better under development work, provided this is your original work. To be accepted, you need to provide proof of work, but also link to your related github repository along with relevant recent commit links.
You can contact us on Discord.
[utopian-moderator]
No problem, but I have a few questions.
Development
andSub-Project
categories? Could you describe what exactly you want to see in theSub-Project
category? For example, if I will develop something like DTube, ChainBB, or dMania, what will category be right?Sub-Project
category is for launching projects whileDevelopment
is for further development. For example, if I launch a new project then I write a post inSub-Project
about reasons to create it and detailed description (about an audience, roadmap, main features, how to use, etc). But when I add new features or fix bugs then I need write about it inDevelopment
. Is it not so?Sub-Project
should be in theDevelopment
category. They all are related to the development. Am I not right?Hey no worries. Thanks for inquiring
sub-projects describe, in great details, major functionalities "suggested" to be implemented as part of an existing project.
If you check out Utopian Rules you will see the top rule as
A Sub-Project is a set of new unseen features having a great impact on the project.
keywords are unseen, and great impact.
If you check out the remaining rules, as I mentioned earlier, the features need to be very well described, with screenshots, illustrations where possible,...
Sub-projects are suggestions for projects that you do not own, it doesn't make sense to come up with a proposal to yourself :)
Development category on the other hand is for work that has been already developed by you, with proof of work and links to github, PR, and commits.
Hope this clarifies it well, and feel free to reach out if you have further questions
You can contact us on Discord.
[utopian-moderator]
Ok, got it. And did update the post.
The last question: the difference between
Suggestion
andSub-Project
is only in a size of suggesting features? Minor features are inSuggestion
, big features are inSub-Project
. Right?Yes that is correct.
We actually still need one item from you. Since your utopian name is different than the github one, we need proof of work, confirming that you are the author of the github.
You might want to edit the readme file and include this detail, and attach a screenshot confirming this here.
Thanks for the explanation.
About the confirmation — just to clarify, rules say:
And I provided this screenshot. Also, an author of this post provided exactly what you want but his post was rejected.
Why are requirements of all moderators so difference? Anyway, I did update the repo description.
![image.png](https://steemitimages.com/640x0/https://res.cloudinary.com/hpiynhbhq/image/upload/v1513480524/msqgkwkqycujxnhhcp00.png)
Thanks, accepted now.
Mods have general rules to follow. Sometimes they make mistakes, we are all human. In any such case, feel free to reach out.
Perfect post.. Thank you very much for the very useful contribution
This post has received a 11.34 % upvote from @booster thanks to: @emptyname.
Congratulations! This post has been upvoted from the communal account, @minnowsupport, by emptyname from the Minnow Support Project. It's a witness project run by aggroed, ausbitbank, teamsteem, theprophet0, someguy123, neoxian, followbtcnews/crimsonclad, and netuoso. The goal is to help Steemit grow by supporting Minnows and creating a social network. Please find us in the Peace, Abundance, and Liberty Network (PALnet) Discord Channel. It's a completely public and open space to all members of the Steemit community who voluntarily choose to be there.
If you would like to delegate to the Minnow Support Project you can do so by clicking on the following links: 50SP, 100SP, 250SP, 500SP, 1000SP, 5000SP. Be sure to leave at least 50SP undelegated on your account.
This post has received a 0.18 % upvote from @drotto thanks to: @banjo.
Congratulations @emptyname! You have completed some achievement on Steemit and have been rewarded with new badge(s) :
Click on any badge to view your own Board of Honor on SteemitBoard.
For more information about SteemitBoard, click here
If you no longer want to receive notifications, reply to this comment with the word
STOP
Hello @emptyname, this is really something useful :-D
Could I suggest 2 improvements :
Anyway, great job and thanks for sharing.
This tool is easy to use, I really like it.
Hey @oroger, thanks for the feedback and suggestions!
NOLOCK
and did read posts of @arcange about it. But now it is optionalFuture features
section of the post).Hey @emptyname I am @utopian-io. I have just upvoted you!
Achievements
Community-Driven Witness!
I am the first and only Steem Community-Driven Witness. Participate on Discord. Lets GROW TOGETHER!
Up-vote this comment to grow my power and help Open Source contributions like this one. Want to chat? Join me on Discord https://discord.gg/Pc8HG9x