Analysis : Posts created per day and Organic Ratio

Repository

https://github.com/steemit/steem

Introduction

Hearing the call of @eastmael in Utopian discord channel, I started to prepare necessary scripts and extract data from Steem block-chain.

This analysis is searching for answers to the following questions :

• How many posts are posted to Steemit in an average day?
• How many of these posts are organic ( no bid-bot upvoted )

Scope of Analysis

• Analysis date: 27 May 2018
• Analysis timeframe: 2018-05-21T13:42:39 --> 2018-05-22T13:30:15
• Post Quantity: 40 000 posts
• Bots list: All the upvote bots registered to SteemBotTracker by @yabapmatt (99 bots)
  bot list
• All times are in UTC.

Tools

• Finding the posts

A javascript code is written using steemjsAPI to find the posts that is created 5 days ago.
5 days is chosen because :

1. If the posts are paid, the script used doesn't show them, so they must be pending payout status posts
2. If the posts are recent, the author can still buy votes from bid-bots and this will give a false result
3. Mainly none of the major bid-bots accept posts older than 3.5 days, so 5 days is safe.

function search(){
     var query = {
        start_author:"adsactly", 
        start_permlink:"adsactly-tech-news-does-increased-access-to-technology-increase-societal-inequality",
        limit: 100
    };
    steem.api.getDiscussionsByCreated(query, function(err, result) {
        console.log(err, result);
        console.log(query);
        postarray=postarray.concat(result);
        calculate_rest();
    });
}

With above script, starting from any post that is 5 days old, following 100 posts are taken sorted accroding to their posting time.

It is observed that 100 posts are created in approximately 3.5 minutes.
So to cover a range of 24 hours, it is calculated that approximately 40 000 posts must be analyzed.

Below script is written to extend the search to 40 000 posts ( we can get posts in packs of 100's so 400 API request have been sent )

 function calculate_rest(){
        count=(i*100)-1;
        if(i<400){
            



            var query = {
                tag: "",
                start_author:postarray[count].author,
                start_permlink:postarray[count].permlink,           
                limit: 100
            };

            steem.api.getDiscussionsByCreatedAsync(query, function(err, result) {
                //console.log(err);
                if(!err){
                    postarray=postarray.concat(result);
                    i+=1;
                    calculate_rest();
                }else{
                    calculate_rest();
                }
                
                
            });   
            
    }else{calculate_bots();}
    console.log(postarray);     
        }

It has been some time to process this in javascript.
As the 40 000 posts are received it is observed that the calculation was valid, 40 000 posts covered exactly 24 hours time frame.
The first post is at 2018-05-21T13:42:39 and the last post is at 2018-05-22T13:30:15.
Missing only 12 minutes in 24 hours.

The result is put in the Google spreadsheet

• Eliminate the posts that used bid-bots

Using the bot list all the posts are filtered if any of the voter is in the botlist with below code.

function calculate_bots(){
var author=[];
var permlink=[];
var time=[];
var activevotes=[];
var control;
    console.log(postarray);
      for (let i = 0; i < postarray.length; i++) { 
      control=0;
      author.push(postarray[i].author);
      permlink.push(postarray[i].permlink);
      time.push(postarray[i].created);
         for (let j = 0; j < postarray[i].active_votes.length; j++) {
         activevotes = postarray[i].active_votes; 
         if (bots.includes(activevotes[j].voter)) {
            control = 1;
          }
         
          }
         if (control==0){
            postarraybot.push(postarray[i]);
         }
      
    }
    console.log(author);
    console.log(permlink);
    console.log(time);
    console.log(postarraybot);
    
}

Now we have the organic posts list that hasn't been upvoted by any of the bid-bots.

Results

Analyzing the 40 000 posts, the following results are found

• 40 000 posts are posted in 24 hours (- 12 minutes)

• 40 000 posts are posted by 20 362 users ( average 2 posts per author )

• The *rush-hours and slow-hours for posts are as follows
  
  Rush-hours are from 15:30 to 19:30 UTC with 2000+ posts / hour
  Slow-hours are 22:30 to 02:30 UTC with less than 1400 posts/hour

• With an average of 2 post/day the distribution of posting frequency of uses are as follows.
  
  

There is even one user with 256 posts/day.... just crazy!

• Among 40 000 posts, the posts that got upvote from bid-bots are 2959 posts. ( in-line with previous findings).
  This is %7 of all the posts.
  

Contact

• @FireDream - Steemit
• @firedream#3528 - Discord

Links

SteemJs GitHub
SpreadSheet for Data