(Previous Post: Part 8)

Previously, I introduced the getState() API function that is used by the Condenser app (a.k.a., Steemit.com) to retrieve content for a given URL path, such as 'trending/steemdev'. That function is great in order to get a page-worth of posts and other information, such as Account objects and the current feed price, all in one call. But, in order to retrieve more than just the first 20 posts, you must use a different API function.

Today, we'll look into the various getDiscussionsBy* functions.

The get_discussions_by API Functions

A lot of the documentation for the Steem API only exists in the form of source code - that's part of the reason why I started this series of posts (to document what I find as I explored the source code while trying to figure out how the API works). The database_api header file is a good place to start, since header files tend to be better documented than the rest of the source.

In that file, you'll find a group of functions with the same signature:

vector<discussion> get_discussions_by_payout(const discussion_query& query )const;
vector<discussion> get_discussions_by_trending( const discussion_query& query )const;
vector<discussion> get_discussions_by_created( const discussion_query& query )const;
vector<discussion> get_discussions_by_active( const discussion_query& query )const;
vector<discussion> get_discussions_by_cashout( const discussion_query& query )const;
vector<discussion> get_discussions_by_votes( const discussion_query& query )const;
vector<discussion> get_discussions_by_children( const discussion_query& query )const;
vector<discussion> get_discussions_by_hot( const discussion_query& query )const;
vector<discussion> get_discussions_by_feed( const discussion_query& query )const;
vector<discussion> get_discussions_by_blog( const discussion_query& query )const;
vector<discussion> get_discussions_by_comments( const discussion_query& query )const;
vector<discussion> get_discussions_by_promoted( const discussion_query& query )const;

Note: In the C API, snake_case is used, while Steem.js uses camelCase. Simply remove the underscores and capitalize each subsequent word to figure out what the function name needs to be for JavaScript.

So, let's say that you used getState('trending/steemdev') to retrieve the first 20 posts, and you want to fetch the next 20. How would you do that with one of these API functions?

First, you need to know which database index you were really using for the getState() call. From a pure data point of view, you can check the data returned by getState() to see which array under discussion_idx is populated.

In this case, it was 'trending', so it's pretty easy to figure out that you probably want the get_discussions_by_trending(query) function (or, rather, getDiscussionsByTrending(query) in JavaScript). But, what's that query argument all about?

discussion_query Struct

Again, referring to the header file, we see that discussion_query is defined as:

/**
 *  Defines the arguments to a query as a struct so it can be easily extended
 */
struct discussion_query {
   void validate()const{
      FC_ASSERT( filter_tags.find(tag) == filter_tags.end() );
      FC_ASSERT( limit <= 100 );
   }

   string           tag;
   uint32_t         limit = 0;
   set<string>      filter_tags;
   set<string>      select_authors;     ///< list of authors to include, posts not by this author are filtered
   set<string>      select_tags;        ///< list of tags to include, posts without these tags are filtered
   uint32_t         truncate_body = 0;  ///< the number of bytes of the post body to return, 0 for all
   optional<string> start_author;
   optional<string> start_permlink;
   optional<string> parent_author;
   optional<string> parent_permlink;
};

Essentially, this is a structure that is used by a lot of the API functions, but functions only take what they need, so some properties will be ignored even if you provided data.

A JavaScript Example

Here's an example of how a call into getState() can be followed by a call into getDiscussionsByTrending():

let index = 'trending'
let tag = 'steemdev'

steem.api.getStateAsync(`${index}/${tag}`)
  .then(function (o) {
    let posts = o.discussion_idx[tag][index]
    let last = o.content[_.last(posts)]

    let query = {
      tag: tag,
      limit: 20, 
      start_author: last.author,
      start_permlink: last.permlink
    }

    steem.api.getDiscussionsByTrendingAsync(query) 
      .then(r => console.log(JSON.stringify(r,null,2)))
        .catch(console.log)          
      })
  })
  .catch(console.log)

Note: As @pilcrow points out in his Angular tutorial that also introduces the getDiscussionsBy* functions, the function itself returns a promise. I'm a creature of habit, so that's why my code shows *Async() in the function name because it explicitly tells me that the function was promisified by Bluebird.js under the covers (and that's a Bluebird convention). You'll get the same result with or without Async in the function name if you follow the function call with a .then().

So, what's happening here? Well, first we get the top 20 posts of trending/steemdev using getState(). The sorted list of posts will be in the .discussion_idx.steemdev.trending property (a string array of permlinks):

"discussion_idx": {
  "steemdev": {
    "category": "",
    "trending": [
      "good-karma/esteem-filters-community-input-om0vmqj9sw",
      "ausbitbank/steemvids-alpha-update",
      "steemreports/steemreports-outgoing-votes-analysis-tool",
      "ontofractal/glasnost-v0-12-released-now-with-postgresql-realtime-and-7-day-lookback-comments-data-sync-open-source-app-server-for-steem",
      "morning/just-another-wordpress-steem-plugin-also-introducing-steemeasy-com",
      "good-karma/good-karma-witness-update-22nd-july-2017-ulf0cx9y6o",
      "almost-digital/dsteem-playground",
      "rycharde/proposal-for-new-rules-regarding-self-votes-and-voting-rings",
      "adept/steemh-com-a-hacker-news-styled-interface-for-steemit",
      "steepshot/the-practice-of-programming-using-graphene-based-blockchains",
      "davidk/steemphp-v0-2-released",
      "djvidov/osteem-chrome-extension-it-s-alive-and-need-20-40-alpha-testers",
      "good-karma/esteem-calling-for-volunteer-translators-16-get-reward-lqpst47n77",
      "calamus056/extensive-curation-stats-overview-since-hf19-june-20th-2017",
      "dez1337/steemj-v0-3-1-has-been-released-update-13",
      "recrypto/wordpress-steem-1-0-2",
      "klye/klye-witness-update-07-22-2017",
      "freyman/esteem-steem-no-mobil-preguntas-frecuentes-faq",
      "djvidov/osteem-first-alpha-version-its-almost-ready-its-time-to-create-an-chrome-developer-account",
      "jfollas/write-a-steemit-web-app-part-8-retrieving-content-with-getstate"
    ],
    "payout": [],
    "payout_comments": [],
    "trending30": [],
    "updated": [],
    "created": [],
    "responses": [],
    "active": [],
    "votes": [],
    "maturing": [],
    "best": [],
    "hot": [],
    "promoted": [],
    "cashout": []
  }
}

From this array, the code picks the last one to use as the start_author and start_permlink for the continuation list of the next 20.

Note that the maximum limit is 100, so there's no real reason to only retrieve 20 posts at a time if you need more. Also note that the results of the getDiscussionsByTrending() call will start with the last post that you already have - so be prepared to handle duplicate data if you are going to merge the results of the two calls.

Unlike getState(), the getDiscussionsByTrending() function will return just an array of posts - if you need the Author's Account metadata, etc, then you will need to make subsequent API calls to fetch that data.

Also, while getState() truncates the post body at 1024 characters, getDiscussionsBy* will not truncate the body unless you provide a truncate_body value in the query struct.

What about Comments?

Stay tuned - I'll cover how to retrieve (and post) Comments next time.

(Next Post: Part 10)