Crossfilter Tutorial #1 : Crossfilter Teminology, basic commands

What Will You Learn?

In this tutorial series you'll learn crossfilter.js, how to handle data in crossfilter and then pass it to the charting library in order to develop the charts for your data.

Requirements

• Basics of HTML and CSS
• Essentials of JavaScript
• Understanding of multi-dimensional data

Difficulty

• Intermediate

Tutorial Contents

What is Crossfilter.js

Crossfilter - a JavaScript Library developed by Mike Bostock for Square, it allow you to group, aggregate and filters the raw data over the web. It is useful with web-based charting dashboard when there is interaction between charts, updating the one chart results in update to other charts. It is like a client-end OLAP server that use to answer your multi-dimensional analytical (MDA) queries.

Crossfilter didn't works alone, it combined to work with another JavaScript libraries dc.js (you can learn dc.js from our tutorials on dc.js), d3.js or the Google Visualization API.

Crossfilter doesn't have its own UI and doesn't draw charts or graphs, this only organizes your raw data. This is why we used other charting libraries (mentioned above) to draw charts with the help of organized data provided by crossfilter. The charts are developed with other libraries and the whole work related to data is done through crossfilter.

The crossfilter is a JavaScript Library, the goals you achieve with crossfilter can be achieved simply with JavaScript, but Crossfilter is quick and really really fast, supports extremely fast (<30ms) interaction with coordinated views. So, this is reason we really recommend you to use this library when you want to explore large multivariate datasets.

Now, we'll start to work with Crossfilter...

Crossfilter Terminology:

Before we jump to start our work with crossfilter we'll have to understand the terminology behind the crossfilter.There are three terms in crossfilter you need to understand first the facts , dimension and group.

Facts

First we talk about facts, if you have some data records then each row in the record would be called a fact in crossfilter terminology. For Example if you record you sales of the day for the month, then the record a day is said to be a fact. Here we have a payment transaction of a restaurant,

              var paymentsRecord = [
              {date: "2011-11-14T16:17:54Z", quantity: 2, total: 190, tip: 100, type: "tab"},
              {date: "2011-11-14T16:20:19Z", quantity: 2, total: 190, tip: 100, type: "tab"},
              {date: "2011-11-14T16:28:54Z", quantity: 1, total: 300, tip: 200, type: "visa"},
              {date: "2011-11-14T16:30:43Z", quantity: 2, total: 90, tip: 0, type: "tab"},
              {date: "2011-11-14T16:48:46Z", quantity: 2, total: 90, tip: 0, type: "tab"},
              {date: "2011-11-14T16:53:41Z", quantity: 2, total: 90, tip: 0, type: "tab"},
              {date: "2011-11-14T16:54:06Z", quantity: 1, total: 100, tip: 0, type: "cash"},
              {date: "2011-11-14T16:58:03Z", quantity: 2, total: 90, tip: 0, type: "tab"},
              {date: "2011-11-14T17:07:21Z", quantity: 2, total: 90, tip: 0, type: "tab"},
              {date: "2011-11-14T17:22:59Z", quantity: 2, total: 90, tip: 0, type: "tab"},
              {date: "2011-11-14T17:25:45Z", quantity: 2, total: 200, tip: 0, type: "cash"},
              {date: "2011-11-14T17:29:52Z", quantity: 1, total: 200, tip: 100, type: "visa"}
            ];

A single row in the record is a fact, hope you gets the idea what is a fact in crossfilter terminology.

Dimension

Now we'll move to what is a dimension in crossfilter?

You can see the facts in the above data, if we say that how many customers you've in each hour of the day then you've to group the transactions by hour. Then sales by hour is an aspect of the data, you can create various aspects from your data. Each aspect is known as a dimension. So, hour would be a dimension.

If you didn't get it, let's make it easier for you. For Example: a person works 8 hours a day for a month and earn some amount, If you want to see that how many hours he work in a week, then you've grouped your data into weeks, Then the week would be a dimension of the data.

Group

And at last the group, we think you got it what is group, if not then above in dimension we group our data into week. Its the grouping of the data. In the above data example you can group our data by the payment type or by quantity or something else.

So, these are the three important terminologies used in the crossfilter that you've to understand before dive into corssfilter.

Setting the Environment:

Before to start writing code in corssfilter we'll have to set the environment to work with corssfilter. As you mentioned above crossfilter is a JavaScript, so, we will use the .html file to write our JavaScript code. Open you text editor, create a file with name index.html and save it in your working directory. Now write the basic HTML code, necessary to work with .html files.

<!DOCTYPE html>
<html lang="EN">
    <head>
      <meta charset="utf-8">
      <title>Crossfilter</title>
    </head>
    <body>
    </body>
</html>

Now you've to load crossfilter.js library in you files, that we able to use the crossfilter. To load this library in you files there are two methods

• Through CDN
  In this method we just use the URL of the file and paste it in our working file. But it require the internet connection to work. If you want to use this method then copy and paste this code in you <head> tag.

      <script type="text/javascript" src="https://cdnjs.cloudflare.com/ajax/libs/crossfilter/1.3.12/crossfilter.min.js"></script>

• Locally downloading in your computer
  In this method you download the crossfilter.js file in your computer and then load it in you file, it doesn't require the internet connection to work. To use this method first you've to download the crossfilter.js file form github. Download the file into you working directory and write the code to load it into your file.

      <script type="text/javascript" src="crossfilter.js"></script>

At last we require some data with which we want to perform crossfilter operations, here we copy the data from crossfilter API, If you've your own data then start work with that data, if not then follow us with this dummy data.

        <script type="text/javascript">

            var paymentsRecord = [
            {date: "2011-11-14T16:17:54Z", quantity: 2, total: 190, tip: 100, type: "tab"},
            {date: "2011-11-14T16:20:19Z", quantity: 2, total: 190, tip: 100, type: "tab"},
            {date: "2011-11-14T16:28:54Z", quantity: 1, total: 300, tip: 200, type: "visa"},
            {date: "2011-11-14T16:30:43Z", quantity: 2, total: 90, tip: 0, type: "tab"},
            {date: "2011-11-14T16:48:46Z", quantity: 2, total: 90, tip: 0, type: "tab"},
            {date: "2011-11-14T16:53:41Z", quantity: 2, total: 90, tip: 0, type: "tab"},
            {date: "2011-11-14T16:54:06Z", quantity: 1, total: 100, tip: 0, type: "cash"},
            {date: "2011-11-14T16:58:03Z", quantity: 2, total: 90, tip: 0, type: "tab"},
            {date: "2011-11-14T17:07:21Z", quantity: 2, total: 90, tip: 0, type: "tab"},
            {date: "2011-11-14T17:22:59Z", quantity: 2, total: 90, tip: 0, type: "tab"},
            {date: "2011-11-14T17:25:45Z", quantity: 2, total: 200, tip: 0, type: "cash"},
            {date: "2011-11-14T17:29:52Z", quantity: 1, total: 200, tip: 100, type: "visa"}
          ];

        </script>  

To see the result in the console write the command.

console.log(paymentsRecord);

You can see the data in browser console.

Getting Started with crossfilter

Everything in Crossfilter is scoped under the crossfilter namespace, which is also the constructor. So, our first line of code that we rung in crossfilter is the crossfilter(); and pass it the the data variable.

          var facts = crossfilter(paymentsRecord);

We named the crossfilter variable as facts, we already discussed what is a fact, every row in the our data is a fact.

Now, we'll move to do some practical stuff with crossfilter. But before that you need to know about the methods of crossfilter that we are going to use in our tutorial series.

Crossfilter come up with five important commands that we'll try to cover in this series.

crossfilter
    size()
    remove()
    add()
    groupAll()
    dimension()     

The above three commands of the crossfilter you can run in the console directly, so we'll first achieve these commands.

size()

The method returns us the total number of records that we passed to the crossfilter. The size() method is independent of any filter. We'll discuss later in the tutorial what is a filter.

        <script type="text/javascript">

            var paymentsRecord = [
            {date: "2011-11-14T16:17:54Z", quantity: 2, total: 190, tip: 100, type: "tab"},
            {date: "2011-11-14T16:20:19Z", quantity: 2, total: 190, tip: 100, type: "tab"},
            {date: "2011-11-14T16:28:54Z", quantity: 1, total: 300, tip: 200, type: "visa"},
            {date: "2011-11-14T16:30:43Z", quantity: 2, total: 90, tip: 0, type: "tab"},
            {date: "2011-11-14T16:48:46Z", quantity: 2, total: 90, tip: 0, type: "tab"},
            {date: "2011-11-14T16:53:41Z", quantity: 2, total: 90, tip: 0, type: "tab"},
            {date: "2011-11-14T16:54:06Z", quantity: 1, total: 100, tip: 0, type: "cash"},
            {date: "2011-11-14T16:58:03Z", quantity: 2, total: 90, tip: 0, type: "tab"},
            {date: "2011-11-14T17:07:21Z", quantity: 2, total: 90, tip: 0, type: "tab"},
            {date: "2011-11-14T17:22:59Z", quantity: 2, total: 90, tip: 0, type: "tab"},
            {date: "2011-11-14T17:25:45Z", quantity: 2, total: 200, tip: 0, type: "cash"},
            {date: "2011-11-14T17:29:52Z", quantity: 1, total: 200, tip: 100, type: "visa"}
          ];

          var facts = crossfilter(paymentsRecord);
          console.log("The total records are " + facts.size());

        </script>

You can see in the console it returns us the 12, because we have total 12 records.

add()

The add() method adds the new record or append the new record in your data. To add new data first create an array[] and pass it the new object.

          var newRecord = [{date: "2011-11-14T17:50:00Z", quantity: 3, total: 400, tip: 200, type: "cash"}];

You can add multiple records at the same time, just separate them by comma "," .

          var newRecord = [{date: "2011-11-14T17:50:00Z", quantity: 3, total: 400, tip: 200, type: "cash"},
            
               {date: "2011-11-14T17:53:00Z", quantity: 1, total: 100, tip: 0, type: "visa"}, 
            
               {...}, {...}, {...} 

];

Here we just add only one record.

          var newRecord = [{date: "2011-11-14T17:50:00Z", quantity: 3, total: 400, tip: 200, type: "cash"}];
          facts.add(newRecord);

You can see now we have the 13 records.

remove();

The method remove the all the records that match the current filters from this crossfilter. The noticable thing is here it didn't remove a sigle record, it removes the all the corssfilter that crossfilter have. So, be careful before using remove() command in crossfilter. Here you can see

          facts.remove();

<!DOCTYPE html>
<html lang="EN">
    <head>
      <meta charset="utf-8">
      <title>Crossfilter</title>
      <script type="text/javascript" src="https://cdnjs.cloudflare.com/ajax/libs/crossfilter/1.3.12/crossfilter.min.js"></script>
    </head>
    <body>



        <script type="text/javascript">

            var paymentsRecord = [
            {date: "2011-11-14T16:17:54Z", quantity: 2, total: 190, tip: 100, type: "tab"},
            {date: "2011-11-14T16:20:19Z", quantity: 2, total: 190, tip: 100, type: "tab"},
            {date: "2011-11-14T16:28:54Z", quantity: 1, total: 300, tip: 200, type: "visa"},
            {date: "2011-11-14T16:30:43Z", quantity: 2, total: 90, tip: 0, type: "tab"},
            {date: "2011-11-14T16:48:46Z", quantity: 2, total: 90, tip: 0, type: "tab"},
            {date: "2011-11-14T16:53:41Z", quantity: 2, total: 90, tip: 0, type: "tab"},
            {date: "2011-11-14T16:54:06Z", quantity: 1, total: 100, tip: 0, type: "cash"},
            {date: "2011-11-14T16:58:03Z", quantity: 2, total: 90, tip: 0, type: "tab"},
            {date: "2011-11-14T17:07:21Z", quantity: 2, total: 90, tip: 0, type: "tab"},
            {date: "2011-11-14T17:22:59Z", quantity: 2, total: 90, tip: 0, type: "tab"},
            {date: "2011-11-14T17:25:45Z", quantity: 2, total: 200, tip: 0, type: "cash"},
            {date: "2011-11-14T17:29:52Z", quantity: 1, total: 200, tip: 100, type: "visa"}
          ];

          // Feeding data to crossfilter
          var facts = crossfilter(paymentsRecord);

          // Adding new record to crossfilter data
          var newRecord = [{date: "2011-11-14T17:50:00Z", quantity: 3, total: 400, tip: 200, type: "cash"}]
          facts.add(newRecord);

          // Remove all the data in crossfilter
          facts.remove();

          // outputing the data in console
          console.log("The total records are " + facts.size());

        </script>
    </body>
</html>

It'll remove all the record

You can see now we've zero records.

groupAll()

Now the last command we can use directly in crossfilter is groupAll(), this method groups your all data and returns a single value to you. Usually we use this command in crossfilter when we need to count the rows in our data or count the sum of one field.

This command come up with its child commands

groupAll()
    value()
    reduceCount()
    reduceSum()

• groupAll().value()

As you know here we dealing with the some transnational data of a restaurant, If you want to know that how many transactions you've done then groupAll().value() will do this for you. It counts all the rows you have in your records.

          var groupAll = facts.groupAll().value();

          console.log(groupAll);

Output:

You can see it counts the total rows for us , so, this is the another method to count records in crossfilter.

• groupAll().reduceCount().value()

It'll return us the same result as in the above method we discussed, because we didn't specify the reduceCount() method and by default it returns the total row counts.

          var reduceCount = facts.groupAll().reduceCount().value();

          console.log(reduceCount);

• groupAll().reduceSum().value()

.reduceSum() method sums the record for us using the specified value accessor function. If you want to count the total amount payment you recieved from our data then this method will do it for you.

This method required a specified value accessor function perform some stuff. So, here we want to count the total payments

          var reduceSum = facts.groupAll().reduceSum(function(d){ return d.total;}).value();

          console.log(reduceSum);

Output:

You can see It Sums all the total payments for us.

If you want to sum the tips then write this code...

          var reduceSumTip = facts.groupAll().reduceSum(function(d){ return d.tip;}).value();

          console.log("Sum Tips: " + reduceSumTip);

Here, Above we learn, what is crossfilte, what is the terminology of crossfilter and the simple commands you can directly run in the crossfilter. Hope you get all this and want to learn more about corssfilter, If you want, then follow this tutorial series and support us.

Source Code:

<!DOCTYPE html>
<html lang="EN">
    <head>
      <meta charset="utf-8">
      <title>Crossfilter</title>
      <script type="text/javascript" src="https://cdnjs.cloudflare.com/ajax/libs/crossfilter/1.3.12/crossfilter.min.js"></script>
    </head>
    <body>



        <script type="text/javascript">

            var paymentsRecord = [
            {date: "2011-11-14T16:17:54Z", quantity: 2, total: 190, tip: 100, type: "tab"},
            {date: "2011-11-14T16:20:19Z", quantity: 2, total: 190, tip: 100, type: "tab"},
            {date: "2011-11-14T16:28:54Z", quantity: 1, total: 300, tip: 200, type: "visa"},
            {date: "2011-11-14T16:30:43Z", quantity: 2, total: 90, tip: 0, type: "tab"},
            {date: "2011-11-14T16:48:46Z", quantity: 2, total: 90, tip: 0, type: "tab"},
            {date: "2011-11-14T16:53:41Z", quantity: 2, total: 90, tip: 0, type: "tab"},
            {date: "2011-11-14T16:54:06Z", quantity: 1, total: 100, tip: 0, type: "cash"},
            {date: "2011-11-14T16:58:03Z", quantity: 2, total: 90, tip: 0, type: "tab"},
            {date: "2011-11-14T17:07:21Z", quantity: 2, total: 90, tip: 0, type: "tab"},
            {date: "2011-11-14T17:22:59Z", quantity: 2, total: 90, tip: 0, type: "tab"},
            {date: "2011-11-14T17:25:45Z", quantity: 2, total: 200, tip: 0, type: "cash"},
            {date: "2011-11-14T17:29:52Z", quantity: 1, total: 200, tip: 100, type: "visa"}
          ];

           // Feeding data to crossfilter
           var facts = crossfilter(paymentsRecord);

          // Adding new record to crossfilter data
          var newRecord = [{date: "2011-11-14T17:50:00Z", quantity: 3, total: 400, tip: 200, type: "cash"}]
          facts.add(newRecord);

          // Remove all the data in crossfilter
          facts.remove();

          // outputing the data in console
          console.log("The total records are " + facts.size());

         // ####### groupAll() ###########

          var groupAll = facts.groupAll().value();

          console.log(groupAll);

          var reduceCount = facts.groupAll().reduceCount().value();

          console.log(reduceCount);

          var reduceSumTotal = facts.groupAll().reduceSum(function(d){ return d.total;}).value();

          console.log("Total Payments: " + reduceSumTotal);

          var reduceSumTip = facts.groupAll().reduceSum(function(d){ return d.tip;}).value();

          console.log("Sum Tips: " + reduceSumTip);



        </script>
    </body>
</html>

Curriculum

• DC.JS Tutorial Series

---

Posted on Utopian.io - Rewarding Open Source Contributors

---