The Making of Great Hypothesis

How Bill and Melinda Gates’ favorite book Factfulness will leapfrog your data science practice

Motivation

So far, data science education focus heavily on data wrangling, hypothesis testing and causal inference. Very rarely do we hear about the preceding step — hypothesis formation.

Forming the wrong hypothesis is costly

Once you’re set on testing a hypothesis, you can spend hours, day, months or even years gathering data, cleaning it, visualizing it, constructing fancy predictive and causal models to no avail. Your pet hypothesis simply did not work— nothing is significant!

That marks the beginning of your identity crisis. You start feeling sorry for the time you’ve wasted. You begin to question your smartness for the very first time in your life. “Maybe I’m not as smart as my 99th percentile SAT/GRE/LSAT indicates,” you dreaded. But you’ve invested so much. Can you really admit defeat so easily? You fingers just can’t resist the urge to tweak the design every so slightly, and see if version 1000 would be the magically design that conform to your theory. You think you’re an honest, upstanding citizen. But in your persistent pursuit of significance, you’ve landed yourself squarely in the land of p-hacking.

It happens to the best of us

The question is: how can we form better hypothesis? Last week, I bumped into just the right book Factfulness. It talks about “10 reasons we’re wrong about the world, and why things are better than you think”. I have personally committed quite a few mistakes on the list. The key to forming smarter hypothesis, I think, lies in developing a deep understanding of our systemic biases. By side-stepping these tempting yet misguided hypothesis, we can save ourselves a lot of time and anguish.

Disclaimer

This blogpost is a cheatsheet, created for self use and sharing with friends. If you find it useful, I strongly encourage you to purchase the book on Amazon. I simply cannot do justice to the greatest paperback I’ve read in years. I’m sure you will be delighted by the masterful data-based story-telling. You might even find a lot of insights that are not encompassed in this cheatsheet.