Inventing the “Google” for predictive analytics

From MIT News:

Platform analyzes big data to answer plain-language business queries in minutes instead of months

Now MIT spinout Endor has developed a predictive-analytics platform that lets anyone, tech-savvy or not, upload raw data and input any business question into an interface — similar to using an online search engine — and receive accurate answers in just 15 minutes.

The platform is based on the science of “social physics,” co-developed at the MIT Media Lab by Endor co-founders Alex “Sandy” Pentland, the Toshiba Professor of Media Arts and Sciences, and Yaniv Altshuler, a former MIT postdoc. Social physics uses mathematic models and machine learning to understand and predict crowd behaviors.

Users of the new platform upload data about customers or other individuals, such as records of mobile phone calls, credit card purchases, or web activity. They use Endor’s “query-builder” wizard to ask questions, such as “Where should we open our next store?” or “Who is likely to try product X?” Using the questions, the platform identifies patterns of previous behavior among the data and uses social physics models to predict future behavior. The platform can also analyze fully encrypted data-streams, allowing customers such as banks or credit card operators to maintain data privacy.

Machine learning is used for complex computational problems that are relatively static, such as image recognition and voice recognition. Written and spoken English, for instance, has been essentially unchanged for centuries.

It isn’t immediately clear what clusters represent, just that there is a strong correlation. Querying the data, however, provides context. With customer data, for instance, someone might query which customers are most likely to buy a specific product. Using keywords, the platform matches behavioral traits — such as location and spending habits — of customers who have bought that product with those who haven’t. This overlap creates a list of possible new customers that are apt to buy the product.

To test the platform, the researchers worked early on with the U.S. Defense Advanced Research Project Agency (DARPA) to analyze mobile data in certain cities in times of civil unrest to show how emerging patterns can help predict future riots. Altshuler also spent a couple months in Singapore analyzing taxi ride data to predict traffic jams in the city.