MIT Researchers have developed an algorithm that can find predictive patterns in unfamiliar data sets better than most humans. In other words, the algorithm can make a computer more intuitive than a human.
The researchers from MIT’s Computer Science and Artificial Intelligence Laboratory created algorithms that can identify interesting features hidden in massive data sets and pools of figures, in order to take some of the strain out of analysis. For example, in a database of a company’s weekly profits and the dates of its sales, the most valuable insight from the data may not be the dates of the sales themselves, but the average profits across the span of the sales, or even just the spans between the sales.
Gaining such insight from data is easier for humans than it is for computers. Machines may be able to process much more data far more quickly, but humans can understand the data’s importance. The ability to glean valuable insight from data sets is precisely what the researchers wanted their algorithms to be able to do. In other words, they wanted a computer program that could read between the lines.
To test the first prototype of their system, the researchers made it compete against humans to find predictive patterns in unfamiliar data sets. Out of the 906 teams it took on over the course of three competitions, the Data Science Machine finished ahead of 615. Its predictions were 94%, 96%, and 87% as accurate as the winning submissions. While the human teams labored for months on their findings, the Data Science Machine took only between two to 12 hours for each of its entries.
In other words, it was more intuitive than about two-thirds of competing humans.
“Cognitive computing will completely take over how you find things online and how digital innovation interacts with you,” says Tom Ajello, Founder/Chief Creative, Makeable. “Machine learning, behavioral learning and more importantly what the machine does for you with that learning is the literal ‘next big thing,’ as they say. Since cognitive systems infer, hypothesize, adapt and improve over time without direct programming or reprogramming, the technology will scale quite rapidly once it is in place. In just the HealthCare industry alone it is predicted that 80 percent of what doctors do (diagnostics) will be replaced by machines, medicine will become tailor-made for each patient and consumer-driven technology will create better incentives to keep people healthy. All based on machine cognition.”
The algorithm has a few ways to replicate human intuition. First, it’s designed to use a database’s structure to generate a myraid of new metrics for comparison, and then finds correlations between them. Second, it pays careful attention to categorical data, like a brand name, and then analyzes the relationship between it and the new metrics.
“We view the Data Science Machine as a natural complement to human intelligence,” said Max Kanter, whose MIT master’s thesis in computer science is the basis of the Data Science Machine. “There’s so much data out there to be analyzed. And right now it’s just sitting there not doing anything. So maybe we can come up with a solution that will at least get us started on it, at least get us moving.”