Crowdsourced Competition Produces Heart Disease Diagnostic Algorithm

March 31, 2016 – Two days ago I wrote about how crowdsourcing is being used to help with medical diagnoses. It, therefore, is appropriate to point out, yesterday, when the results of the second annual Data Science Bowl, hosted by the crowdsourcing site Kaggle and its partner Booz Allen Hamilton, that crowdsourcing to find a solution for cardiologists was won by a former hedge fund analyst and employee of robotics start up, neither with any medical experience.

The contest challenge was to accurately measure end-systolic and diastolic volumes by reding cardiac MRI images, something that can take a trained cardiologist 20 minutes to measure. Ejection volumes from the heart’s chambers (see illustrations below) and the ejection fraction present a clear indicator of whether a heart is healthy or diseased. MRIs are defined as the “gold standard” in making this assessment.

The National Heart, Lung, and Blood Institute, in the United States, provided 1,000 MRI images, similar to the one seen below, to all contest entrants.

States Josh Sullivan, Senior Vice-President at Booz Allen Hamilton, “this particular challenge was one that the medical community had not been able to solve.” Coming up with a means by which a computer could measure the volumes just by looking at a MRI image would mean incredible time and money savings. Sullivan points out that turning to crowdsourcing, drawing on entries from sources medical or non-medical seemed like a good way to go because of the nature of the challenge which was really about analyzing large amounts of data presented as images.

Who were the winners?

Tencia Lee and Qi Liu. Lee had recently left a Los Angeles capital management company to join a robotics start up. Liu had worked for the hedge fund Two Sigma. Both entered the contest individually and in the preliminary rounds placed in the top 10. They then found each other when they sent a Kaggle help request and with the answer received decided to team up. As a team they tackled the problem choosing to use deep learning technology.

What is deep learning?

Deep learning is artificial intelligence that gives a computer the means to teach itself. This is the way IBM’s Watson learns. And deep learning is ideal for sifting through and analyzing large amounts of complex data such as MRI images.

Lee and Liu devised what they call a “convolutional neural network” to mimic animal vision. They used their financial background solving problems derived from large data sets and applied this to analyzing the MRI images. To develop the algorithm they chose an open source software tool which they downloaded from GitHub. Using it they defined the problem and then fed their neural network hundreds of heart MRI examples. It took the better of three months working in their spare time to come up with the winning algorithm.

How were the winners picked?

A point system defined solution rankings. Lee and Liu earned top performance ranking among the 1,392 entries composed of 293 individuals and 192 teams. The total prize money offered amounted to $200,000 U.S. Lee and Liu’s first place finish netted them $125,000 from that total.

For Kaggle, the crowdsourcing competition site with more than 500,000 active members coming from 194 countries, this contest further validates the use of the Internet and social media to solve complex problems. In its first National Data Science Bowl that ran in 2014 a PhD student in neural networks was the winner. Today that student is employed by DeepMind, Google’s deep learning company. If you are unfamiliar with DeepMind it was their software that beat the leading human player of the board game Go in the last month.

What will happen to Lee and Liu’s algorithm? Expect the leading manufacturers of MRI equipment to incorporate the tool once it has received FDA approval.