- ONC today announced the three winners of the Patient Matching Algorithm Challenge.
Winning submissions were selected from a pool of over 140 competing teams and nearly 7,000 algorithm submissions using an ONC-provided dataset. Algorithms were designed to identify and match data about a patient held by one healthcare provider with data about the same patient contained within the same system or other systems.
“Many experts across the healthcare system have long identified the ability to match patients efficiently, accurately, and to scale as a critical interoperability need for the nation’s growing health IT infrastructure,” stated National Coordinator for Health IT Don Rucker, MD in a list serv. “This challenge was an important step towards better understanding the current landscape.”
Winners were chosen for each of three categories. Winners selected in the best F-Score category were measured highest for accuracy in both precision and recall.
Best in Category prizes were given to participants based on best precision (fewest amount of mismatched patients,) best recall (fewest amount of missed matches), and best F-Score (combination of best precision and best recall.) Because competitors were given 100 attempts to score their matching solution, a prize was also given for best first run, or attempt.
Winners in each category are as follows:
· First Place ($25,000): Vynca
· Second Place ($20,000): PICSURE
· Third Place ($15,000): Information Softworks
Best First Run ($5,000): Information Softworks
Best Recall ($5,000): PICSURE
Best Precision ($5,000): PICSURE
PICSURE used an algorithm based on the Fellegi-Sunter method for probabilistic record matching first introduced in 1969. The competitor performed a significant amount of manual review.
Vynca used a stacked model that combined the predictions of eight different models into one. The team reported they manually reviewed less than .01 percent of patient health records.
Information Softworks also used the Fellegi-Sunter-based enterprise master patient index (EMPI) SYSTEM, but incorporated some additional tuning. Despite using a similar method as PICSURE, Information Softworks reported extremely limited manual review.
The Patient Matching Algorithm Challenge was launched in June.