- The Clinical Decision Support (CDS) Coalition released a summary of submitted stakeholder feedback surrounding its voluntary guidelines for software developers designing CDS systems following a public comment period earlier this summer.
Particularly, a common thread spanning the feedback in the contention that the guidelines are overly prescriptive and complex as well as in need of further transparency.
The guidelines were partly designed to prevent FDA regulation over CDS software to fulfill provisions in the 21st Century Cures Act.
The coalition shared its comments with its network and also solicited comments from FDA.
“Specifically, we met with FDA’s Digital Health Task Force to present these guidelines,” it wrote. “During that meeting, we received useful suggestions from the Agency about how we could improve the guidelines.
The coalition addressed 15 consistent recommendations and criticisms raised by responding stakeholders. While many stakeholders pointed to areas where the guidelines should be improved, most agreed with the guidelines basic principle.
“Through the comments, we received almost unanimous support for the basic principle that enabling a healthcare professional to independently review the basis for recommendations the software makes requires transparency by the software, competent human intervention, and time to 2 reflect,” wrote the coalition. “Those are the three pillars of the guidelines, and there seems to be ample agreement on that front.”
The coalition processed stakeholder recommendations and made changes throughout the guidelines to address comments submitted by the public.
Stakeholders also suggested the guidelines exceed what the 21st Century Cures Act would require, and recommended the guidelines incorporate ONC’s approach to transparency.
In response to comments criticizing the guidelines for being too prescriptive, the coalition revised the guidelines to note that the recommended approach to CDS is only one approach and the availability of other existing approaches to using CDS systems.
The coalition declined to adopt the ONC approach to transparency. ONC includes links in its guidelines to show stakeholders were information comes from. The CDS Coalition stated ONC’s approach to transparency was “adopted for a different, more general purpose.”
Several comments from stakeholders also focused on machine learning.
“In that context, they argued transparency really doesn’t so much mean providing an explanation of how the software arrived at its conclusion, but rather demonstrating that the software has been validated to the point where the user can take comfort that the recommendations are likely correct,” stated the coalition.
In response to this comment, the coalition acknowledged that machine learning is an emerging technology and researchers are still developing ways to address its challenges. However, the coalition offered five steps developers can take to optimize the transparency of CDS recommendations:
- Explain what can be explained
- Communicate the quality of the machine learning algorithms
- Describe the data sources used for learning
- State the association as precisely as possible
- Convey the confidence level
Using these five steps, developers can increase their level of confidence that a recommendation is likely valid.
The revised guidelines are ultimately aimed at ensuring healthcare providers are using CDS software in a way that boosts their ability to provide accurate patient care without relying too heavily on the software’s recommendations.