A study of certification records for popular electronic health record (EHR) systems shows that more than a third of vendors failed to follow a usability testing regulation established by the Office of the National Coordinator for Health IT (ONC). This lack of adherence to testing standards may be a major factor contributing to poor usability of EHRs.
Researchers, led by Raj Ratwani, PhD, of MedStar Health in Washington, D.C., published their findings in the Sept. 8 issue of the Journal of the American Medical Association. They started with the premise that many EHRs fall short in the area of usability, leading to frustration on the part of clinician users and potential patient safety risks.
ONC set up certification requirements to promote usability practices by EHR vendors as part of the meaningful use program. ONC-authorized bodies are responsible for certifying EHR products. Vendors must attest to using user-centered design (UCD), a process that focuses software developers on the needs of the frontline user. ONC certification calls for formal usability testing to be carried out on 8 different capabilities to ensure the product meets performance objectives.
EHR vendors must provide a written statement naming the UCD process they used, along with results of usability testing for their system. ONC endorses guidelines from the National Institute of Standards and Technology specifying that usability testing should include at least 15 representative end-user participants.
Ratwani and colleagues analyzed reports meeting ONC’s 2014 certification requirements for the 50 EHR vendors with the highest number of providers attesting to meaningful use requirements between April 2013 and November 2014. The study authors extracted from each report the stated UCD process, as well as the number and clinical background of usability test participants.
Certified reports were not publicly available from nine vendors, so the researchers limited their study to the 41 reports available for review. They found that 34 percent (14 vendors) had not met the ONC certification requirement of stating their UCD process. Forty-six (46) percent did use an industry standard, while 15 percent used an internally developed UCD process.
The researchers also found variability in the number of participants enrolled in the usability tests. Of the 41 studied vendors, 63 percent used less than the standard of 15 participants, and only 22 percent used at least 15 participants with clinical backgrounds. One vendor used no clinical participants in its usability testing. Overall, more than half of the available vendor reports lacked required demographic details.
“Enforcement of existing standards, specific usability guidelines and greater scrutiny of vendor UCD processes may be necessary to achieve the functional and safety goals for the next generation of EHRs,” the JAMA study concludes.
During a meeting of the federal Health IT Policy Committee on Sept. 9, committee member Christoph Lehmann, a professor in biomedical informatics at Vanderbilt University School of Medicine, cited the JAMA report on UCD testiing in reference to rising numbers of providers who switched EHR vendors during the 2014 reporting year for meaningful use. “We need to look deeper at how meaningful those changes that have been implemented in EHRs truly are usable for clinicians,” Lehmann commented.