Standard Metrics Needed to Measure After-Hours Clinician EHR Work

Vendor-derived EHR use metrics likely underestimate the time clinicians spend on after-hours EHR work, according to an article published in The Annals of Family Medicine.

EHR vendors Epic and Cerner provide customers with measures that aim to quantify work outside of work (WOW).

For example, Epic’s Signal tool measures:

• Time Outside Scheduled Hours (TOSH) on days with scheduled patient appointments
• Time on Unscheduled Days (TUSD)
• Pajama Time (weekends and weekdays outside 7:00 AM to 5:30 PM)
• Time Outside of 7:00 AM to 7:00 PM on days with scheduled appointments

Cerner’s Lights On reporting platform provides a similar metric to Epic’s Pajama Time called Time After Hours (weekends and weekdays outside 6:00 AM to 6:00 PM).

The authors noted that Epic’s TOSH and TUSD measures vary in whether they reference a clinician’s schedule to determine time spent in the EHR outside time scheduled with patients. Additionally, Cerner’s Time After Hours metric and Epic’s measures for Pajama Time and Time Outside 7:00 AM to 7:00 PM predominantly use clock time without reference to time scheduled with patients.

“Even measures that reference time scheduled with patients do not strictly adhere to the patient care schedule, however, as Epic’s TOSH excludes 30 minutes before the first and after the last scheduled appointments in a day as well as any active EHR time during interruptions between scheduled appointments,” the authors noted.

To better capture time working in the EHR outside time scheduled with patients, the authors recommend separating all time working outside of time scheduled with patients from time working in the EHR during time scheduled with patients.

“Attributing all EHR work outside time scheduled with patients to WOW, regardless of when it occurs, will produce an objective and standardized measure better suited for use in efforts to reduce burnout, set policy, and facilitate research,” the authors wrote.

The article also points out that the EHR vendors’ measures differ in determining active EHR use regardless of when clinicians complete work. Epic uses a 5-second inactivity period, while Cerner uses a proprietary method based on 45-second periods of inactivity and a combination of clicks, keystrokes, and mouse movements.

“Both methods likely undercount activities such as note reading, and neither method has been publicly validated,” the authors said.

Currently, there are no standardized and publicly validated methods of quantifying active EHR use by clinicians across EHR vendors and health systems.

“Whether the EHR is being used during or outside time scheduled with patients, we recognize the value and need to identify when a clinician steps away from the EHR to engage in other clinical or nonclinical tasks while remaining logged into the EHR,” the authors wrote. “We also recognize there are different interruptions (in terms of both frequency and duration) depending on the setting in which the EHR work is completed.”

“We therefore want to reiterate that it is important for the research and vendor communities to develop a standardized and validated vendor-agnostic measure for active EHR time by comparing methods of determining such EHR use from event logs with durations derived from direct observation,” they said.

The authors noted that an independently developed set of vendor-agnostic specifications would avoid the inherent biases with vendor-defined methods while accommodating the variety of work patterns that could otherwise dramatically underestimate EHR use.

“Independent measure development, however, is costly, still necessitates vendor involvement for validation, and is not practical for most institutions,” they acknowledged. “For the short term, we realize that most institutions will by necessity use vendor definitions of active EHR time, although they should recognize that these likely underestimate the actual time their clinicians are engaged with the EHR.”