The Reliability of Quality Metrics in the Time of Value-Based Payment

In an article published in the American Journal of Medical Quality, a VSSL team from Dr. Linnaea Schuttner (lead author) and Dr. Joshua Liao (senior author) report findings about quality metric reliability in the context of the nationwide shift toward value-based payment.

As Drs. Schuttner, Liao, and colleagues note in their abstract:

"Quality metrics are fundamental to value-based payment reforms. Because metrics are key components used to drive performance, health care organizations participating in payment reforms should consider metric reliability—a measure of true performance versus statistical 'noise.'"

The team conducted a cross-sectional study examining reliability, variation from patient and clinician characteristics, and volume thresholds for 9 ambulatory quality metrics between 2015-2017 in a health system engaged in value-based payment reforms. (For those more methodologically included, the team used a set of hierarchical mixed models).

Overall findings:

"Reliability was lower for all metrics at the clinician level (range 6%-64%) than at the clinic level (84%-99%), with little variation related to patient or clinician characteristics. Few clinicians, but the majority of clinics, contributed sufficient volumes of patient encounters to meet a 70% reliability threshold."

The VSSL team concluded the following about their findings: "that clinic-level performance measurement may be more appropriate than individual clinician-level measurement, particularly in low-volume contexts."

Read the full article here. Fore more about issues of metrics in the context of health care value, see here for an article anchored by Liao about metrics quantifying low-value care.