Think You Know Everything You Need To Know About Measuring Quality? Think Again

Fast on the heels of this recent Disease Management Care Blog post on a serious shortcoming of mainsteam quality measurement (the routine failure to account for patient preferences) comes this more far reaching and eye-opening JAMA commentary titled "Sudden Acceleration of Diabetes Quality Measures." Authored by Leonard Pogach and Avid Aron, it efficiently summarizes what’s wrong when it comes to trying to do the right thing for populations with diabetes. Thanks to checking in with the DMCB, you’ll do yourself right by familiarizing yourself with the handy summary below on the downsides of our current mainstream approach to “outcomes.”

While readers may believe that only nincompoops and ne'er-do-wells would dare to second guess the National Committee for Quality Assurance’s (NCQA) and Leapfrog Group’s approach to quality for diabetes mellitus, Drs. Pogach and Aron assure us that’s not the case. The Diabetes Quality Improvement Project (DQIP) and its successor, the National Quality Improvement Alliance (mentioned in this AHRQ summary) also reviewed the science of diabetes control and concluded there was insufficient evidence to warrant recommend an A1c of 7% or a blood pressure of less than 130/80. In the years that followed, they may have turned out to be correct. The ACCORD study showed that an A1c of 6.4% resulted in an increased death rate (here) and a blood pressure of 119 systolic was ultimately no better (here) than a blood pressure of 133 systolic.

So, how did the NCQA get it wrong? According to the authors, measuring quality in populations demands a more nuanced, conservative and go-slow approach. To wit, what is needed is a measurement methodology that:

is continuous - right now, most measures are "binary" i.e., measures are boiled down to meeting a single measurement threshold. Since benefit and risk are non-linearly associated with diabetes control, quality assessment should recognize that there is a difference, for example, between an A1c of 7% versus 8% versus 9% and higher.

accounts for selection bias - not all populations that selected for measurement are the same, so comparisons of quality between groups may be prone to bias.

is amenable to case-mix adjustment - if any known sources of bias are present, they should be statistically accounted for.

accommodates patient preferences - take the DMCB's word for it, a significant number of highly informed dabetes patients do not want a low A1c or blood pressure if it means taking more pills.

minimizes measurement variability - blood pressure measurement is very operator dependent and even blood tests can also vary.

is not used until potential harms like hypoglycemia (low blood sugar reactions) and polypharmacy (too many pills) are fully accounted for.

The authors recommend that the Agency for Healthcare Research and Quality take things over. They're more likely to be accountable, take other viewpoints into account, be fully transparent and would probably agree to have everything televised before any quality measure is formalized for widespread use.

The DMCB isn't sure that it necessarily agrees with blowing things up and starting over, especially if it involves a government agency. While it thinks this through, one common sense suggestion may be for the science of quality improvement to adopt the principles above for the NEW measures that are coming out. Hopefully, the NCQA is paying attention.