The Risk of (Improper) Risk Adjustment: Why The Doctor May See You Now

The stats guys go to work

The Disease Management Care Blog continues to welcome blog posts from outside authors. This is another one, courtesy of Erik Tollefson, who works in the health policy field. He can be reached at erikDOTmDOTtollefsonATgmailDOTcom

As payers increasingly pursue reimbursement strategies that share risk with health care providers, risk adjustment is emerging as an important topic in health policy research.  Indeed, for providers, the ability to properly identify the underlying health risks of a specific population could mean the difference between earning savings rather than paying penalties.  As the importance of risk adjustment ascends grows, increasing attention is being given to the robustness sophistication of adjustment methodologies. That includes including the data used to estimate the risk of a covered population risk. 

In a recent study published in the British Medical Journal (BMJ), John Wennberg and co-authors explore how under-recognized “patient observation bias” can contaminate risk adjustment results.

Previous studies have identified many pitfalls of using observational data as a proxy of a population’s true disease burden. One common example is the phenomenon of “upcoding.” This practice not only adjusts diagnostic codes to maximize reimbursement, it is a well-documented source of bias that spuriously increases variation between geographical regions and complicates risk adjustment.

Wennberg and colleagues show that patient observation bias is another potential pitfall in the science of risk adjustment that has not received such attention. The phenomenon can be quantified by using numerous proxies such as frequency of visits by physicians and the intensity of diagnostic tests (both laboratory and imaging) ordered. Previous research has shown that geographical areas with higher patient observation bias independently correlate with higher rates of comorbidity after being adjusted for all the other relevant patient factors that underlie true risk.

Using a sample of Medicare claims data, the authors computed two different risk adjustment measures: 1) using standard methods; 2) adjusting for potential observation bias.  Overall, the adjusted method (using visit intensity) emerged as a more accurate predictor of the population’s underlying burden of illness. Thanks to their ability to identify a source of bias that explained a greater portion of variation in important categories such as sex, age, and race, but was the authors were able, compared to the usual risk adjustment methods, to reduce the variation levels between different geographic regions.

The importance of accurate risk adjustment is not just limited to its potential impact on increasingly high financial stakes in the health care sector. The ability of policy makers to understand why variation occurs in health care costs and utilization across geographical areas has emerged as one of the key questions that may ultimately improve delivery and cut costs in a bloated health care system. This study offers important evidence that part of the answer may indeed lay in the supply-inducing doctors self-referring patients for unnecessary diagnostic procedures that littered Atul Gawande’s account of rising health care costs in McAllen, Texas.

Another important take home lesson, however, also applies: the quality of underlying data. With continued histrionics surrounding access to ever greater amounts of health care data, it helps to remember that qualitatively understanding what one is actually measuring is still as important as the accuracy of the underlying statistical methods.