More On "Why No One Believes the Numbers" and the Uncertainty of Measuring Return on Investment in Disease Management

Measuring ROI

In yesterday's posting on Al Lewis' book Why No One Believes the Numbers, the Disease Management Care Blog pointed out that the measurement of population health management (PHM) outcomes remains an inexact and still evolving science. While that can be a source of endless fascination for the DMCB, the inability of the industry to rustle up credible "return on investment" numbers has prompted some observers to condemn PHM as a waste of money.

The search for simple answers explains much of the appeal of this book.

According to author, one important solution is the "dummy year analysis" (DYA). This relies on repeated year-over-year measurements of utilization that use multiple comparison pairings of all patients with the condition of interest. When that's combined with a "plausibility" check list, Mr. Lewis says purchasers of the Patient Centered Medical Home (PCMH), disease management or wellness programs should be able to get a better fix on whether they saved any money. You can a sense of that perspective here.

The DMCB isn't too sure about that because a) other factors that have nothing to do with population health management can also impact utilization during and after the dummy years, making it difficult to assign an attributable ROI and b) entire health plan populations can likewise regress toward a regional or national mean.

The DMCB also sees three additional reasons why there may be less to this book's methodology than meets the eye:

1. When employers, health plans, accountable care organizations or other buyers have a list of names that have been through a care program, they typically want to understand the outcomes for the individuals on that list. If that's the case, the challenge is to find an adequate comparator that portrays what would have happened in the absence of the care program. Multiple options for identifying a parallel comparator have been used in published science for decades. That's difficult, imperfect, but not broken.  It remains an option.

2. While the book is replete with examples of "actuaries behaving badly," it is impossible to underestimate the influence of actuarial science and trending on premium rate setting, statutory accounting, and the regulation of insurance. As a result, if the actuaries say money is - or is not - being saved, health system leaders ignore their insights at their peril.

3. Isolating the impact of PCMH, disease management or wellness program out of all the other "noise" of a changing economy, evolving consumerism, benefit changes, electronic health record databases, medical advances, inflation and the news media is a function of an increasingly sophisticated and changing statistical sciences and computational technology. It's ironic, but one outcome has been a better description and measurement of the uncertainty surrounding a result.
 
To the author's credit, Why No One Believes the Numbers is not being promoted as the single best methodology that will lead PCMH, disease management and wellness programs to outcomes certainty. Rather, it is one option among many in asking whether a program had any financial impact.

Ultimately, therefore, that's why the DMCB advises that measuring outcomes in PHM - absent an ironclad methodology - comes down to using multiple approaches to triangulate on the truth. After reading Why No One Believes the Numbers, some readers may choose it as one of those approaches.