More on Penn State's Wellness Woes and The Evolving Science of Evaluating Health Promotion Program Outcomes: There Is No Gold Standard

Aside from keeping up to date with work buddies, the Disease Management Care Blog doesn't really use LinkedIn all that much. But when "Support our Penn State Colleagues in their Fight to Prevent Coercive Junk-Science Wellness Programs" postings began to appear in a Discussions board, the DMCB couldn't resist. It rose in support of its alma mater (College of Medicine, '77) faster than a med-mal attorney can calculate a contingency fee.

 As noted in this prior DMCB posting, Penn State University launched a rather routine health promotion program that prompted some nasty and very public teaching faculty resentment. Calls for "civil disobedience" and sinister references to "eugenics" made the DMCB wonder how much of the reported push-back was mainstream employee opinion vs. mainstream media's biased reporting. That distinction didn't stop the LinkedIn board from running a mostly one-sided dialogue on the matter.

So, undeterred by the unfairness of so many vs. just one, the contrarian DMCB naturally jumped right in.  Among the issues raised:

The RAND Study on wellness casts doubts on the merits of employer sponsored wellness programs:

Actually, RAND found employer-sponsored programs lead to statistically significant increases in exercise levels as well as reductions in tobacco abuse and body weight.  To the disappointment of wellness vendors everywhere, however, these programs did not lead to statistically significant reductions in health insurance claims expense.  The ever-optimistic DMCB points out that that means that these health improvements occurred without an increase in health care costs.

While cost neutrality alone is good news, the DMCB also believes that an emerging generation of wellness programs will do a far better job of identifying persons with 1) actionable risk and 2) who are willing to take action.  By husbanding wellness resources for subpopulations where it will have the greatest "bang," program costs will go down and claims savings will achieve statistical significance.

The author of the widely quoted Health Affairs paper on the merits of employer sponsored wellness programs has back-pedaled away ("too early to tell") from her study's original conclusions.

Actually, the original Health Affairs paper said that the finding of a $3.27 return on every dollar spent is subject to:

 "(f)urther study.... to elucidate the time path of return on investment.... The assumption of a linear trend in savings from the beginning to the end of program evaluation may not reflect the reality of behavior change within organizations."

The point is that nuanced and calibrated conservatism is typical of excellent peer-reviewed research and, taken in context, the authors are being quite consistent in-print and on-air.  Academics will always say more research is needed.  Skeptics will over read that.

There are powerful arguments against the common wisdom that "wellness saves money," suggesting that the health promotion industry has been intentionally ripping employers off.

Actually, when it comes to wellness outcomes, there is no agreement on "the" measurement "gold standard." Without any consensus on which assessment approach (for e.g., this vs. this) is truly "better," only one thing is certain: much like the Betamax vs. VHS wars, the future owner of "the" standard stands to reap a consultant's bonanza. Until we declare a winner, assessing the truth will be a messy mix of triangulating on means, medians, confidence intervals, imperfect reference controls, suspect generalizability, human judgment, moving targets and evolving interventions.

What about [insert name of wellness program here] that is an obvious sham?

There have been women who have had mammograms with missed cancer, victims of car crashes who have died despite seat belts and times when the DMCB did something really dumb despite the advice of the DMCB spouse.  That doesn't mean mammogram, seat belts or advice are worthless.  The plural of anecdotes is not data.

Coda: By the way, the statistically significant "value of  0.05" is more of a consensus than a gold standard.  Why is a 5% chance that an observed result is not the result of randomness wiser than a 6% chance or a 4% chance?

Which of These Four News Reports Is False? Insights from the Wacky World of Health Care Reform

Baron Von Munchhausen

Despite host Peter Segal's occasionally highbrow insider cleverness, The Disease Management Care Blog remains a loyal fan of NPR's "Wait Wait... Don't Tell Me" radio show.  While the DMCB has its suspicions about the ratio of truly spontaneous wit to pre-planned ripostes, that won't stop it from turning to a part of the show called 'Bluff the Listener' for bloggy inspiration.

BtL has guests try to guess which of three funny stories is based on a real true news report. The DMCB thinks health care is so wacky that it'd be more challenging to guess which of the four stories below is false.

Unfortunately, if you win, getting the DMCB to put its voice on your home answering machine is unlikely to impress anyone. However, if you can pick out which story is a complete Munchhausenesque fabrication, you will deserve the respect of your friends and co-workers.

Ready to try to get some bragging rights?

+++++

Even doltish man-trolls know better than to try to organize an all-male blogging conference. Unable to reach out to that demographic, HHS Secretary Kathleen Sebelius did what's best: appeared before the annual "BlogHer" Conference in an appeal to women bloggers to tout the benefits of Obamacare. Her outreach supplements plans to rely on celebrities to help with a nationwide drive to increase enrollment through the insurance exchanges. Next up will be effort to recruit motor scooter owners to sport pro-Obamacare ads on the back of their helmets.  Then it's on to asking members of the European Beret Society to host recruitment drives at their monthly chardonnay tastings.

Answer here.

+++++

Al Lewis and Vik Khanna condemned the wellness industry in a Wall Street Journal editorial when they proclaimed that "workplace programs don't work." They went on to say that they are "ineffective at reducing costs, lack support in the medical literature, are unpopular enough to require incentives and are occasionally even harmful." Yet, the Khanna On Health Blog's “workplace wellness consulting” page suggests the authors’ unique consulting insights can help potential customers “do wellness right.” Did the DMCB mention that both individuals are lawyers?

Answer here.

+++++

Writing in a separate issue of the Wall Street Journal, former Vermont Governor and Democratic National Committee Chair Howard Dean actually attacked Obamacare by criticizing its Independent Payment Advisory Board as a rate setting enterprise that is doomed to failure. Brazenly using Tea Party terms such as "bureaucrats" and "health rationing," Dr. Dean's liberal-progressive apostasy prompted ACA architect Peter Orszag to curiously opine in a separate article in Bloomberg that the argument favoring IPAB is that it will be a much better rate setting body than Congress. If this keeps up, even labor unions will start criticizing Obamacare.

Answer here.

+++++

While partisan blood continues to spill over Obamacare in Washington DC, there is much good news outside the beltway.  It's been announced that the IRS will not only rely on self-reporting of income levels in setting premium subsidies. Even better, individuals who qualify for tax credits while buying their health insurance with the on-line exchanges will get a two-fer: 1) the option of applying the rebates to reduce their monthly premiums, and 2) confidence that there won't be any tax liability "claw backs" should their final income be higher than anticipated. Interest and penalties will be optional.

Answer here

Medicaid Disease Management: No Impact on Emergency Room Utilization or Inpatient Costs for Enrollees with Diabetes?

Regular readers of the Disease Management Care Blog know that Medicaid is coming. While many of the nation's Governors have declined President Obama's invitation to run Medicaid the Affordable Care Act way, others have agreed to use the ACA's generous funding to enroll millions of their indigent citizens into this vastly expanded public insurance program.

"No problem!" says the disease management vendors.  For years, they've been offering their services to state Medicaid programs and would be happy to expand their contracts.

Unfortunately, an article by Matthew Conti that was just published in the journal Health Services Research suggests that that may not be a good idea.  The article's title is Effect of Medicaid Disease Management Programs on Emergency Admissions and Inpatient Costs.  The only thing that's missing are the words "The Lack of Any" at the front of that sentence.
   
The article studied the impact of diabetes "opt-out" disease management on diabetic patients' emergency room utilization and admissions in three states' Medicaid programs: Washington (started in 2002), Texas (started in 2004), and Georgia (started in 2005).

These states with were compared to states without diabetes disease management.  These control states were selected on the basis of baseline Medicaid enrollment trends that were similar to the three study states. These control states were Hawaii, Kentucky, Massachusetts, Maryland, Maine, North Carolina, Nebraska, South Carolina and Tennessee.

To perform the comparison, Dr. Conti used the Agency for Health Care Research and Quality's (AHRQ) National InPatient Sample (NIS) from the Health Care Cost and Utilization Project ("HCUP"). These databases contain patient-level and longitudinal hospital information on inpatient stays, including cost, payer, admission type (e.g., emergency, urgent and elective), age, gender, primary payer, and total charges. The span of data that was used went from 2000 through 2008.

A complicated pre-post "difference in differences" model was used to compare baseline vs. follow-up:

1) total inpatient charges/Medicaid enrollment (which averaged $430 per diabetic enrollee, with a 95% confidence interval of $265 to $700) and

2) emergency admissions/inpatient admissions (a ratio of 0.37 per admission with a standard deviation of plus or minus 0.12)  All Medicaid enrollees with diabetes were included in the analysis, whether or not they had been enrolled or opted out.  The author used this approach figuring that if a statewide disease management program enrolled up to a third of eligible persons with diabetes (that was the case in Texas), there should have been an observable impact on the entire population. That's the approach favored by the Disease Management Purchasing Consortium.
 
The results?  No state with disease management had lower emergency room utilization or inpatient costs for their Medicaid enrollees with diabetes. The DMCB couldn't find a table with numbers, but the figures (which can't be reproduced without permission) show little impact over time.

What can readers conclude?  Assuming that, during the period of study, the three states' Medicaid programs suffered from the program's endemic issues of underpayment to providers with a relative lack of access to primary care:

1. "Blanket" call-everyone telephonic disease management cannot make up for fee-for-service Medicaid's shortfalls.  It remains to be seen if the ACA's revitalization of Medicaid will make up for this and increase the parallel impact of disease management. 

2) This also means that Medicaid's experience with disease management can't be generalized to other types of insurance with better provider payment rates and patient access to care.

That being said, the DMCB has two concerns:

1. If the DMCB is reading this right, it appears all persons of any age with diabetes were included in the study, including Type 1 diabetics.  If that's correct, that could have also blunted the impact of any disease management program, since children are over-represented in Medicaid and the impact of remote telephonic coaching in Type 1 is widely viewed (even among the disease management vendors) to be ineffective.  Insulin-requiring kids need lots of face-to-face hands-on care.

2. The DMCB is unfamiliar with the three study states' disease management programs, but if they were set up the "old fashioned way" to contact all persons with diabetes without the modern regard to future risk and "impactibility," then it's little wonder that the programs failed.  State-of-the-art population health management tailors its programs by focusing on subsets of persons with chronic conditions that are most likely to benefit.  Any impact on emergency room use or inpatient charges for these patients would be lost in the data "noise" of everyone else's utilization.

Should Medicaid programs that are facing huge jumps in enrollment abandon Medicaid as a result of this study?  Based on this study, the DMCB doesn't think so.  The findings are interesting, but more research is needed.

More on Al Lewis' Book "Why No One Believes the Numbers"

Remember the DMCB's reviews (here and here) of Al Lewis' book Why No One Believes the Numbers?  The DMCB can't either, which is why you may want to check out this review by Christobel Selecky over at the excellent DMCB sister blog, The Doctor Weighs In.  It's worthwhile reading, especially since Ms. Selecky was the CEO of the star crossed disease management company LifeMasters. She knows of what she speaks and has the scars to prove it.

The DMCB finds her 6th paragraph slightly off topic but worthy of further discussion. Say what you like about the "evidence" supporting disease management, but let's at least be consistent and apply the same high standard to the patient centered medical home, the electronic health record and accountable care organizations.  Patients deserve nothing less.

More On "Why No One Believes the Numbers" and the Uncertainty of Measuring Return on Investment in Disease Management

Measuring ROI

In yesterday's posting on Al Lewis' book Why No One Believes the Numbers, the Disease Management Care Blog pointed out that the measurement of population health management (PHM) outcomes remains an inexact and still evolving science. While that can be a source of endless fascination for the DMCB, the inability of the industry to rustle up credible "return on investment" numbers has prompted some observers to condemn PHM as a waste of money.

The search for simple answers explains much of the appeal of this book.

According to author, one important solution is the "dummy year analysis" (DYA). This relies on repeated year-over-year measurements of utilization that use multiple comparison pairings of all patients with the condition of interest. When that's combined with a "plausibility" check list, Mr. Lewis says purchasers of the Patient Centered Medical Home (PCMH), disease management or wellness programs should be able to get a better fix on whether they saved any money. You can a sense of that perspective here.

The DMCB isn't too sure about that because a) other factors that have nothing to do with population health management can also impact utilization during and after the dummy years, making it difficult to assign an attributable ROI and b) entire health plan populations can likewise regress toward a regional or national mean.

The DMCB also sees three additional reasons why there may be less to this book's methodology than meets the eye:

1. When employers, health plans, accountable care organizations or other buyers have a list of names that have been through a care program, they typically want to understand the outcomes for the individuals on that list. If that's the case, the challenge is to find an adequate comparator that portrays what would have happened in the absence of the care program. Multiple options for identifying a parallel comparator have been used in published science for decades. That's difficult, imperfect, but not broken.  It remains an option.

2. While the book is replete with examples of "actuaries behaving badly," it is impossible to underestimate the influence of actuarial science and trending on premium rate setting, statutory accounting, and the regulation of insurance. As a result, if the actuaries say money is - or is not - being saved, health system leaders ignore their insights at their peril.

3. Isolating the impact of PCMH, disease management or wellness program out of all the other "noise" of a changing economy, evolving consumerism, benefit changes, electronic health record databases, medical advances, inflation and the news media is a function of an increasingly sophisticated and changing statistical sciences and computational technology. It's ironic, but one outcome has been a better description and measurement of the uncertainty surrounding a result.
 
To the author's credit, Why No One Believes the Numbers is not being promoted as the single best methodology that will lead PCMH, disease management and wellness programs to outcomes certainty. Rather, it is one option among many in asking whether a program had any financial impact.

Ultimately, therefore, that's why the DMCB advises that measuring outcomes in PHM - absent an ironclad methodology - comes down to using multiple approaches to triangulate on the truth. After reading Why No One Believes the Numbers, some readers may choose it as one of those approaches.

Why Everybody Should Read Why Nobody Believes the Numbers

Music of the Spheres

Back in the early 1900s, Albert Einstein had a problem. Sophisticated instruments were unexpectedly showing that the measured speed of light was the same if the source or the observer were moving or stationary.  In other words, if one were moving away from a bullet, it should look (to the observer) that the bullet had slowed down. Light's refusal to conform to the prevailing common sense about how the universe should work ultimately forced Einstein in 1905 to conclude that, in order for the speed of light to be constant, time and mass had to be elastic. This ushered in a new field of relativity mathematics that is still being used to plumb the known universe's Music of the Spheres.

While the controversies surrounding the effectiveness of "population health management" (PHM) are quite minor compared to Einstein's Theory of Relativity, the comparison is still instructive. The similar mismatch between what is assumed, what is observed and how to mathematically describe the ultimate truth also underlies Al Lewis' book, Why Nobody Believes the Numbers.  In other words, we assume care management-based patient coaching always yields savings, increasingly sophisticated observations often fail to show it and, as a result, we need new mathematics to reconcile what we assume and what we observe.

Interestingly, author Al Lewis of the Disease Management Purchasing Consortium never doubts the speed of light or that high quality PHM ultimately can save money. While PHM vendors may interpret his long history of skepticism as some sort of shakedown, Al's passion is clearly evident: Why Nobody Believes the Numbers is ultimately driven by a search for the truth. For that he deserves a lot of credit.

The good news is that Mr. Lewis does a masterful job of examining the prevailing assumptions underlying the PHM universe by relying on layman's logic, simple examples, real world anecdotes and clever insights. As a result, even the mathematically challenged can come away with a better grasp of the pitfalls that surround selection bias, regression to the mean, invalid comparators and calculation of trend. As a result, the first chapter on "Actuaries Behaving Badly" is arguably "must reading" for human resources managers, sales personnel or C-suite types that are contemplating the "return on investment" from a company wellness or a disease management program.

That bad news is that Al Lewis is no Einstein. He suggests that a solution is at hand thanks to a simplistic "dummy year adjustment" methodology that is based on serial observations over a long period of time that includes all patients with the index condition.  When this is combined with a series of common-sense based "plausibility" tests, Al proclaims his mix of common sense and fundamental mathematics will yield a single, yes or no, black or white, it did or did not reduce insurance-claims expense-truth.

Unfortunately that ain't necessarily so. Even Einstein's insights couldn't explain all of the sublime harmonics that make up the Spheres. There'll be a future Disease Management Care Blog posting with more on how Why Nobody Believes the Numbers falls short.  That will address the unavoidable impreciseness that surrounds measures of central tendency, the challenges of measuring subgroups and the moving-target realities of an insurance industry that continue flummox those of us who are trying to explaining the health care universe.

In the meantime, if you're a buyer or a vendor, the DMCB recommends Why Nobody Believes the Numbers for your bookshelf.  DMCB readers will come away with a better grasp of the good, the bad and the ugly of outcomes measurement, understand what it can and cannot tell us and appreciate the underlying and still evolving debate over the ultimate value of the PHM industry.