The (Irresistible) Rise of “Shadow” Cost-Effectiveness Analysis?

The shadow knows.... more

The Disease Management Care Blog continues to welcome blog posts from outside authors. This one is courtesy of Erik Tollefson, who works in the health policy field. He can be reached at erikDOTmDOTtollefsonATgmailDOTcom.

Although public and private health care payers officially eschew the use of formal cost-effectiveness analysis in approving medical treatments, a growing number of examples illustrate that cost-effectiveness principles are seeping into medical decision-making. Indeed, Memorial Sloan Kettering Cancer Center decided not to give patients Zaltrap (a drug) for late-stage colorectal cancer due to cost concerns; at least three health insurance companies, including most recently Blue Shield of California, have decided not to cover proton beam therapy for early-stage prostate cancer due to its high price.

These decisions symbolize that “shadow” cost effectiveness analysis, whereby payers make informal value calculations based on the price and efficacy of a treatment, may become an increasingly common feature of the payment landscape. 

It should be noted that payers have likely made similar calculations during initial coverage decisions: health insurance companies and hospitals have historically negotiated with drug makers and medical device manufacturers on price and value compared to existing treatments. Blue Shield California, however, has agreed not to cover proton beam therapy any longer due to cost concerns, although that reasoning is not complete: the decision is based on the therapy’s inability to demonstrate equal or better outcomes, while boasting a price tag several times above the benchmark treatment modality.  As Marcus Thygeson, the senior vice president and chief health officer at Blue Shield of California, stated in a letter to oncology and radiology practices in the state:

“The preponderance of medical evidence clearly shows that the treatment has about the same clinical outcomes as other forms of radiation, but it's a lot more expensive…because it's not cost effective, we're not going to cover it."

The rise of shadow cost effectiveness analysis is not surprising in the current economic environment.  While there are still strident concerns regarding overall spending on medical care, even as cost growth has moderated during the recession, substantial pressure exists at the firm level (e.g., insurance plans and hospitals) as margins compress and risk-sharing agreements increase.

The fragmented nature of the nation’s public-private health care payment system has also contributed: Medicare, one of the largest and most influential payers, cannot reject coverage of medical treatments explicitly based on cost due to restrictions in the program’s enabling statute. This puts the onus on private payers and hospitals to exercise greater authority in the rejection of expensive, innovative treatments with limited efficacy; traditionally, however, this power has not been aggressively exercised, leading to coverage of treatments that might only provide marginal benefit.

This schism in payer assessment of treatments in the US where private insurance plays the leading role is unlike that found in other industrialized countries where public payers play the dominant role: In the UK NICE decides which drugs or medical devices meet the “cost-effective” threshold for coverage by NHS via systematic cost-effectiveness analysis. This process makes a palpable difference in the availability of treatments: While there are currently 12 proton beam therapy centers in the US (with numerous more planned), there are currently none in the UK; the first two centers are planned to come online in 2018. 

Overall, there are both positive and negative elements to the emerging phenomenon of shadow cost-effectiveness analysis. First, the discussion of tradeoffs between cost and outcomes of medical treatments is notably more productive than the prevailing focus on merely “constraining costs.” Indeed, a medical delivery system that focuses on cutting costs, but does not focus on the actual value of treatments, is literally one of little value. Second, the use of rudimentary cost-effectiveness principles calls into question what actually constitutes “innovation” in the medical space, and may give pause to the inevitable “arms race” that follows coverage decisions.  Indeed, if insurance companies continually reassess (and reverse) coverage decisions based on emerging clinical evidence, it may lead to better medical decision making.

On the negative side, shadow cost effectiveness has limited efficacy without a full array of analytical tools. That is, while it is useful in assessing (and stopping) egregiously non-cost effective interventions, it is less effective in dealing with similarly valueless interventions that may have similar efficacy as existing interventions but cost marginally more or less.

Image from Wikipedia

Do Employer-Based Wellness Programs Work?

Reuters tackles worksite wellness outcomes

Mrs. Smith (name changed) was overweight.  She knew it and her physician, the Disease Management Care Blog, knew that she knew it.  Since the DMCB was one of the two persons on the planet who knew her true weight, she could talk to it behind closed doors about diet, exercise, fads, over the counter meds, prescription appetite suppressants and even bariatric surgery. 

What the DMCB quickly discovered was that Mrs. Smith's weight-loss goals were not only unrealistic but, like many women struggling with weight, driven more by the prospect of how she'd look in a bathing suit than any real health benefit.

Mrs. Smith wasn't alone. This seminal study demonstrated just how unrealistic women's weight loss goals (in the range of 50 lbs.) can be.  Think of the popularity of The Biggest Loser and it's easy to see why persons think thinness is just a matter of a few months of dieting and exercise, and that being skinny leads to health and happiness, 

Easy, right?

DMCB readers know otherwise. That's why they're not going to be impressed by the tone of this May 24 Reuters article on worksite wellness.  RAND, in a not-quite released report to Congress, examined the impact of several employee based programs and found, in the words of the Reuters reporter, only a "modest effect."  Average weight loss was "only" three lbs., tobacco cessation rates were significant but "short term," average cholesterol levels were unchanged and reductions in health insurance claims expense failed to achieve statistical significance.

Researchers have known for years that conservative dietary and lifestyle therapy typically results in weight loss in the range described above. In addition, cholesterol reduction as a primary prevention intervention is low yield when it comes to health. On the other hand, even short term tobacco cessation is a good thing. When it comes to the ability of wellness to reduce health care costs, weight reduction is unlikely to drive claims expense for a health insurer within two to three years, the impact of obesity on overall mortality rates is not as large as you'd think and "prevention" rarely saves money.

What's more, these programs were able to achieve their "modest" outcomes without increasing claims expense.  Participants lost weight and stopped smoking at no additional cost to the system.  Now that is something.

Mrs. Smith and Reuters are very similar.  Both are struggling with nrealistic expectations thanks to dubious fashion trends, media misinformation and scientific ignorance.  Fortunately, Mrs. Smith had access to a resource that could help her better manage her weight.  The DMCB can only hope that Reuters has access to a resource that can help it manage its lack of background knowledge.

The $440 Billion Question for the Value of Cancer Treatment Such As Erbitux Gets Even MORE Complicated

In a prior posting, the Disease Management Care Blog made a statistically based argument about the plus-or-minus distribution or 'spread' of cancer treatment-related survival around an average. It referred to a specific Erbitux research paper, where there were two treatment groups, each with an 'average' survival with a standard deviation. The DMCB compared and contrasted the two standard deviations, not the average results, and went on to state that cancer patients and their oncologist physicians are more interested in the upper and lower bounds of possibility, not what happened to most patients in the middle.

Well, wouldn't you know, a statistician emailed the DMCB with some insights about its faulty logic. It likes to hear from statisticians almost as much as from actuaries, especially when there's learning to be done. This reminds the DMCB of a key difference between statisticians and actuaries: the former tells you how you were wrong, while the latter tells how you're going to be wrong.

Cody L. Custis is employed by the State of Montana and also teaches mathematics at UM Helena. He points out that he is speaking as an individual and not for any of his employers. The DMCB points out it doesn't feel so bad if it took these kinds of credentials to uncover a mistake.

Here' the email:

As a professional statistician, I wanted to raise two objections about the conclusions in your blog's posting on Erbitux. In it, you say:

“Check out this real life clinical trial that is available on line. It showed Erbitux resulted in a median survival of 12 months with a confidence interval ranging from about 8 ½ to just over 15 months versus just over 9 months of survival with a confidence interval extending from about 7 ½ months to just under 12 months without Erbitux. This means the real bottom line in this trial is that getting Erbitux may result in a life expectancy as high as 15 months versus a life expectancy as low as 7 ½ months without Erbitux.”

Based upon the study referenced, I assume that these conclusions come from the following statement in the Butts et. al. paper:

"Median survival time was 11.99 months in the cetuximab arm (95% CI, 8.80 to 15.18) and 9.26 months in the platinum/gemcitabine arm (95% CI, 7.43 to 11.79)."

First, if a clinical trial involving two groups results in two sets of data, there will be two separate averages or means. If one group’s average result is compared to the other group’s average, it is important to not only know the difference between the two means, but also the confidence interval for that difference. In contrast, your blog compared two separate confidence intervals. In the Butts et. al. paper, the authors did not construct a confidence interval for the increase in life expectancy in the Cetuximab study; rather, they calculated two separate intervals for two treatments. Because confidence intervals are given for the two treatments separately, rather than for the difference, cancer patients and their physicians cannot know if a difference in the life expectancy of 1.5 months is really significant. Mathematically, if the confidence interval extends to zero, the 1.5 months could be the outcome of random chance. As the authors of the Butts et. al. paper state in their conclusion : 'The major limitation of the study was its noncomparative design, not statistically powered to demonstrate significant differences between treatment arms.'

Second, your blog's conclusion takes the worst possible case for one treatment and the best possible case for another treatment. Thus, while your blog is correct to focus on the importance of confidence intervals rather than point estimation, the conclusions are based on unfair assignment of best and worst case outcomes of two treatments. A skeptic could just as fairly state the conclusion: this means the real bottom line in this trial is that getting Erbitux may result in a life expectancy as low as 8 months versus a life expectancy as high as 12 months without Erbitux. Both conclusions unfairly take extreme outcomes.

The DMCB says good points. In the paper referenced, patients would be better served by knowing the distribution around the average difference in survival and should also be made aware of the up and down sides of any treatment option.

That being said, the DMCB also thinks, based on experience, that most patients and their oncologists tend to believe in the most optimistic treatment scenarios. If those scenarios fall with the reasonable (plus or minus) bounds of possibility, it's hard for insurers, policy makers, comparative effectiveness researchers, regulators and legislators to say no.

The DMCB thanks Cody Custis for the insights.

The $440 Billion Question for the Value of Cancer Treatment Such As Erbitux Gets More Complicated

It seems the contrast between the often staggering cost of treatment and the modest yield in life expectancy caught the attention of the editors and academics over in the Journal of the National Cancer Institute (JNCI). The DMCB thanks them for confirming an issue raised previously in this blog. How so, you ask? Read on.

Erbitux (generic name cetuximab) is one of several manufactured proteins that in turn bind to a protein component found on various human cell surfaces that regulate growth (called, appropriately enough, ‘growth factors’). When combined with standard chemotherapy for cancers such as lung or colon, Erbitux has been shown to modestly increase life expectancy. Unfortunately, Erbitux is a very sophisticated product that required years of expensive testing and development, demand for the drug is very high, setting prices for medications is very arcane and the manufacturer has a patent.

As a result, it’s not unusual for a treatment course consisting of several IV infusions of Erbitux to cost tens of thousands of dollars. When projected over the number of persons that would qualify for treatment, authors Tito Fojo and Christine Grady apparently (access to the full text is restricted) determined that the total cost to our nation’s health care system could add up to a whopping $440 billion per year.

In our age of printing up trillions in dead Presidents, that’s not necessarily the problem. The addition of Erbitux, according to summary news reports, results in a paltry average of 1.2 months of added life expectancy. This poor value proposition troubles the authors, who recommend that expected cost and life expectancy should be used as a criteria for the funding of future cancer treatment research.

According to the Wall Street Journal, actual patients may disagree. They point out that today’s expensive advance will be cheaper tomorrow, big breakthroughs lead to other breakthroughs, persons can use the 1.2 months as a bridge to other treatments and, well, it’s only money.

For the record, the prescient DMCB raised this issue just hours before the JNCI release and believes Drs. Fojo and Grady are confirming that stomping out ‘waste’ and ‘medical mistakes’ pale in comparison to our need to reconcile the high cost of technology versus the actual yield. The issue has been around for a long time. One old example is this comparison of the expensive clot buster tPA versus cheap clot buster streptokinase for heart attack. As for our chances of reconciling high cost vs. modest yield: good luck, even in today’s reform-minded environment.

However, while the mainstream media have focused on the validity of the $440 billion price tag, the contrarian DMCB thinks ‘1.2 months’ is statistically unfair and does a poor job of reflecting the real issues faced by cancer patients. Check out this real life clinical trial that is available on line. It showed Erbitux resulted in a median survival of 12 months with a confidence interval ranging from about 8 ½ to just over 15 months versus just over 9 months of survival with a confidence interval extending from about 7 ½ months to just under 12 months without Erbitux.

Most scientific studies report confidence intervals to give you an idea of the distribution of the results. In other words, there is a 'plus minus' ‘spread’ around the average in how any population of patients will respond to treatment. That is determined by myriad clinical factors but, once the numbers are added up, it acts in typical random 'Gaussian' behavior. This means the real bottom line in this trial is that getting Erbitux may result in a life expectancy as high as 15 months versus a life expectancy as low as 7 ½ months without Erbitux.

Depending on the real price of a course of Erbitux, that may place it within reach of the standard threshold of cost effectiveness. Put another way, if you were told your life expectancy could double to 15 months for, say, 20 grand, would you go for it? Saying yes at an individual and policy level is not that unreasonable. The DMCB says the JNCI editors' failure to recognize that real world calculus is unreasonable.

Oh, and one more thing: the JNCI authors suggest that ‘oncologists must offer clear guidance for the conduct of research, interpretation of results, and prescription of chemotherapies.’ The DNCB’s experience dealing with oncologists and their drugs as a medical director in a highly regarded health plan taught it otherwise. So does some peer review literature. The JNCI authors are not only also unreasonable, they're being naive.