The $440 Billion Question for the Value of Cancer Treatment Such As Erbitux Gets Even MORE Complicated

In a prior posting, the Disease Management Care Blog made a statistically based argument about the plus-or-minus distribution or 'spread' of cancer treatment-related survival around an average. It referred to a specific Erbitux research paper, where there were two treatment groups, each with an 'average' survival with a standard deviation. The DMCB compared and contrasted the two standard deviations, not the average results, and went on to state that cancer patients and their oncologist physicians are more interested in the upper and lower bounds of possibility, not what happened to most patients in the middle.

Well, wouldn't you know, a statistician emailed the DMCB with some insights about its faulty logic. It likes to hear from statisticians almost as much as from actuaries, especially when there's learning to be done. This reminds the DMCB of a key difference between statisticians and actuaries: the former tells you how you were wrong, while the latter tells how you're going to be wrong.

Cody L. Custis is employed by the State of Montana and also teaches mathematics at UM Helena. He points out that he is speaking as an individual and not for any of his employers. The DMCB points out it doesn't feel so bad if it took these kinds of credentials to uncover a mistake.

Here' the email:

As a professional statistician, I wanted to raise two objections about the conclusions in your blog's posting on Erbitux. In it, you say:

“Check out this real life clinical trial that is available on line. It showed Erbitux resulted in a median survival of 12 months with a confidence interval ranging from about 8 ½ to just over 15 months versus just over 9 months of survival with a confidence interval extending from about 7 ½ months to just under 12 months without Erbitux. This means the real bottom line in this trial is that getting Erbitux may result in a life expectancy as high as 15 months versus a life expectancy as low as 7 ½ months without Erbitux.”

Based upon the study referenced, I assume that these conclusions come from the following statement in the Butts et. al. paper:

"Median survival time was 11.99 months in the cetuximab arm (95% CI, 8.80 to 15.18) and 9.26 months in the platinum/gemcitabine arm (95% CI, 7.43 to 11.79)."

First, if a clinical trial involving two groups results in two sets of data, there will be two separate averages or means. If one group’s average result is compared to the other group’s average, it is important to not only know the difference between the two means, but also the confidence interval for that difference. In contrast, your blog compared two separate confidence intervals. In the Butts et. al. paper, the authors did not construct a confidence interval for the increase in life expectancy in the Cetuximab study; rather, they calculated two separate intervals for two treatments. Because confidence intervals are given for the two treatments separately, rather than for the difference, cancer patients and their physicians cannot know if a difference in the life expectancy of 1.5 months is really significant. Mathematically, if the confidence interval extends to zero, the 1.5 months could be the outcome of random chance. As the authors of the Butts et. al. paper state in their conclusion : 'The major limitation of the study was its noncomparative design, not statistically powered to demonstrate significant differences between treatment arms.'

Second, your blog's conclusion takes the worst possible case for one treatment and the best possible case for another treatment. Thus, while your blog is correct to focus on the importance of confidence intervals rather than point estimation, the conclusions are based on unfair assignment of best and worst case outcomes of two treatments. A skeptic could just as fairly state the conclusion: this means the real bottom line in this trial is that getting Erbitux may result in a life expectancy as low as 8 months versus a life expectancy as high as 12 months without Erbitux. Both conclusions unfairly take extreme outcomes.

The DMCB says good points. In the paper referenced, patients would be better served by knowing the distribution around the average difference in survival and should also be made aware of the up and down sides of any treatment option.

That being said, the DMCB also thinks, based on experience, that most patients and their oncologists tend to believe in the most optimistic treatment scenarios. If those scenarios fall with the reasonable (plus or minus) bounds of possibility, it's hard for insurers, policy makers, comparative effectiveness researchers, regulators and legislators to say no.

The DMCB thanks Cody Custis for the insights.

The $440 Billion Question for the Value of Cancer Treatment Such As Erbitux Gets More Complicated

It seems the contrast between the often staggering cost of treatment and the modest yield in life expectancy caught the attention of the editors and academics over in the Journal of the National Cancer Institute (JNCI). The DMCB thanks them for confirming an issue raised previously in this blog. How so, you ask? Read on.

Erbitux (generic name cetuximab) is one of several manufactured proteins that in turn bind to a protein component found on various human cell surfaces that regulate growth (called, appropriately enough, ‘growth factors’). When combined with standard chemotherapy for cancers such as lung or colon, Erbitux has been shown to modestly increase life expectancy. Unfortunately, Erbitux is a very sophisticated product that required years of expensive testing and development, demand for the drug is very high, setting prices for medications is very arcane and the manufacturer has a patent.

As a result, it’s not unusual for a treatment course consisting of several IV infusions of Erbitux to cost tens of thousands of dollars. When projected over the number of persons that would qualify for treatment, authors Tito Fojo and Christine Grady apparently (access to the full text is restricted) determined that the total cost to our nation’s health care system could add up to a whopping $440 billion per year.

In our age of printing up trillions in dead Presidents, that’s not necessarily the problem. The addition of Erbitux, according to summary news reports, results in a paltry average of 1.2 months of added life expectancy. This poor value proposition troubles the authors, who recommend that expected cost and life expectancy should be used as a criteria for the funding of future cancer treatment research.

According to the Wall Street Journal, actual patients may disagree. They point out that today’s expensive advance will be cheaper tomorrow, big breakthroughs lead to other breakthroughs, persons can use the 1.2 months as a bridge to other treatments and, well, it’s only money.

For the record, the prescient DMCB raised this issue just hours before the JNCI release and believes Drs. Fojo and Grady are confirming that stomping out ‘waste’ and ‘medical mistakes’ pale in comparison to our need to reconcile the high cost of technology versus the actual yield. The issue has been around for a long time. One old example is this comparison of the expensive clot buster tPA versus cheap clot buster streptokinase for heart attack. As for our chances of reconciling high cost vs. modest yield: good luck, even in today’s reform-minded environment.

However, while the mainstream media have focused on the validity of the $440 billion price tag, the contrarian DMCB thinks ‘1.2 months’ is statistically unfair and does a poor job of reflecting the real issues faced by cancer patients. Check out this real life clinical trial that is available on line. It showed Erbitux resulted in a median survival of 12 months with a confidence interval ranging from about 8 ½ to just over 15 months versus just over 9 months of survival with a confidence interval extending from about 7 ½ months to just under 12 months without Erbitux.

Most scientific studies report confidence intervals to give you an idea of the distribution of the results. In other words, there is a 'plus minus' ‘spread’ around the average in how any population of patients will respond to treatment. That is determined by myriad clinical factors but, once the numbers are added up, it acts in typical random 'Gaussian' behavior. This means the real bottom line in this trial is that getting Erbitux may result in a life expectancy as high as 15 months versus a life expectancy as low as 7 ½ months without Erbitux.

Depending on the real price of a course of Erbitux, that may place it within reach of the standard threshold of cost effectiveness. Put another way, if you were told your life expectancy could double to 15 months for, say, 20 grand, would you go for it? Saying yes at an individual and policy level is not that unreasonable. The DMCB says the JNCI editors' failure to recognize that real world calculus is unreasonable.

Oh, and one more thing: the JNCI authors suggest that ‘oncologists must offer clear guidance for the conduct of research, interpretation of results, and prescription of chemotherapies.’ The DNCB’s experience dealing with oncologists and their drugs as a medical director in a highly regarded health plan taught it otherwise. So does some peer review literature. The JNCI authors are not only also unreasonable, they're being naive.