An Outstanding Review Appears in JAMA about Population Based Outcomes Studies

For your consideration. Canucks Christopher Booth and William Mackillop of the Queen's University in Ontario have an on-spot article in the latest issue of JAMA. After citing three telling examples of the contrast between randomized clinical trials (RCTs) and population based outcomes studies (PBOSs), the authors provide some needed guidance.

The three examples are 1) an exquisitely performed New England Journal study that demonstrated spironolactone (a special type of diuretic) caused persons with chronic heart failure to live longer and stay out of the hospital more often. Use of this drug increased (including among the DMCB's patients) until a follow-up PBOS showed no change in inpatient stays for CHF but an alarming increase in the incidence of a life-threatening spironolactone side effect: high blood potassium levels; 2) after the publication of several studies, the National Cancer Institute announced chemotherapy should be used in persons with cancer of the cervix. A follow-up PBOS confirmed that women with this kind of cancer were indeed living longer; 3) lacking an ability to perform a RCT, a PBOS pre-post study was used to assess the relationship of inpatient versus outpatient care for persons with mini-strokes. Inpatient care was associated with a better outcome, leading to this approach becoming the standard of care.

The authors correctly caution that the entire population has to be included (to avoid referral bias, ie the selection of persons most likely to benefit), other events that may influence events have to be accounted for, sorting patients by condition not treatment is necessary and, last but not least, a comprehensive registry is vital. There is also a dig at the real purpose here: not citations or publications (or grants that fund Departments, or attending scientific meetings filled with other scientists, or getting rank and tenure), but real meaningful improvements in patient and society well being.

The authors are not trashing RCTs, merely putting them into perspective. They have their place. However, they show PBOSs can add additional insights and can help guide therapies when RCTs are not possible or when RCTs fail to provide the whole answer.

Implications for the disease management community? For starters, we should put this reference in all those citations at the end of our glossy marketing literature. But more importantly, we should be challenging ourselves as well as government and commercial payers to use our/their population-based data bases to invest in and perform PBOSs; if investing in them is not possible (though Kaiser tells us otherwise), making them publically available on-line is another option. In addition, there is a growing science around PBOSs. It's time to get exquisitly familiar with the methodologies and install in it our DNA. Last but not least, we need to caution our policy makers about the merits and risks of having an information monopoly centered in a national center for comparative effectiveness that favors RCTs.

Unfortunately, access to the full manuscript and its bibliography is available by subscription only. It's still worth trying to get a copy.

Booth CM, MacKillop WJ: Translating new medical therapies into societal benefit. The role of population-base outcome studies. JAMA 2008; 300(18): 2177-2179