What’s up with disease management organizations and their ‘white papers?’ Typically paired with some inflated press release or a laudatory web site, these downloads (also labeled as ‘research,’) are typically engineered to convince the reader a novel and important advance is at hand. Capitalizing on the company’s supposed scientific ‘player’ status, we’re expected to believe that a ‘patented approach,’ or a ‘validated methodology’ or a ‘proprietary system’ backed by ‘industry leaders’ increases quality, saves money, attains physician buy-in and restores karma across the healthcare landscape. Alas, while there may be some exceptions, most of the time, those assertions are not supported by the data..With the increasing visibility of the Disease Management Care Blog, it is fast becoming the target of company sponsored white papers, analyses, reviews and postings. It has generally avoided commenting on them because it has found many of the papers have astonishingly shallow research designs, truncated bibliographies and self-serving conclusions. Recall how disease management organizations published all that junk years ago? While the reviewers and editors finally caught up with them, like whack-a-moles, this pseudoscience still has a lingering safe zone on the web-pages of and spam-like email attachments from even some of the more reputable disease management companies.
Ah, you say, this is just the plain ol’ marketing, what’s the big deal and everyone does it? The big deal is that the disease management industry has a unique reputation for lousy analyses that have repeatedly violated multiple fundamental statistical and analytic rules. The audience of clinical leaders, policy makers, politicians’ aides, physicians and insurance executives has been sensitized to this and are likely to see these white papers for what they are: the same old garbage. As a result, these glossy faux articles paradoxically cause the industry to lose credibility. And since it’s not hard to find the old white papers out there, the arrival of even more makes the DMCB think there must be a manuscript zombies about, spreading their pestilence thanks to naïve marketing departments, poorly trained scientific writers and high level executives who are impatient with science and the pace of peer review.
The DMCB thinks the blow-back toxicity of the white papers run amok is enough to warrant a moratorium. The DMCB is humbly resigned, however, to the likelihood that the industry will not follow its recommendations. So, it offers up the DMCB Seven - If You Must - Rules to White Paper Management:
1. Think: is the scientific news that your company wants to share really so critically important that you can’t accommodate the peer review process? If it’s that important, many Journals will 'fast track' your findings. If it’s not that important, then it’s probably marketing, not a white paper. Treat it as such and don’t insult the intelligence of your customers by pretending otherwise.
2. There are some outstanding Journals that have an outstanding turn-around time with volunteer expert peer reviewers that can help sharpen the conclusions, spot inappropriate assertions and help you with a high acceptance rate. Your company won’t have to wait long to see its findings in print with all the advantages of a white paper with fewer gaffes.
3. If you must issue a white paper, strive to make it good enough to pass peer review anyway. In fact, it’s possible to submit it to a journal, skip the publication (though that’s very rude if they offer to take it) and use the reviewers’ comments to make the manuscript better.
4. The DMCB calls your bluff if you go with a white paper. Try using your readership as peer reviewers by open sourcing your (HIPAA compliant) data so that others can confirm your findings.
5. While you may believe your company is aided by claims of proprietary expertise, transparency is a stronger suit long term.
6. Minimize the involvement of sales and marketing until after the scientific writing is completed.
7. Think about getting involved with the PHI Institute. These guys have a process that helps companies assure that their in-house analyses follow the basic rules when it comes to correctly attributing causality between your program on one end that the observed outcomes at the other end.
Posts
2 comments:
The DMCB should be commended for questioning the value of vendor-supplied white papers. As a researcher and healthcare executive, I continue to be puzzled as to why DM and other healthcare companies believe they are creating any meaningful sales or brand value from white papers. My experience strongly suggests that peer-reviewed publications have far greater marketing value than white papers.
Leading research and product development at a PBM for many years, I issued white papers very infrequently and in small proportion relative to peer-reviewed publications. In deciding whether to use a white paper, I had a few simple guidelines that I would encourage other organizations to consider: 1) Is the research trying to make a causal claim? If yes, the paper should go through the review process of a peer-reviewed journal. However, if the study is descriptive in nature, (i.e., not trying to make a causal claim) and the findings are NOT promotional in their conclusions (e.g., changes in medical spending over time), then a white paper could be considered. 2) Any white papers must have full transparency of methods. While most readers are not interested in methodological specifics, the full detail should be made available. 3) Case studies should not be positioned as white papers but left to the realm of marketing materials while still undergoing proper review by research teams before dissemination. These guidelines will prevent readers/plan sponsors from confusing white papers with validation of outcomes, which should come through the peer-review process.
None of these guidelines guarantee rigorous white papers, which is why DMCB’s recommendation to make the underlying data available is an excellent suggestion. The concept has merit, not just for white papers, but even for peer-reviewed publications, which are fraught with inconsistent quality in peer- reviews. The field of economics has been making data available for years, creating much greater accountability for the quality of the methodology and conclusions. While many will raise concerns over confidentially and proprietary information, I believe these arguments are red herrings and that both issues can be satisfactorily addressed. This is an idea that warrants further discussion.
The only point on which I disagreed with DMCB’s commentary on white papers is the quality of the peer-review process and articles published in disease management-focused journals. Generally speaking, I find the quality of the work in these journals to be lacking. Many studies make causal claims without even basic quasi-experimental designs that include appropriate control groups, and the study conclusions are frequently not supported by the study data and methodology. Plan sponsors, the real audience for these publications, would be better served by raising the quality of the articles published in these journals.
Brenda Motheral, BPharm, MBA, PhD
President, CareScientific
I like the guidelines from Dr. Motheral. I suspect that if I had thought through the original post over several days (instead of just one - the typical time horizon for a keeping a blog), the 3 simple rules would have bubbled to the top.
In a separate email, another colleague pointed out that peer review is not all that it should be. I'm surprised to hear that, since my publications have benefitted immensely from them - and as a reviewer, I like to think I've helped improve the quality of the scientific discourse. I don't mind poor data, design and conclusions so much, so long as (the Editors force) the author(s) take the time to point out the weaknesses in their "Discussion" AND if readers remember the motto of caveat emptor.
Post a Comment