By Jack D. McCue MD
At the end of 2018 a simple search in PubMed (the online US National Library of Medicine at the National Institutes of Health) with the criteria “cannabis or cannabinoids or marijuana” will identify more than 38,000 citations in bioscientific journals and books, dating back to 1846.
The total number of citations is now growing by more than 3,000 each year with a shrinking doubling time of seven years.
Most of the cited papers are trivial, dated, and ill-informed, and many are polemics by ignorant editorialists irresistibly drawn to cute titles such as “Medical Marijuana: All Smoke and Mirrors?” or “High Time for Medical Marijuana?
But there are many thousands of serious scientific papers, some providing profound insights into the functioning of the most important signaling system in the mammalian body, and irrefutable documentation of the medical benefits of cannabinoids and terpenes.
The challenge for the doctor is how to find the useful and reliable published material from thousands of clinical studies from dozens of countries written over the past century .
Evidence-Based Medicine (EBM) to the rescue? Regrettably, not yet.
What is Evidence-Based Medicine?
Beginning in the 1960s, an interest in how physicians made decisions developed into a scholarly discipline that employedinsights from sophisticated statistical analyses, clinical epidemiology, epistemology, psychology, and economics.1
The pioneers of Evidence-Based Medicine include David Eddy (who first used the term “evidence-based medicine” and developed much of its methodology), Alvan Feinstein (who popularized Venn diagrams —see graphic above), Archie Cochrane (who devised a ranking system for quality of design in published papers), and David Sackett (who developed the first program in clinical epidemiology at McMaster University).
My brush with EBM began in the early 1970s at Beth Israel Hospital and the Harvard Community Health Plan. Anthony Komaroff, MD, the leader of our research group, believed that evaluation and in some cases, treatment, of common complaints (for example, urinary tract infection, cough, chest pain) could be reduced to binary decisions (yes-no answers on a “check-box practice guideline” template), and each decision would be guided by a rigorous assessment of the evidence. Hence, evidence-based medicine. The resulting algorithms could be used to enable nurse practitioners and physician assistants to handle minor problems and triage more complex ones for physician evaluation. Or at least that was the grand idea.
Unfortunately, clinicians still consistently preferred to use their own clinical judgment. Me, too.
At its best, evidence-based medicine merges scientific research findings with clinicians’ best medical judgment based on their experience and what the patient is seeking from their medical care.
Of course, it doesn’t work that way. Perhaps inevitably, the techniques of decision analysis, meta-analysis (which employs statistical methods to combine the patient numbers and results of disparate clinical trials into a single, more powerful conclusion), and systematic reviews of the medical literature moved away from analyses designed to be helpful to practitioners, and progressively began to emphasize the mechanics of EBM —the statistical playground of PhDs rather than clinicians.
Cannabis has been spared the cold analytic glare of EBM until recently. There were just not enough randomized clinical trials (RCTs) to analyze.
Input of physician judgment was systematically avoided as being unreliable. (The PhDs do have their point, but medical practice would be paralyzed if we could only make decisions based on presumed-reliable analytic data produced by PhDs).
Cannabis has been spared the cold analytic glare of EBM until recently. There were just not enough randomized clinical trials (RCTs) to analyze. That changed, for better or worse, in June 2015, when “Cannabinoids for Medical Use: A Systematic Review and Meta-analysis” by Whiting et al was published in the Journal of the American Medical Association.2
The meta-analysis found that for two diagnoses —chronic pain and spasticity due to multiple sclerosis— there was at least moderately good evidence of medical effectiveness of cannabinoids and cannabis. (See table on next page.)
The authors concluded that the evidence supporting effectiveness of cannabinoids in the treatment of nausea and vomiting, HIV/AIDS, sleep disorders, anxiety, depression, psychosis and Tourette syndrome was credible, but weak or very weak.
This was the only nearly-comprehensive EBM paper of the cannabis literature using proper meta-analytic techniques, and doctors and politicians gave it more credibility than it deserved. As an indication of how often it has been used to summarize clinical research, the landmark report of the National Academy of Sciences, Engineering, and Medicine (NAS) released in January 2017 referenced Whiting et al on 21 occasions.3
It does, however, provide “exhibit A” in our assessment of whether EBM currently offers the best techniques for analyzing the clinical evidence for the widely held beliefs by cannabis users that it is effective for treatment of a large number of medical conditions.
Cannabis is being widely used in the treatment of many more than 10 conditions.
The Whiting EBM analysis was funded by the Swiss Federal Office of Public Health, which determined the 10 diagnoses to be studied. Cannabis is being widely used in the treatment of many more than 10 conditions, and from the clinician’s perspective, it would be helpful to assess the relevant literature on all of them. The problem is a lack of adequate RCTs for EBM.
The choices by Whiting et al were not necessarily bad ones, but EBM is only as useful as the topics chosen for examination. It is not clear why only 10 conditions were reviewed, or why those 10 were chosen. (Oddly, they included glaucoma, which has no RCTs to analyze.)
For example, migraine, epilepsy, opioid dependence, stress, and fibromyalgia have a useful supporting medical literature —but no solid RCTs. A few others, such as inflammatory bowel disease do have persuasive small RCTs and would have earned a weak or very weak rating.4
But the core methodology of EBM is that only RCTs are taken as serious data. Unless the study is an RCT, the best EBM will give it is a low or very-low quality rating —if that. EBM excludes non-clinical studies (i.e., animal or lab studies), which is appropriate, because drugs should not be used based on non-human studies. Thus 99.7% of the scientific articles that the JAMA authors’ literature search uncovered were discarded! Stating it differently, the entire analysis was based on only 79 out of the 23,754 articles they identified!
In the JAMA paper by Whiting et al, nearly all the RCTs that made it through the screening criteria were pharmaceutical company trials or used synthetic CBD and THC —very few used what our patients actually use —herbal Cannabis.
And no experienced cannabis clinician thinks that a single synthetic cannabinoid molecule can replicate the beneficial “entourage” effects of the cannabis plant. At least one Israeli RCT confirms the superior efficacy of THC-rich whole-plant extracts in the treatment of IBD, and CBD-rich extracts were strikingly superior to purified CBD in multiple preclinical studies.5
Trying to corral messy plants like cannabis inside the tight, transparent boundaries of the randomized placebo-controlled clinical trial in humans is harder —and harder to find funding for— than studying a synthetic single molecule in rats. Ethan Russo has concisely summarized the barriers to clinical research encountered when the active medication being tested is botanical cannabis.6 (See box at bottom of page.)
Seven of eight studies on pain deemed “adequate” by the JAMA authors were Sativex trials.
Most drug studies that meet the stringent requirements for inclusion in EBM are ones aimed at the FDA drug approval process. Seven of eight studies on pain deemed “adequate” by the JAMA authors were Sativex trials. All six studies on spasticity were either Sativex (four), Marinol, or a concoction of synthetic THC and CBD. Sativex is constituted as a whole-plant extract of multiple cultivars, but even the basic studies needed to demonstrate Sativex’s equivalence with whole-plant smoke or vapor in humans have not been performed —and it may not be.
Even when EBM is done well, there is room for interpretation and the results may not agree with other EBM reviews. Hence, recommendations based on seemingly reliable RCTs may differ widely.
Post-traumatic stress disorder (PTSD) is an apt example, in part because it is a disabling problem for which cannabis is obviously superior to other therapies. (For shame, VA.) A careful, clear-eyed commentary on the conflicting practice guidelines for treating PTSD drawn up by six august medical bureaucracies and professional bodies, all of which were supposedly based on a systematic review of all available RCTs, sniffed: “It is notable that all of these (conflicting) reports are essentially reviews of the same research but have drawn different conclusions.”7
Are There Alternatives to EBM?
What happens when the crusty professor who has spent a 40+ year career educating students and post-graduate medical residents in evidence-based clinical decision-making (not to mention decades as a CMO, using EBM to berate the poor surgeons to cut costs) fails his retirement? In an attempt to avoid going stir-crazy, I took a part-time job in rural Northern California advising more than 3,000 cannabis-using patients —nearly all of whom, incidentally, grow their own weed and know an awful lot about cannabis (in dramatic contrast to the doctor-professor who knew nothing at all).
Seriously! I had never heard of THC or CBD, and I thought my neighbor in Marin County had a bad skunk problem. Well, the first thing you do is a lot of quiet listening, and you learn the humility it takes to be taught by knowledgeable salt-of-the-earth people, many of whom had not made it through high school. And you quickly learn that the medical literature on cannabis is shallow and terribly biased, and that the seemingly authoritative researchers are actually as ignorant as I was.
My instinct, turning to EBM for knowledge and thoughtful answers, was a joke. The EBM-worthy research wasn’t there, and what was there was almost always unhelpful. (Dumb was the first adjective that came to mind.).
EBM is a very powerful, necessary tool for making sense out of an overwhelming amount of often conflicting data, but it is only as good as the studies it analyzes.
For example, just because the data from a well-designed study shows statistically significant differences does not mean that it is medically correct. There may be hidden flaws in the study design. Let’s say you discover that people who take vitamins live longer. But unfortunately, you have missed the possibilities that people who buy vitamins are in a higher socioeconomic group or maybe people who take vitamins are less likely to smoke tobacco, and that is why they live longer.
Or, if the drug being tested is a harsh synthetic like Marinol, failure to demonstrate statistically significant results may be related to the poor choice of the intervention itself or poor dosing. And as Russo has pointed out, the placebo effect in cannabis studies is large and growing, and the benefits actually provided by cannabis are influenced by patients’ expectations that it will be very effective.
Another (preferable, but still flawed) option is the “systematic review,” which attempts to avoid some of the serious shortcomings of EBM. The systematic review process is exemplified by the impressive NAS publication (468 pages!), “The Health Effects of Cannabis and Cannabinoids: The Current State of Evidence and Recommendations for Research,” which was designed to be a comprehensive review of the literature by panels of 31 experts supported by a large staff of research associates and clerical assistants, plus some uber-experts overseeing it all.
The staff and experts determine which scientific articles are worthy of review with more of an “inclusive” process based on potential relevance of content of articles, rather than the “exclusive” process of EBM in which all articles are initially included based on medical content and then subsequently excluded, based on research methodology rather than content. The NAS systematic review looked at 21 conditions.
The NAS review, however, reached many of the same conclusions, in part because, with a very few exceptions, the experts actually had no experience with medical cannabis and did not actually take care of patients who use cannabis medically. Furthermore, as noted, they relied heavily on the Whiting review, citing it 21 separate times. There was no indication that very many of the reviewers actually looked up or read the studies that Whiting et al included or excluded from that meta-analysis. Indeed, in many cases the NAS was just a re-hashing of the Whiting review, incorporating its potential flaws and biases.
Turning to statistically flawed studies is sometimes surprisingly helpful. For one thing, the researchers behind weaker studies usually know more about cannabis than those behind the FDA-ready studies.
Let’s look at some examples of weak studies that reached the correct conclusions, but could never aspire to the standards of EBM.
Migraine. As any doctor who has monitored cannabis use by migraine-sufferers can tell you, it works —often dramatically. Yet until very recently there were no human studies to validate or deny those anecdotal reports.
DN Rhyne and colleagues in the family practice program at the University of Colorado recently published a retrospective chart study of 121 patients (unfortunately, statistically speaking, out of the 262 patients they identified).8 They found that use of cannabis was associated with a reduction in migraine episodes by more than half (with one chance in a thousand that their results could be a random outcome). Methodologically, this study had so many design flaws that it would reduce an EBM researcher to tears. But it is the only study in the medical literature, and it just happens to be correct —without a doubt.
Conclusion #1: Just because a study is poorly designed and executed does not mean that it is wrong. If it is all you have, and the intervention is relatively harmless, cannabis may be worth a try. If it works…it works!
Insomnia. Studies of cannabinoids in insomnia barely made it past the lowest standard of EBM (“very low” reliability). The JAMA meta-analysis found only two insomnia studies worthy of inclusion and both used a rather harsh synthetic THC-like drug, Nabilone, developed by Eli Lilly in the 1970s.
In contrast, a previous comprehensive review9 found 39 studies that merited inclusion in their meta-analysis. (Eleven were Sativex trials, which generally have decent study designs). This review properly concluded that the human studies of medical cannabis and insomnia varied greatly in design and were generally of poor quality. In one pathetically bad study,10 30 mg of Marinol (!) was given to two brothers for 14 nights. Its effect on sleep was “poorly understood,” the authors concluded. Oy veh!
Overall, the comprehensive review concluded that while the study outcomes were highly variable, when a consistent measurement tool for insomnia was used, the results were more consistent. Duh.
Never considered in either of the reviews2,9 were the alternative treatments for insomnia.
Zolpidem (Ambien) has bizarre and dangerous adverse effects that have landed insomniacs in jail, and is a drug that reliably induces durable dependence. Benzodiazepines cause mental numbness and hangovers, and reliably result in dependence. Low doses of the antidepressant amitriptyline result in lingering drowsiness and can cause fatal overdoses; antihistamines and melatonin simply don’t work. Set in context with this collection of losers, the cannabis studies look rather promising!
Conclusion #2: If the alternatives are dangerous and ineffective, analyzing the quality of studies concluding that cannabis might be effective is kind of beside the point. It is worth a try until more statistically convincing studies find it to be ineffective. And the fact is, for a substantial minority (maybe about 30%), it does work well when all the other options have failed.
Cancer. It is incontrovertibly true that cannabis is effective for many of the symptoms of cancer (pain, appetite loss) and side effects of cancer treatments (nausea and vomiting, painful neuropathy). And it is almost certainly helpful for many other associated secondary symptoms (anxiety, insomnia, depression, feelings of hopelessness). But does it actually treat the malignancy itself?
The complexity of this seemingly simple question is beyond daunting. The anecdotal examples of malignancies that have improved or been cured can be powerfully persuasive, but they also can be incorrect. Still, a patient with an unresectable glioblastoma has less than a coin-flip chance that he or she will live more than a few months. As long as adjunctive treatment with cannabis does not interfere with the treatment of the cancer itself (and there are legitimate concerns for some cancers), what do you have to lose?
The obvious answer is that some preclinical (test-tube and rat) trials of THC show worsening of cancers, sometimes in high doses and sometimes in low doses. Worsening has not been demonstrated with CBD. Cannabis should always be used as adjunctive therapy, and probably CBD-dominant cultivars should be favored.
The most secure recommendation is that if cannabis can be used to treat symptoms of the cancer or the chemotherapy effectively, arguments against also trying to use it as possible treatment of tumor look rather thin. Unfortunately, the question then becomes “Well, if I am hoping for an anti-tumor effect, how do I use it?” The answer to that question is simply unknown.
Conclusion #3: Lack of good studies notwithstanding, if you are running out of treatment options, have little or nothing to lose, and the risk of harm is low, why not try it?
Inflammatory Bowel Disease (IBD). Conventional treatments of IBD (predominantly ulcerative colitis and Crohn’s disease) work for most patients, but a relatively small percentage of cases are resistant to treatment, or patients cannot tolerate (or afford) some of the most effective therapies. In an informal pilot study, Hergenrather documented that patients with IBD clearly believe that cannabis relieves symptoms, improves quality of life, and allows them to discontinue some of the toxic drugs routinely used for treatment.11
There is also a series of small studies by Timna Naftali and colleagues in Israel who strongly assert that (plant) cannabis treatment is effective, including one that is a credible RCT.4 Unfortunately, all the studies are small, highly susceptible to bias, and illustrate several of the barriers to high quality cannabis RCTs pointed out by Russo.
Nevertheless, anecdotal experiences of success with self-treatment of IBD with cannabis, such as Hergenrather’s, are consistent with the recent Israeli RCT showing complete remission in five of 11 patients, complete weaning of corticosteroids in three, and statistically significant improvement in appetite, sleep, and quality of life.
A follow-up, larger study has been initiated, but these small studies are all we have for the near future. They probably would not have made it past the exclusion criteria, even if the JAMA meta-analysis had included IBD among its diagnoses, as it should have. But neither can they be ignored.
Conclusion #4. Pretty good research just has to be good enough while we wait for statistically solid studies to be done. The irrational blocks to legitimate cannabis research must be removed so good RCTs can be done.
In summary, EBM’s standards are goals for clinical researchers who are objectively examining medical benefits of cannabis to strive for. Under-funded, ill-designed studies carried out under siege by government agencies can hardly meet those standards.6 Until the obstacles to good research are removed, the scientific community must ask whether the EBM-style criteria that work for pharmaceutical research are the best way to evaluate plant-based therapies. Doctors and researchers must learn how to use plant cannabis with proper dose titration and reliable delivery systems before consistent results can be expected.
The needed clinical trials will emerge slowly and most research results will continue to be frustratingly inconsistent. That does not excuse us from doing the best with what we have, for the benefit of our patients who need our help now.
*Available online as a free download or for order of the hard copy.
References:
1. Masic I, Muhamedagic B. Evidence based medicine – new approaches and challenges. Acta Inform Med 2008;16:219-25.
2. Whiting PF, Wolff RF, Deshpande S, Di Nisio M, Duffy S, Hernandez AV, Keurentjes JC, Lang S, Misso K, Ryder S, Schmidlkofer S, Westwood M, Kleijnen J. Cannabinoids for medical use: A systematic review and meta-analysis. JAMA 2015;313:2456-73.
3 National Academy of Sciences, Engineering, and Medicine 2017. The Health Effects of Cannabis and Cannabinoids: The Current State of Evidence and Recommendations for Research. Washington, DC: The National Academies Press.
4. Naftali T, Mechulam R, Lev LB, Konikoff FM. Cannabis for inflammatory bowel disease. Dig Dis 2014;32:468-74.
5. Gallily R, Yekhtin , Hanus LO. Overcoming the bell-shaped dose-response of cannabidiol by using Cannabis extract enriched in cannabidiol.Pharmacology & Pharmacy 2015;6:75-85.
6. Russo EB. Current therapeutic cannabis controversies and clinical trial design issues. Front Pharmacol 2016;7:309.
7. Difede J, Olden M, Cukor J. Evidence-based treatment of post-traumatic stress disorder. Annu Rev Med 2014;65:319-32.
8. Rhyne DN, Anderson SL, Gedde M, Borgelt LM. Effects of medical marijuana on migraine headache frequency in an adult population. Pharmacotherapy 2016;36:505-10.
9. Gates PJ, Albertella L, Copeland J. The effects of cannabinoid administration on sleep: A systematic review of human studies. Sleep Med Rev 2014;18:477-87.
10. Freemon FR. The effect of chronically administered delta-9-tetrahydrocannabinol upon the polygraphically monitored sleep of normal volunteers. Drug Alcohol Depend 1982;10:345-53.
11. https://beyondthc.com/wp-content/uploads/2013/03/SCC-Crohns-Study.pdf
12. Cannabis and Cannabinoids. JAMA 2016:316:2424-5.
To read this article as a pdf: McCue on EBM
