by Jack McCue, MD, will run in the print edition of O’Shaughnessy’s. We are posting it online because it is so pertinent to breaking news, notably the release of the NAS Report and the publication of clinical evidence by Sulak et al in Epilepsy & Behavior. Here is ‘Evidence-Based Medicine vs. Medicinal Cannabis laid out for the paper. And here are some excerpts:

What is Evidence-Based Medicine?

Beginning in the 1960s, an interest in how physicians made decisions developed into a scholarly discipline that employed insights from sophisticated statistical analyses, clinical epidemiology, epistemology, psychology, and economics. 

The pioneers of Evidence-Based Medicine include David Eddy (who first used the term “evidence-based medicine” and developed much of its methodology), Alvan Feinstein (who popularized Venn diagrams —see graphic below), Archie Cochrane (who devised a ranking system for quality of design in published papers), and David Sackett (who developed the first program in clinical epidemiology at McMaster University).

My brush with EBM began in the early 1970s at Beth Israel Hospital and the Harvard Community Health Plan. Anthony Kamaroff, MD, the leader of our research group, believed that evaluation and in some cases, treatment, of common complaints (for example, urinary tract infection, cough, chest pain) could be reduced to binary decisions, each of which would be supported by assessment of the evidence. The resulting algorithms could be used by nurse practioners and physician assistants to handle minor problems and refer more complex ones for physician evaluation.

Unfortunately, clinicians consistently preferred to use clinical judgment. Me, too.

At its best, Evidence-Based Medicine merges scientific findings with clinicians’ best medical judgment based on their experience and what the patient is seeking from their medical care.

Of course, it doesn’t work that way. Perhaps inevitably, the techniques of decision analysis, meta-analysis (which employs statistical methods to combine the patient numbers and results of disparate clinical trials into a single, more powerful conclusion), and systematic reviews of the literature moved away from analyses designed to be helpful to practitioners, and began to progressively emphasize the statistics and mechanics of EBM —the playground of PhDs rather than MDs. Physician judgment was systematically avoided as being unreliable. (The PhDs do have their point, but medical practice would be paralyzed if we could only make decisions based on presumed-reliable data).

Cannabis has been spared the cold analytic glare of EBM until recently. There were just not enough randomized clinical trials (RCTs) to analyze. That changed, for better or worse, in June 2015, when “Cannabinoids for Medical Use: A Systematic Review and Meta-analysis” by Whiting et al was published in the Journal of the American Medical Association.2

…

The core methodology of EBM is that only RCTs are taken seriously. Unless your study is an RCT, the best EBM will give it is a low or very low quality rating —if that. EBM excludes non-clinical studies (i.e., animal or lab studies), which is appropriate, because drugs should not be used based on non-human studies. Thus 99.7% of the scientific articles that the JAMA authors’ literature search uncovered were discarded! The entire analysis was based on only 79 out of the 23,754 articles they identified.

In the JAMA paper by Whiting et al, nearly all the RCTs that made it through the screening criteria were pharmaceutical company trials or used synthetic CBD and THC —not herbal Cannabis.

…

EBM is a very powerful, necessary tool for making sense out of an overwhelming amount of often conflicting data, but it is only as good as the studies it analyzes.

For example, just because the data from a well-designed study shows statistically significant differences does not mean that it is medically correct. There may be hidden flaws in the study design. For example, if you discover that people who take vitamins live longer, you may have missed the point that people who buy vitamins are in a higher socioeconomic group and are less likely to smoke tobacco, and that is why they live longer.

…

In summary, EBM’s standards are goals for clinical researchers who are objectively examining medical benefits of cannabis to strive for. Under-funded, ill-designed studies carried out under siege by government agencies can hardly meet those standards. Until the obstacles to good research are removed, the scientific community must ask whether the EBM-style criteria that work for pharmaceutical research are the best way to evaluate plant-based therapies. Doctors and researchers must learn how to use plant cannabis with proper dose titration and reliable delivery systems before consistent results can be expected.

The needed clinical trials will emerge slowly and most research results will continue to be frustratingly inconsistent. That does not excuse us from doing the best with what we have, for the benefit of our patients who need our help now.