Driving around Hamilton the past few weeks I couldn’t help but notice the signs sprinkled around the city for the “Natural Health Expo” which is to take place here this week end. As I perused the website for this event yesterday I was disturbed by the large number of anti-scientific “treatments” that will be showcased. Like my co-blogger Grant who has already posted on this, I was troubled by the amount of misinformation that will be leveled directly at consumers.
As I was pondering how to answer the bewildering array of AltMed that will be promoted I checked my email and found a great little article just published in Chiropractic & Osteopathy (made available through the open access publisher BioMed Central).
The paper, “Why do ineffective treatments seem helpful? A brief review” written by Steve E Hartman, looks at how practitioners and patients can fool themselves into thinking that ineffective medical interventions actually work. An excellent example of Evidence Based Medicine 101, Steve covers the cognitive biases that hinder our ability to draw logical conclusions in the medical sphere such as the Post Hoc, Ergo Propter Hoc logical fallacy, confirmation bias and cognitive dissonance. Also covered are explanations of how it can seem that a treatment has been directly responsible for improvement in a patient’s condition when it may not have been.
The paper touches on disease natural history, which simply refers to how a particular malady might be expected to progress without treatment. Self limiting diseases such as colds, headaches and fatigue can be expected to get better on their own . If a patient is taking a treatment at the time, the treatment (rather than their own immune system) might erroneously be given the credit.
This combined with the overlapping arenas of the placebo effect and regression to the mean can be a powerful confounding factor when treatments are not being considered in light of scientifically controlled settings. The placebo effect is referred to frequently in common culture but regression to the mean is a less well known entity for the layman. Steven does a good job of explaining the concept, essentially people experience a variety of different intensities in their symptoms. Also they will tend to seek medical help when the symptoms become severe, knowing that the severity of the symptoms will tend to cluster about a mean value it is likely that whether treatment is sought or not the patient’s condition will tend to get better.
Thus the patient will feel relief and attribute that relief to what ever modality they are using at the time. Practitioners are not immune to these effects either and will in their practice see time and again that patients are getting better after their pet therapy is applied. In which case they will feel justified in proclaiming it works in the absence of confirming studies (or even in the face of disconfirming evidence).
The one aspect that I felt was missing from the paper, although it may have been obliquely implied, is the role of prior plausibility in evaluating treatments. Many modalities that will be on offer at the Natural Health expo are not only unusual they fly in the face of currently understood science. Scientific plausibility is our compass, without it we can become lost in the wilderness of fanciful ideas without any method of discerning the way forward. This concept is what separates Science Based Medicine from simply Evidence Based Medicine. The former takes the plausibility of a treatment into account when deciding the threshold of evidence needed before it can be considered effective. The later only measures outcomes and so is less able to distinguish true effects from chance outcomes.
Consider the following scenario: I claim to be able to influence the outcomes of coin tosses by virtue of what I had for breakfast on a particular day. If I have eggs then tails with predominate, lettuce produces more heads. Now without considering the plausibility of the setup we could run a trial, perform statistical analysis and find that my predictions are correct. But given that there is no good reason to suspect that my diet can influence a coin toss the positive is more likely to be because of chance than because of a real effect. In this case then a higher standard of evidence would need to be achieved than if I had said I could alter the probabilities be sticking a piece of gum to one side.
All-in-all though this a very nice paper and my complaint is a small one, given the probable readership of the journal the inclusion of plausibility may even have alienated those that might otherwise have been receptive to the other points presented. I recommend reading it for yourself, it is a very easy and informative read.
RSS - Posts
