Caution needed when drawing links between improving symptoms and unproven remedies: Study

"Post-hoc bias can play tricks on patients that can eventually lead to serious disappointments – and for health-care workers, it can ultimately lead to shortfalls in care"

People tend to continue with unproven treatments even if there’s no evidence to suggest an initial marginal improvement in symptoms is anything more than a potential coincidence, a new study has found.

"I've noticed many of my patients take unnecessary vitamins, pills or alternative remedies with little evidence to inform their choice, leading to a lot of distraction, wishful thinking and wasted money,” says senior study author Donald Redelmeier, a staff internist and senior scientist at Sunnybrook Health Sciences Centre, and professor in the department of medicine in the University of Toronto’s Temerty Faculty of Medicine.

“Perhaps even more concerning is a false belief that leads to a missed diagnosis that later becomes incurable.”

The study, published in the journal JAMA Network Open, explores “post-hoc bias” – a tendency in reasoning that causes many patients to continue taking dubious or unreliable treatments. The bias encourages a popular misconception: that because one action preceded another later event, the first must have caused the second since it occurred in sequence.

But medical science, the researchers note, stresses that the order of two events does not prove a cause-and-effect since coincidences are frequent. The implication for medical care is that a patient who improved after a treatment is not necessarily a patient who improved because of the treatment.

Instead, other potential explanations include withdrawal from an adverse activity, added rest or the body’s own healing powers.

To test bias across a variety of clinical cases, the researchers ran multiple experiments using hypothetical clinical scenarios administered by a randomized survey of pharmacists and members of the community.

The scenarios described a patient with fatigue or another vague symptom who feels a bit better after trying a vitamin, shampoo, sugar pill or other treatment.

“We found that most respondents suggested continuing the treatment indefinitely even though the change in symptoms might be pure random chance," says Redelmeier, who is also affiliated ICES and the Institute for Health Policy, Management and Evaluation in ��˾��ֱ��’s Dalla Lana School of Public Health.

"The post-hoc bias can play tricks on patients that can eventually lead to serious disappointments – and for health-care workers, it can ultimately lead to shortfalls in care."

While attributing an initial improvement in – or lack of – symptoms to a treatment is a quick and intuitive approach, the researchers say the study reinforces the need for both patients and clinicians to be cautious when drawing conclusions.

“An awareness of post-hoc bias will not make it disappear, however we suggest patients and clinicians need to think twice and stay mindful of alternative explanations.”

The study was supported by the Alfred P. Sloan Foundation, a Canada Research Chair in Medical Decision Sciences, Canadian Institutes of Health Research, the PSI Foundation of Ontario and the National Science Foundation.

This story was originally posted at Sunnybrook Research Institute