Pioneering treatment or quackery? How to decide

mother was only slightly older than I am now when she died of emphysema
(chronic obstructive pulmonary disease). It’s a progressive condition for which
there is no cure, though it can be managed by use of inhalers and oxygen. I am
still angry at the discomfort she endured in her last years, as she turned from
one alternative practitioner to another. It started with a zealous nutritionist
who was a pupil of hers. He had a complicated list of foods she should avoid: I
don’t remember much about the details, except that when she was in hospital I protested
at the awful meal she’d been given - unadorned pasta and peas - only to be told
that this was at her request. Meat, sauces, fats, cheese were all off the menu.
My mother was a great cook who enjoyed good food, but she was seriously
underweight and the unappetising meals were not helping. In that last year she
also tried acupuncture, which she did not enjoy: she told me how it involved
lying freezing on a couch having needles prodded into her stick-like body.
Homeopathy was another source of hope, and the various remedies stacked up in
the kitchen. Strangely enough, spiritual healing was resisted, even though my Uncle
Syd was a practitioner. That seemed too implausible for my atheistic mother,
whose view was: “If there is a God, why did he make us intelligent enough to
question his existence?”

time to time, friends and relatives of mine have asked my advice about other
treatments that are out there. There is, for instance, the Stem Cell Institute in Panama,
offering treatment for multiple sclerosis, spinal cord injury, osteoarthritis,
rheumatoid arthritis, other autoimmune diseases, autism, and cerebral palsy.  Or nutritional therapist Lucille Leader,  who has a special interest in supporting patients
with Parkinson's Disease, Multiple Sclerosis and Inflammatory Bowel Disease. My
mother would surely have been interest in AirEnergy,
a “compact machine that creates 'energised air' that feeds every cell in your
body with oxygen that it can absorb and use more efficiently”.

source of queries are parents of the children with neurodevelopmental disorders
who are the focus of my research. If you Google for treatments for dyslexia you
are confronted by a plethora of options. There is the Dyslexia
Treatment Centre
, which offers Neurolinguistic Programming and hypnotherapy
to help children with dyslexia, dyspraxia or ADHD. Meanwhile the Dore Programme markets a set of “daily
physical exercises that aim to improve balance, co-ordination, concentration
and social skills” to help those with dyslexia, dyspraxia, ADHD or Asperger’s
syndrome. The Dawson
offers vibrational kinesiology to correct imbalances in the body’s
energy fields.  I could go on, and on,
and on….

how on earth can we decide which treatments to trust and which are useless or
even fraudulent? There are published lists of warning signs (e.g. ehow Health, Quackwatch),
but I wonder how useful they are to the average consumer. For instance, the
cartoon by scienceblogs
will make skeptics laugh, but I doubt it will be much help for anyone with no
science background who is looking for advice. So here’s my twopennyworth.
First, a list of things you need to ignore
when evaluating a treatment.

The sincerity of the practitioner. It’s a mistake to assume all purveyors of
ineffective treatments are evil bastards out to make money of the desperate.
Many, probably most,  believe honestly in
what they are doing. The nutritionist who advised my mother was a charming man
who did not charge her a penny - but still did her harm by ensuring her last
months were spent on an inadequate and boring diet. The problem is if
practitioners don’t adopt scientific methods of evalulating treatments they
will convince themselves they are doing good, because some people get better
anyway, and they’ll attribute the improvement to their method.

The professionalism of the website. Some dodgy treatments have very slick
marketing. The Dore Treatment, which
I regard as of dubious efficacy
, had huge success when it first appeared.
Its founder, Wyford Dore was a businessman who had no background in
neurodevelopmental disorders but knew a great deal about marketing. He ensured
that if you typed ‘dyslexia treatment’ into Google his impressive website was
the first thing you’d hit.

Fancy-looking credentials. These can be misleading if you aren’t an expert -
and sometimes even if you are. My bugbear is ‘Fellow the Royal Society of
Medicine’, which sounds very impressive - similar to Fellow the Royal Society
(which really is impressive).  In fact, the
threshold for fellowship is pretty low
, so much so that fellows are told by
the RSM that they should not use FRSM
on a curriculum vitae. So when you see this on someone’s list of credentials,
it means the opposite of what you think: they are likely to be a charlatan.
It’s also worth realising that it’s pretty easy to set up your own organisation
and offer your own qualifications. I could set up the Society of Skeptical
Quackbusters and offer Fellowship to anyone I choose. The letters FSSQ might
look good, but carry no guarantee of anything.

Testimonials. There is evidence (reviewed
) that humans trust testimonials far more than facts and figures. It’s
a tendency that’s hard to overcome, despite scientific training. I still find
myself getting swayed if I hear someone tell me of their positive experience
with some new nutritional supplement, and thinking, maybe there’s something in
it. Advertisers know this: it’s one thing to say that 9 out of 10 cats prefer
KittyMunch, but to make it really effective you need a cute cat going ecstatic
over the food bowl. If you are deciding whether to go for a treatment you must
force yourself to ignore testimonials. For a start, you don’t even know if they
are genuine: anyone who regards sick and desperate people as a business
opportunity is quite capable of employing actors to pose as satisfied customers.
Second, you are given no information about how typical they are. You might be
less impressed by the person telling you their dyslexia was cured if you knew
that there were a hundred others who paid for the treatment and got no benefit.
And the cancer patients who die after a miracle cure are the ones
you won’t hear about

Research articles. Practitioners of alternative treatments are finding that the
public is getting better educated, and they may be asked about research
evidence. So it’s becoming more common to find a link to ‘research’ on websites
advertising treatments. The problem is that all too often this is not what it
seems. This was recently illustrated by an analysis of research
publications from the Burzynski clinic
, which offers the opportunity to
participate in expensive trials of cancer treatment. I was interested also to
see the research
listed on the website of FastForword
, a company that markets a computerized
intervention for children’s language and literacy problems. Under a long list
of Foundational Research articles, they list one of my papers that fails to support their theory that
phonological and auditory difficulties have common origins. More generally, the
reference list contains articles that are relevant
to the theory behind the intervention, but don’t necessarily support it. Few
people other than me would know that. And a recent meta-analysis of
randomized controlled trials
of FastForword is a notable omission from the
list of references provided. Overall, this website seems to exemplify a
strategy that has previously been adopted in other areas such as climate
, impact of
or sex
, where you create an impression of a huge mass of scientific
evidence, which can only bbe counteracted if painstakingly unpicked by an expert
who knows the literature well enough to evaluate what’s been missed out, as
well as what’s in there. It’s similar to what Ben Goldacre has termed ‘referenciness’,
or the ‘Gish gallop’ technique
of creationists. It’s most dangerous when employed by those who know enough
about science to make it look believable. The theory behind FastForword is not
unreasonable, but the evidence for it is far less compelling than the website
would suggest.

those are the things that can lull you into a false sense of acceptance. What
about the red flags, warning signs that suggest you are dealing with a dodgy
enterprise? None of these on its own is foolproof, but where several are
present together, beware.

  1. Is there any theory
    behind the intervention, and if so is it deemed plausible by mainstream
    scientists? Don’t be impressed by sciency-sounding theories - these are
    often designed to mislead. Neuroscience terms are often incorporated to
    give superficial plausibility: I parodied this in my latest
    , with the invention of Neuropositive Nutrition, which is based
    on links between nutrients, the thalamus and the immune system. I suspect
    if I set up a website promoting it, I’d soon have customers. Unfortunately,
    it can be hard to sort the wheat from the chaff, but NHSChoices is good for
    objective, evidence-based 
    information. Most universities have a communications office that may
    be able to point you to someone who could indicate whether an intervention
    has any scientific credibility.  

  2. How specific is the
    treatment? A common feature of dodgy treatments is that they claim to work
    for a wide variety of conditions. Most effective treatments are rather
    specific in their mode of action.

  3. Does the
    practitioner reject conventional treatments? That’s usually a bad sign,
    especially if there are effective mainstream approaches.

  4. Does the
    practitioner embrace more than one kind of alternative treatment? I was
    intriguted when doing my brief research on Fellows of the Royal Society of
    Medicine to see how alternative interventions tend to cluster together.
    The same person who is offering chiropractic is often also recommended
    hypnotherapy, nutritional supplements and homeopathy.  Since modern medical advances have all
    depended on adopting a scientific stance, anyone who adopts a range of
    methods that don’t have scientific support is likely to be a bad bet.

  5. Are those
    developing the intervention cautious, and interested in doing proper
    trials?  Do they know what a
    randomised controlled trial is? If they aren’t doing them, why not? See this
    for an accessible explanation of why this is important.

  6. Does it look as
    though those promoting the intervention are deliberately exploiting people’s
    gullibility by relying heavily on testimonials? Use of celebrities to
    promote a product is a technique used by the advertising industry to
    manipulate people’s judgement. It’s a red flag.

  7. Are costs
    reasonable?  Does the website give
    you any idea of how much they are, or do you have to phone up for
    information? (bad sign!). Are people tied in to long-term treatment/payment
    plans? Are you being asked to pay to take part in a clinical trial? (Very
    unusual and ethically dubious). Do you get a refund if it doesn’t work? If
    yes, read the terms and condition very
    carefully so you understand exactly the circumstances under which you get
    your money back. For instance, I’ve seen a document from the Dore
    organisation that promised a money-back guarantee on condition there was
    ‘no physiological change’. That was interpreted as change on tests of
    balance and eye movements. These change with age and practice, and don’t
    necessarily mean a treatment has worked. Failing to improve in reading did
    not qualify you for the refund.

  8. Can the
    practitioner answer the question of why mainstream medicine/education has
    not adopted their methods? If the answer refers to others having competing
    interests, be very, very suspicious. Remember, mainstream practitioners
    want to make people better, and anyone who can offer effective treatments
    is going to be more successful than someone who can’t.