Thought experiment: Suppose that scientists want to compare a new therapy for children with Autism. They’ll need to compare the New Therapy to a control condition and evaluate it over time using multiple different outcome measures. I’m going to describe this because I want to talk about the effects of “recovery” in Autism in the control group, the perception of the effectiveness of complimentary and alternative therapies, and the placebo effect.
Let’s say the researchers designed a longitudinal study in which very many young children meeting a certain cut score on a clinical scale for Autism were assigned to either an experimental or a placebo-control condition. The children in the experimental condition received a new therapy combining a special biological compound and high-fidelity, intensive, behaviorally-based, language and social intervention.
Because the experimenters could not ethically give the control children nothing, they arranged to make available an array of popular therapies (including massage, diet, therapeutic recreation, and so forth; in short, they were told about any complimentary therapy that wasn’t dangerous and they were free to get any other they found). The researchers also gave each family in the control condition vouchers roughly equivalent to the cost of the experimental therapy to pay for the cost of alternative therapies.
The researchers invited the families to visit the labs twice a year and administer a battery of assessments in a pleasant atmosphere. How are the children progressing? How close to developmental milestones are they? They also, of course, interview the parents, service providers, and others, keeping track of what therapies each child is receiving (the type, the “dosage”), and so forth. Thus, the researchers can track growth of the individuals and the average of the two groups over time.
Of course, the main question is whether the experimental treatment results in better outcomes than the placebo control. Of course, there are lots of “ifs” and “buts” that readers are probably harboring regarding that question. I’ll discuss some in later paragraphs.
First, however, Let’s focus on the outcomes in the control condition, because that’s why I’m writing this post. There’s going to be variation in the outcomes of the controls. Some are going to do better than others. Some few are actually even going to recover nearly all or a great deal of functions, even in the absence of intensive, systematic, therapy. For all the world, they are going to look like the horseback therapy worked!
Placebo effects occur when a presumably inert treatment appears to have a benefit or, according to Merriam-Webster “improvement in the condition of a patient that occurs in response to treatment but cannot be considered due to the specific treatment used.”
I read an article by Mandi Carozza in the Daily Iowan, the newspaper of the University of Iowa, entitled, “Alternative autism therapies prove successful in local programs,” that set me along this path.
After citing Iowa Department of Education data about the number of individuals with Autism in Iowa, Ms. Carozza reported about children’s participation in a local alternative therapy program.
Of these children, roughly 250 have taken part in Waterloo’s Aspire program since the its inception in 1998, program director Sara Card said.
While there is no evidence that treatment through therapeutic horseback riding can eliminate symptoms of the disorder, many have witnessed positive behavioral changes.
“We had one noncommunicative autistic boy who would only speak to us when prompted,” Card said. “We would have to rub a finger in the middle of his chest to get him to repeat what we were saying, but he said it all by himself after 12 weeks [of therapeutic horseback riding] … It’s pretty spectacular when you see things like that.”
As Ms. Carozza continued her exploration of the topic, she reported interviews with people who had used massage therapy. She told their stories of great outcomes, wondrous recoveries. To her credit, she noted that these therapies had not formally been proven effective.
But, if children who received these therapies spontaneously improved, might one not be tempted to attribute the improvement to the therapy? Post hoc, ergo propter hoc? That is, after this, therefore because of this, or because the outcome followed that action, mustn’t that action have caused the outcome?
Not necessarily! Some children with Autism have “recovered.” As Molly Helt and colleagues discussed in their excellent 2008 examination of this topic, somewhere between 3% and 25% of individuals who have a childhood diagnosis of Autism do not have the distinguishing characteristics of the disorder in adolescence or young adulthood. They may still have still have tics, depression, or phobias, and problems with higher-order communication and attention; but the major problems are gone and they no longer qualify for the diagnosis. Was it the massages?
Spontaneous recovery may make one think that the placebo was effective.
Now, as promised, some other comments: Some readers might have some questions the research design, especially about those vouchers. I’ve posed a couple here and provided answers. If you have others, please pose them in the comments, and we can discuss them.
Q: What if some of the parents in the control condition bought “ABA” for their children? Wouldn’t having something effective like that invalidate the study?
A: Not necessarily.Because they would be tracking the services used, the researchers would be able to identify those who did use some effective therapy. Presumably, only some of the control group would do so. If the number employing such services is small, then their success would not be enough to make the control group’s average rise much. Knowing who did and did not use a specific therapy, the researchers could analyze and report the data with and without those children’s scores included, too.
Q: Would recommending these kind of therapies to the parents in the control group be immoral?
A: I struggled with this as I thought about this imaginary experiment. I didn’t want to recommend any therapy that I thought might be considered near the border of abusive (e.g., Zazlow’s “Rage Reduction Therapy,” aka “Attachment Therapy” (which should not be confused with Attachment Theory), for example. Massages, exercising, riding horses, and such things are (in and of themselves) probably worthwhile things. So, no, I don’t think that recommending them is bad. Selling them as having curative powers is a different story, so the researchers would have to be careful in how they presented the alternatives to the families.
Q: If the New Therapy actually worked, how would the researchers know whether it was the biological component or the social & language intervention that made the difference?
A: Because the two are linked in the design I described, they wouldn’t. The two are a package. To assess a question about them separately, they would need another design; one way to do so would be a design with participants assigned randomly to each of four groups: Group A (just the bio component) vs. Group B (just the social and language component) vs. Group C (both components) vs. Group D (placebo control).
Q: Wouldn’t all the various different choices parents would make in the control group create chaos for the researchers?
A: In a way, yes. More technically, there would likely be a lot of variability in the outcomes for the control group. Some might select effective therapies and get good outcomes (yay!) but others would not (rats!). So, the range of scores would be wide and the standard deviation high or variance great. The bad part for the researchers is that increased variance makes statistics a little more difficult; but, as noted previously, they can conduct different analyses (with and without subgroups) to tell us what’s happened.
They could also test within the control group. Suppose 20% of the control group chose a certain pattern of therapies, another 30% chose a different pattern, and another 30% chose yet a third pattern. Even though they were not created randomly, the researchers could compare the outcomes for those three quasi-experimental groups.
Please pose other questions in the comments.
Elsewhere in her article, Ms. Carozza referred (somewhat obliquely) to an important research article in Journal of Child Psychology and Psychiatry to which I shall return in another post tomorrow. For your convenience, here’s a link to “Alternative autism therapies prove successful in local programs” from the Daily Iowan by Ms. Carozza.
Helt, M., Kelley, E., Kinsbourne, M., Pandey, J., Boorstein, H., Herbert, M., & Fein, D. (2008). Can children with autism recover? If so, how? Neuropsychology Review, 18, 339-366. doi:10.1007/s11065-008-9075-9