total number of people who were sober and still attending AA was only 31 percent of the whole group. Despite this figure, and despite the fact that the study involved only people who had self-selected into 12-step programs and that nearly 35 percent of subjects had dropped out entirely, biasing the results toward more positive outcomes, the authors state, “We conclude that the probability of a positive effect is sufficient to recommend participation in self-help groups as a supplement to drug addiction treatment.” This study was repeated in our popular press as proof positive that AA was a success.
In 2012, yet another longitudinal and observational study, conducted in Sacramento, California, by Jane Witbrodt and colleagues, found essentially the same results as the prior studies, namely, that the people who are still in AA at the end of many years tend to be admirably sober and well. 25 Once again, however, there were familiar issues: 25 percent of the study subjects had dropped out by year 9, and of those remaining, only 25 percent were “high” attenders of AA, which was the group with the best outcomes. (Even within this high-attending group, 22 percent were still drinking.) Like its cousins, this study relied on self-reporting and, like its cousins, acknowledged a major caveat: “We suspect that the higher abstinence in our ‘high’ class may be due in part to this being a more stably insured or employed population.” The authors also acknowledged that they “lacked baseline measures for prior 12-step involvement and treatment episodes. Undoubtedly, these prior exposures may have influenced subsequent attendance for some study participants.” It appears from this acknowledgment that the authors were aware that there was a pro-AA bias in the selection of their sample.
In fact, every one of the subjects in this paper had already been through the eight-week Chemical Dependency Recovery Program at the Kaiser Permanente facility in Sacramento, which is a 12-step-based program. In other words, all of the subjects of this study had already been exposed to the AA philosophy and actively encouraged to attend before they were followed up to determine if AA treatment would be helpful. It would be hard to imagine a clearer example of selection bias. If the authors had titled their article “Abstinence among People Intensively Exposed to AA Doctrine Who Then Chose to Continue with AA,” they would have been on more solid ground.
The authors did, however, acknowledge that the big question was whether AA was helping people or not. To this point, they added a familiar caveat:
Still, the direct relationship between 12-step attendance and abstinence remains uncertain in part because randomized clinical trials that direct and restrict attendance are difficult to conduct with such a freely available source of support as AA (and NA/CA). This relationship becomes even more blurred when attendance is studied over longer follow-up periods and as people transition in and out of both formal treatment programs and 12-step groups. In addition, only scant research has focused on outcomes other than actual alcohol and drug use (e.g., abstention status, percent days abstinent, drinks per drinking day).
Like the Moos and Fiorentine researchers before them, the Witbrodt researchers failed to address the simple fact that one cannot prove that any medical intervention works without a control group. (Interestingly the real “control” group for Alcoholics Anonymous—people who seek no treatment at all—have their own impressive rate of recovery, which I will discuss shortly.)
One paper has tried to tackle the question of whether we can determine causality in a direct way. A 2003 study, conducted by J. McKellar and colleagues of the Palo Alto Health Care System and published in the
Journal of Consulting and Clinical Psychology
, attempted to do a difficult thing—determine causality retroactively using statistical techniques and a
In 2012, yet another longitudinal and observational study, conducted in Sacramento, California, by Jane Witbrodt and colleagues, found essentially the same results as the prior studies, namely, that the people who are still in AA at the end of many years tend to be admirably sober and well. 25 Once again, however, there were familiar issues: 25 percent of the study subjects had dropped out by year 9, and of those remaining, only 25 percent were “high” attenders of AA, which was the group with the best outcomes. (Even within this high-attending group, 22 percent were still drinking.) Like its cousins, this study relied on self-reporting and, like its cousins, acknowledged a major caveat: “We suspect that the higher abstinence in our ‘high’ class may be due in part to this being a more stably insured or employed population.” The authors also acknowledged that they “lacked baseline measures for prior 12-step involvement and treatment episodes. Undoubtedly, these prior exposures may have influenced subsequent attendance for some study participants.” It appears from this acknowledgment that the authors were aware that there was a pro-AA bias in the selection of their sample.
In fact, every one of the subjects in this paper had already been through the eight-week Chemical Dependency Recovery Program at the Kaiser Permanente facility in Sacramento, which is a 12-step-based program. In other words, all of the subjects of this study had already been exposed to the AA philosophy and actively encouraged to attend before they were followed up to determine if AA treatment would be helpful. It would be hard to imagine a clearer example of selection bias. If the authors had titled their article “Abstinence among People Intensively Exposed to AA Doctrine Who Then Chose to Continue with AA,” they would have been on more solid ground.
The authors did, however, acknowledge that the big question was whether AA was helping people or not. To this point, they added a familiar caveat:
Still, the direct relationship between 12-step attendance and abstinence remains uncertain in part because randomized clinical trials that direct and restrict attendance are difficult to conduct with such a freely available source of support as AA (and NA/CA). This relationship becomes even more blurred when attendance is studied over longer follow-up periods and as people transition in and out of both formal treatment programs and 12-step groups. In addition, only scant research has focused on outcomes other than actual alcohol and drug use (e.g., abstention status, percent days abstinent, drinks per drinking day).
Like the Moos and Fiorentine researchers before them, the Witbrodt researchers failed to address the simple fact that one cannot prove that any medical intervention works without a control group. (Interestingly the real “control” group for Alcoholics Anonymous—people who seek no treatment at all—have their own impressive rate of recovery, which I will discuss shortly.)
One paper has tried to tackle the question of whether we can determine causality in a direct way. A 2003 study, conducted by J. McKellar and colleagues of the Palo Alto Health Care System and published in the
Journal of Consulting and Clinical Psychology
, attempted to do a difficult thing—determine causality retroactively using statistical techniques and a
Similar Books
