Psychologist Drew Westen, Ph.D., director of the adolescent and adult personality program of the Center for Anxiety and Related Disorders at Boston University, and his colleague, Kate Morrison, Ph.D., have written an ambitious, multidimensional meta-analysis of 34 studies on empirically validated psychotherapies published between 1990 and 1998 in top peer-reviewed journals. The study has raised controversy in its reassessment of previously published data and with its suggestion that these data are not always strong enough to warrant the status of these therapies as the treatment of choice for their respective diagnoses.
Westen and Morrison (2001) looked at 17 trials of panic disorder, five trials of generalized anxiety disorder (GAD) and 12 trials of depression, and included a total of 2,414 subjects. Another 23 trials were considered but excluded because they "did not meet minimal criteria for randomized controlled trials."
In general, Westen and Morrison argued that researchers need to report a broader range of outcome indices to provide a "comprehensive, multi-dimensional portrait of treatment effects." They found a serious dearth of long-term follow-up that could attest to the lasting benefits of treatment; their search identified only nine experimental studies with follow-up at 12 to 18 months and only four studies with follow-up at 24 or more months for all three disorders combined. Exclusion rates of between 64% and 68% of subjects screened, lack of standardization of exclusion criteria and failure to report exclusion criteria call into question the studies' generalizability to real-world patients.
The meta-analysis revealed the most impressive results for the treatment of panic disorder, where a substantial proportion of patients who completed treatment (63%) were shown to improve and remain improved. For depression, 54% were deemed improved, but the existing data did not support long-term effects. For GAD, the improvement rate was less -- 52% -- and there were virtually no published long-term follow-up data.
In one of four critical commentaries published alongside the report, Peter E. Nathan, Ph.D., (2001) called the meta-analysis a "tour de force that deserves serious consideration." In an interview with Psychiatric Times, David H. Barlow, Ph.D., author of several of the anxiety and panic studies Westen and Morrison examined, summed up the findings this way: "What [Westen and Morrison] basically say is that we need to do three things: 1) look at long-term outcomes, at least two years down the road; 2) look at a range of indicators of measures of outcome, not just one narrow one; and 3) broaden our inclusion criteria." But Barlow also thinks it is important to note that the suggested criteria "go way beyond the [U.S. Food and Drug Administration's] criteria for proof in drug studies. If these same criteria were applied to drug [trials], there would be no drugs considered effective."
In another commentary, Deane E. Aikins, Ph.D., and colleagues (2001) advised caution in interpreting the results of any meta-analysis, a procedure that usually collapses results from a number of trials to re-calculate efficacy (usually in the form of effect size) based on the total. By pooling a large number of subjects, a meta-analysis gains power and may provide the bigger picture of a treatment's overall efficacy. But a meta-analysis may also lump together treatments whose "mechanisms of change" are not comparable.
In the case of Westen and Morrison's GAD data, Aikins et al. found that the "analysis of effect size was based on four CBT [cognitive-behavioral therapy] conditions, two relaxation conditions, a combination of cognitive therapy and relaxation, behavior therapy, brief supportive expressive psychotherapy, analytic psychotherapy and anxiety-management training." While these all may qualify as empirically validated therapies, Aikins et al. do not consider them comparable. They performed their own mini meta-analysis of the four cognitive treatments and found a more impressive pre- and post-effect size of 1.18 relative to the two wait-list conditions (0.15) (Aikins et al., 2001).
Aikins DE, Hazlett-Stevens H, Craske MG (2001), Issues of measurement and mechanism in meta-analyses: comment on Westen and Morrison (2001). J Consult Clin Psychol 69(6):904-907 [comment].
DeRubeis RJ, Stirman SW (2001), Determining the pertinence of psychotherapy outcome research findings for clinical practice: comment on Westen and Morrison (2001). J Consult Clin Psychol 69(6):908-909 [comment].
Nathan PE (2001), Deny nothing, doubt everything: a comment on Westin and Morrison (2001). J Consult Clin Psychol 69(6):900-903 [comment].
Westen D, Morrison K (2001), A multidimensional meta-analysis of treatments for depression, panic, and generalized anxiety disorder: an empirical examination of the status of empirically supported therapies. J Consult Clin Psychol 69(6):875-899 [see comments pp900-913].