DSM-5 Field Trials: What Was Learned

With DSM-5 now approved by the APA Board of Trustees—and, to the dismay of this reader, all discussion removed from the DSM-5 Web site—how are we to evaluate the results of the field trials for the end product? I suggest beginning with the short piece published in Psychiatric News, “What We Learned from DSM-5 Field Trials.”¹ Authors David Kupfer and Helena Kraemer wrote, “We ultimately tested the criteria for 23 disorders. The question we asked was a straightforward one: In the hands of regular clinicians, assessing typically symptomatic patients in no different way than they would during everyday practice, was a particular disorder reliable?”^1(p.10)

In responding to this statement, let me begin with a confession of ignorance. Since I was not a participant in the field trials—neither for DSM-5 nor for previous versions of the manual—I didn’t appreciate that the major purpose of the field trials was to test reliability of the diagnostic constructs. I labored under a naïve illusion that since the ongoing crisis in the DSM has to do with diagnostic validity, field testing would somehow involve working on validity of new diagnoses. The “Guidelines for Making Changes to DSM-V” did hold out a little hope for work on validity with the statement: “Although some preliminary ‘convergent validity’ may be obtained in the field test (clinicians find the criteria fit syndromes expressed by real patients), a full test of the validity of these criteria will occur after DSM-V is published.”^2(p.10)

A little history
Let me give a quick reminder of the relevant history. The goal of DSM-III was to establish diagnostic reliability through operational definitions and diagnostic criteria. It was assumed that psychiatric science would fill in and establish validity for the DSM-III diagnostic constructs. Thirty years later, as we were preparing for DSM-5, everyone acknowledged the great shock of the previous 3 decades: diagnostic validity for the DSM-III/IV diagnostic categories was in shambles. As the DSM-5 leaders wrote in the 2002 white paper, “A Research Agenda for DSM-V”³:

In the more than 30 years since the introduction of the Feighner criteria by Robins and Guze, which eventually led to DSM-III, the goal of validating these syndromes and discovering common etiologies has remained elusive. Despite many proposed candidates, not one laboratory marker has been found to be specific in identifying any of the DSM-defined syndromes. Epidemiologic and clinical studies have shown extremely high rates of comorbidities among the disorders, undermining the hypothesis that the syndromes represent distinct etiologies. Furthermore, epidemiologic studies have shown a high degree of short-term diagnostic instability for many disorders. With regard to treatment, lack of treatment specificity is the rule rather than the exception.^3(p.xviii)

The authors’ response to this crisis was a vague appeal for a “paradigm shift”:

All these limitations in the current diagnostic paradigm suggest that research exclusively focused on refining the DSM-defined syndromes may never be successful in uncovering their underlying etiologies. For that to happen, an as yet unknown paradigm shift may need to occur. Therefore, another important goal of this volume is to transcend the limitations of the current DSM paradigm and to encourage a research agenda that goes beyond our current ways of thinking to attempt to integrate information from a wide variety of sources and technologies.^3(p.19)

In the ensuing years, advances in neuroscience and genetics were still not validating the DSM-IV categories. The perfectly reasonable directives for the Work Groups outlined in the Guidelines mentioned above could not accomplish much without clear support from basic science. In the face of this limitation, the DSM-5 leadership moved to dimensional measures as one solution to the validity problem. In 2009 Regier and colleagues⁴ wrote:

The single most important precondition for moving forward to improve the clinical and scientific utility of DSM-V will be the incorporation of simple dimensional measures for assessing syndromes within broad diagnostic categories and supraordinate dimensions that cross current diagnostic boundaries. Thus, we have decided that one, if not the major, difference between DSM-IV and DSM-V will be the more prominent use of dimensional measures in DSM-V.^(4,p.649)

As the dimensional measures were being challenged (they were dismissed by the Board of Trustees in May of 2012), Kupfer and Regier⁵ moved to the NIMH Research Domain Criteria project as the ultimate salvation of DSM-5—almost implying that the RDoC project, developed independently of DSM-5 and the DSM categories because the categories don’t work for research, was somehow developed in conjunction with the DSM work:

A logical extension of those discussions, as detailed in our Research Agenda articles, is the Research Domain Criteria (RDoC) initiative recently launched by the National Institute of Mental Health (NIMH) . . . This NIMH objective is consistent with our research planning conferences and conclusions, which underscored our commitment to examining evidence from neurobiology and assessing the readiness of proposed revisions for DSM-5. We are pleased with the work on RDoC that is being undertaken, and we believe this initiative will be very informative for subsequent versions: DSM-5.1, DSM-5.2, and beyond…It is important to emphasize that DSM-5 does not represent a radical departure from the past, nor does it represent a radical separation from the goals of the RDoC.^(5,p.2)

Back to 1980?
So here we are, waiting for the appearance of DSM-5 in May, and being reassured that those new diagnoses that have survived the chopping block of the Board of Trustees and the Scientific Review Committee have been field tested and proven to be reliable. Isn’t this where we were in 1980 with the appearance of DSM-III—now another handful of new diagnoses with demonstrated reliability and minimal evidence of scientific validity? Since we still don’t have adequate validity for the existing ton of DSM-IV diagnostic categories, is there any reason to think that the new handful won’t suffer the fate of the DSM-III/IV categories—heterogeneity of presentation, fuzzy boundaries, excessive comorbidity, and failure of neuroscientific and genetic foundation. And we really can’t count on the RDoC to save the day for DSM-5.1, DSM-5.2, and beyond, given that the RDoC researchers are not working with DSM categories and have no expectation that the constructs for which they are seeking disruptions of neural circuitry will match up with existing diagnostic categories. If you check the working grid of RDoC, you won’t find a column for “DSM diagnosis.”^6-8

What did we learn from the DSM-5 field trials?
So what did we really learn from the DSM-5 Field Trials? We learned that with diagnostic criteria, it’s not that hard to assure diagnostic reliability. And that we now have such reliability for another small group of new diagnoses. But didn’t we already know from DSM-III that diagnostic reliability is fairly easy? How can one not respond to Kupfer and Kraemer: is that it?