For now we see through a glass, darkly . . .
–1 Corinthians 13: 12-13
When I was growing up in the late 1950s, “science” was all the rage among my young classmates. The manned space program was in its infancy, and most of my friends had fashioned “space helmets” out of 5-gallon ice-cream tubs and collected picture cards showing the seven Mercury astronauts. Fifty years later, I am amazed at how the cachet of “science” has dominated the recent debate over psychiatric diagnosis, the DSM-5, and the issue of psychiatry’s supposed “medicalization” of normality.1 Ironically, both friends and foes of psychiatry invoke the prestige of “science” in asserting their views. Critics of the DSM-5 insist that the new manual’s modifications are not “scientific”, while the DSM-5 itself informs us that it has incorporated “scientific findings from the latest research.”2 Meanwhile, the National Institute of Mental health promotes its “RDoC” (Research Domain Criteria) project—emphasizing neurocircuits--as the new “scientific agenda” for psychiatry.3
But for the philosophically inclined, these confident pronouncements on “science” present more questions than answers. For example: what, exactly, does the word “science” mean? What is denoted by the term “scientific”? Is “scientific” a binary term—for example, is a particular claim, or diagnostic category, either “scientific” or “not scientific”? Or can claims and categories be more or less “scientific”? Similarly, is there a continuum of scientific validity? Are biological “tests” and observable neuropathology necessary to validate a psychiatric diagnosis? Does diagnosis in general medicine and psychiatry typically involve the identification of a well-defined lesion or pathoanatomical “entity”? The remainder of this essay will offer some very provisional answers to these questions.
What is science and the scientific method?
The nature of “science” and the meaning of “scientific” are far from settled matters—even among scientists. Prof. Sean Carroll, a theoretical physicist at the California Institute of Technology, writes that,
“Defining the concept of “science” is a notoriously tricky business. In particular, there is long-running debate over the demarcation problem, which asks where we should draw the line between science and non-science.”4
Nevertheless, Carroll makes clear that science is not a particular set of facts or lab tests, but a three-part process: 1. Developing several hypotheses about some aspect of the world. 2. Carefully examining that aspect of the world and collecting relevant data; and 3. Choosing the hypothesis that best “fits” or explains the data, whenever possible. This, essentially, is what Western science calls, “the hypothetico-deductive method.” But Sean Carroll notes that “every one of these three steps is highly problematic in its own way”,4 particularly the third step. That’s because any set of data may yield several quite reasonable hypotheses, any one of which may “explain” the phenomenon in question--what philosophers refer to as the under-determination of the data.
To take a recent example from the DSM-5, consider “Jimmy,” a 7-year-old boy who is nearly always irritable or angry, most of the day, nearly every day. Let’s say Jimmy also shows severe, recurrent, and sometimes violent temper outbursts three times per week, which are felt to be inconsistent with his developmental level. Let’s say Jimmy has shown these features for the past 18 months, and has never met criteria for a manic or hypomanic episode. We might offer two quite different hypotheses to account for this child’s presentation.
The first says, “Jimmy is a normal, 7-year-old child who, like many children his age, often has bad moods and temper tantrums. He’s probably upset by problems within his family, feels ignored, and hasn’t learned appropriate ways of expressing anger.” The second hypothesis says, “Jimmy has a pathological condition called Disruptive Mood Dysregulation Disorder (DMDD), which can be distinguished from both “normal” moodiness and bipolar disorder. DMDD is often a precursor of a depressive or anxiety disorder, and is associated with specific attentional problems not seen in normal children or those with bipolar disorder.” The second hypothesis is roughly the basis for the controversial new DSM-5 category of DMDD.5
Now, we can certainly question whether the DSM-5 work group was justified in positing DMDD as a specific disorder, despite their review of the available epidemiological, clinical, and neuropsychological data. And, it might turn out that, indeed, Jimmy is just a normal but temperamental child. But, as Prof. Carroll construes the scientific method, it would be unfair to say that the DSM-5 work group has been “unscientific” in its deliberations on DMDD.
Indeed, the British Science Council (BSC) has defined science as, quite simply, .”..the pursuit of knowledge and understanding of the natural and social world following a systematic methodology based on evidence.” In my view, the most serious and debilitating psychiatric disorders encountered in clinical practice—schizophrenia, bipolar disorder, and melancholic major depressive disorder--are scientifically-based, using the BSC definition. While the specific DSM criteria have changed somewhat from DSM-III to DSM-5, our basic clinical descriptions of these conditions are grounded in many decades of careful observations, as well as thousands of systematic research studies.
This doesn’t mean that the DSM work groups are never swayed by political, economic or social pressures.6 Other medical specialties, too, are sometimes motivated by “extra-scientific” considerations and values. For example, the recent recommendation by the American Medical Association to declare obesity a “disease” appears to have been driven primarily by the wish to encourage better treatment of this condition—a laudable goal, surely, but not a purely “scientific” decision.7
Finally, it’s important to note that use of the scientific method doesn’t guarantee that our diagnostic categories are valid or clinically useful. After all, many careful scientific observations led to the hypothesis that a small planet—Vulcan, no less!--existed somewhere between Mercury and the Sun. Yet ultimately, this claim was invalidated. It is certainly possible that DMDD and other DSM-5 categories will suffer the same fate—but this is all part of the scientific process.**
What properties confer validity on a diagnosis?
Clinicians want their diagnostic categories to be both reliable and valid. Reliability refers to the degree of inter-rater agreement that can be achieved with a particular set of diagnostic criteria, represented by the designation “kappa.” So, let’s say two people viewing a picture of a horse-like creature with a horn in the middle of its forehead agree that the picture represents a unicorn, and agreement occurs 100% of the time with all other observer pairs. This picture elicits perfect inter-rater agreement, and would have a kappa of 1.0. But this tells us nothing about the existence of unicorns!
Validity is quite another matter. Very broadly, validity describes the “is-ness” of a diagnosis or set of criteria; ie, the degree to which a diagnostic category actually identifies something “real”—or at least, a clinically useful and meaningful diagnostic entity. How is validity established in clinical medicine and psychiatry?
Dr Bernard Carroll’s description of “convergent validity” helps answer this question. He explains that our disease constructs take shape through a process of “convergent validation.” This entails “. . . iterative attention to signs, symptoms, course of illness, response to treatments, family history, and laboratory data.”8
Like the term “scientific,” validity is not a binary term. Diagnostic categories may have varying degrees of validity. As I wrote several years ago, in order for a diagnostic category to gain a least a modicum of validity,
“ . . . the criteria for “Disease X” must be “sharp” enough to distinguish its sufferers from those with Disease Y or Z. Its “elements” must cohere, in the way we would expect the pieces of a jigsaw puzzle to fit together. For example, if Disease X is defined by the presence of auditory hallucinations, dry skin, elevated blood pressure, and tremor, one would expect high degrees of concordance and overlap among these features. One would also expect a good correlation between this symptom picture and the course, outcome, and response to treatment of Disease X...”9
How many DSM-5 diagnostic categories will meet these fairly stringent tests of “validity”? No one knows—maybe a handful, maybe a few dozen. I believe that, at a minimum, schizophrenia, bipolar disorder, panic disorder, obsessive-compulsive disorder, and the melancholic subtype of major depressive disorder will make the grade. (My colleague, Nassir Ghaemi, MD, estimates that, among DSM-III categories, “about two dozen” were based on “decent scientific evidence.”6 [see footnote].
But even by these harsh lights, there is no foundation for the claim that all psychiatric diagnoses, across the board, lack validity. Even DSM categories not yet fully validated are not necessarily in-valid. Indeed, the same kinds of rigorous studies that would validate a DSM category would be required to invalidate it. Furthermore, diagnostic validity is always provisional and probabilistic. Thus, our level of confidence in a set of disease criteria may increase or decrease, as new data or discoveries emerge. (I believe the “DSM-5.1”—ie, expected updates of the DSM-5—must provide ongoing validity data that support its newer categories, or the credibility of the manual will continue to be questioned).
Nevertheless, even if some of DSM-5’s categorical distinctions prove invalid, this in no way invalidates the construct of psychiatric disease, understood in neuropathological terms. For example: the most recent investigations of schizophrenia and psychotic bipolar disorder suggest that there is substantial neurobiological overlap of these conditions, with respect to white matter abnormalities.10 If these preliminary findings are replicated, we might reasonably infer that—in the case of these afflictions-- Nature is not “carved at its joints” so much as shaded at its borders. But such “fuzziness” at the schizophrenia-bipolar border does not render the aforementioned white matter abnormalities—much less, the patient’s suffering and incapacity!--any less “real.” As the philosopher Ludwig Wittgenstein observed, a fuzzy beam of light is just as real as a sharply focused one.11
Are biomarkers or lab tests necessary for a diagnostic category to be valid?
Once again, Dr Bernard Carroll hits the mark:
“We need to be clear that the existence of disease is not predicated on having a biological test. It’s nice when we do have one, but there are many areas in medicine where there is no conclusive diagnostic test. Think migraine. Think multiple sclerosis. Think chronic pain. Indeed, clinical science correctly recognized many diseases long before lab tests came along for confirmatory diagnostic application. Think Parkinson’s disease, Huntington’s disease, epilepsy . . . it’s a long list.” 8
Acknowledgments: My deep appreciation to Dr S. Nassir Ghaemi and Dr Bernard J. Carroll, for their helpful comments on this article. The views presented here, however, are my own.
** My colleague, Dr Nassir Ghaemi, has suggested that “in clinical diagnosis, the specific scientific criteria are the five [Robins and Guze] criteria of validity”; and that one cannot justifiably “separate science from validity when discussing nosology.” (personal communication, 8/29/13). (The Robins & Guze criteria (Am J Psychiatry. 1970 Jan;126(7):983-7) comprise clinical description, laboratory study, exclusion of other disorders, follow-up study, and family study—often considered the fundamental elements of “construct validity”). To be sure: “science” is intimately related to construct validity—but they are distinct concepts. “Construct validity” is essentially an outgrowth and subset of the long-established scientific method, which dates from Roger Bacon’s work in the 13th century. An empirical claim may be proved “invalid” yet be eminently “scientific.” For example, the “steady-state” theory of the universe was unquestionably grounded in the scientific method, but was eventually invalidated and replaced by the “Big Bang Theory.” By the same token, the DSM-5’s decision to eliminate the subtypes of schizophrenia (paranoid, disorganized, etc.) might someday be invalidated, based on new data—but it would be unfair, retrospectively, to call the DSM-5’s decision “unscientific.” (see Highlights of Changes from DSM-IV-TR to DSM-5, American Psychiatric Association, 2013).
^^These different aspects of the patient’s condition have been discussed in terms of “disease” (pathoanatomical entity) and “illness” (the patient’s subjective reaction) by Dr Arthur Kleinman.
1. Pies R: Psychiatry and the myth of medicalization. Psychiatric Times. April 18, 2013. http://www.psychiatrictimes.com/depression/psychiatry-and-myth-%E2%80%9C....
2. American Psychiatric Association. Diagnostic and Statistical Manual of Mental Disorders, Fifth Edition, 2013, p. xlii
3. Cuthbert B: Concept Clearance - First-Generation RDoC Standard Data Elements http://www.nimh.nih.gov/funding/grant-writing-and-application-process/concept-clearances/2013/concept-clearance-first-generation-rdoc-standard-data-elements.shtml?utm_source=rss_readers&
4. Carroll S: What is Science? July 3, 2013 http://www.preposterousuniverse.com/blog/2013/07/03/what-is-science/
5. Leibenluft E, Severe Mood Dysregulation, Irritability, and the Diagnostic
Boundaries of Bipolar Disorder in Youths. Am J Psychiatry. 2011;168: 129–142
6. Ghaemi SN: Requiem for DSM. Psychiatric Times, July 17, 2013. http://www.psychiatrictimes.com/dsm-5-0/requiem-dsm
7. Pies R: With Obesity, A New Disease is Born: Its Profound Implications for Psychiatry. http://psychcentral.com/blog/archives/2013/06/22/with-obesity-a-new-disease-is-born-its-profound-implications-for-psychiatry/
8. Carroll BJ. Comment on Medscape. http://www.medscape.com/viewarticle/804408?nlid=31347_421&src=wnl_edit_medp_psyc&spon=1
9. Pies R: What Should Count as a Mental Disorder in DSM-V? Psychiatric Times. April 14, 2009
10. Skudlarski P, Schretlen DJ, Thaker GK, Stevens MC, Keshavan MS, Sweeney JA, Tamminga CA, Clementz BA, O’Neil K, Pearlson GD. Diffusion tensor imaging white matter endophenotypes in patients with schizophrenia or psychotic bipolar disorder and their relatives. Am J Psychiatry. 2013 Aug 1;170(8):886-98.
11. Wittgenstein L: The Blue and Brown Books, New York, Harper Torchbooks; 1965
12. Kroenke K: Diagnostic Testing and the Illusory Reassurance of Normal Results. JAMA Intern Med 2013; 173:416-17.
13. Pies R: Mental illness is no metaphor: five uneasy pieces. Psychiatric Times, Sept. 13, 2012. http://www.psychiatrictimes.com/articles/mental-illness-no-metaphor-five-uneasy-pieces?iframe=true&width=90%25&height=90%25]
14. Pies R, Thommi S, Ghaemi SN: Getting It From Both Sides: Foundational and Antifoundational Critiques of Psychiatry Psychiatric Times, July 1, 2011 http://www.psychiatrictimes.com/display/article/10168/1895157
15. Cohen H: The nature, methods and purpose of diagnosis. The Lancet, 1943; 241: 23-25
16. Malterud K: Diagnosis—A tool for rational action? A critical view from family medicine. Atrium, Winter, 2013, pp. 26-35.
17. Ventegodt S, Kandel I, Merrick J ; A short history of clinical holistic medicine. The Scientific World Journal 2007; 7, 1622–1630
18. Pies RW, Geppert C: Ethical issues in the psychiatric treatment of the religious ‘fundamentalist’ patient. Medscape Psychiatry. March 19, 2013. http://www.medscape.com/viewarticle/780839
20. Ghaemi SN: Paradigms of Psychiatry: Eclecticism and Its Discontents. Curr Opin Psychiatry. 2006;19(6):619-624. http://www.medscape.com/viewarticle/547497_9.
21. Lishman WA: Organic Psychiatry: the Psychological Consequences of Cerebral Disorder. 3rd edition, Wiley-Blackwell, 1998.
22. Ghaemi SN: The Rise and Fall of the Biopsychosocial Model: Reconciling Art and Science in Psychiatry. Johns Hopkins University Press, 2012.
For further reading:
Pies R: DSM-5’s Validity: Non Sumus Angeli! Medscape Psychiatry. June 12, 2013 http://www.medscape.com/viewarticle/805365
Markova IS, Berrios GE: Epistemology of psychiatry. Psychopathology 2012; 45:220-227 [This paper discusses the concept that psychiatry is a “hybrid discipline” whose objects of inquiry are themselves “hybrid” constructs]