Diagnostic Validators and the Question of What Really Exists

CONCEPTS IN PSYCHIATRY

In the first 2 installments of this series, we argued that psychiatry has gradually lost sight of psychopathology and replaced it with an increasingly technical, but conceptually thin, diagnostic enterprise. We also suggested that the Diagnostic and Statistical Manual (DSM) approach, despite its practical utility, rests on a mistaken understanding of what psychiatric diagnosis is supposed to accomplish. In this piece, we turn to a related and foundational question: how do we determine whether a psychiatric diagnosis corresponds to something real?

Some people claim that validity does not matter. They either do not understand what validity is, or they accept the idea that falsehood is fine. Validity means truth or reality, and its opposite is falsehood. If you say validity does not matter, then you are saying falsehood is acceptable. Since we assume most clinicians and patients would like to practice truthfully, we conclude that validity matters. So how do we determine whether a psychiatric diagnosis is true, and not false—that it corresponds to something real and not fictional?

The answer traditionally given in psychiatry is diagnostic validity, not mere descriptive agreement. Yet over the past half-century, validity has quietly receded from view and has been displaced by a far narrower concern with symptoms and reliability. To understand how this happened, and why it matters, we must return briefly to history.

Modern Scientific Medicine

Modern scientific medicine began in the 19^th century with an important innovation: the use of pathology at autopsy to confirm or refute clinical diagnosis. This approach was the basis of the fame of William Osler, author of the main textbook of medicine from around 1900 to about 1940.¹ Osler is famous for his clinical acumen, and his textbook consists mostly of detailed diagnostic description, but he based it all on knowing the truth at pathology and autopsy. Clinical debates had existed for millennia, but modern medicine became scientific when the pathologist could prove the truth on a microscopy slide. This “clinicopathologic” approach is the core of modern medicine, and it has been successful in almost all specialties, except psychiatry. Neurologists and psychiatrists, like Theodor Meynert and Alois Alzheimer and Emil Kraepelin, tried to apply pathology to psychiatric clinical syndromes, but they found no abnormalities.² Thus, the clinical debates continued into the 20^th century in psychiatry, and they continue today.

DSM-III and the Origins of Diagnostic Validators

In the 1970s, a group at Washington University in St Louis, led by Eli Robins, MD, and Samuel Guze, MD, came up with a solution to the above dilemma. This solution remains the core of accepted research in scientific nosology in psychiatry today. Since pathology was not available, they replaced it with validation of clinical syndromes based on multiple independent lines of evidence that were unrelated to symptoms. In their landmark 1970 paper, “Establishment of Diagnostic Validity in Psychiatric Illness,” Robins and Guze argued that psychiatric diagnoses should be evaluated using multiple independent validators, not simply by agreement among clinicians.³ These validators included:

• characteristic clinical features,
• delimitation from other disorders (symptom specificity),
• follow-up studies establishing course and outcome,
• family studies demonstrating aggregation (genetic specificity), and
• laboratory or biological correlates, when available.

The importance of this framework cannot be overstated. It represented a decisive rejection of purely impressionistic diagnosis and a move toward a scientific, hypothesis-testing model. Diagnoses, on this view, were provisional constructs whose validity could be strengthened, or weakened, over time by empirical study. They also were not acceptable if purely proposed based on symptoms. Like the rest of medicine, psychiatry could become scientific only if it went beyond a pure symptom-based approach.

These ideas strongly influenced the early planning of DSM-III. However, during the DSM-III process itself, a fundamental disagreement emerged. The Washington University group emphasized that reliability without validity was meaningless.⁴ A diagnosis could be applied consistently and still fail to correspond to a real disease entity. Robert Spitzer and others, by contrast, argued that improving interrater reliability was the necessary first step. Without reliability, they reasoned, validity could never be tested.

This compromise, often misunderstood in retrospect, was never intended to elevate reliability as an end in itself. DSM-III diagnoses were meant to be reliable starting points, not final truths. They were to be subjected, over time, to exactly the sort of validation studies Robins and Guze had outlined.

What Happened to Validity?

More than 4 decades later, the results of this grand experiment are sobering.

Despite an enormous research literature, only a small number of psychiatric diagnoses robustly satisfy the classic validators.

Disorders such as schizophrenia, bipolar disorder, obsessive-compulsive disorder, and borderline personality disorder show reasonably consistent findings across course, outcome, genetics, and, at least in part, biological correlates. Many other diagnoses do not.

Instead of confronting this reality, psychiatry has largely responded by lowering the bar for what counts as validity, or by redefining the problem altogether. Increasing emphasis has been placed on symptom counts, cross-sectional comorbidity, and statistical covariation. The original validators, especially course of illness and genetics, have receded into the background.

This shift reflects a deeper conceptual change. Psychiatric diagnoses have come to be treated less as hypotheses about disease entities and more as administrative or descriptive labels. Once that happens, the question “Does this diagnosis refer to something real?” is quietly replaced by the question “Is this diagnosis useful?” Utility matters, of course. But utility and validity are not the same thing, and confusing them has serious consequences for both research and clinical practice.Ideas can seem useful but be completely wrong: for much of the 20^th century psychiatrists found it useful to view homosexuality as a mental illness. Was that acceptable? If validity (what is true) does not matter, then we should be fine with pathologizing homosexuality.

Dimensional Models and the Problem of Symptoms

The rise of dimensional models such as HiTOP illustrates this problem vividly. Dimensional approaches promise to solve the limitations of categorical diagnosis by focusing on continua of symptoms rather than discrete disorders. In practice, however, they often double down on the very reductionism they claim to transcend.

Symptoms are treated as the primary data of psychiatry, while course of illness, longitudinal stability, genetics, and treatment response receive far less attention. Disorders dissolve into statistical clusters of symptoms, with little concern for whether these clusters correspond to biological diseases or other psychological pathology.

From the standpoint of diagnostic validation, this represents a regression rather than an advance. Robins and Guze never claimed that symptoms alone could establish validity. On the contrary, symptoms were only 1 validator among several, and arguably not the most important one. Course of illness—whether a condition is episodic or chronic, progressive or remitting—often tells us more about what a disorder is than a checklist of symptoms at a single point in time. This latter point should not be underestimated. Kraepelin held that prognosis is diagnosis, and much of his work has held up over time.

If critics do not like Kraepelin, we can refer to dozens of other leaders in scientific medicine in other specialties, where course of illness is central to diagnosis. In infectious disease, for instance, it matters if the clinical features are acute or not, last a few days or weeks or months, resolve spontaneously or not, happen in certain climates or not. In cardiology it matters if the chest pain is acute or not, lasts days or weeks, is chronic or episodic, occurs with exertion or not. Course of illness is central to diagnosis in all of scientific medicine, but many psychiatrists simply ignore it.

The question of dimensions versus categories is also irrelevant to the question of validity. Science should provide the answer, not the opinions of nosologists. Scientific research on diagnostic validators may show that some conditions, like manic-depressive illness (including mood temperaments), are dimensional and not categorical (in the false sense of the bipolar/major depressive disorder dichotomy). Scientific research may find that other conditions, like schizophrenia, are categorical and not dimensional. We realize that some will quibble with that statement; for instance, our view is that schizoaffective illness is a false, invalidated concept. Another column can address this idea.

By privileging cross-sectional symptom data, dimensional models risk producing constructs that are psychometrically elegant but ontologically empty. They describe patterns in data, not diseases in nature. They continue debates about symptoms that never end. They represent unscientific premodern medicine.

What Diagnostic Validators Are For

The enduring value of diagnostic validators lies in what they attempt to measure: whether a diagnosis corresponds to a real biological disease or other real psychological pathology (such as posttraumatic stress disorder). Validators force psychiatry to ask hard questions. Does this condition run a characteristic course? Is it genetic? Is it distinguishable, over time, from neighboring syndromes? Does it have a unique biological basis?

These questions cannot be answered by reliability statistics alone, nor by ever more sophisticated factor analyses of symptom data. They require longitudinal observation, genetics research, and a willingness to accept that some diagnostic constructs may fail. Reliability was never meant to replace validity—it was meant to facilitate it. The tragedy of contemporary psychiatric diagnosis is not that DSM-III emphasized reliability, but that the field largely stopped at that step. It did so because it fell in love with itself, with DSM as an end in itself. Reliability was claimed to produce a dictionary, but then it was called a Bible. Psychiatry worshiped its dictionary and converted it into a Bible. Once it became a Bible, it was no longer a science. Instead of refuting hypotheses, it became sacrilegious to question diagnoses. Our criteria for major depressive disorder, generalized anxiety disorder, borderline personality disorder, narcissistic personality disorder, and many others have hardly changed since 1980. That’s not science. That’s a religion.

Concluding Thoughts

If psychiatry is to recover a meaningful concept of diagnosis, it must return to the logic of diagnostic validation. This does not require reverting to pre-DSM impressionism. It requires remembering what diagnosis is for: not merely to label, but to identify real diseases or valid conditions that unfold over time characteristically and reflect underlying pathophysiology, even when that pathophysiology is only partially understood.

In future installments of this series, we will explore how a renewed focus on diagnostic validity might reshape psychiatric research, clinical practice, and the ongoing debates about the future of psychiatric nosology.

Dr Ghaemi is a lecturer on psychiatry at Harvard Medical School, Cambridge Health Alliance, and is employed by Bristol Myers Squibb. The views expressed in this article are solely those of the authors and do not necessarily reflect the official policy or position of their employers.

Dr Ruffalo is an assistant professor of psychiatry at the University of Central Florida College of Medicine in Orlando and an adjunct assistant professor of psychiatry at Tufts University School of Medicine in Boston, Massachusetts.

References

1. Ghaemi SN. In the tradition of William Osler: a new biohumanistic model of psychiatry. Perspectives in Biology and Medicine. 2023;66(4):520–534.

2. Shorter E. A History of Psychiatry: From The Era of the Asylum To The Age of Prozac. John Wiley & Sons; 1997.

3. Robins E, Guze SB. Establishment of diagnostic validity in psychiatric illness: its application to schizophrenia. American Journal of Psychiatry.1970;126(7):983–987.

4. Decker HS. The Making of DSM-III: A Diagnostic Manual's Conquest of American Psychiatry. Oxford University Press; 2013.