The Importance of Measuring Outcomes in Clinical Practice

BRIEF COMMUNICATION

Imagine going to your primary care physician with a fever and symptoms of an upper respiratory tract infection. The doctor puts his or her palm to your forehead and agrees that you feel warm. A course of treatment is recommended, you return in a couple of days, and he or she again feels your forehead and notes that you are cooler. Would you be happy with this approach to care? Would you continue to see a doctor who evaluated your body temperature in this way? We would not accept this level of care from an internist, a family practitioner, or a pediatrician-and yet this is the community standard of care provided by most behavioral health clinicians when treating psychiatric disorders.

To determine the impact of treatment, it is necessary to evaluate outcome. In mental health clinical settings, this typically is based on unstructured interactions that yield unquantified judgments of progress. This is at variance with other areas of medical care in which outcome is determined, in part, on the change of a numerical value. Body temperature, blood pressure, cholesterol values, blood sugar levels, cardiac ejection fraction, and white blood cell counts are examples of quantifiable variables that are used to evaluate treatment progress. In the mental health field, standardized, quantifiable outcome measures exist for most major psychiatric disorders, yet they are rarely used in routine clinical practice. To determine the impact of treatment it is not simply a matter of evaluating outcome, but rather a matter of measuring outcome.

The results of two surveys of psychiatrists, however, suggest that standardized scales are not being used to evaluate outcome in clinical practice.

Gilbody and colleagues¹ surveyed 340 psychiatrists in the United Kingdom regarding their use of outcome measures. Only 11.2% of the psychiatrists routinely used standardized measures to assess outcome when treating depression and anxiety disorders. More than half of the clinicians indicated that they never used standardized measures to evaluate outcome. The authors did not ask the respondents why they were disinclined to use scales to measure outcome; however, they noted that several respondents included comments on the questionnaires indicating that they thought such scales were simplistic, not useful in clinical practice, of questionable reliability and validity, or overly burdensome and costly to implement routinely.

Zimmerman and McGlinchey² conducted a similar survey of 314 psychiatrists in the US. They, too, found that the vast majority of psychiatrists did not routinely use clinical scales to monitor outcome of treating depression. More than half of the psychiatrists indicated that they never or rarely used scales to monitor outcome, and fewer than 10% almost always used scales to monitor outcome of depression treatment. They compared the characteristics of psychiatrists who reported using scales frequently or almost always with the rest of the group and found no differences between the two groups in sex, age, years of practice, or practice setting.

Psychiatrists who reported never, rarely, or only sometimes using scales to monitor outcome were asked the reasons for not routinely using scales in their clinical practice. More than one-quarter of them indicated that they did not believe using scales would be clinically helpful, that they take too much time to use, or that they were not trained in their use (Table).

The results of these two surveys found that psychiatrists typically do not use standardized scales of established reliability and validity when treating depression. One issue identified as an obstacle in their use is the perceived burden of scale completion. If the payers of the delivery of mental health treatment increasingly encourage, or require, the measurement of outcome, then the user-friendliness of measurement tools, as well as their reliability and validity, will be critical to their widespread adoption. Clinicians are already overburdened with paperwork, and adding to this load by requiring repeated detailed evaluations with such instruments as the Hamilton Rating Scale for Depression³ is unlikely to meet with success.

Self-report questionnaires are a cost-effective option because they are inexpensive in terms of professional time needed for administration, and they correlate highly with clinician ratings. To be sure, there are also limitations with self-report questionnaires such as response set biases, and their use may be limited by the readability of the scale and literacy of the respondent. However, self-report scales are free of clinician bias and are therefore free of clinician overestimation of patient improvement (which might occur when there are incentives to document treatment success).

This article was originally posted on 9/4/2014 and has since been updated.

Disclosures:

Dr Zimmerman is Director of Outpatient Psychiatry and the Partial Hospital Program, Rhode Island Hospital, and Professor of Psychiatry and Human Behavior at Brown University in Providence, Rhode Island. The author reports no conflicts of interest concerning the subject matter of this article.

References:

1. Gilbody SM, House AO, Sheldon TA. Psychiatrists in the UK do not use outcomes measures. National survey. Br J Psychiatry. 2002;180:101-103.
2. Zimmerman M, McGlinchey JB. Why don’t psychiatrists use scales to measure outcome when treating depressed patients?J Clin Psychiatry. 2008;69:1916-1919.
3. Hamilton M. A rating scale for depression. J Neurol Neurosurg Psychiatry. 1960;23:56-62.