Measuring Outcome in Psychiatric Private Practice Using Outpatient Self-Reports

Measuring Outcome in Psychiatric Private Practice Using Outpatient Self-Reports

Increased demand for accountability
is requiring more clinicians
to supplement their judgments
of patient outcome with standardized
and objective protocols. In 2004,
Massachusetts required its behavioral
health contractor to have all its providers
conduct outcome assessments.1 At the
federal level, the Health Care Finance
Authority mandated in 1998 that
contractors include outcome evaluations2
and now Medicare is moving
rapidly toward differential payments
based on performance and outcome.3

Although the pressure for standardized
assessment comes from external
forces, clinical and scientific benefits
will result. Repeated assessments will
enable dynamic adjustment of treatment,
using either formal algorithms
of evidence-based practice or individualized
clinical pathways; and such
assessments will help answer the key
question in all health interventions are
our patients getting better?

This question is not easy to answer
in real world (ie, nonacademic) psychiatry,
4,5 where patients differ greatly in
compliance, comorbidity, and other
parameters that are well controlled in
academic research. Moreover, dissemination
is imperfect. Even when
evidence-based educational interventions
(EBEI) are delivered, the impact
on clinician behavior varies greatly.6,7
For example, a recent study showed that
only half of the clinicians who received
EBEI training applied the knowledge,
while half had a “knowledge-behavior
gap.”8 Thus, truly unfiltered translation
of best practices from the laboratory to
the field is only rarely possible.4

Since treatment and patient factors
in real-life practice environments vary
from those in laboratory studies,9 knowing
whether real-world patients get
better requires assessment in the realworld
environment. Knowledge about
what works in the real world (effectiveness)
would complement the increasing
knowledge about what works
in the laboratory (efficacy).

Outcome evaluation
in the real world

Outcome evaluation has not been widespread
in psychiatric practices in the
United States, because the costs have
generally been viewed as outweighing
the benefits. Even when outcome evaluation has been instituted by public agencies, it has sometimes been curtailed
because of fiscal constraints.10 When
such evaluation becomes required,
providers have objected to the increased
demands, since they are unaccompanied
by sufficient fees.1 As psychiatrists face
increasing pressures ranging from reduced
reimbursement (with compensatory
increase in the number of patients
seen) to increased paperwork and regulations,
the time to measure outcome
becomes ever harder to carve out. Moreover,
the decision to evaluate outcome
subsumes more demanding detailed
questions,11,12 including:


  • What variables should be measured
    level of distress (level of anxiety
    or depression), functioning (work,
    school, or social situations), or quality
    of life?
  • Should the same data be collected
    from all patients or individualized
    (are the data from patients with
    agoraphobia the same as from patients
    with major depression)?
  • Data source—patient (self-report instrument), significant other, or
    clinician (structured interview, clinician
    rating scale).
  • When should the data be collected
    pretreatment, frequency thereafter,
    termination, follow-up.
  • Properties of instrument--reliability,
    validity, existence of appropriate
    norms, sensitivity, specificity, ability
    to measure change.


  • Costs of instrument, copyright,
    licensing fees.
  • Ease and time to complete--how
    well tolerated is the instrument and
    how will this contribute to compliance
    and accuracy?
  • Method of data collection, storage
    and analysis (all electronic vs all
    paper and pencil vs combination).

Finally, as Klein and Smith4 point
out, “if patient evaluation is not unobtrusively
incorporated into normal clinical
activities, it is impossible to know
if outcome findings are an artifact of
the supervening assessment process.”

It is no surprise, then, that psychiatrists
and the mental health profession
in general have not rushed to embrace
outcome evaluation. Presently, the fragmented
system of health care delivery
and reimbursement in America (fee–for-
service, HMOs, capitated models,
public sector) provides varying incentives
and disincentives to conduct outcome
evaluation. Capitated practices
that compete for contracts, based on
service, outcome, and price, provide a
strong incentive to assess outcome.

Alabama Psychiatric Services (APS),
a multioffice private practice providing
psychiatric care for more than 1 million
covered individuals with a capitated
model has been routinely doing outcome
evaluations for 5 years. To illustrate one
way to consistently integrate meaningful
outcome evaluation into clinical
care, we show how APS addressed the
methodologic and practical questions
outlined above and describe our protocol
for data collection.


Loading comments...
Please Wait 20 seconds or click here to close