Be Wary of P Values: Ask About Effect Size

P values tell half of the story. A result that is statistically significant may actually be clinically irrelevant, says Leslie Citrome, MD, MPH, Clinical Professor of Psychiatry and Behavioral Sciences at New York Medical College, Valhalla, NY.

In order to understand a clinical trial result, we need to know the size of the treatment effect. If the treatment effect size is small, then it does not matter if the result was statistically significant because the effects size was clinically irrelevant.

We can measure effect size by the number of points improved on a rating scale, but that is difficult to interpret. Is a 4-point average improvement important? Hard to tell! Better off using a metric such as Number Needed to Treat (NNT) which can answer the question: "How many patients do you need to treat with one agent versus the other before expecting to encounter one additional responder?"

Dr Citrome was a speaker at the 2019 Psych Congress® in San Diego, CA, in a presentation titled “When does a difference make a difference? Everything you need to know about effect sizes, but were afraid to ask.”

Disclosures:

The author reports that he is or has been a consultant of Acadia, Alkermes, Allergan, Avanir, BioXcel, Eisai, Impel, Indivior, Intra-Cellular Therapies, Janssen, Lundbeck, Luye, Merck, Neurocrine, Noven, Osmotica, Otsuka, Pfizer, Shire, Sunovion, Takeda, Teva, Vanda; a speaker at Acadia, Alkermes, Allergan, Janssen, Lundbeck, Merck, Neurocrine, Otsuka, Pfizer, Sage, Shire, Sunovion, Takeda, Teva; held stocks (small number of shares of common stock) with Bristol-Myers Squibb, Eli Lilly, J & J, Merck, Pfizer purchased > 10 years ago; and receives royalties: Wiley (Editor-in-Chief, International Journal of Clinical Practice), UpToDate (reviewer), Springer Healthcare (book).

The articles and interviews featured on this page were developed by Psychiatric Times editorial staff and contributors based on presentations delivered at Psych Congress®, an annual conference produced by HMP Global, LLC. Psychiatric Times is an independent publication and is not affiliated with, endorsed by, or sponsored by HMP Global, LLC. All content on this page reflects the independent editorial judgment of Psychiatric Times and does not represent the views, positions, or communications of HMP Global, LLC.