Be Wary of P Values: Ask About Effect Size

P values tell half of the story. A result that is statistically significant may actually be clinically irrelevant, says Leslie Citrome, MD, MPH, Clinical Professor of Psychiatry and Behavioral Sciences at New York Medical College, Valhalla, NY.

In order to understand a clinical trial result, we need to know the size of the treatment effect. If the treatment effect size is small, then it does not matter if the result was statistically significant because the effects size was clinically irrelevant.

We can measure effect size by the number of points improved on a rating scale, but that is difficult to interpret. Is a 4-point average improvement important? Hard to tell! Better off using a metric such as Number Needed to Treat (NNT) which can answer the question: "How many patients do you need to treat with one agent versus the other before expecting to encounter one additional responder?"

Dr Citrome was a speaker at the 2019 Psych Congress in San Diego, CA, in a presentation titled “When does a difference make a difference? Everything you need to know about effect sizes, but were afraid to ask.”

Disclosures:

The author reports that he is or has been a consultant of Acadia, Alkermes, Allergan, Avanir, BioXcel, Eisai, Impel, Indivior, Intra-Cellular Therapies, Janssen, Lundbeck, Luye, Merck, Neurocrine, Noven, Osmotica, Otsuka, Pfizer, Shire, Sunovion, Takeda, Teva, Vanda; a speaker at Acadia, Alkermes, Allergan, Janssen, Lundbeck, Merck, Neurocrine, Otsuka, Pfizer, Sage, Shire, Sunovion, Takeda, Teva; held stocks (small number of shares of common stock) with Bristol-Myers Squibb, Eli Lilly, J & J, Merck, Pfizer purchased > 10 years ago; and receives royalties: Wiley (Editor-in-Chief, International Journal of Clinical Practice), UpToDate (reviewer), Springer Healthcare (book).