Key questions to consider when evaluating a digital therapeutic for anxiety.
As the need grows for equitable, accessible, and affordable mental health treatment, digital therapeutics—app-based treatments—show promise for filling these gaps, particularly to target anxiety. The high prevalence of anxiety disorders “vastly exceeds the capacities of mental health services” to provide treatment for all those affected.1
Anxiety has risen since the beginning of the COVID-19 pandemic. The US Census Bureau reported that adults were 3 times more likely to screen positive for anxiety disorders in 2020 compared with 2019 (31% vs 8%). A cross-sectional survey of individuals conducted in China February 2020 found the prevalence of generalized anxiety disorder (GAD) to be 35.1%; a meta-analysis found an average anxiety prevalence of 32% during the pandemic (17 studies, n = 63,439).2,3 A study of patients diagnosed with COVID-19 between January and August 2020 (n = 62,000+) found that 1 in 5 patients with no psychiatric history received a new mental health diagnosis, with the greatest health risk being for anxiety disorders.4
Digital therapeutics can be designed to specifically target a behavioral or psychological mechanism of action, and they can be developed with user-centered design principles to maximize engagement. However, to date, very few digital therapeutics have been tested for efficacy, and even fewer have been studied regarding mechanism of action. There seem to be 2 categories of digital therapeutics emerging: those that are commercially developed with little supporting evidence or plans for evaluation and those with academic or government backing (and often not easily available to consumers).1 The former has the advantage of maximizing curb appeal: Resources can be devoted toward developing engaging user experiences using the latest design techniques, and companies can push a digital therapeutic to market rapidly. Scientists have developed evidence-based digital therapeutics, yet few are publicly available. They appear more slowly than their commercial counterparts, and those that do appear usually fail to attract users and user engagement.5 However, both face the same fate: No matter how shiny an app is, it still takes time to run rigorous clinical trials to determine efficacy (and mechanism).
John Torous, MD, Digital Section Editor for Psychiatric TimesTM and chair of the American Psychiatric Association’s Smartphone App Evaluation Task Force, put it bluntly when interviewed for an article on digital therapeutics in Nature: “Right now it almost feels like the Wild West of health care.”6
How can the busy health care clinician keep up-to-date on the latest trends, determine which digital therapeutics are evidence based, and give informed answers when patients ask which apps to use? Is there enough evidence for prescribers to pull out their proverbial digital prescription pad to recommend an app-based treatment for anxiety?
After considering the recent literature, there are a few questions clinicians should consider.
Why Should There Be an Effect? What Mechanism Is Being Targeted?
A good place to start is to consider if a digital therapeutic is mechanistically based. Although many apps assert that they are scientifically based, very few live up to this claim. The National Institutes of Health established a common fund program called the Science of Behavior Change (SOBC) to focus on mechanisms of change in behavioral interventions as a way to both strengthen intervention development and “increase rigor, transparency, and dissemination of common terminology, methods, and measures.”7 The basic idea behind SOBC is to measure if a behavioral intervention engages a hypothesized mechanism of change and if engaging that target leads to change in a health outcome. If this approach sounds similar to that taken for the development of any new drug, it should, because it is. It was modeled after the experimental medicine approach. However, a significant research-practice gap exists in the digital mental health field, especially in the field of anxiety interventions.8
Findings of a recent study looking at the frequency of evidence-based treatment elements in apps for anxiety showed that the most commonly included elements were psychoeducation (53%), relaxation (47%), meditation (41%), and mindfulness (29%) (Figure).8 Yet, the inclusion of an element does not automatically translate to targeting a specific mechanism or therapeutic benefit. Another recent review found that 67% of apps for anxiety lacked involvement of any health professional in their development, and just 4% had been empirically evaluated.9
This unfortunate lack of focus on mechanism is not due to an absence of information on how anxiety can develop and be perpetuated. For example, decades-old research has suggested potential mechanisms of action, such as targeting worry-driven negative reinforcement, yet only recently have these been incorporated into digital therapeutic devopment.10-13
Who Is the Studied Population?
With apps that can reach thousands, and even millions, of potential patients, it is much easier to collect data from a convenience sample, look for reductions in some type of scale, and then claim it to be an evidence-based treatment for anxiety. However, there is a world of difference between someone who is mildly anxious (after all, who is not these days?), and someone whose anxiety is severe enough to land them firmly in the realm of needing treatment for a DSM-diagnosed anxiety disorder. For example, a meta-analysis of randomized controlled trials of mindfulness apps reported significant effect sizes for reductions in anxiety, but every study recruited individuals from app users and the general population and specific nonclinical groups (largely students and employees).14 Of these, only a single study included individuals with mild to moderate anxiety using the General Anxiety Disorder-7 (GAD-7) cutoff of 14 to exclude individuals who reported severe anxiety (GAD-7: 5-9, mild; 10-14, moderate; 15+, severe anxiety).
To date, only 1 study of internet-based cognitive behavioral therapy (iCBT) used GAD-7 scores of ≥ 8 as a minimum cutoff to recruit individuals with mild to moderate symptoms of anxiety, individuals in the United Kingdom’s National Health System (NHS) and “who may be eligible and require iCBT” as part of the Improving Access to Psychological Therapies (IAPT) program.15 Additionally, 2 studies of app-based mindfulness training for anxiety (Unwinding Anxiety) have used GAD-7 scores of ≥ 10 as minimum cutoff for inclusion of anxious physicians and individuals with GAD, respectively.13,16 Taken together, these results suggest that the majority of studies are drawing convenience samples from nonclinical populations, and that can lead to problematic inferences on the efficacy and utility of digital therapeutics in individuals with moderate to severe anxiety—those that would be most likely to seek or be referred to treatment—unless they have been specifically studied in these populations.
What Is the Evidence? Is There a Clinically Relevant Reduction in Symptoms?
Popularity does not equal evidence for efficacy. A number of app-based programs have put a primary focus on increasing their user base rather than developing digital therapeutics for anxiety. Although there is nothing inherently wrong with this approach—it is just a different focus—seeing a lot of individuals use an app can lead to a false sense of efficacy. Most studies to date are small pilot studies, often run by the app developers rather than by independent researchers, and the results have yet to be replicated. Additionally, many studies are run relative to wait-list control groups. Although this is better than no control group, it does not assess basic confounding components that are covered by a treatment as usual control group, such as receiving some type of treatment and expectancy effects (which, if performed well, will include the usual clinical care delivered by their therapist or doctor).
As previously mentioned, only a few published studies have examined digital therapeutics in individuals with moderate to severe anxiety: 1 in the general population (mainly recruited from users who were already signed up for the app Pacifica) with baseline GAD-7 scores of between 5 and 14; 1 (iCBT) in the UK’s NHS in individuals with baseline GAD-7 scores of ≥ 8; 1 (Unwinding Anxiety) in anxious physicians with baseline GAD-7 scores of ≥ 10; and 1 (Unwinding Anxiety) in individuals with GAD with baseline GAD-7 scores of ≥ 10.13,15-17
In the first study, a randomized controlled trial that compared the Pacifica app to a wait-list control, only 35% of the app and 47% of wait-list participants completed the study. A composite anxiety score, using an average score of standardized GAD-7 and Depression and Anxiety Stress Scale-21 (anxiety component) scores, showed a statistically significant drop in both groups after 4 weeks (wait-list control, 0.23 composite point; Pacifica, 0.66 composite point), which was reported as statistically significantly different. In the second study, a randomized controlled trial of iCBT (Silvercloud) vs wait-list control, individuals in the iCBT group showed a statistically significant 35% reduction in GAD-7 scores at 8 weeks (baseline, 12.7; 8-week follow-up, 8.2) relative to a 15% reduction in the control group (baseline, 12.7; 8-week follow-up, 10.8). Retention was 82% and 76%, respectively.
In the third study, a single-arm trial of the Unwinding Anxiety app, anxious physicians demonstrated a statistically significant reduction in GAD-7 scores of 48% and 57% after 1 and 3 months, respectively (baseline, 11.5; 1-month, follow-up, 6.0; 3-month follow-up, 5.0). Retention was 60%. In the fourth randomized controlled trial looking at Unwinding Anxiety vs treatment as usual (TAU), individuals randomized to Unwinding Anxiety showed a statistically significant 67% reduction in GAD-7 scores (baseline, 12.0; 2-month follow-up, 3.0) relative to a 14% reduction in the TAU group at 2 months (baseline, 13.0; 2-month follow-up, 10.0). Retention was 88% for the Unwinding Anxiety group and 100% for the TAU group.
How Risky Is This App For My Patient?
When prescribing a medication or suggesting that a patient try a behavioral treatment, we fall back on the ideals of the Hippocratic oath that we took at the beginning of medical school: First, do no harm. Although digital therapeutics do not carry the traditional adverse effect risk profile of a medication, it is important to approach information in a parallel manner: Ingesting information is not free of unintended consequences. For example, a review of apps for bipolar disorder found the presentation of information that could lead to harm (eg, encouraging individuals to drink liquor to help them sleep during manic episodes).18 Additionally, privacy is a concern, as apps can easily collect personal information ranging from symptoms to geolocation.1
Digital therapeutics have the potential to live up to their promise: Mechanistic and evidence-based treatments that can be widely disseminated at low cost. To date, very few have demonstrated efficacy, and these studies would benefit from replication and extension of their findings. As clinicians consider recommending app-based treatments, it is important to stay abreast of emerging study results, and as they evaluate apps for their patients, they must keep those aforementioned important questions in mind (Table).
Dr Brewer is an associate professor of psychiatry at the Warren Alpert Medical School of Brown University, executive medical director of behavioral health at Sharecare Inc, and author of Unwinding Anxiety. Twitter: @judbrewer. He owns stock in, and serves as a paid consultant for, Sharecare Inc, the company that owns the Unwinding Anxiety app described in this article. The financial interest has been disclosed to and is being managed by his institution, Brown University, in accordance with its Conflict of Interest and Conflict of Commitment policies.
1. Firth J, Torous J, Carney R, et al. Digital technologies in the treatment of anxiety: recent innovations and future directions. Curr Psychiatry Rep. 2018;20(6):44.
2. Huang Y, Zhao N. Generalized anxiety disorder, depressive symptoms and sleep quality during COVID-19 outbreak in China: a web-based cross-sectional survey. Psychiatry Res. 2020;288:112954.
3. Salari N, Hosseinian-Far A, Jalali R, et al. Prevalence of stress, anxiety, depression among the general population during the COVID-19 pandemic: a systematic review and meta-analysis. Global Health. 2020;16(1):57.
4. Taquet M, Luciano S, Geddes JR, Harrison PJ. Bidirectional associations between COVID-19 and psychiatric disorder: retrospective cohort studies of 62 354 COVID-19 cases in the USA. Lancet Psychiatry. 2021;8(2):130-140.
5. Bowland S, Hensley M, Johnson B, Fleming A. Consumer focus groups: a key to transforming behavioral health systems? Int J Ment Health. 2010;39:16-28.
6. Anthes E. Mental health: there’s an app for that. Nature. 2016;532(7597):20-23.
7. Keller C, Ferrer RA, King RB, Collier E. Future directions of the National Institutes of Health Science of Behavior Change Program. Transl Behav Med. 2021;10:ibab029.
8. Wasil AR, Venturo-Conerly KE, Shingleton RM, Weisz JR. A review of popular smartphone apps for depression and anxiety: assessing the inclusion of evidence-based content. Behav Res Ther. 2019;123:103498.
9. Sucala M, Cuijpers P, Muench F, et al. Anxiety: there is an app for that. A systematic review of anxiety apps. Depress Anxiety. 2017;34(6):518-525.
10. Brewer JA. Why targeting entrenched habits can treat anxiety. Psychology Today. March 19, 2021. Accessed June 21, 2021.
11. Brewer JA, Roy A, Deluty A, et al. Can mindfulness mechanistically target worry to improve sleep disturbances? Theory and study protocol for app-based anxiety program. Health Psychol. 2020;39(9):776-784.
12. Brewer JA. Unwinding Anxiety: New Science Shows How to Break the Cycles of Worry and Fear to Heal Your Mind. Avery; 2021.
13. Janes AC, Datko M, Roy A, et al. Clinical efficacy and psychological mechanisms of an app-based digital therapeutic for generalized anxiety disorder. Under review.
14. Gál É, Ştefan S, Cristea IA. The efficacy of mindfulness meditation apps in enhancing users’ well-being and mental health related outcomes: a meta-analysis of randomized controlled trials. J Affect Disord. 2021;279:131-142.
15. Richards D, Enrique A, Eilert N, et al. A pragmatic randomized waitlist-controlled effectiveness and cost-effectiveness trial of digital interventions for depression and anxiety. NPJ Digital Medicine. 2020;3(1):1-10.
16. Roy A, Druker S, Hoge EA, Brewer JA. Physician anxiety and burnout: symptom correlates and a prospective pilot study of app-delivered mindfulness training. JMIR Mhealth Uhealth. 2020;8(4):e15608.
17. Moberg C, Niles A, Beermann D. Guided self-help works: randomized waitlist controlled trial of Pacifica, a mobile app integrating cognitive behavioral therapy and mindfulness for stress, anxiety, and depression. J Med Internet Res. 2019;21(6):e12556.
18. Nicholas J, Larsen ME, Proudfoot J, Christensen H. Mobile apps for bipolar disorder: a systematic review of features and content quality. J Med Internet Res. 2015;17(8):e198.