AI and Augmented Reality: A Transdiagnostic Tool for Real-World Function

amai line art/AdobeStock

When teaching residents, I remind them of the 2 essential DSM diagnostic criteria required for any mental illness diagnosis: dysfunction and distress. Yet most current treatments—pharmacological or therapeutic—focus predominantly on distress.¹ In my area of expertise, trauma and anxiety, medications reduce autonomic arousal and nightmares, while psychodynamic, cognitive, and exposure therapies target guilt, shame, and traumatic memories. Still, real-world functioning too often remains untouched. Posttraumatic stress disorder (PTSD) exposure therapy for example, is to trauma memories, and real-world avoidances are left for the patient to deal with alone. Too often a year into treatments, patients have less nightmares and flashbacks, but still avoid grocery stores, social gatherings, and the workplace.

This neglect of real world social, family, academic, and occupational functioning in the clinic arises from a lack of effective tools. Exposure therapy, the gold standard for avoidance in phobias, social anxiety disorder, agoraphobia, and PTSD, is constrained by the reality that most feared stimuli cannot be reproduced in the office.² We do not have dogs, spiders, snakes, or sex, age, race, diverse humans in a grocery store or a restaurant readily available in the clinic. As a result, therapy often becomes “hey what’s up” check-ins or, at best, develops self-guided exposure homework that is difficult for patients to implement.³

It is hard to convince a patient with fear of dogs to do exposure on their own. If you bring a dog to your clinic (which is time consuming), the patient might not even agree to come. Or the dog might get too excited and terrify them at the beginning of the exposure, when they are not ready for it. If patient gets to feel comfortable with your Labrador, facing a Pitbull out there can still be terrifying, as treatment is not generalizable to different breeds of dogs.

Avoidances involving people—a hallmark of PTSD, social anxiety disorder, and agoraphobia—are even harder to treat. You do not have access to 10 humans of diverse age, sex, race in your office to practice exposure with your patient. If you go to the grocery store with your patient, they may panic at the sight of someone resembling the perpetrator of their trauma, making in vivo exposure often infeasible, at least for the beginning. Roleplaying with the clinic staff is limited by time constraints, and there is a lack of diversity for optimal effectiveness and generalizability.⁴ Furthermore, because patients are aware that roleplaying is not real, the emotional intensity of the interaction is often diminished, limiting its therapeutic effectiveness. Because of these limits, patients too often remain disabled for years. That is why I have spent a decade developing a technology to just address this gap.

What is Augmented Reality?

Augmented reality (AR) is the newest human computer interface that blends the real and the digital into a single immersive experience. Imagine Tony Stark’s JARVIS system: digital information seamlessly overlaid onto the physical world. That is the essence of AR. Devices like the Microsoft HoloLens and Apple Vision Pro are designed to do just that—project interactive digital objects into real-world environments. Unlike virtual reality (VR), AR does not replace reality; it enhances it. Users can see their own bodies, walk through actual rooms on their own feet, interact with real people, and simultaneously engage with digital objects—whether it is a spider, a dog, a human figure, or a restaurant table.

It was this powerful, reality-integrated potential that got me excited about AR nearly a decade ago. Back then, the technology was in its infancy and largely confined to developers. I devoted the years since then to forging a new tool that merges cutting-edge technology with clinical need, for a path we previously could not walk through.

AR for Real-World Psychiatric Rehabilitation

Over the years, we have discovered AR’s remarkable ability to elicit strong objective, subjective, and physiological fear responses to digitally rendered feared objects. Patients with arachnophobia for example, showed fear of AR spiders, similar to a real tarantula. In a recent clinical trial, all patients with arachnophobia were able to touch a real live tarantula or the tank containing it in a 1-hour session.⁵ Effects persisted or further improved at 1 month follow-up without additional treatment. An ongoing clinical trial focused on fear of dogs is currently replicating these findings.

Only in the past couple of years has AR technology advanced enough to address more complex high impact challenges, such as fear of humans or human interactions, as seen in PTSD, social anxiety disorder (SAD), agoraphobia, and fear of public speaking. Many patients with PTSD become so avoidant they remain homebound, unable to go to a party, restaurant, or a grocery store.

Augmented reality example

Partnering with leading AR experts and animation artists, and using the most advanced AR hardware available, we developed what is essentially a clinical holodeck—cutting edge, immersive, and easy to use. Originally designed to help first responders with PTSD confront avoidances in real-life settings like restaurants, police stations, or firehouses, I quickly realized the platform’s focus on real-world functioning made it ideal for transdiagnostic utility.

Simply put, the technology connects the clinician’s computer to the wireless AR headsets worn by the patient (currently Apple Vision Pro) via Wi-Fi or hotspot. The clinician defines the physical boundaries of the treatment space, and plugs the patient into an immersive exposure experience with 3 components:

1. Preset Scenarios: These predesigned modules simulate common public or occupational environments (eg, a restaurant, grocery store, firehouse, police roll call). Each scenario begins with 1 or 2 characters entering a far corner of the room. For example, in the grocery store scene, shelves appear next to the room’s real walls, followed by a grocery store employee walking in through a digital door. As the patient’s anxiety decreases, the therapist can gradually introduce more characters with varying age, race, gender, body type, and behavior, until the room is realistically crowded. The patient can walk freely, listen to ambient conversations, and even pretend picking items off the shelves.

Augmented reality example

In our PTSD focused modules, a roll call room may begin sparsely, then fill with officers. At higher exposure levels, a critical incident is announced on a radio, and characters leave the room to respond. Clinicians can play background triggering sounds like fireworks, car honking, people arguing, or baby crying. You can watch the demo video here.

2. Scene Generator: This component basically offers a flexible movie studio, enabling the clinician to create individualized experiences for each patient’s unique avoidances or social needs. A library of nearly 100 characters diverse in age, sex, race, body type, and clothing, along with props such as bars, chairs, desks, food items, and restaurant tables. These elements can be arranged to simulate anything from a quiet meeting to a lively house party.

Clinicians control each character’s nonverbal behavior—smiling, frowning, gesturing—and can type dialogue that is instantly spoken by AI-generated voices. This allows for therapist-guided conversations through the character. For example, I helped a student with fear of public speaking practice her presentation by populating the room with seated digital peers, some smiling, some frowning, and having them ask spontaneous questions.

3. AI Driven Humans: This is the most dynamic and exciting feature for many patients. Highly adaptive AI brains can be placed in any of the AR characters and verbally be told who they are, defining their personality, background, and conversational style. These characters are fully automated and capable of natural, unscripted interactions—ideal for practicing diverse complex human interactions.

Augmented reality example

For example, in this video, I ask Amy to act as a pushy friend pressuring me to drink and do acid, while I practice abstinence. The characters can adjust their behavior on the fly, escalating the intensity of the conversation. For example, in a simulation for a firefighter practicing deescalation with an AI character who is acting confused and agitated, I asked the AI character to be more agitated, and it started yelling demanding he leave the room! Other examples include a job interview, interviewing a patient with PTSD, and police officer training deescalation.

At the end of the session, the AI character can offer performance feedback, adding an extra layer of realism and therapeutic utility. The full details of the technology can be found in a methods paper published recently.⁶

Clinical Vignette

Initially, I had concerns that digital humans might not provoke the same level of fear response as digital spiders. Clinical experience quickly proved otherwise. I often find myself reminding patients: “You know these are not real, right?” Objective, subjective, and physiological fear responses are intense, mirroring those elicited by real humans. This is promising, as effective exposure therapy requires fear activation that closely resembles real-life avoidances.

The first time a sex crime unit detective with PTSD entered an AR house party, it took 30 minutes before she could tolerate the presence of 1 additional character beyond the 2 women standing in the far corner of the room, chatting about their children.

“I know it’s not real,” she said, “but I can’t feel that it’s not real.” This is something that I have heard many times from many people since. As the functional gains became clear to her, she began requesting specific scenarios—like a grocery store or house party—because, as she put it, “I’m spending too much on grocery delivery,” or “my sister invited me to a party.” Over time, she was able to shop in person and attend social events.

Her return to work was a challenge. Due to intense avoidance of work-related environments, it was not until the sixth session that she could tolerate the presence of a single digital female officer in the room. Eventually, she was able to have a basic interaction with a virtual police officer as I typed in the dialogue. Later, she had a conversation with a friendly AI-driven peer officer—an interaction that helped her transition back into part-time employment.

Another patient, a police sergeant with PTSD who had been avoiding all social situations, began coming to the clinic to practice picking items off virtual grocery shelves. He then spent time sitting at a virtual bar in the restaurant scenario, learning to tolerate having his back to a digital crowd—something he had not felt safe doing in years. Currently, he is able to do 3 real world activities a week, including going to a grocery store, a public event, or a restaurant.

Reflections and Future Directions

What makes AR such a uniquely powerful therapeutic tool is its capacity to blur the boundaries between real and unreal. First, patients can see their own body, the physical walls of the room, and the clinician. This creates a deep sense of immersion, where the boundaries of real and unreal begin to blur in the user’s mind. Second, because they walk on their own feet in a familiar space, the experience feels far more embodied and real than VR.⁷ Third, while patients intellectually know the scenario is not real, their bodies—the animal brain—cannot differentiate real from unreal. This cognitive-emotional disconnect enables participation from those who might otherwise be too fearful to engage in in vivo exposure. Finally, because AI characters do not carry the same perceived intent as real humans, patients often find their interactions with them feel surprisingly authentic—and powerfully activating.

For decades, psychiatric treatment has largely missed the real-world functional disabilities of anxiety and trauma related disorders. AI enhanced AR offers a unique and long-overdue opportunity to bridge this gap. By focusing on social and occupational functioning, and dynamic adaptation to each patient’s individual needs, this technology holds immense transdiagnostic potential. In addition to trauma related and anxiety disorders, it can be used for autism spectrum disorder and chronic mental illness for social skills training. The wireless design opens the door to telemedicine access for those who are too disabled or geographically isolated to attend in-person sessions.

I foresee a near future—perhaps within a year or 2—when clinicians will simply describe an avoided scenario, and AI will generate the entire scene, characters, and events, tailored to the needs of the individual patient. This is not science fiction anymore; it is a fast-approaching unreal becoming reality, that may redefine how we deliver psychiatric care.

Disclosure: This technology has potential for commercialization. Patent and license is owned by Wayne State University.

Dr Javanbakht is the director of the Stress, Trauma, and Anxiety Research Clinic in the Department of Psychiatry and Behavioral Neurosciences at Wayne State University School of Medicine.

References

1. American Psychiatric Association. Diagnostic and Statistical Manual of Mental Disorders. 5th ed. American Psychiatric Publishing; 2013.

2. Foa EB, Hembree EA, Rothbaum BO, Rauch S. Prolonged Exposure Therapy for PTSD: Emotional Processing of Traumatic Experiences. 2nd ed. Oxford University Press; 2019.

3. Deacon BJ, Farrell NR, Kemp JJ, et al. Assessing therapist reservations about exposure therapy for anxiety disorders: the therapist beliefs about exposure scale. J Anxiety Disord. 2013;27(8):772-780.

4. Rosen CS, Matthieu MM, Wiltsey Stirman S, et al. A review of studies on the system-wide implementation of evidence-based psychotherapies for posttraumatic stress disorder in the Veterans Health Administration. Admin Policy Ment Health. 2016;43(6):957-977.

5. Javanbakht A, Madaboosi S, Grasser LR. Real-life contextualization of exposure therapy using augmented reality: a pilot clinical trial of a novel treatment method. Ann Clin Psychiatry. 2021;33(4):220-231.

6. Javanbakht A, Hinchey L, Gorski K, et al. Unreal that feels real: artificial intelligence-enhanced augmented reality for treating social and occupational dysfunction in post-traumatic stress disorder and anxiety disorders. Eur J Psychotraumatol. 2024;15(1):2418248.

7. Tsai CF, Yeh SC, Huang Y, et al. The effect of augmented reality and virtual reality on inducing anxiety for exposure therapy: a comparison using heart rate variability. J Healthc Eng. 2018;2018:6357351.