OR WAIT null SECS
The world is facing a cognition crisis. Here’s how to face it with measurement-based cognitive care.
The world is in the midst of an unfolding cognition crisis, with devastating personal, societal, and economic implications. The expanding elderly population requires better cognitive care; we are also coming to understand the importance of cognitive interventions and support in early childhood. Furthermore, the COVID-19 pandemic has brought global attention to cognitive issues, with over 50% of individuals recovering from the infection experiencing fatigue and brain fog. Initial studies indicate lasting deficits in executive function, impaired cognitive control, and reduction in global cognition leading to disability or reduced function.1,2 There are fears that COVID-induced deficits may last for years beyond the infection.
Cognitive skills—such as attention, memory, perception, language, and reasoning3,4—are declining across the whole population. Beyond the effects of COVID-19, there is an alarming increase in the number of individuals around the world with serious cognitive impairments. These impairments co-occur with disorders including depression, anxiety, schizophrenia, autism, posttraumatic stress disorder, dyslexia, obsessive-compulsive disorder, bipolar disorder, attention-deficit/hyperactivity disorder (ADHD), addiction, dementias, and more.5 The severity, type, duration, and course of cognitive dysfunction differ between these disorders. The aging world population will heighten the cognition crisis. By 2060, the number of individuals aged 65 and older is projected to double.6
The Importance of Assessing Cognition
Cognition is often only assessed when there is a specific, subjective complaint from patients, family members, or caregivers. In this context, there are many factors which suboptimize the value. For example, there are lengthy waiting lists to see neuropsychologists or cognitive neurologists, and various modalities used to assess cognitive dysfunction in later life are inaccessible to most (eg, positron emission tomography and advanced magnetic resonance imaging scans). Furthermore, there is no consistent clinical assessment algorithm for how to use cognitive assessment tools (eg, when to use pencil and paper tests, digital, blood-based, or neuroimaging). When there is no specific, subjective cognitive complaint, cognition is not assessed. This means more subtle and slow-onset forms of cognitive dysfunction may be missed. This can delay treatment and mean more progressive brain changes. If cognition is assessed, it is often not addressed, supported, or followed up in the long-term. Further, in many developing countries, cognition is rarely—if ever—screened, resulting in complications such as high rates of undiagnosed dementia.7
However, since impairments in cognition occur in disorders across the lifespan, it is critical that cognition is assessed and monitored across disorders and throughout life. Addressing and building cognition is especially crucial during early childhood, when cognitive difficulties can be caught before they have too much detrimental impact on a child’s developmental trajectory.
Examples of Cognitive Decline in Psychiatric Conditions
Consider depression: More than 264 million individuals around the world suffer from depression, making it the leading cause of disability worldwide and a major contributor to the overall global burden of disease.8 Depression is commonly associated with a lowering of a patient’s mood and energy for long periods of time. Depression also impairs cognition, including affecting attention, memory, information processing, decision-making skills, cognitive flexibility, and executive functioning.9,10 Indeed, 85 to 94% of individuals with acute depression experience significant cognitive impairments.11,12 Even when depression has remitted, around 40 to 45% of patients report dysfunction in one cognitive domain.13 Cognitive impairments are not addressed by first-line treatment options for depression,14-16 however cognitive remissionis now proposed as a novel treatment target for depression.17 Cognitive remediation therapy has provided encouraging results for the management of cognitive deficits in major depressive disorder (MDD).17 The incorporation of biobehavioral strategies (eg, exercise) and multimodal treatment approaches (eg, cognitive training, antidepressant therapy, and neuromodulation) is likely to generate therapeutic benefit.17
Another example is cancer. It is estimated that at the end of 2020 there were over 70 million cancer survivors worldwide.18,19 Cognitive impairments—including deficits in visual working memory, sustained attention, new learning, and more—are commonly observed in patients with cancer and those in remission.20,21 Though cancer- and chemotherapy-related cognitive impairments (eg, confusion, lapses in memory and attention, difficulty concentrating, etc) are often known colloquially as chemo brain or chemo fog, up to 30% of patients with cancer exhibit subtle cognitive impairments prior to treatment, suggesting the body’s physiological response to a tumor (eg, inflammatory processes or vascular changes around the tumor) may be disruptive to cognition.22,23 Neuroimaging and neuropsychological studies have validated the role of chemotherapy contributing to cognitive issues, with chemotherapy patients having a more difficult time during memory recall than those cancer patients who did not receive the drugs, which suggests that their neural networks had been altered.23,24 Indeed, 75% of cancer patients have cognitive impairments during treatment, and 35% of survivors have cognitive impairments that continue in the months and years after treatment.21
In addition to depression and cancer, there are many other disorders across the lifespan that have been shown to cause cognitive impairments, including ADHD,25 diabetes,26 anxiety disorders,27 chronic obstructive pulmonary disease (COPD),28,29 Parkinson’s disease,30 multiple sclerosis,31,32 and more. The Figure depicts a small subset of disorders with associated cognitive problems.33
New Approaches to Cognitive Issues
Innovative measurement-based approaches need to be leveraged to better understand and treat the impacts of cognitive impairments from disorders across the lifespan. Measurement-based care can be defined as the practice of basing clinical care on client data collected throughout treatment. Measurement-based cognitive care, therefore, includes ensuring that cognition is assessed and monitored at all stages of the treatment process across a multitude of disorders—not merely Alzheimer disease (AD) and late-life cognitive decline.
For example, with cancer patients, cognition needs to be assessed before treatment begins and then regularly during and after treatment. Cognitive assessments must also be easy for the patient to access. Since these patients will already be saturated with physical and mental health interventions, it is critical that any cognitive assessments and additional interventions are able to fit into their lives as seamlessly as possible.
Regular assessment and monitoring of cognitive health is also critical across the entire spectrum of health, including those who do not have a present health concern. Innovative delivery systems—such as at-home digital technologies—can ensure accessibility and awareness that these assessments exist. Regularly assessing cognition throughout the lifespan will enable cognition to be used as a new vital sign. Further, it will provide the opportunity to establish an individual’s cognitive baseline; identify changes that may suggest prodromal stages of disease and allow for early intervention. Cognitive care will adopt a more proactive, preventative, and personalized approach to one’s cognitive lifespan. Psychometric quality is clearly critical (ie, robust assessments of performance norms, reliability, validity, sensitivity, and specificity).34 Ensuring cognitive assessments include quality of life and activities of daily living is also important.
Utilizing New Technologies
New technologies and tools will help make measurement-based cognitive care a reality. New, predictive tools in cognition include facial biomarkers,35,36 vocal biomarkers,35,37 digital biomarkers,38 neuroimaging,39,40 blood tests,41 and text mining from social media.42 Tracking eye-movement has been found to offer a window into cognitive processes and impairments. Preliminary data indicate that eye-tracking tests may detect early stages of AD43,44; characterize cognitive impairments in patients with other neurological disorders, including amyotrophic lateral sclerosis, Parkinson disease, multiple sclerosis, autism spectrum disorders,45 and epilepsy46; and identify and access brain dysfunction in patients—including children—with mental health conditions such as depression.47-50 When implemented in a digital setting, some novel tools have been found to have a myriad of benefits.
For example, Akili Interactive developed the first and only video game-like digital therapeutic that has been cleared by the US Food and Drug Administration to treat adolescent ADHD and improves attention impairments.51 Akili’s digital therapeutic for adolescent ADHD has been found to improve objectively measured inattention as determined by the Test of Variable Attention Attention Performance Index (API) in adolescent patients with ADHD.52 Other digital therapeutics have been found to reduce cognitive impairments in adults living with MDD53 and are being tested to treat COVID-19 brain fog.54 Tools are also being developed to promote the development of cognitive skills during childhood.55 Some smart phone-based technologies may be deployed worldwide including to low- and middle-income countries, allowing greater equity in cognitive care.56
The necessity of validated remote evaluations—enabled by many of these new biomarkers—is apparent, underscored by the recent limitations on in-person visits secondary to the COVID-19 pandemic. Digital technologies are well positioned for a role in remote evaluation to be used synchronously or asynchronously and over telemedicine platforms.57
Measurement-Based Cognitive Care
We must re-engineer the modern health care team when moving towards measurement-based cognitive care. Just as genetic counselling was established as a new field in 1969 to help individuals navigate genetic risk and testing, a new field needs to be established to help patients navigate cognitive risk and testing. Many new cognition biomarkers provide a risk score or predictive analytic, which like predictive genetics can bring a host of new questions and worries: What does my score mean? How do you interpret a black-box algorithm? What can I do about my risk for cognitive impairment after chemotherapy? Am I developing AD or am I experiencing cognitive problems due to depression? Will these results create issues for my health or life insurance policies?
Cognition counselors are needed as part of the modern care team to 1) ensure that cognition is addressed in disorders where it is commonly overlooked (eg, postsurgery, cancer, diabetes, depression, etc); 2) help patients navigate the new anxieties and questions that arise from new technologies, cognitive assessment, and risk scores; and 3) discuss and coordinate care options and provide referrals to educational services, advocacy and support groups, novel technologies, other health professionals, and community or state services. Cognition counselors will also help overcome the barriers of implementing digital technology in both a clinical and educational setting, increasing acceptance and compliance. Further, existing care team members—such as clinical neuropsychologists—could also be trained or upskilled to fulfill the role of cognition counselors, helping to rethink and shape the field of neuropsychology.58
To optimize the likelihood that measurement-based cognitive care can be instituted, leveraged tools should be brief (≤ 10 minutes), highly scalable, able to be administered by the patient or trained clinical staff, and able to automatically generate an interpretative report with instant electronic medical record integration.59 To further help realize measurement-based cognitive care, health systems must be prepared to move quickly when a new biomarker is seen to add clinical value. We applaud the work of the Davos Alzheimer Collaborative for proposed frameworks to accelerate such preparedness.60
Finally, design thinking principles must be strictly adhered to in the development of novel biomarkers given these tools can be deployed across disorder type and the lifespan.61 For example, elderly individuals may experience changes and challenges that span disability segments, including visual, cognitive, speech, mobility, and neural impairments. From deterioration in vision to difficulty touching screens to everything in between, novel biomarkers and tools must account for these changes and be designed to be accessible to all. The fields of disability studies and assistive technologies will likely be informative for these design challenges.62-63 Further, measures must be taken to mitigate practice effects from repeated use of measurement-based cognitive care tools, ensuring that increasing familiarity and exposure to cognitive tests does not become a confounding factor in scores.64
Measurement-based cognitive care is the urgently needed to address to the unfolding cognition crisis. Measurement-based cognitive care will bring an era of medicine where cognition is proactively addressed and better understood for disorders across the lifespan; technologies and tools underpin advancements in cognitive assessment and monitoring; and care teams are equipped to help patients navigate the new uncertainties that arise and receive proper care. In doing so, individuals will be able to improve their quality of life and have their capacity to thrive increased. Placing measurement-based cognitive care as a critical component of medicine will bring widespread personal, medical, and societal benefits.
Erin Smith is a member of the Steering Committee of the Organization for Economic Co-operation and Development (OECD)-PRODEO Institute Neuroscience-inspired Policy Initiative, and an Atlantic Fellow for Brain Equity at the Global Brain Health Institute at UC San Francisco (UCSF). Dr Cummings is the Joy Chambers-Grundy Professor of Brain Science and Director of the Chambers-Grundy Center for Transformative Neuroscience, University of Nevada Las Vegas (UNLV). Dr Bellgrove is a Professor in Cognitive Neuroscience and Director of Research in the Turner Institute for Brain and Mental Health, School of Psychological Sciences, Monash University, Australia. Dr Robertson is the T. Boone Pickens Distinguished Scientist at the Center for BrainHealth, a codirector at Global Brain Health Institute, professor emeritus at Trinity College Dublin, and founding director of its Institute of Neuroscience. Dr Wolfe is a Senior Clinical Advisor for Cohen Veterans Bioscience’s Trauma Research Programs. She serves as Strategic Advisor co-leading CVB’s Suicide Strategy Initiative. Dr Kirk is an Early Career Research Fellow, Turner Institute for Brain and Mental Health, Neurodevelopment Program School of Psychological Sciences, Monash University. Dr Lavretsky is the professor of psychiatry, Director, Late-life Mood, Stress and Wellness, and Integrative Psychiatry research programs UCLA Semel Institute for Neuroscience and Human Behavior. Dr Eyre is cofounder of the PRODEO Institute, co-lead of the OECD-PRODEO Institute Neuroscience-inspired Policy Initiative, an instructor in brain health diplomacy at the Global Brain Health Institute. He holds adjunct positions with IMPACT at Deakin University and Baylor College of Medicine.
1. Ortelli P, Ferrazzoli D, Sebastianelli L, et al. Neuropsychological and neurophysiological correlates of fatigue in post-acute patients with neurological manifestations of COVID-19: Insights into a challenging symptom. J Neurol Sci. 2021;420:117271.
2. Agadjanyan MG, Petrovsky N, Ghochikyan A. A fresh perspective from immunologists and vaccine researchers: active vaccination strategies to prevent and reverse Alzheimer's disease. Alzheimers Dement. 2015;11(10):1246-1259.
3. Gazzaley A. The Cognition Crisis: The Perils and Promise of Technology and the Brain. Digital Media and Developing Minds National Conference. October 17, 2018. Accessed July 14, 2017. https://www.childrenandscreens.com/2018-conference/talks-and-panels/the-cognition-crisis-the-perils-and-promise-of-technology-and-the-brain/
4. Goetz CG. Textbook of Clinical Neurology, Volume 355. Elsevier Health Sciences; 2007.
5. Gazzaley A, Rosen LD. The Distracted Mind: Ancient Brains in a High-Tech World. MIT Press; 2016.
6. ACL Administration for Community Living. Profile of Older Americans: 2017. April 30, 2018. Accessed July 14, 2021. https://acl.gov/news-and-events/announcements/announcing-2017-profile-older-americans
7. Jitapunkul S, Chansirikanjana S, Thamarpirat J. Undiagnosed dementia and value of serial cognitive impairment screening in developing countries: a population‐based study. Geriatr Gerontol Int. 2009;9(1):47-53.
8. World Health Organization. Depression. January 30, 2020. July 14, 2021. https://www.who.int/news-room/fact-sheets/detail/depression
9. McIntyre RS, Xiao HX, Syeda K, et al. The prevalence, measurement, and treatment of the cognitive dimension/domain in major depressive disorder. CNS Drugs. 2015;29(7):577-89.
10. Millan MJ, Agid Y, Brüne M, et al. Cognitive dysfunction in psychiatric disorders: characteristics, causes and the quest for improved therapy. Nat Rev Drug Discov. 2012;11(2):141-68.
11. Perini G, Ramusino MC, Sinforiani E, et al. Cognitive impairment in depression: recent advances and novel treatments. Neuropsychiatr Dis Treat. 2019;15:1249-1258.
12. Conradi HJ, Ormel J, de Jonge P. Presence of individual (residual) symptoms during depressive episodes and periods of remission: a 3-year prospective study. Psychol Med. 2011;41(6):1165-74.
13. Advances in Understanding Neurobiological Underpinnings and Treatment Interfaces of Cognition in Depression.Psych Congress 2020. 2020. Accessed July 14, 2021. https://virtual.psychcongress.com
14. Shilyansky C, Williams LM, Gyurak A, et al. Effect of antidepressant treatment on cognitive impairments associated with depression: a randomised longitudinal study. Lancet Psychiatry. 2016;3(5):425-435.
15. Cartreine J. More than sad: depression affects your ability to think. Harvard Health Blog. May 6, 2016. Accessed July 14, 2021. https://www.health.harvard.edu/blog/sad-depression-affects-ability-think-201605069551
16. Richardson L, Adams S. Cognitive deficits in patients with depression. The Journal for Nurse Practitioners. 2018;14(6):437-443.
17. Carvalho AF, Alves GS, Köhler CA, McIntyre RS. Cognitive enhancement in major depressive disorder. In Keefe RS, Reichenberg A, Cummings J, eds. Cognitive Enhancement in CNS Disorders and Beyond. Oxford University Press; 2017.
18. Weiss B. Chemobrain: a translational challenge for neurotoxicology. Neurotoxicology. 2008;29(5):891-898.
19. Hodgson KD, Hutchinson AD, Wilson CJ, Nettelbeck T. A meta-analysis of the effects of chemotherapy on cognition in patients with cancer. Cancer Treat Rev. 2013;39(3):297-304.
20. Janelsins MC, Heckler CE, Peppone LJ, et al. Longitudinal trajectory and characterization of cancer-related cognitive impairment in a nationwide cohort study. J Clin Oncol. 2018;36(32):3231-3239.
21. Pendergrass JC, Targum SD, Harrison JE. Cognitive impairment associated with cancer: a brief review. Innov Clin Neurosci. 2018;15(1-2):36-44.
22. Olson B, Marks DL. Pretreatment cancer-related cognitive impairment—mechanisms and outlook. Cancers. 2019;11(5):687.
23. Palmer C. Cognition and cancer treatment. American Psychological Association. 2020;51(2):42.
24. Kesler SR, Bennett FC, Mahaffey ML, Spiegel D. Regional brain activation during verbal declarative memory in metastatic breast cancer. Clin Cancer Res. 2009;15(21):6665-6673.
25. Rubia K. Cognitive neuroscience of attention deficit hyperactivity disorder (ADHD) and its clinical translation. Front Hum Neurosci. 2018;12(100).
26. Munshi MN. Cognitive dysfunction in older adults with diabetes: what a clinician needs to know. Diabetes Care. 2017;40(4)461-467.
27. Robinson OJ, Vytal K, Cornwell BR, Grillon C. The impact of anxiety upon cognition: perspectives from human threat of shock studies. Front Hum Neurosci. 2013;7:203.
28. Torres-Sánchez I, Rodríguez-Alzueta E, Cabrera-Martos I, et al. Cognitive impairment in COPD: a systematic review.J Bras Pneumol. 2015;41(2):182-190.
29. Andrianopoulos V, Gloeckl R, Vogiatzis I, Kenn K. Cognitive impairment in COPD: should cognitive evaluation be part of respiratory assessment? Breathe. 2017;13(1):e1-e9.
30. O'Callaghan C, Lewis SJG. Cognition in Parkinson's disease. Int Rev Neurobiol. 2017;133:557-583.
31. Sumowski JF, Benedict R, Enzinger C, et al. Cognition in multiple sclerosis: state of the field and priorities for the future. Neurology. 2018;90(6):278-288.
32. Benedict RH, Amato MP, DeLuca J, Geurts JJG. Cognitive impairment in multiple sclerosis: clinical management, MRI, and therapeutic avenues. Lancet Neurol. 2020;19(10):860-871.
33. Aging2.0 OPTIMIZE Conference: Savonix Presentation. 2018.
34. Charalambous AP, Pye A, Yeung WK, et al. Tools for app-and web-based self-testing of cognitive impairment: systematic search and evaluation. J Med Internet Res. 2020;22(1):e14551.
35. Williamson JR, Quatieri TF, Helfer BS, et al. Vocal and facial biomarkers of depression based on motor incoordination and timing. Proceedings of the 4th International Workshop on Audio/Visual Emotion Challenge. 2014:65-72.
36. Ali MR, Myers T, Wagner E, et al. Facial expressions can detect Parkinson's disease: preliminary evidence from videos collected online. arXiv. December 9, 2020. Accessed July 14, 2021. https://arxiv.org/abs/2012.05373
37. Cummins N, Scherer S, Krajewski J, et al. A review of depression and suicide risk assessment using speech analysis.Speech Communication. 2015;71:10-49.
38. Dagum P. Digital biomarkers of cognitive function. NPJ Digit Med. 2018;1(1):1-3.
39. Hart J, Kraut MA, Womack KB, et al. Neuroimaging of cognitive dysfunction and depression in aging retired National Football League players: a cross-sectional study. JAMA Neurol. 2013;70(3):326-335.
40. Liotti M, Mayberg HS. The role of functional neuroimaging in the neuropsychology of depression. J Clin Exp Neuropsychol. 2001;23(1):121-136.
41. Zetterberg H, Blennow K. Blood biomarkers: democratizing Alzheimer’s diagnostics. Neuron. 2020;106(6):881-883.
42. Bull CN, Asfiandy D, Gledson A, et al. Combining data mining and text mining for detection of early stage dementia: the SAMS framework. 2016. Accessed July 14, 2021. https://eprints.lancs.ac.uk/id/eprint/79004/1/combining_data_mining.pdf
43. Mengoudi K, Ravi D, Yong KX, et al. Augmenting cognitive assessment with instruction‐less Eye‐tracking tests: a machine learning approach for detecting abnormal oculomotor biomarkers. Alzheimer's & Dementia. 2020;16:e045318.
44. Crutcher MD, Calhoun-Haney R, Manzanares CM, et al. Eye tracking during a visual paired comparison task as a predictor of early dementia. Am J Alzheimers Dis Other Demen. 2009;24(3):258-266.
45. Falck-Ytter T, Carlström C, Johansson M. Eye contact modulates cognitive processing differently in children with autism. Child Dev. 2015;86(1):37-47.
46. Tao L, Wang Q, Liu D, et al. Eye tracking metrics to screen and assess cognitive impairment in patients with neurological disorders. Neurol Sci. 2020;41(7):1-8.
47. Harrison AJ, Gibb BE. Attentional biases in currently depressed children: an eye-tracking study of biases in sustained attention to emotional stimuli. J Clin Child Adoles Psychol. 2015;44(6):1008-1014.
48. Sanchez A, Vazquez C, Marker C, et al. Attentional disengagement predicts stress recovery in depression: an eye-tracking study. J Abnorm Psychol. 2013;122(2):303.
49. Gotlib IH, Joormann J. Cognition and depression: current status and future directions. Annu Rev Clin Psychol. 2010;6:285-312.
50. Itti L. New eye-tracking techniques may revolutionize mental health screening. Neuron. 2015;88(3):442-444.
51. EndeavorRx. 2021. Accessed July 14, 2021. https://www.endeavorrx.com/
52. Kollins SH, DeLoss DJ, Cañadas E, et al. A novel digital intervention for actively reducing severity of paediatric ADHD (STARS-ADHD): a randomised controlled trial. Lancet Digit Health. 2020;2(4):e168-e178.
53. Lovett L. Akili’s video game therapy linked to reduced cognitive impairments in patients with depression. Mobi Health News. December 12, 2019. Accessed July 14, 2021. https://www.mobihealthnews.com/news/north-america/akilis-video-game-therapy-linked-reduced-cognitive-impairments-patients
54. Akili. Akili collaborates with Weill Cornell Medicine, NewYork-Presbyterian Hospital and Vanderbilt University Medical Center to study digital therapeutic AKL-T01 as treatment for COVID brain fog. News release. April 7, 2021. Accessed July 14, 2021. https://www.akiliinteractive.com/news-collection/akili-collaborates-with-weill-cornell-medicine-newyork-presbyterian-hospital-and-vanderbilt-university-medical-center-to-study-digital-therapeutic-akl-t01-as-treatment-for-covid-brain-fog
55. Tali Health. 2021. Accessed July 14, 2021. https://talidigital.com/about-us/
56. Bonnechère B, Sahakian BJ. Can mobile technology help prevent the burden of dementia in low- and mid-income countries? Front Public Health. 2020;8:554938.
57. Staffaroni AM, Tsoy E, Taylor J, et al. Digital cognitive assessments for dementia: digital assessments may enhance the efficiency of evaluations in neurology and other clinics. Practical Neurol. 2020;2020:24-45.
58. Postal K. Disruptive technology initiative. American Academy of Clinical Neuropsychology. May 25, 2016. Accessed July 14, 2021. https://theaacn.org/disruptive-technology-initiative/
59. Sabbagh MN, Boada M, Borson S, et al. Early detection of mild cognitive impairment (MCI) in primary care. J Prev Alzheimers Dis. 2020;7:165-170.
60. Davos Alzheimer’s Collaborative. Healthcare system preparedness. 2021. Accessed July 14, 2021. https://www.davosalzheimerscollaborative.org/healthcare-system-preparedness
61. Altman M, Huang TT, Breland JY. Design thinking in health care. Prev Chronic Dis. 2018;15:e117.
62. Mankoff J, Hayes GR, Kasnitz D. Disability studies as a source of critical inquiry for the field of assistive technology. Proceedings of the 12th International ACM SIGACCESS Conference on Computers and Accessibility. January 2010. Accessed July 14, 2021. https://www.researchgate.net/publication/221652175_Disability_Studies_as_a_Source_of_Critical_Inquiry_for_the_Field_of_Assistive_Technology
63. Nusem E. A design-led approach to innovation in aged care. Queensland University of Technology. 2016. Accessed July 14, 2021. https://eprints.qut.edu.au/96318/
64. Goldberg TE, Harvey PD, Wesnes KA, et al. Practice effects due to serial cognitive assessment: implications for preclinical Alzheimer’s disease randomized controlled trials. Alzheimers Dement (Amst). 2015;1(1):103-11.