Wearables and Smart Phones with Sensing Technologies: A Plan ‘To Reshape Mental Health Care’ by Better Visualizing Early Warning Signs
By Lori Cameron
Published 05/31/2018
Share this on:
For the roughly 450 million people worldwide who suffer from serious mental illness, a new sensing technology proposal promises to offer some relief.
Wearables and smartphones may soon be able to help people who suffer from depression, bipolar disorder, schizophrenia, suicidal thoughts, and other disorders by monitoring their activity, physical symptoms, and social interactions for early warning signs of trouble.
“Serious mental illnesses often don’t have life-long cures; however, appropriate intervention and management can ensure long-term patient well-being. Effective illness management requires granular symptom monitoring. Specifically, identifying early-warning signs in patients can result in timely clinical interventions and, thus, prevent relapse onset and hospitalization,” say Saeed Abdullah of Penn State University and Tanzeem Choudhury of Cornell University in their article “Sensing Technologies for Monitoring Serious Mental Illnesses.” (Login may be required for full text.)
Abdullah and Choudhury propose a new sensing technology system that uses wearables and smart phones to capture a variety of behavioral, physiological, and social data related to serious mental illnesses. The chart below details the sensing methods, types of technology required, data features, and the types of patients most helped by the method.
Sensing technologies for capturing behavioral, physiological, and social data relevant to serious mental illnesses.
Behavioral Signals
The behavior of mentally ill individuals can reveal a lot about their mental state—speech patterns, activity level, use of social media, and where they do (or don’t) go.
Abdullah and Choudhury offer a number of examples of what these behaviors reveal about a person:
Location features such as total distance traveled, distance from home, and unpredictable locations are strongly correlated with disease symptoms in patients with schizophrenia. In addition, a sedentary lifestyle is closely associated with depression.
Vocal jitter—short-time fluctuations in the fundamental frequency—is less prominent in high-risk suicidal patients.
Higher use of social apps correlates with lower stress, and increased use of entertainment apps is associated with higher mood level in people suffering from bipolar disorder.
In bipolar disorder, mania is marked by over-activity, and depression by lower activity. Reduced level of physical activity is also a marker of increased symptom severity in patients with schizophrenia.
Physiological Signs
Physical properties that can reveal a person’s mental state include facial expressions, heart rate variability (HRV), eye movement, and electrodermal activity—changes in the skin caused by sweating.
Facial expressions like smiling, frowning, raised eyebrows, and head movement—which can all be recorded by a computer or smart phone camera—are all indicators of a person’s emotional state. For example, research has shown that patients with schizophrenia tend to show reduced facial expressivity.
Other examples include eye-tracking dysfunction among those suffering from schizophrenia and melancholic depression, and electrodermal activity clues in those contemplating suicide.
Social Signals
A lack of social interaction can be a sign of depression, while inappropriate interaction can predict the onset of psychosis in bipolar and schizophrenic patients.
Communication patterns can also signal a shift between manic and depressive episodes in patients with bipolar disorder. During depressive episodes, patients send fewer text messages; during manic episodes, they send more.
Social media platforms such as Facebook and Twitter also provides a wealth of information about people’s mental state. “Specifically, social media data can be useful in determining social engagement, social network characteristics, mood, and emotion,” write Abdullah and Choudhury.
In addition to posts, images and video—and even the use of emoticons—can be used to measure a person’s mental condition.
Sensing technologies with ‘the potential to reshape mental health care’
Multimedia technology can help people with mental illness, the authors assert.
“We believe that the multimedia community can build on sensing technologies to enable efficient clinical decision-making in mental health care. Specifically, innovative multimedia systems can help identify and visualize personalized early-warning signs from complex multimodal signals, which could lead to effective intervention strategies and better preemptive care,” they say.
But there is more to be done before sensing technologies become a reality and an integral part of mental health care, according to Abdullah and Choudhury.
“Before these technologies can be fully integrated into existing healthcare infrastructure, we must address numerous key challenges, including the lack of clinical evidence, integration of multi-modal data streams, privacy issues, and long-term user engagement. Successful integration of sensing technologies has the potential to reshape mental health care—making it preemptive, patient-centered, and cost-effective while extending its reach to a global population,” they say.
Research related to mental health in the Computer Society Digital Library
Lori Cameron is a Senior Writer for the IEEE Computer Society and currently writes regular features for Computer magazine, Computing Edge, and the Computing Now and Magazine Roundup websites. Contact her at l.cameron@computer.org. Follow her on LinkedIn.
