Mental Distress Detection in Hebrew Texts
A real world solution for improving mental care among teenagers
Hava Doron-Soferman (4girls.co.il), Natalie Shapira (BIU),
Michal Komem (Sapir College),
Amit Shkolnik (4girls.co.il)
Hava Doron-Soferman (4girls.co.il), Natalie Shapira (BIU),
Michal Komem (Sapir College),
Amit Shkolnik (4girls.co.il)
Within 10 years, from 2004-2015 suicidal incident among UK teenagers’ girls raised by 65% (31% among boys), 3 symptoms of emotional distress reached 37% (boys 15%), anxiety 12% (boys 8.6%)[1]. Researches from other countries indicating similar findings[2].
While those dramatic changes are attributed to the changes in lifestyle, and mainly to the role of digital medias & social networks, very little solutions were offered to tackle those mediums.
Teenagers spend 4-10 hours a day online. In order to offer them better mental care, we first need to detect those individuals needed it. Manual monitoring social network is overwhelming task. Therefor an artificial intelligence model combining machine learning and natural language processing methods seemed to be adequate solution if the given model is accurate enough (F1=0.8+), flexible enough to be transferred to any social network (e.g. dynamic model), and can handle short text as well as long one, with all their relative issue- slang, shortcuts, emojis, etc.
www.4girls.co.il is a unique social network site serving only teenagers’ girls (age 12-18). Girls are writing anonymously, so they can express them self freely, including intimate and personal areas. Site’s database includes data from 2004 through 2019: 60K Posts, 1.2M direct chat messages, 520K chat rooms messages, 12.5K forum discussion with 250K talkbacks. Utilizing the database, and 4GIRLS users to build and test a mental distress detection model seemed to be promising.
Our goal was to build a real-world model for detecting distress in posts.
A dataset of 4,200 posts, including 1,500 posts indicating mental distress were tagged for supervised model. The mental distress include 6 types: suicidal, violence, depression, self-esteem/ body image, loneliness and others.
As text are all in Hebrew, we used YAP [3] as Hebrew natural language processing pipe-line. Several models were tested, finally Logistic Regression is in use. Model is using lemnmatized posts and talkback texts, as well as meta-features and TF-IDF vectorization.
Results
Precision: 85.6, Recall: 85.7, F1: 0.856, AUC=91.7.
This AUC score is slightly above Patient Health Questionnaire (PHQ)[4] and Hospital Anxiety and Depression Scale (HADS)[5]
System is now in production and inspection on 4Girls site.
ML&NLP model were previously evaluated for mental illness detection for academic goals[6]. In this work we are showing that it can be used for real world detection and improving mental care in. It is also preliminary work of its kind in Hebrew- as far as we know.
• Moving exiting model to neural network by using existing dataset as weak learner and for unsupervised data augmentation.
• Move to dynamic model to allow easy adaption to any social media network
• New model for short text analysis- forums, chats, 1X1 chats
• Prediction of mental distress in early stages
• Prediction of mental distress in naïve mediums like Facebook
Avoiding severe mental distress via community support in early stages.
[1] Carli Lessof, Andy Ross, Richard Brind, Emily Bell, Sarah Newton – TNS BMRB: Longitudinal Study of Young People in England cohort 2: health and wellbeing at wave 2
[2] Jean M. Twenge, Thomas E. Joiner, Megan L. Rogers, Gabrielle N. Martin: Increases in Depressive Symptoms, Suicide-Related Outcomes, and Suicide Rates Among U.S. Adolescents After 2010 and Links to Increased New Media Screen Time.
[3] More Amir, Seker Amit, Basmova Victoria, Tsarfaty Reut :Joint Transition-Based Models for Morpho-Syntactic Parsing: Parsing Strategies for {MRL}s and a Case Study from Modern {H}ebrew
[4] Lo¨ we B, Kroenke K, Herzog W, Gra¨ fe K: Measuring depression outcome with a brief self-report instrument: sensitivity to change of the Patient Health Questionnaire (PHQ-9). J Affect Disord 2004, 81:61-66.
[5] Zigmond AS, Snaith RP: The hospital anxiety and depression scale. Acta Psychiatr Scand 1983, 67:361-370.
[6] Sharath Chandra Guntuku, David B Yaden, Margaret L Kern, Lyle H Ungar, Johannes C Eichstaedt : Detecting depression and mental illness on social media: an integrative review