Multimodal Deep Learning based Framework for Detecting Depression and Suicidal Behaviour by Affective Analysis of Social Media Posts

Malhotra, Anshu and Jindal, Rajni (2020) Multimodal Deep Learning based Framework for Detecting Depression and Suicidal Behaviour by Affective Analysis of Social Media Posts. EAI Endorsed Transactions on Pervasive Health and Technology, 6 (21). e1. ISSN 2411-7145

[img]
Preview
Text
eai.13-7-2018.164259.pdf
Available under License Creative Commons Attribution No Derivatives.

Download (3MB) | Preview

Abstract

INTRODUCTION AND OBJECTIVES: Currently, no social media platform has deployed a real time system which can analyse users’ state of mind based on day to day posts on continual basis and detect the onset of depression, suicidal or self harming behaviour etc. Platforms majorly rely on manual reporting of suicidal and self harming behavior. In this paper, we propose a real time, deep learning based system for affective analysis of a user’s online social media posts of multimodal nature, with the objective of detecting onset of depression and suicidal or self harming behaviour; as depression often drives people to commit suicide or harm themselves physically.

METHODS: Joint representations are obtained by fusing the individual vector representations of multiple modalities from user’s social media feed: text, image and videos. These vector representations are in turn obtained through state of the art approaches for each modality e.g. VGG-16 for feature extraction from images, word2vec for text and Faster R-CNN on video frames. These joint representations are used to obtain weighted average score which can be used for making the final classification using the Softmax prediction layer.

SIGNIFICANCE AND IMPACT: To the best of our knowledge, this is the first research where the use of deep learning techniques has been proposed for real time detection of onset of depression and suicidal behaviour by analysing multimodal user generated content.

Item Type: Article
Uncontrolled Keywords: Multimodal Deep Learning, Social Media Analysis, User Generated Content, Mental Health, Depression, Suicidal and Self harming Behaviour, Psychology
Subjects: Q Science > QA Mathematics > QA75 Electronic computers. Computer science
QA75 Electronic computers. Computer science
Depositing User: EAI Editor II.
Date Deposited: 04 Feb 2021 14:26
Last Modified: 04 Feb 2021 14:26
URI: https://eprints.eudl.eu/id/eprint/950

Actions (login required)

View Item View Item