Novel AI to avert the mental health crisis in COVID-19: Novel application of GPT2 in Cognitive Behaviour Therapy

The effect of the COVID-19 pandemic on mental health is substantial. The World Health Organization has called for action to avert an impending mental health crisis. To respond to this call, this paper contributes a novel application of Deep Learning in Natural Language Generation (NLG) to seed healthy thoughts for mental health therapy. For the 1 st time in literature, a transfer learning capable large neural network with more than 100 million parameters for a NLG based mental health therapy application is proposed & demonstrated. This AI is designed to address scalable impact for millions of families with a timely health intervention in a privacy-safe approach. To the best of our knowledge, this is the first research paper to apply GPT2 (Generative Pretrained Transformer) for Cognitive Behavior therapy (CBT). Further, the paper demonstrates the proposed neural network architecture with a lab prototype implementation with reproducible results. This paper demonstrates this AI’s ability to generate conditional synthetic human-like text intended to seed a healthy mental outlook. This is accomplished by fine tuning a pre-trained GPT2 language model. The source code and video demonstration is contributed at https://sites.google.com/view/ai-in-mental-health.Also, for the 1 st time in literature, a novel idea of NLU (Natural Language Understanding) activated NLG therapy is demonstrated with reproducible results using a BERT based classifier to activate the GPT2 based therapy. Performance of GPT2 models of three different sizes (124, 355, 774 million parameters) was the same for a very small dataset, thus a small GPT2 model is suggested for on-device AI inference. This AI is a step forward in responding to WHO’s call for action to avert the crisis. Towards addressing all the three dimensions of the monumental challenge, the paper designed a novel AI architecture by taking advantage of both BERT & GPT2. It also demonstrated the feasibility of Transformers-based AI for developing a mental health therapy solution. Further, this paper contributed an open-source AI prototype to support research communities to transform global mental wellness.