International Journal of Mental Health Promotion

DOI: 10.32604/ijmhp.2022.015706

ARTICLE

Digital Social and Emotional Literacy Intervention for Vulnerable Children in Brazil: Participants’ Experiences

Louise Freeland1, Michelle O’Reilly2, Juliana Fleury3, Sarah Adams4 and Panos Vostanis1,*

1Department Neuroscience, Psychology and Behaviour, University of Leicester, Leicester, LE1 7RH, UK
2Department of Sociology, University of Leicester, Leicester, LE1 7RH, UK
3Associação pela Saúde Emocional de Crianças (ASEC), São Paulo, 01000, Brazil
4Department of Education, University of Leicester, Leicester, LE1 7RH, UK
*Corresponding Author: Panos Vostanis. Email: pv11@le.ac.uk
Received: 03 January 2021; Accepted: 08 March 2021

1  Introduction

Social emotional literacy (SEL) can be defined as the ability to understand, empathize and express emotions [1]. Growing evidence illustrates that children with developed SEL skills typically have a greater resilience to adversity and stressors, and are more likely to achieve positive life outcomes [2]. SEL empowers children to learn and grow socially, thus respond adaptively to negative life experiences [3]. Conversely, a significant association has been established between impaired SEL skills with emotional, behavioural or social difficulties, which can persist into later life without early intervention [4]. The likelihood of experiencing such mental health difficulties tends to be more pronounced among children living in disadvantage and those exposed to various traumatic experiences like maltreatment [5].

1.1 SEL Interventions in High-Income Countries

Although SEL has long been viewed as central to children’s adaptive development, recent years have seen increasing interest in its preventive functions. SEL skills have, therefore, been incorporated in mental health promotion programmes, predominantly implemented through schools [6,7]. A range of approaches have been developed to target SEL, some of which are universal in scope, whilst other target children at risk of developing mental health difficulties [8–11]. Evidence indicates variable degrees of improved outcomes in enhancing well-being and academic attainment, promoting the development of social and emotional skills, and reducing mental health difficulties and risk behaviours [12]. Furthermore, through the active involvement of parents, impact can be generated across different life domains [13,14]. This evidence largely originates from high-income countries, although some programmes were designed to be needs-led and culturally sensitive, thus demonstrated positive effects among disadvantaged and ethnic minority children [11,15].

1.2 Evidence from Low- and Middle-Income Countries (LMIC)

More than 80% of the world’s population of children live in LMIC and are exposed to a range of risk factors for poor mental health such as poverty, malnutrition and lack of resources [16]. Meeting children’s mental health needs is further compounded by stigma, lack of culturally appropriate interventions, and limited resources [17]. Yet, evidence suggests that SEL can also function as a protective factor in the face of contextual adversities [18,19]. Limited evaluation of SEL interventions in LMIC established positive psychosocial outcomes along different life domains [20]. For example, in India children reported stress reduction through effective techniques to manage stressful situations; whilst their parents observed transferability of benefits to the home environment, and teachers reported improved academic attainment [21]. To achieve beneficial outcomes, stakeholder involvement is particularly important in LMIC in engaging communities and families, and in challenging stigma [22]. Schools have the potential to reach vast numbers of children, who find it difficult to access appropriate help [23,24]. Increasing opportunities through digital technology can further enhance reach.

1.3 Application of Digital Technology in SEL Interventions

In recent years, the use of digital technology has led to the design, implementation and evaluation of tools that can improve children’s learning, cognitively, emotionally, and socially [25]. Consequently, digital approaches are increasingly being used as stand-alone or to supplement face-to-face interventions. Indeed, different digital mental health interventions have been shown to variably improve mental health awareness, emotional literacy, stress management, problem-solving skills, coping strategies, and help-seeking [26]. Craig et al. [27] found that an interactive online game could translate SEL strategies into social problem-solving in the virtual world, thus improve emotional regulation and impulse control. Digital modalities can help tailor interventions to children’s individual needs [28], and enhance the benefits of face-to-face learning [29]. Emerging findings indicate that guided facilitation (face-to-face or web-based) is a key predictor of engagement with and efficacy of remotely accessed programmes [30].

Although children in LMIC, especially those living in disadvantage, are often faced with additional barriers to accessing digital technologies, such provision has the potential to compensate for adequate service provision on the ground. Growth in mobile phone usage and internet access in LMIC can enable children with mental health problems to reach support that was previously unavailable to them. Despite the global ‘digital divide’, the gap is closing over time [28]. For example, mobile phone subscriptions exceed 80% of the population in Latin America, Africa and South Asia countries [31]. Technological support systems can thus enable children and their families to overcome barriers towards seeking mental health support in LMIC such as stigma, distance, transport, or lack of resources.

In summary, there is a gap in the implementation of SEL interventions in LMIC, but also an opportunity to enhance their accessibility and uptake through digital approaches. This research gap informed the rationale for this study.

2  Methods

The aim of this study was to explore the perceptions and experiences of children, parents and facilitators of the potential value of addressing social emotional literacy via a digital intervention. Two core research questions underpinned this aim:

1. To what extent did these key stakeholders experience that social emotional literacy needs could be addressed through digitally-mediated interventions?

2. What were the perceived challenges and benefits of a digital adaptation of an existing social emotional literacy intervention?

2.1 Sociocultural Context

In Brazil, 32 million children and young people (or 60%) are affected by poverty [32]. During 2017, 11,804 adolescents were assassinated (32 on average per day), the highest number globally in absolute terms. Children under 13 years account for more than half of the increasing cases of sexual victimization. While access to education is high, there are still two million children aged 4-to-17 years out of school, with an additional 3.5 million students dropping-out or repeating their grade annually. Several studies established high rates of mental health needs among children and young people in the country [33], which were related to the above risk factors [34,35]. There has been a steady increase in youth suicide rates in recent years, with the highest rates among males aged 15–19 years, and with a strong association with unemployment [36].

2.2 Participants

In total, 208 children aged 6–10 years took part in the intervention. These children were invited to apply learning provided by the intervention through the networks of a host non-governmental organization (NGO) based in Rio de Janeiro, Brazil, and their sponsors, by digital invitation and social media. Following the delivery of the intervention described below, a sample of children who had completed the course (target population n = 120), their parents, and facilitators (volunteer psychologists, total n = 17) were invited to participate in the study. We adopted a purposeful sampling frame in inviting consecutive children and their parents, until the selected children’s sample reached thematic saturation [37]. All children lived with at least one parent within their household. Parents provided written informed consent, whilst children gave additional verbal assent to participate in the study. Thirteen children attended remote semi-structured interviews with an independent researcher. In addition, nine parents (mothers) and nine facilitators (volunteer psychologists) were interviewed separately on their experiences and perspectives. At the time of the study, all participants were isolating due to the COVID-19 pandemic lockdown.

2.3 Social Emotional Literacy Intervention

We created an intervention based on the broad principles of the school-based approach ‘Zippy’s Friends’ [38], which was infused with a transactional model of stress and coping [39] and the circle time theory of fostering a school ethos that respects children and increases their self-worth [40]. This is one of the largest internationally used school-based SEL interventions, which has been implemented in over 50 countries worldwide and translated into 12 languages. The objective of the intervention is to improve children’s social emotional literacy, problem-solving and coping skills, thus strengthen their resilience when faced with life stressors [41–43]. It is usually delivered by teachers, and can incorporate activities outside of the classroom, e.g., at home collaboratively with the family, to enable transferability of skills.

2.3.1 Development and Facilitator Training

The intervention was delivered by a non-governmental organization (NGO) in Brazil by volunteer psychologists (facilitators). This NGO operates through a global network of 33 members to promote children’s mental health. The NGO nationally provides a range of preventive activities, largely in the school environment and in vulnerable contexts, as well as developing teaching capacity. The intervention was tailored, adapted and informed by the theoretical framework, philosophy and principles of a school-based SEL (skills for life) programme delivered face-to-face over the years by the host NGO. The programme included different components for different age groups.

The intervention was also tailored by the NGO provider to the context of the COVID-19 pandemic, by adapting its content, to enable children to deal with specific stressors related to the pandemic such as fears of illness and loss, and self-isolation; and its format to a digital mode, to ensure safe delivery. All facilitators received the same one-day training by an experienced NGO trainer member, and three follow-up support sessions throughout the implementation of the intervention. Additionally, all were provided with learning resources and manualized guidelines to ensure fidelity.

2.3.2 Delivery of Intervention

Every three participating children were allocated to one small group and one facilitator. Facilitators were not previously familiar with the children, and each was involved with one small group. Their preparation, training and support are described above. All children received the intervention via a personal computer or by smartphone. There were no required specifications of the computers, other than having an accessible webcam device and internet access. The digital platform Whereby (similar to Zoom) was used. As the intervention was implemented during a period of lockdown, all children attended remotely from their home environment. Not all children attended all the sessions, occasionally resulting in some smaller groups.

The intervention was designed to include nine weekly 45-minute sessions. In contrast with previous face-to-face school-based programmes involving large groups of children, often at classroom level, the digitally adapted intervention targeted small groups of three children and one facilitator (volunteer psychologist). By running the intervention in small groups it was hoped this would maximize opportunities for children to express themselves, practice activities, and interact with each other as well as with the facilitator. Each session focused on a particular theme, in the following sequence: 1–importance of dealing with one’s feelings; 2–sadness; 3–anger; 4–missing favourite situations and dealing with fears; 5–empathy and solidarity with others; 6–communication of feelings; 7–preparing for return to school; 8–facing changes; and 9–love and gratitude.

Up to the fifth session, the aim was for children to identify and articulate their feelings, and to amplify coping skills and internal resources to promote self-care. The next three sessions broadened and generated these skills and strategies in a social context, i.e., being empathic to other people’s feelings, communicating their strategies to both help and receive support, and adapting to external changes within their peer group. In the final session, children revisited their feelings and learning points, and concluded by focusing on positive emotions such as love and gratitude. Each session had a similar structure used in the Zippy’s Friends intervention [38] and other digital modular mental health promotion programmes [29,44]. It started with an icebreaker and welcome, followed by a brief review of issues arising from the previous session. The majority of the session focused on its core theme within a virtual circle time, and a film, photo or play activity to generate examples, discussion and practice. The session finished with an invitation to generate a fun activity at home, and children sharing their reflections within the group.

2.4 Analysis

To promote a person-centred focus of analysis that accounted for the existing knowledge base, we utilized the codebook thematic approach, as this is a category-based coding process that prioritizes participant narratives [45]. Three coders independently worked through the data and, via a verification process and open dialogue, merged the data into a final coding frame. In total, the process resulted in three overarching themes, related subthemes, and 42 operational codes.

3  Results

The emerging themes and subthemes are summarized in Table 1 below. These are discussed in detail, with supporting quotes by different stakeholders.

3.1 Theme 1: Empowerment

Young participants discussed extensively how they felt empowered, especially through the group process. They described enabling factors, which are common to a range of group modalities. All children stated that they felt comfortable with the intervention being offered online. Some parents and facilitators attributed engagement to children already being versed in online formats.

“I felt comfortable, I felt heard.”Child 1

“I saw him very comfortable, expressing his feelings, and talking about things that had nothing to do with the subject.”Parent 1

“As the sessions went by, she became more comfortable.”Facilitator 1

Children reported feeling listened to by both the facilitator and their peers. Their comments included their ability to express difficult emotions and not being judged. In relation to the online platform, they perceived that the facilitators were sensitive to these moments, and that they adapted digital features to promote children listening to each other.

“I felt like I could say anything that came to my mind. I felt I could vent. I vented to her a lot there.”Child 6

“Whenever we wanted to speak, (facilitator) would tell everyone to mute their microphones to let the other person speak.”Child 5

This facilitation and group ethos fostered a sense of inclusion and respect between the children. Consequently, they stated that they developed a sense of belonging, which was a pre-requisite to working through difficult personal issues.

Most children stated that the online format was used sensitively, in a way that still made them feel welcome; thus, despite not being in the same room, they still felt the presence of each other online. Consequently, they were able to display the same respect that they would in a classroom, by listening and taking turns. Parents corroborated that their child was enabled to feel like they belonged in the group, and that this was largely down to the facilitator’s skills in online engagement. Facilitators reported that, although many of the children did not know each other before the programme, they allowed each other time to speak and waited for their turn, even though their interactions were not face-to-face.

“I felt like I belonged.”Child 2

“I thought each of them had their own place, their own value.”Parent 7

“The counsellor did not leave anyone aside. She interacted with the three of them equally.”Parent 1

“I felt they respected each other, giving each other time to speak, talking to each other. One child found it more difficult, but faced her difficulties.”Facilitator 6

All three stakeholder groups claimed that the children were made to feel supported by both their peers and the facilitator throughout the online sessions. In turn, such mutual support encouraged them to share openly with the group and removed any potential barriers to communication related to the remote platform. Children thus gradually took a more active role in the discussions and activities.

“They helped each other. The children were also facilitators.”Facilitator 1

Child and adult participants shared their perceptions of whether and how the intervention had positive or negative impact. Having fun together, playing and making friends were key therapeutic components. However, some children remarked that, whilst virtual sessions were enjoyable, given the choice, they would opt for face-to-face interactions.

“They (sessions) were cool.”Child 3

“When it was face-to-face, we did not play video games, we just played. We played with our erasers and pencils at school. It is nicer to play face-to-face.”Child 5

Although this programme was not designed to regularly involve parents, they were given the option to be present during the sessions. This seemed to result in mixed responses on engaging most children, whilst occasionally inhibiting some. Most facilitators stated that the child’s parents were available should they be needed, but they also respected the child’s privacy.

“One of the girls kept leaving to ask for her father’s help.”Facilitator 2

“A girl insisted on her mother being there with her. The little boy, in turn, was very bothered by the presence of his mother. At the end, his mother told me he was very closed off at home.”Facilitator 2

“The family was around but they did not interfere, they did not distract them… She then realised that her daughter needed to have this private space. Our meetings were never interrupted, I think the mother understood that the girl was having a hard time speaking about certain things, but since she spoke to me, the mother was very respectful of this time we spent together.”Facilitator 6

There appeared to be a divide between the parents on their participation. Some parents felt that the intervention was aimed at the child, and they should be given privacy to engage, without feeling uncomfortable. In contrast, other parents actively joined in with the sessions and included themselves in different activities.

“I left her alone, I chose the room she would be at, and then I would leave her by herself. I would give her some space, so we could talk about it later.”Parent 2

When parents were involved during or between sessions, they encouraged children to practice activities at home, and adapt those to their developmental capacity or their home environment.

“The parents helped to adapt the material by giving every-day examples, saying ‘remember that day we did such a thing?’ …the smaller children needed help with the language.”Facilitator 4

Interestingly, even if this was not a planned primary therapeutic objective, most children felt that they had used the lessons learnt from the intervention to improve their relationships with their family. Some children covertly adopted new strategies, whilst others communicated with their family what they learnt from the sessions, who then reinforced their application at home. Some children taught their family what they had learnt from the intervention and described how they were then able to help them emotionally. Generalizable skills usually related to emotional regulation and dealing with anger with peers and parents.

“I wasn’t very grateful before (with my sister), you know?”Child 2

“When we were angry, we would draw. My mother was the one who taught me that…we learnt that in the meetings.”Child 9

“My mother said she used the ‘smelling the flower and blowing the candle’ idea…my father, I am not sure, but he is in the kitchen, and my sister used the ‘smelling the flower and blowing the candle’ too.”Child 5

Other children were content to practice activities on their own at home, without being prompted or involving other family members.

“What really impressed me was…the concepts she learned: empathy, solidarity…she wrote down everything she learned, and she has put it into practice in her day-to-day life.”Parent 2

3.2 Theme 2: Participatory Aspects

The intervention normatively included participatory activities, to facilitate children’s engagement during the sessions. These activities required some adaptation to be delivered online. Nonetheless, children and facilitators generally received them positively and commented on the modes, activities and ways they had to be adapted to engage the children to convey therapeutic messages, and encourage them to practice and generate skills beyond the sessions. Activities were used interchangeably at the beginning of each session as warm-up tools, i.e., to promote interaction, sharing and rapport-building with the facilitator. They became more focused as the session progressed, again without losing their participatory functions and fun.

“That activity where they had to choose an object in the house and explain why they liked it, I thought it helped break the ice. I also chose something from my house, and I talked about myself. And that activity helped us connect.”Facilitator 6

“In general, the children really liked the whole framework. We started off with some relaxation, then a chat, a conversation circle. Then, there was some teaching, some new word for them to add to their vocabulary or to reframe, and finally some other activity. They really enjoyed it.”Facilitator 4

An important aspect of the intervention was that it drew on storytelling, with hidden meanings to engage children with the content of each session. Most children appeared able to connect with the emotional literacy lessons from these stories. Facilitators recognized the emotional effect and connection that the stories successfully had on the children. Whilst some addressed that a story itself may not have always been appropriate for an individual child, the emotional connection that the child had with it, by relating to their earlier experiences, was still apparent.

“There was the story of the angry chick…the boy’s mother told him what he had to do. One day he ran away from home. Then his mother got really upset and only found him the next day…she was feeling sad because he had run away. He was feeling sad too, because she kept telling him what he was supposed to wear, what he was supposed to do.”Child 6

“There was a lesson in which I told a story and he had to continue, the story of a dinosaur who was friends with a boy. The dinosaur was a monster, and the boy became friends with him, and realized that the dinosaur was big in size but was a good friend. He was able to establish that connection. He realized little by little that the boys were not that monstrous. He would always bring up the story of how he was bullied. He said he had seen a counsellor in the past, but they didn’t do as many activities. I think the activities awakened something in him.”     Facilitator 3

“(name of child) went on and on about the loss of a puppy…she was very emotional. The red chick’s story, although it was not an engaging theme for her because of her age, really touched her. It brought back memories and emotions.”     Facilitator 2

Take-home messages were often built around games. Children could identify with those games, which in turn increased their interactions. Many parents observed their children repeating or completing the games outside the sessions, and felt that this gave them opportunities to reinforce therapeutic objectives by joining in with their child.

“What I liked most was the games, and what was most useful I think was to be entertained. (name of sponsor) is great, and by doing this project, they entertained the children during the lockdown.”     Child 4

“The games, the interaction…they sang songs, they danced…always working on their emotions. All the activities focused on emotions. By the way, she was having a hard time with a friend from school, they were not getting along well, and she was able to let it all out during the meetings.”     Parent 5

Interestingly, facilitators observed that children often used games proactively to demonstrate, re-enact or process a topic that made an impression on them.

“The acting exercise was the most surprising to me. I said we were getting to the end of the sessions and asked them to think of one activity they had truly enjoyed, and to tell me during the next session…so, they told me through a short play. They worked hard, they acted it out, it was awesome!Facilitator 6

Children appeared particularly engaged with the emotional elements involved in the arts and crafts activities, and found them to be particularly useful. These were also easy to reproduce at home when experiencing similar emotions. Both parents and facilitators warned that some children’s anxieties about technology or creative skills involved might put them off using them. Consequently, they stressed the importance of direction and support, within and between sessions. Also, remaining flexible if a child found a particular activity difficult to follow.

“When I’m sad, when I’m angry, when I’m missing someone or something, I make a drawing and that calms me down.”Child 4

“We asked for drawings, these types of things, one child always did it. Another, the one who missed the meeting, she was a little more disorganized, I believe she would need someone with her to follow-up on her for her to do the assignment.”     Facilitator 7

“I had a child who didn’t want to participate in the relaxation session.”Facilitator 4

Overall, parents felt that the children were able to understand and engage with the emotional meanings behind the activities. One parent became aware that one activity affected her child in a way that she evaluated her own parenting. In contrast, a risk was highlighted in some parents misconstruing the purpose of the activities, and/or linking them to other tasks and behaviours. This could result in children perceiving the activities negatively and disengaging from the intervention. A practical implication is the importance of ongoing clarity, communication and consistency between facilitators and parents.

“I overheard part of the conversation and I noticed she managed to let her feelings out.”Parent 6

“I myself started to be more aware at home, to take a closer look at my behaviour as a mother. This activity made me pay more attention to this.”     Parent 6

“Today he will learn to customize a T-shirt, so he keeps asking about it. But if I ask him to do his schoolwork, he never wants to.”      Parent 4

Some facilitators commented on the participatory aspects of the intervention in relation to their own training. Most found the materials and their usage of language well suited to the objectives, which did not usually required any modification. Such training issues are important in easily sharing and rolling out the intervention in the future.

“The material was very easy to work with. It used appropriate language, was very well explained and well developed, delightful to work with.”      Facilitator 3

3.3 Theme 3: Digital Adaptation

In the third theme, participants explored how the intervention had been adapted for a digital format. The participants considered the challenges of delivering an intervention remotely and how this might compare to face-to-face. From this discussion, the participants were able to suggest several recommendations for how to improve the intervention in the future.

Many children felt that the online platform did not compromise the verbal support offered during the sessions. Nevertheless, some children also experienced that physical aspects of this support were not digitally translatable.

“I think the advantage, one of them, is that I met new people, new things…we talk about the same things, whether we are on the computer or not.”Child 2

“It’s different, because we can’t hug our friends…I think…contact…we should create a virtual application for ‘high five’ to greet a friend.”Child 7

Facilitators generally approached the online aspects with an open mind. However, those who had not previously delivered an intervention remotely initially found this notion alien. Their main concern was whether they could provide the same quality of care online. They were also anxious about the technical aspects, i.e., whether they could master the technology and whether it would work during the session.

“Counselling children in a face-to-face setting is one thing. I wondered what it would be like to do that online.”Facilitator 2

“Except for connection problems, which sometimes require that you repeat something that was said.”     Facilitator 9

After completing the intervention, facilitators had largely changed their views about providing support digitally by discovering it was just as useful as it previously was in person. One facilitator described how they perceived online support as overcoming certain practical barriers of face-to-face sessions, as the online medium was more accessible to many children, who were already well-versed in it.

“I don’t see any difference, there is no difference in the way they report their emotional pain. When patients speak through a remote channel, it is the same as if they were speaking to you in person”.      Facilitator 9

“This generation is already immersed in the technological world.”     Facilitator 9

Most parents shared children’s and facilitators’ impressions of the online benefits. In addition, some parents valued the opportunity of children receiving support while confined at home during the unusual situation of the pandemic. This approach could improve access and use, as being time- and resource-effective.

“I can only see the advantages. When the meetings started, his school was not yet giving online classes. His online classes started the following week and he already had that experience.”Parent 1

“I believe it’s a good thing, particularly because of the time constraints imposed by the daily rush in our lives. I have a very busy day-to-day, I work nights, and end up having to rush against time. So, I think it’s a good idea, not having to stop and drive to the counsellor’s office. This virtual world has been helping a lot. He would have never had the time to go there.”      Parent 3

In relation to different web-based and social media platforms, all children were comfortable with using their mobile phones and social media to socialize with others and engage in activities. Some also interacted with their parents this way, whereas others were discouraged from using them at home. Participating parents reflected these mixed attitudes. Some parents raised how schools were using digital technology to ensure that children continued to receive education remotely during the pandemic. However, some parents actively encouraged their child to stay in touch through technology, whilst others felt that the increased use of technology was a negative influence at the expense of usual human interactions and social learning.

“For example, playing a game, we enter the meeting through Zoom and my mother, my father, my godmother, my godfather, my grandmother, everyone joins! We even threw a party on the computer these days.”Child 4

“At school, they are using Google Classroom; for the English classes, Zoom.”Parent 5

Facilitators described digital technology they had previously used to enhance their skills. They highlighted how the current climate encouraged them to begin applying technology to their practice, by enabling them to continue working remotely. Participants listed different forms of social media, and how they adapted them to suit specific areas of their work.

“I am using WhatsApp, I know it is not the safest, but it is the easiest for my clients. I use Skype. And I use Instagram with a patient who was based in Massachusetts.”Facilitator 8

“Many people only have WhatsApp and video calls, others also have Skype, Zoom, Meet, Hangout. Some people say they didn’t even know that this form of social interaction, of meeting, existed before. I think these technologies, these Apps, have suddenly boomed. So many people are using them now!”Facilitator 5

“The children watch TV, YouTube, Netflix. She has WhatsApp too, but she prefers these other Apps. Sometimes she chats with her cousin on WhatsApp.”Facilitator 8

Despite the largely positive responses to the digital adaptation, all stakeholder groups drew comparisons with face-to-face support. Given the choice, most children favoured the latter option. Reasons for their preference included more intimate and meaningful interactions through the physical presence of others, sharing fun activities, and being able to communicate more openly. When explored further, these reasons largely related to being able to read and share emotions, which was at the core of the intervention objectives.

“Real life is better, because we can see people’s faces, look into their eyes.”Child 8

“In real life, you can talk in different ways, with affection…on the computer, you can only talk.”     Child 8

“Chatting in real life is much cooler.”Child 2

These views were shared by parents, albeit to a lesser extent.

“If it was face-to-face, she would open up a little bit more.”Parent 2

“I think it’s about human warmth, physical interaction, not being able to hug, to be close to others…the only thing missing is being able to be close to others.”Parent 5

Facilitators approached the delivery mode from different angles. Some participants stated that face-to-face interactions enabled them to observe more subtle non-verbal postures and behaviours, create a more active group dynamic, and feel more at ease with their role. Nevertheless, others concluded that learning and intervention fidelity were not compromised online. A pragmatic view was that whatever the constraints, digital delivery offered opportunities to reach more children, especially in times of crisis.

“A face-to-face meeting is more enjoyable, I feel more comfortable.”Facilitator 7

“Face-to-face sessions are warmer, they have a specific energy. You can look them in the eye, read their body language”.     Facilitator 1

“In terms of learning, it is possible, because they showed they were interested, they interacted, I didn’t feel any resistance from them from being on a screen.”Facilitator 7

The exploration of perceived impact, previous experience of interventions, and alternative models of delivery led to stakeholder views on needs for additional supports, for the intervention as a whole and for individual children. Children may have had recognized needs that required further help or specialist services or, as in the case of the parent below, needs became apparent because of the SEL intervention.

“The course helped me realize that I had to look for a counsellor…in one of the sessions, the counsellor asked him what made him sad and what made him happy. He recalled a fact that marked him and started to cry. I realized he misses those who have passed away. This led me to look for another counsellor to help him, because I noticed he was stifling his feelings and needed help.”      Parent 1

Similarly, some parents recognized the importance of being actively involved to generate positive outcomes at home, or to address their own emotional issues. The COVID-19 pandemic was an example of collective and ongoing trauma exposure.

“Maybe offer something to family members, allowing caretakers to also take part in the sessions, because adults are also suffering with this situation and, in these sessions, perhaps they can find ways to cope with their own emotions and also help children cope”.Facilitator 9

4  Discussion

This study explored the views of children, their parents, and facilitators on their experiences of a digitally adapted SEL intervention during the first COVID-19 pandemic lockdown in Brazil. Participants’ perspectives related to the empowering components, participatory tools, benefits and challenges of online delivery, perceived impact outside the sessions, and recommendations on its future use. Children articulated a detailed understanding of what the intervention involved. They were described by adults as often being pro-active in co-facilitating activities during the sessions, and applying new strategies at home. Crucially, children appeared to generate strategies outside the sessions, during the prolonged stress-inducing period of the COVID-19 pandemic.

Indeed, children described a range of empowering factors such as being listened to and not being judged. They particularly elicited group processes like respect, inclusion, sharing and belonging, which were transferrable to a small online group through sensitive and skilled facilitation. These factors have been found to predict both engagement and positive outcomes through different programmes [46]. Children also focused on the specific objectives of SEL interventions in understanding, expressing, and regulating emotions [47]. These newly learnt adaptive strategies were viewed as important in making impact outside the intervention, and some parents corroborated that children applied those at home; for example, when feeling sad or lonely, or to resolve arguments.

The digital adaptation was an important aspect of the intervention, being responsive to global change and public health need. Children’s digital literacy skills were useful in such an adaptation, as they were well-versed in digital technology, and a range of media at home and school. Facilitators initially held some reservations on the transferability of the SEL approach, but were largely more positive after its implementation, because they implemented all participatory tools and activities (stories, games, drawing and videos) as originally planned. The development of similar interventions and staff training should thus ensure the combination of technical confidence and programme fidelity.

Although parent involvement was not built in this intervention, it seemed to occur ad hoc for different reasons and with mixed experiences. As families were inevitably at home because of the pandemic lockdown, some parents were variably present or involved to offer re-assurance or get ideas on how to help their child, whilst others viewed it as the child’s private space. Nevertheless, there were some unintended perceived benefits in parents observing more closely their child’s emotional processing, reinforcing new skills, being actively involved after the sessions, and reflecting on their parenting role. We should also acknowledge certain risks if parents are not purposefully involved from the outset, predominantly in misconstruing the purpose of some activities, or using them selectively as reward or punishment for their child’s behaviour. As previous studies established the positive impact of parent involvement in SEL interventions [48] and, considering parents’ positive outlook to technology in this study, such an extension of the intervention would be valuable, with a degree of direct contact between parents and facilitators.

Overall, and despite the positive responses to the digital adaptation and the small group size of three children per facilitator, children and facilitators still favoured face-to-face interaction, because of the advantages of sustained eye and emotional contact, following non-verbal clues, and more spontaneous interactions. These findings are not necessarily mutually exclusive, instead they highlight the potential for combined, engaging and resource-effective approaches. Face-to-face and digital elements could thus be complemented for different levels of needs. For example, SEL activities could be introduced universally online, whilst children with more complex needs such as emotional dysregulation may require direct facilitation, for example, at schools, community or care settings.

These findings need to be interpreted in the context of certain limitations. The study was conducted under the unique circumstances of the COVID-19 pandemic, although many lessons can be adapted post-pandemic. The sample carried certain characteristics, notably having access to NGO networks, volunteering to receive the intervention, and capacity to use a digital platform (laptop, tablet, or mobile phone). Consequently, modified approaches, engagement strategies and digital infrastructure should be designed for disadvantaged groups of children. The study captured stakeholders’ perspectives, especially children’s voices. A quantitative approach is also required to measure outcomes such as in children’s emotional regulation, coping strategies and wellbeing, in line with the objectives of a SEL intervention [5,7]. Future research could consider the influence of the intervention ‘dosage’, as the exact number of sessions attended by each child were not made available to the research team in this qualitative study. Further studies should also evaluate the adaptation of this intervention for different types of literacy, and its implementation with different vulnerable groups such as refugee children. Although no culture-specific suggestions were made by the participants in this study, implementation in various sociocultural contexts would improve our understanding of requirements for cultural adaptation [49]. Indeed, previous research showed that SEL programmes can be tailored to different cultural groups [17].

Evolving approaches under the implementation science framework can increase our understanding of processes underpinning such interventions [50,51]. These approaches entail establishing whether an intervention meets a number of pre-defined outcomes and identifying mechanisms through which these outcomes are achieved for certain groups and in certain contexts. Implementation science is particularly appropriate for complex interventions in global health settings [52,53], by exploring whether, how and in which contexts pre-determined multiple objectives are met. Emerging findings can thus lead to the development of a programme theory by uncovering generative mechanisms that lead to these outcomes [54,55].

The intervention was provided during an unprecedented global crisis, when schools had closed, and children were confined at home. Using digital technology, it equipped children with strategies and coping skills, thus bridged an educational and service gap during this challenging period. Lessons arising from this study can inform future mental health promotion and responsive programmes beyond these unique circumstances.

Indeed, the body of available evidence indicates that children can build coping strategies, social skills, and resilience through a range of SEL interventions [56]. These can be adapted to digital formats, which offer a wider reach to large groups of children, as long as they can access online platforms, predominantly through their schools [57]. It has been shown that the appeal of digital interventions such as based on casual video games [58,59] increases children’s engagement and uptake [60]. Co-design with young people enhances their appeal [61]. These interventions can be implemented in advance of stressful experiences such as exams [62] and in response to stressors [58,63].

A blended facilitated approach could maximize staff capacity, whilst maintaining some direct contact with children. Skilled facilitation and building alliance with the facilitator are strong predictors of child and youth engagement and outcomes, both in face-to-face and digital formats of delivery [30,64]. Therefore, such adaptations should not be viewed as ‘cheap’ or easy to deliver, instead they require investment in development, training, delivery, supervision, and evaluation. Blended SEL interventions could be adapted within a tiered model of universal mental health promotion, targeted prevention, and clinical interventions, with increasing face-to-face input by skilled staff at higher levels of need. As children have unique knowledge of both their emotional needs and the innovative use of technology, they should be central to the design and monitoring of these interventions.

In conclusion, these findings can inform the design and implementation of interventions at different levels. Similar approaches can be included in the school curriculum, as part of mental health promotion, as well in community-based preventive activities. Digital modules can ‘boost’ targeted interventions, in self-help or facilitated formats. Young participants can overcome geographical barriers by sharing experiences and learning across regional, national and even national youth networks. Specific attention and digital support should be given to groups who cannot easily access other resources, predominantly in LMIC and disadvantaged communities.

Parallel awareness initiatives maybe required in these contexts for children, parents, schools and communities to embrace the importance of SEL skills in relation to learning and resilience across different life domains. The involvement of youth, community and religious groups, as well as NGOs, in partnership with schools and agencies on the ground can help overcome such challenges and enhance children’s engagement. The objectives, content and activities of the intervention could be adapted to address different types of literacy such as multicultural, recreational or functional literacy; and other stressful conditions such as bullying, community violence or immigration.

Funding Statement: This study was funded by a research grant by the Leicester Institute for Advanced Studies in the UK. The adaptation and delivery of the intervention was supported by the Pampili Company in Brazil.

Conflicts of Interest: The authors declare that they have no conflicts of interest to report regarding the present study.

References

 1.  Steiner, C., Perry, P. (1997). Achieving emotional literacy. London: Bloomsbury. [Google Scholar]

 2.  Knowler, C., Frederickson, N. (2013). Effects of an emotional literacy intervention for students identified with bullying behaviour. Educational Psychology, 33(7), 862–883. DOI org/10.1080/01443410.2013.785052. [Google Scholar]

 3.  Goldberg, J., Sklad, M., Elfrink, R. (2019). Effectiveness of interventions adopting a whole school approach to enhancing social and emotional development: A meta-analysis. European Journal of Psychological Education, 34, 755–782. DOI org/10.1007/s10212-018-0406-9. [Google Scholar]

 4.  Reinke, W., Splett, J., Robeson, E., Ouffutt, C. (2008). Combining school and family interventions for the prevention and early intervention of disruptive behavior problems in children: A public health perspective. Psychology in the Schools, 46(1), 33–44. DOI 10.1002/pits.20352. [Google Scholar] [CrossRef]

 5.  Dvir, Y., Ford, J., Hill, M., Frazier, J. (2014). Childhood maltreatment, emotional dysregulation, and psychiatric comorbidities. Harvard Review of Psychiatry, 22(3), 149–161. DOI 10.1097/HRP.0000000000000014. [Google Scholar] [CrossRef]

 6.  Snell, T., Knapp, M., Healey, A., Goglani, S., Lacko, S. et al. (2013). Economic impact of childhood psychiatric disorder on public sector services in Britain: Estimates from national survey data. Journal of Child Psychology & Psychiatry, 54(9), 977–979. DOI org/10.1111/jcpp.12055. [Google Scholar]

 7.  Korpershoek, H., Harms, T., de Boer, H., van Kuijk, M., Doolaard, S. (2016). A meta-analysis of the effects of school-management strategies and classroom management programs on students’ academic, behavioral, emotional, and motivational outcomes. Review of Educational Research, 86(3), 643–680. DOI 10.3102/0034654315626799. [Google Scholar] [CrossRef]

 8.  Haddon, A., Goodman, H., Park, J., Crick, R. (2005). Evaluating emotional literacy in schools: The development of the school emotional environment for learning survey. Pastoral Care in Education, 23(4), 5–16. DOI 10.1111/j.1468-0122.2005.00346.x. [Google Scholar] [CrossRef]

 9.  Blackstock, K., Ingram, J., Burton, R., Brown, K., Slee, B. (2010). Understanding and influencing behaviour change by farmers to improve water quality. Science of the Total Environment, 48(23), 5631–5638. DOI 10.1016/j.scitotenv.2009.04.029. [Google Scholar] [CrossRef]

10. Humphrey, N., Wigelsworth, M. (2016). Making the case for universal school-based mental health screening. Emotional & Behavioural Difficulties, 21(1), 22–42. DOI 10.1080/13632752.2015.1120051. [Google Scholar] [CrossRef]

11. Barry, M., Clarke, M., Dowling, K. (2017). Promoting social and emotional well-being in schools. Health Education, 117(5), 434–451. DOI 10.1108/HE-11-2016-0057. [Google Scholar] [CrossRef]

12. Weare, K. (2015). What works in promoting social and emotional well-being and responding to mental health problems in schools? London: National Children’s Bureau. https://developyourchild.co.uk/wp-content/uploads/sites/21/2016/03/NCB-well-being-mental-health-report-2015.pdf. [Google Scholar]

13. Sheridan, S., Smith, T., Moorman, E., Beretvas, S., Park, S. (2019). A meta-analysis of family-school interventions and children’s social-emotional functioning: Moderators and components of efficacy. Review of Educational Research, 89(2), 296–332. DOI 10.3102/0034654318825437. [Google Scholar] [CrossRef]

14. Wong, B., Cripps, D., White, H., Young, L., Kovshoff, H. et al. (2020). Primary school children’s perspectives and experiences of Emotional Literacy Support Assistant (ELSA) support. Educational Psychology in Practice, 36(3), 313–327. DOI 10.1080/02667363.2020.1781064. [Google Scholar] [CrossRef]

15. Holen, S., Waaktaar, T., Lervag, A., Ystgaard, M. (2012). The effectiveness of a school-based programme on coping and mental health. Educational Psychology, 32(5), 657–677. DOI 10.1080/01443410.2012.686152. [Google Scholar] [CrossRef]

16. McCoy, D., Peet, D., Ezzati, M., Danaei, G., Black, M. et al. (2017). Early childhood developmental status in low and middle-incomes countries: National, regional and global prevalence estimates using predictive modelling. PLoS Medicine, 14(1). DOI 10.1371/journal.pmed.1002034. [Google Scholar] [CrossRef]

17. Patel, V., Saxena, S., Lund, C., Thornicroft, G., Baingana, F. et al. (2018). The Lancet Commission on global mental health and sustainable development. Lancet, 392(10157), 1553–1598. DOI 10.1016/S0140-6736(18)31612-X. [Google Scholar] [CrossRef]

18. Cortina, M., Khan, K., Fazel, M., Hlungwani, T., Tollman, S. et al. (2007). School-based interventions can play a critical role in enhancing children’s development and health in the developing world. Child: Care, Health and Development, 34(1), 1–3. DOI 10.1111/j.1365-2214.2007.00820.x. [Google Scholar] [CrossRef]

19. Sullivan, L., Simonson, R. (2016). A systematic review of school-based social-emotional interventions for refugee and war-traumatized youth. Review of Educational Research, 86(2), 503–530. DOI 10.3102/0034654315609419. [Google Scholar] [CrossRef]

20. Xu, T., Tomokawa, S., Gregorio, R., Mannava, P., Nagai, M. et al. (2020). School-based interventions to promote adolescent health: A systematic review in low- and middle-income countries of WHO Western Pacific Region. PLoS One, 15(3). DOI 10.1371/journal.pone.0230046. [Google Scholar] [CrossRef]

21. Parikh, R., Michelson, D., Sapru, M., Sahu, R., Singh, A. et al. (2019). Priorities and preferences for school-based mental health services in India: A multi-stakeholder study with adolescents, parents, school staff, and mental health providers. Global Mental Health, 6, e18. DOI 10.1017/gmh.2019.16. [Google Scholar] [CrossRef]

22. Engle, P., Fernald, L., Alderman, H., Behrman, J., O’Gara, C. et al. (2011). Strategies for reducing inequalities and improving developmental outcomes for young children in low-income and middle-income countries. Child Development, 378(9799), 1339–1353. DOI 10.1016/S0140-6736(11)60889-1. [Google Scholar] [CrossRef]

23. Fazel, M., Patel, V., Thomas, S., Tol, W. (2014). Mental health interventions in schools in low-income and middle-income countries. Mental Health Interventions in School, 1(5), 388–398. DOI 10.1016/S2215-0366(14)70357-8. [Google Scholar] [CrossRef]

24. World Health Organization (2017). Global accelerated action for the health of adolescents: Guidance to support country implementation. Geneva: World Health Organization. [Google Scholar]

25. Savina, E., Mills, J., Atwood, K., Cha, J. (2017). Digital media and youth: A primer for school psychologists. Contemporary School of Psychology, 21(1), 80–91. DOI 10.1007/s40688-017-0119-0. [Google Scholar] [CrossRef]

26. Fullager, S., Rich, E., Webb, J. (2017). New kinds of (Ab)normal?: Public pedagogies, affect, and youth mental health in the digital age. Digital Age Social Sciences, 6(3), 99. DOI 10.3390/socsci6030099. [Google Scholar] [CrossRef]

27. Craig, A., Brown, E., Upright, J., de Rosier, M. (2016). Enhancing children’s social emotional functioning through virtual game-based delivery of social skills training. Journal of Child & Family Studies, 25, 959–968. DOI 10.1007/s10826-015-0274-8. [Google Scholar] [CrossRef]

28. Fairburn, C., Patel, V. (2016). The impact of digital technology on psychological treatments and their dissemination. Behaviour, Research & Therapy, 88, 19–25. DOI 10.1016/j.brat.2016.08.012. [Google Scholar] [CrossRef]

29. Shum, A., Lai, E., Lung, W. G., Cheng, M., Wong, H. K. et al. (2019). A digital game and school-based intervention for students in Hong Kong: Quasi-experimental design. Journal of Medical Internet Research, 21(4), e12003. DOI 10.2196/12003. [Google Scholar] [CrossRef]

30. Clarke, A., Kuosmanen, T., Barry, M. (2015). A systematic review of online youth mental health promotion and prevention interventions. Journal of Youth & Adolescence, 44(1), 90–113. DOI 10.1007/s10964-014-0165-0. [Google Scholar] [CrossRef]

31. Naslund, J., Aschbrenner, K., Araya, R., Marsch, L., Unutzer, J. et al. (2017). Digital technology for treating and preventing mental disorders in low-income and middle-income countries: A narrative review of the literature. Lancet Psychiatry, 4(6), 427–506. DOI 10.1016/S2215-0366(17)30096-2. [Google Scholar] [CrossRef]

32. UNICEF (2019). Brazil annual report 2019. New York: UNICEF. https://www.unicef.org/about/annualreport/files/Brazil-2019-COAR(1). pdf. [Google Scholar]

33. Fleitlich-Bilyk, B., Goodman, R. (2004). Prevalence of child and adolescent psychiatric disorders in Brazil. Journal of the American Academy of Child & Adolescent Psychiatry, 43(6), 727–734. DOI 10.1097/01.chi.0000120021.14101.ca. [Google Scholar] [CrossRef]

34. Bordin, I., Duarte, C., Ribeiro, W., Paula, C., Coutinho, E. et al. (2018). Violence and child mental health in Brazil. The Itaborai Youth Study methods and findings. International Methods of Psychiatric Research, 27, e1605. DOI 10.1002/mpr.1605. [Google Scholar] [CrossRef]

35. Barbosa, L. P., Quevedo, L., da Silva, G., Jansen, K., Pinheiro, R. T. et al. (2014). Childhood trauma and suicide risk in a sample of young individuals aged 14–35 years in southern Brazil. Child Abuse & Neglect, 38(7), 1191–1196. DOI 10.1016/j.chiabu.2014.02.008. [Google Scholar] [CrossRef]

36. Jaen-Varras, D., Mari, J., Asevedo, E., Borschmann, R., Diniz, E. et al. (2019). The association between adolescent suicide rates and socioeconomic indicators in Brazil: A 10-year retrospective ecological study. Brazilian Journal of Psychiatry, 41(5), 389–396. DOI 10.1590/1516-4446-2018-0223. [Google Scholar] [CrossRef]

37. Francis, J., Johnston, M., Robertson, C., Glidewell, L., Entwistle, V. et al. (2010). What is adequate sample size? Operationalising data saturation for theory-based interview studies. Psychology and Health, 25(10), 1229–1245. DOI 10.1080/08870440903194015. [Google Scholar] [CrossRef]

38. Sloan, S., Gildea, A., Miller, S., Poulton, L., Egar, C. et al. (2017). Evaluation of Zippy’s Friends for addressing socio-emotional and economic outcomes in six to seven year olds. International Journal of Educational Research, 82, 200–209. DOI 10.1016/j.ijer.2017.01.003. [Google Scholar] [CrossRef]

39. Lazarus, R. S., Folkman, S. (1984). Stress, appraisal, and coping. New York: Springer. [Google Scholar]

40. Mosley, J. (2005). Circle time for young children. London: Routledge. [Google Scholar]

41. Mishara, B., Ystgaard, M. (2006). Effectiveness of a mental health promotion programme to improve coping skills in young children: Zippy’s Friends. Early Childhood Research Quarterly, 21, 110–123. DOI 10.1016/j.ecresq.2006.01.002. [Google Scholar] [CrossRef]

42. Monkeviciene, O., Mishara, B., Dufour, S. (2006). Effects of the Zippy’s Friends programme on children abilities during transition from kindergarten to elementary school. Early Childhood Education Journal, 34(1), 53–59. DOI 10.1007/s10643-006-0104-0. [Google Scholar] [CrossRef]

43. Clarke, A., Bunting, B., Barry, M. (2014). Evaluation of a school-based emotional wellbeing programme. Health Education Research, 29(5), 786–798. DOI 10.1093/her/cyu047. [Google Scholar] [CrossRef]

44. Campos, L., Dias, P., Duarte, A., Veiga, E., Dias, C. C. et al. (2018). Is it possible to find space for mental health in young people? Effectiveness of a school-based mental health literacy promotion program. International Journal of Environmental Research and Public Health, 15, 1426. DOI 10.3390/ijerph15071426. [Google Scholar] [CrossRef]

45. Braun, V., Clarke, V. (2019). Reflecting on reflexive thematic analysis. Qualitative Research in Sport, Exercise and Health, 11(4), 589–597. DOI 10.1080/2159676X.2019.1628806. [Google Scholar] [CrossRef]

46. Hauber, K., Boon, A., Vermeiren, R. (2019). Therapeutic factors that promote recovery in high-risk adolescents intensive group psychotherapeutic MBT programme. Child & Adolescent Psychiatry & Mental Health, 13(2). DOI 10.1186/s13034-019-0263-6. [Google Scholar] [CrossRef]

47. Jones, M., Brown, L., Hoglund, G., Aber, L. (2010). A school-randomized clinical trial of an integrated social-emotional learning and literacy intervention: Impacts after one school year. Journal of Consulting & Clinical Psychology, 78(6), 829–842. DOI org/10.1037/a0021383. [Google Scholar]

48. Wilding, L., Claridge, S. (2016). The Emotional Literacy Support Assistant (ELSA) programme: Parental perceptions of its impact in school and at home. Educational Psychology in Practice, 32(2), 180–196. DOI 10.1080/02667363.2016.1146573. [Google Scholar] [CrossRef]

49. Getanda, E. M., Vostanis, P. (2020). Feasibility evaluation of psychosocial evaluation for internally displaced youth in Kenya. Journal of Mental Health. DOI 10.1080/09638237.2020.1818702. [Google Scholar] [CrossRef]

50. Blasé, K., van Dyke, M., Fixsen, D. (2013). Stages of implementation analysis. Chapel Hill: University of North Carolina. [Google Scholar]

51. Nilsen, P. (2015). Making sense of implementation theories, models and frameworks. Implementation Science, 10(53). DOI 10.1186/s13012-015-0242-0. [Google Scholar] [CrossRef]

52. Marchal, B., Dedzo, M. D., Kegels, G. (2010). A realist evaluation of the management of a well-performing regional hospital in Ghana. BMC Health Services Research, 10(24). DOI 10.1186/1472-6963-10-24. [Google Scholar] [CrossRef]

53. Mirzoev, T., Etiaba, E., Ebenson, B., Uzochukwu, B., Manzano, A. (2016). Study protocol: Realist evaluation of effectiveness and sustainability of a community health workers programme in improving maternal and child health in Nigeria. Implementation Science, 11(83). DOI 10.1186/s13012-016-0443-1. [Google Scholar] [CrossRef]

54. Hoffman, T., Glasziou, P., Boutron, I., Milne, R., Perera, R. et al. (2014). Better reporting of interventions: Template for intervention description and replication (TIDieR) checklist and guide. British Medical Journal, 348(1687). DOI 10.1136/bmj.g1687. [Google Scholar] [CrossRef]

55. Wight, D., Wimbush, E., Jepson, R., Doi, L. (2018). Six steps in quality intervention development: Theory and methods. Journal of Epidemiology and Community Health, 70, 520–525. DOI 10.1136/jech-2015-205952. [Google Scholar] [CrossRef]

56. Garwood, J. (2018). Literacy interventions for secondary students formally identified with emotional and behavioral disorders: Trends and gaps in the research. Journal of Behavioral Education, 27, 23–52. DOI 10.1007/s10864-017-9278-3. [Google Scholar] [CrossRef]

57. Chen, W., Zheng, Q., Liang, C., Xie, Y., Gu, D. (2020). Factors influencing college students’ mental health promotion: The mediating effect of online mental health information seeking. International Journal of Environmental Research and Public Health, 17, 4783. DOI 10.3390/ijerph17134783. [Google Scholar] [CrossRef]

58. Fleming, T., Stasiak, K., Moselen, E., Hermansson-Webb, E., Shepherd, M. et al. (2019). Revising computerized therapy for wider appeal among adolescents: Youth perspectives on a revised version of SPARX. Frontiers in Psychiatry, 10(802). DOI 10.3389/fpsyt.2019.00802. [Google Scholar] [CrossRef]

59. Fleming, T., Bavin, L., Lucassen, M., Stasiak, K., Hopkins, S. et al. (2018). Beyond the trial: Systematic review of real-world uptake and engagement with digital self-help interventions for depression, low mood, or anxiety. Journal of Medical Internet Research, 20(6), e199. DOI 10.2196/jmir.9275. [Google Scholar] [CrossRef]

60. Pine, R., Sutcliffe, K., McCallum, S. (2020). Young adolescents’ interest in a mental health casual video game. Digital Health, 6, 1–7. DOI 10.1177/2055207620949391. [Google Scholar] [CrossRef]

61. Fleming, T., Merry, S., Stasiak, K., Hopkins, S., Patolo, T. (2019). The importance of user segmentation for designing digital therapy for adolescent mental health: Findings from scoping processes. JMIR Mental Health, 6(5), e12656. DOI 10.2196/12656. [Google Scholar] [CrossRef]

62. Perry, W., Werner-Seidler, A., Calear, A., Mackinnon, A., King, C. et al. (2017). Preventing depression in final year secondary students: School-based RCT. Journal of Medical Internet Research, 19(11), e369. DOI 10.2196/jmir.8241. [Google Scholar] [CrossRef]

63. Ovensted, K., Ormhaug, S., Shirk, S., Jensen, T. (2018). Therapists’ behaviors and youths’ therapeutic alliance during trauma-focused cognitive therapy. Journal of Consulting Psychology, 88(4), 350–361. DOI 10.1037/t04671-000. [Google Scholar] [CrossRef]

64. Short, K. (2016). Intentional, explicit, systematic: Implementation and scale-up of effective practices for supporting student well-being in Ontario schools. International Journal of Mental Health Promotion, 18(1), 33–48. DOI 10.1080/14623730.2015.1088681. [Google Scholar] [CrossRef]

This work is licensed under a Creative Commons Attribution 4.0 International License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.