Construction and Validity of Chinese Translation of the Universal Mental Health Literacy Scale for Adolescents

Introduction

The incidence of psychological, behavioral problems, and mental disorders among adolescents is on the rise globally, posing a significant public health challenge [1,2]. Adolescent mental health is directly influenced by their level of mental health literacy, with a higher literacy level positively impacting their mental well-being [3]. The concept of “mental health literacy” was introduced by Jorm in 1997 [4] as “knowledge and beliefs essential for identifying, managing, or preventing mental disorders.” Jorm later expanded this concept in 2012 [5], though it remained focused on mental disorders. In recent years, the scope of mental health literacy has been broadened to include stigma, the efficacy of seeking help, and other aspects of mental health promotion [6], a development supported by several scholars [7,8]. Chinese scholar Jiang et al. [9] defined it as “the knowledge, attitudes, and behavioral habits developed to enhance one’s own and others’ mental health, as well as to address personal and others’ mental disorders”.

Throughout their growth, teenagers are particularly susceptible to mental health challenges. In fact, half of all mental disorders starting by age fourteen [10,11]. Tully et al. [12] emphasized in the Australian and New Zealand Journal of Psychiatry the need for a national child mental health literacy initiative to reduce the prevalence of child mental health disorders. Targeting younger teenagers is crucial as early detection and timely intervention can significantly mitigate the long-term impact of their mental health issues [13]. However, China currently lacks tools to measure mental health literacy among younger teenagers. The Mental Health Literacy Scale for Adolescents, compiled by Li et al. [14] with first-year medical students as subjects, is currently used by many Chinese researchers [15,16]. However, the use of some professional terms in the scale entries, such as “post-traumatic stress disorder” and “emotional abuse”, may be beyond the cognitive scope of younger teenagers, resulting in the applicability of the scale in younger teenagers remains to be studied. Therefore, a tool to measure mental health literacy in younger teenagers is needed.

Kutcher et al. [6] posit that mental health literacy for adolescents should enable them to differentiate between normal psychological distress and serious psychological issues, diminish mental illness stigma, and encourage help-seeking behaviors. His definition encompasses four key aspects: acquiring and maintaining sound mental health, understanding psychological disorders and their treatments, reducing stigma associated with psychological disorders, and improving the efficacy of seeking help [6,17]. Adopting Kutcher’s framework can thus not only raise awareness about mental health knowledge but also address stigma, foster a positive outlook on mental health, and encourage a proactive approach to seeking help, ultimately contributing to the betterment of adolescent mental health [7,18,19]. The Universal Mental Health Literacy Scale for Adolescents (UMHL-A), crafted by Kagstrom et al. [13], evaluates the mental health literacy of adolescents aged 10 to 14, encompassing all four elements of Kutcher’s definition. The aim of this study is to investigate the factor structure of the Chinese version of the UMHL-A, and we hypothesize that the Chinese version of the UMHL-A adapted to the cultural background of China may have a similar factor structure as the original UMHL-A based on previous research. Additionally, we will examine the psychometric properties of the Chinese version of the UMHL-A, such as reliability and validity, when used with Chinese adolescents, in order to understand its applicability in the Chinese context.

Method

Participants

A simple and convenient sampling method was adopted to send 1321 questionnaires online to middle school students in Shandong Province of China. During the test, the subjects answered in the classroom. All questionnaires were self-reported. Additionally, we excluded those participants who took an unreasonably short or long time to complete the questionnaires, and who had missing data, and finally obtained 1208 valid data, with the recovery efficiency of 91.44%. Among them, the total sample of valid data included 605 males and 603 females, 562 students in Grade One, 311 students in grade two and 335 students in grade three. The age distribution was between 11 and 15 years old, with an average age of 13.02 ± 0.89 years old. After an interval of 2 weeks, 323 people from the initial sample were selected to retest the scale at random. 301 people were valid, and the recovery rate was 93.18%, including 153 boys, 148 girls, 115 students in grade one, 104 students in grade two, and 82 students in grade three. The demographic characteristics of the participants are shown in Table 1.

Procedure

The researchers reached out to the authors who developed the original UMHL-A questionnaire and communicated with them about the project. Two English-Chinese bilingual researchers who specialized in psychology translated the original questionnaire into Chinese. Then, seven graduate students in the team expressed and corrected the questionnaire to avoid ambiguity, lengthy, and difficult to understand. In response to their feedback, the newly translated version was revised and subsequently sent to two bilingual individuals fluent in English and Chinese, who were then invited to independently perform back-translation. After discussion with nine subjects, the language development of students beyond the comprehension of junior high school students or difficult-to-understand parts were replaced, and the Chinese version of the scale was finally formed. The study was approved by Ethics Committee of School of Education and Psychology at the University of Jinan (IRB number: 202301). All participants signed the informed consent in this study.

Measurements instruments

Universal Mental Health Literacy Scale for Adolescents (UMHL-A). The UMHL-A, developed by Kagstrom et al. in 2023 [13], is based on the definition of mental health literacy by Kutcher et al. It comprises an attitudes subscale and a knowledge subscale, featuring four dimensions: help-seeking attitude, stigmatization attitude, general mental health knowledge, and specific mental illness knowledge, totaling 17 items. The attitudes subscale is evaluated using a 5-point Likert scale with an additional “don’t know” option (1 = strongly agree, 5 = strongly disagree, don’t know = 3 = uncertain). The Knowledge subscale uses the T/F scale, which means subscale uses a “yes” or “no” format, with “don’t know” as an option. A “yes” response scores 1 point, while “no” or “don’t know” responses score 0 points. A higher score on this scale indicates a greater level of mental health literacy.

Self-Stigma of Seeking Help (SSOSH) Scale. Developed by Vogel et al. in 2006 [20] and revised in China [21], the SSOSH Scale consists of 10 items rated on a 5-point Likert scale (1 = strongly agree, 5 = strongly disagree). A higher score indicates more severe self-stigma among research participants regarding seeking professional psychological help. In this study, Cronbach’s alpha was 0.75.

Seek Professional Psychological Help Attitude Scale. This scale, originally compiled by Fischer et al. in 1970 [22] and later revised in China [23], contains 29 items across four dimensions: trust in professionals, societal and individual perceptions, level of self-disclosure, and recognition of the need for psychological help. It utilizes a 5-point Likert scale (1 = strongly agree, 5 = strongly disagree), with higher scores reflecting a more positive attitude towards seeking psychological help. The scale’s Cronbach’s alpha in this study was 0.83.

Mental Health Knowledge Questionnaire. Issued by the General Office of the Ministry of Health of China in 2010 as part of a mental health indicators survey and evaluation program, this questionnaire includes 20 questions [24]. Questions 1 to 16 score 1 point for each correct answer and 0 points for incorrect ones. For questions 17 to 20, selecting “Know” scores 1 point, while “Don’t know” scores 0 points. The total score is the sum of all question scores, with higher scores indicating better knowledge. Cronbach’s alpha in this study was 0.78.

Data analysis

Data processing and analysis were conducted using SPSS 25.0, which included item analysis, reliability analysis, and exploratory factor analysis (EFA) for the UMHL-A. Additionally, confirmatory factor analysis (CFA) was performed using AMOS 24.0.

To explore the factor structure of the UMHL-A, the total sample was randomly divided into two equal parts: one half (n = 604) was used for EFA, and the other half (n = 604) for CFA. For EFA, Kaiser-Meyer-Olkin (KMO) and Bartlett’s test were conducted. Then, principal component analysis, employing the varimax rotation method, was selected to extract common factors and to ascertain the scale’s factor structure. CFA was conducted to validate the factors identified through EFA. The maximum likelihood estimation method was used to assess the fit of the measurement model [25], with indicators including the root mean square error of approximation (RMSEA), standardized root mean square residual (SRMR), comparative fit index (CFI), and Tucker–Lewis index (TLI) used to evaluate model fit [26].

Reliability analysis involved Cronbach’s alpha, split-half reliability, and test-retest reliability. Subsequently, 323 students from the initial sample were retested two weeks later, and the data were analyzed for test-retest reliability.

Construct validity of the sample data was assessed by analyzing the Pearson correlation between the scores of UMHL-A subscales and each dimension’s scores, along with other scale qualities such as help-seeking attitude and stigma. This analysis provided insights into the construct validity of the scale. A threshold p-value of 0.05 was used to determine statistical significance.

Result

Exploratory factor analysis

Since the scale consists of two subscales with differing scoring methods, exploratory factor analysis was conducted separately on the attitude subscale and the knowledge subscale. The exploratory factor analysis of 604 samples revealed that the KMO measure of sampling adequacy for the attitude subscale was 0.83, and the Bartlett’s test of sphericity showed a significant difference (χ2 = 1484.82, p < 0.001). Utilizing principal component analysis with a maximum variance rotation method, two common factors were identified: help-seeking attitude and stigmatization attitude. The cumulative variance explained by these factors was 60.01%, aligning with the original scale structure (Table 2). For the knowledge subscale, the KMO value was 0.84, with Bartlett’s test yielding (χ2 = 1315.98, p < 0.001). Exploratory factor analysis extracted two factors—mental health knowledge and mental illness knowledge—with a cumulative variance contribution of 51.28%, also consistent with the original scale structure (Table 3).

Confirmatory factor analysis

Given the distinct scoring methods of the two subscales, confirmatory factor analysis was performed separately on the attitude subscale and the knowledge subscale. Following the exploratory factor analysis confirmatory factor analysis, confirmatory factor analysis was conducted on the subscale items using the other half of the sample data, according to the determined two-factor structure, and maximum likelihood estimation was employed to evaluate the fit of the measurement model. The results of the CFA indicated that the two-factor model had an acceptable model fit. The standard load range for each item on the attitude subscale varied from 0.52 to 0.80 (Fig. 1), while the standardized load range for each item on the knowledge subscale ranged from 0.39 to 0.76 (Fig. 2). The fit indices, presented in Table 4, indicate that the model satisfactorily met the evaluative criteria and demonstrated a good fit.

Figure 1: Factor loadings of the attitude subscale (n = 604).
Note: HS = Help-seeking. ST = Stigma.

Figure 2: Factor loadings of the knowledge subscale (n = 604).
Note: KMH = Knowledge of mental health. KMI = Knowledge of mental illness.

Item analysis

Critical ratio method and item-total correlation were employed to analyze the items. The sample was sorted by subscale total scores, with the top 27% and bottom 27% forming the “high” and “low” subgroups, respectively. An independent sample t-test was performed for each item score on the scale, comparing the high and low subgroups. The analyses revealed significant differences between high and low subgroups for all items; the specific data are presented in Table 5. The correlation coefficients between the scores of each item and the subscales total score ranged from 0.55 to 0.74 (all p < 0.01), as shown in Table 5. This confirms the scale’s effective differentiation and discrimination capabilities.

Reliability

Reliability analysis (Table 6) reported Cronbach’s alpha values of 0.82 and 0.79 for the attitude and knowledge subscales, respectively. Cronbach’s alpha for each dimension varied between 0.71 and 0.80, indicating the scale’s high reliability. Split-half reliability of the two subscales is 0.87 and 0.82, respectively. And Split-half reliability of each dimension is between 0.70 and 0.84. Test-retest reliability for the subscales was found to be 0.76 and 0.80, respectively, over a two-week period, with dimension-specific reliability ranging from 0.69 to 0.84. All correlation coefficients were significant (p < 0.01), underscoring the scale’s consistent reliability.

Construct validity

The construct validity of the scale was confirmed through significant positive correlations between the help-seeking dimension, attitude subscale, and the scale for seeking professional psychological help (r = 0.51, 0.43, p < 0.01), and negative correlations between the stigmatization dimension, attitude subscale, and the self-stigma scale for seeking psychological help (r = 0.45, 0.51, p < 0.05). Additionally, the mental health knowledge dimension, mental illness knowledge dimension, and knowledge subscale were all positively correlated with the mental health knowledge questionnaire (r = 0.59, 0.55, 0.65, p < 0.01), further validating the scale’s construct validity.

Discussion

The reliability and validity of the Chinese version of the UMHL-A have been effectively confirmed among Chinese adolescents aged 11–15. This scale includes 17 items across two subscales: mental health attitudes and mental health knowledge, which further break down into four dimensions—help-seeking attitude, stigmatization attitude, general mental health knowledge, and specific mental illness knowledge. The item composition aligns with the original scale, satisfying the criteria for factor loading. Both internal consistency reliability and test-retest reliability meet the necessary standards. Furthermore, the comprehensive scores across all subscales and dimensions positively correlate with the overall scores of the mental health and mental health knowledge questionnaire, aligning with the foundational concept of mental health literacy. The aspects of help-seeking attitude, stigmatizing attitude, and the attitude subscale show positive correlations with the propensity towards professional help-seeking and negative correlations with self-stigmatizing tendencies in seeking psychological help, demonstrating strong convergent validity.

The Chinese UMHL-A version serves as an effective tool for evaluating the mental health literacy of young adolescents, enabling researchers to understand their mental health status, pinpoint existing issues, and intervene accordingly. It also offers valuable insights for schools and families to enhance their focus on and nurture the mental health of young individuals. By identifying areas of weakness in adolescents’ mental health literacy, targeted interventions can be applied to foster improvements, thereby supporting their overall physical and mental well-being. Evaluating young adolescents’ mental health literacy allows for a deeper understanding of their essential needs, facilitating more tailored interventions and support for their growth and development.

As an assessment instrument for adolescent mental health literacy, the UMHL-A presents several advantages. It avoids the use of complex professional jargon in its items to prevent comprehension difficulties, ensuring smooth completion of the survey. Focused on attitudes and knowledge, the scale is concise with fewer items compared to questionnaires developed under a broader definition of mental health literacy, resulting in high evaluation efficiency. This efficiency aids in identifying areas where adolescents’ mental health literacy is lacking, enabling targeted interventions. Moreover, to accommodate potential gaps in respondents’ knowledge, the scale includes a “do not know” option, minimizing forced responses and reducing assessment bias [27]. The Chinese version of the UMHL-A is both reliable and valid, making it a suitable tool for assessing the mental health literacy of young adolescents in China.

Limitation

First and foremost, this study’s geographical scope was limited to a single region, with all participants hailing from Jinan City, Shandong Province (China), resulting in relatively high sample homogeneity. To enhance the applicability of our findings, further research involving more nationally representative samples and larger population sizes is recommended. Additionally, the use of a cross-sectional study design in this research does not allow for the examination of the dynamic changes in adolescent mental health literacy over time. Consequently, there is a pressing need for longitudinal studies that delve into the evolution of mental health literacy among adolescents, including its related factors, and potential impacts at various stages of development. Lastly, the reliance on self-reported data in this study introduces certain limitations. Future research should employ a more diverse methodology, incorporating evaluations from family members, teachers, and other relevant individuals to obtain a more comprehensive understanding of the subject matter.

Acknowledgement: The authors would like to thank all the participants in this study.

Funding Statement: This work is supported by National Natural Science Foundation of China, Grant No. 32100856 (to Fanlu Jia), Grant No. 31800913 (to Kaiyun Li); Youth Innovation Team of Shandong Provincial Higher Education Institutions, Grant No. 2022RW019 (to Fanlu Jia).

Author Contributions: Qi Wang and Qi Wang was responsible for the initial draft writing and data analysis. Yuxuan Ji and Kexu Chen provided guidance on data processing and revising the article. Kaiyun Li and FanLu Jia suggested article revisions. Ting Peng managed data processing and article revisions. Kaiyun Li provided suggestions regarding revision and confirmed the final version. All authors reviewed the results and approved the final version of the manuscript.

Availability of Data and Materials: The data will be provided upon request to the corresponding author.

Ethics Approval: The study was approved by Ethics Committee of School of Education and Psychology at the University of Jinan (IRB number: 202301). All participants signed the informed consent in this study.

Conflicts of Interest: The authors declare that they have no conflicts of interest to report regarding the present study.

References

1. Qu Y, Wu XY, Tao FB. Research progress on the association between social jet lag and mental health in adolescents. Chin J Sch Health. 2021;42(8):1265–9 (In Chinese). doi:10.16835/j.cnki.1000-9817.2021.08.033. [Google Scholar] [CrossRef]

2. Harris TB, Udoetuk SC, Webb S, Tatem A, Nutile LM, Al-Mateen CS. Achieving mental health equity: children and adolescents. Surf Coat Technol. 2020;43(3):471–85. doi:10.1016/j.psc.2020.06.001. [Google Scholar] [PubMed] [CrossRef]

3. Zhang JY, Qin SD, Zhou YQ. Research progress of adolescent mental health literacy. Chinese J Health Psychol. 2022;30(9):1412–8. doi:10.13342/j.cnki.cjhp.2022.09.025. [Google Scholar] [CrossRef]

4. Jorm AF, Korten AE, Jacomb PA. “Mental health literacy”: a survey of the public’s ability to recognise mental disorders and their beliefs about the effectiveness of treatment. Med J Aust. 1997;166:182–6. doi:10.5694/j.1326-5377.1997.tb140071.x. [Google Scholar] [PubMed] [CrossRef]

5. Jorm AF. Mental health literacy: empowering the community to take action for better mental health. Am Psychol. 2012;67(3):231–43. doi:10.1037/a0025957. [Google Scholar] [PubMed] [CrossRef]

6. Kutcher S, Wei Y, Coniglio C. Mental health literacy: past, present, and future. Can J Psychiat. 2016;61(3):154–8. doi:10.1177/0706743715616609. [Google Scholar] [PubMed] [CrossRef]

7. Bjornsen HN, Eilertsen MEB, Ringdal R, Espnes GA, Moksnes UK. Positive mental health literacy: development and validation of a measure among Norwegian adolescents. BMC Public Health. 2017;17(1):717. doi:10.1186/s12889-017-4733-6. [Google Scholar] [PubMed] [CrossRef]

8. Ming ZJ, Chen ZY. Mental health literacy: concepts, assessments, interventions and roles. Chin J Public Health. 2020;37(1):1–12. doi:10.11847/zgggws1123418. [Google Scholar] [CrossRef]

9. Jiang GR, Li DY, Ren ZH, Yan YP, Wu XC, Zhu X, et al. Mental health literacy: connotation, measurement, and a new conceptual framework. Acta Psychol Sin. 2022;53(2):182–98. doi:10.3724/sp.J.1041.2021.00182. [Google Scholar] [CrossRef]

10. Kim-Cohen J, Caspi A, Moffitt TE, Harrington H, Milne BJ, Poulton R. Prior juvenile diagnoses in adults with mental disorder: developmental follow-back of a prospective-longitudinal cohort. Arch Gen Psychiat. 2003;60:709–17. doi:10.1001/archpsyc.60.7.709. [Google Scholar] [PubMed] [CrossRef]

11. Kessler RC, Demler O, Frank RG, Olfson M, Pincus HA, Walters EE, et al. Prevalence and treatment of mental disorders, 1990 to 2003. N Engl J Med. 2005;352(24):2515–23. doi:10.1056/NEJMsa043266. [Google Scholar] [PubMed] [CrossRef]

12. Tully LA, Hawes DJ, Doyle FL, Sawyer MG, Dadds MR. A national child mental health literacy initiative is needed to reduce childhood mental health disorders. Aust N Z J Psychiat. 2019;53(4):286–90. doi:10.1177/0004867418821440. [Google Scholar] [PubMed] [CrossRef]

13. Kagstrom A, Pesout O, Kucera M, Jurikova L, Winkler P. Development and validation of a universal mental health literacy scale for adolescents (UMHL-A). Psychiat Res. 2023;320:115031. doi:10.1016/j.psychres.2022.115031. [Google Scholar] [PubMed] [CrossRef]

14. Li DL, Hu J, Huang XX, Xue YN, Chen SX, Wang SS, et al. Development and application of Adolescent Mental Health Literacy Assessment Scale in medical students. Chin J Sch Health. 2021;42(7):1038–41 (In Chinese). doi: 10.16835/j.cnki.1000-9817.2021.07.018. [Google Scholar] [CrossRef]

15. Wang LL, Hou YB, Zhang JL, He JK, LZ Z, Wang L, et al. Analysis of mental health quality, psychological condition and service demand of adolescent students in Ningbo City. Mod Pract Med. 2023;35(7):928–33. doi:10.3969/j.issn.1671-0800.2023.07.028. [Google Scholar] [CrossRef]

16. Zhang ZX, Huang XX, Hu J, Xue YN, Jia LY, Tao FB, et al. The interaction between mental health literacy and non-suicidal self-injury behavior is associated with suicide-related behavior in middle school students. Chin J Public Health. 2022;38(12):1517–22. doi:10.11847/zgggws1138596. [Google Scholar] [CrossRef]

17. Kutcher S, Bagnell A, Wei Y. Mental health literacy in secondary schools: a Canadian approach. Child Adolesc Psychiat Clin N Am. 2015;24(2):233–44. doi:10.1016/j.chc.2014.11.007. [Google Scholar] [PubMed] [CrossRef]

18. Morgan AJ, Ross A, Reavley NJ. Systematic review and meta-analysis of Mental Health First Aid training: effects on knowledge, stigma, and helping behaviour. PLoS One. 2018;13(5):e0197102. doi:10.1371/journal.pone.0197102. [Google Scholar] [PubMed] [CrossRef]

19. Wei Y, McGrath PJ, Hayden J, Kutcher S. Mental health literacy measures evaluating knowledge, attitudes and help-seeking: a scoping review. BMC Psychiat. 2015;15:291. doi:10.1186/s12888-015-0681-9. [Google Scholar] [PubMed] [CrossRef]

20. Vogel DL, Wade NG, Haake S. Measuring the self-stigma associated with seeking psychological help. J Couns Psychol. 2006;53(3):325–37. doi:10.1037/0022-0167.53.3.325. [Google Scholar] [CrossRef]

21. Hao ZH, Liang BY. Revision of the stigma questionnaire for seeking professional psychological help among college students. Chinese Mental Health J. 2011;25(9):646–9. doi:10.3969/j.issn.1000-6729.2011.09.002. [Google Scholar] [CrossRef]

22. Fischer EH, Turner JI. Orientations to seeking professional help: development and research utility of an attitude scale. J Consult Clin Psych. 1970;35(1):79–90. doi:10.1037/h0029636. [Google Scholar] [PubMed] [CrossRef]

23. Hao ZH, Liang BY. Revision of the attitude questionnaire on seeking professional psychological help. Chinese J Clin Psychol. 2007;15(1):1–3+9. doi:10.16128/j.cnki.1005-3611.2007.01.002. [Google Scholar] [CrossRef]

24. Ministry of Health General Office. Ministry of Health General Office survey on the issuance of mental health performance indicators notification of evaluation options; 2010 (In Chinese). doi: 10.16128/j.cnki.1005-3611.2013.02.019. [Google Scholar] [CrossRef]

25. Satorra A, Bentler PM. A scaled difference chi-square test statistic for moment structure analysis. Psychometrika. 2001;66(4):507–14. doi:10.1007/bf02296192. [Google Scholar] [CrossRef]

26. Hooper D, Coughlan J, Mullen M. Structural equation modelling: guidelines for determining model fit. Electron J Bus Res Methods. 2008;6(1):53–60. doi:10.21427/D7CF7R. [Google Scholar] [CrossRef]

27. Luskin RC, Bullock JG. “Don’t know” means “Don’t know”: DK responses and the public’s level of political knowledge. J Politics. 2011;73(2):547–57. doi:10.1017/s0022381611000132. [Google Scholar] [CrossRef]