Mental health literacy measures evaluating knowledge, attitudes and help-seeking: a scoping review

Background Mental health literacy has received increasing attention as a useful strategy to promote early identification of mental disorders, reduce stigma and enhance help-seeking behaviors. However, despite the abundance of research on mental health literacy interventions, there is the absence of evaluations of current available mental health literacy measures and related psychometrics. We conducted a scoping review to bridge the gap. Methods We searched PubMed, PsycINFO, Embase, CINAHL, Cochrane Library, and ERIC for relevant studies. We only focused on quantitative studies and English publications, however, we didn’t limit study participants, locations, or publication dates. We excluded non-English studies, and did not check the grey literature (non peer-reviewed publications or documents of any type) and therefore may have missed some eligible measures. Results We located 401 studies that include 69 knowledge measures (14 validated), 111 stigma measures (65 validated), and 35 help-seeking related measures (10 validated). Knowledge measures mainly investigated the ability of illness identification, and factual knowledge of mental disorders such as terminology, etiology, diagnosis, prognosis, and consequences. Stigma measures include those focused on stigma against mental illness or the mentally ill; self-stigma ; experienced stigma; and stigma against mental health treatment and help-seeking. Help-seeking measures included those of help-seeking attitudes, intentions to seek help, and actual help-seeking behaviors. Conclusions Our review provides a compendium of available mental health literacy measures to facilitate applying existing measures or developing new measures. It also provides a solid database for future research on systematically assessing the quality of the included measures. Electronic supplementary material The online version of this article (doi:10.1186/s12888-015-0681-9) contains supplementary material, which is available to authorized users.


Epidemiology of mental illness
Approximately 70 %-75 % of adult mental health problems and mental disorders start to manifest during adolescence or early adulthood (12)(13)(14)(15)(16)(17)(18)(19)(20)(21)(22)(23)(24)(25) [1,2]. Globally, mental disorders make up about 1/3 of the burden of illness in adolescence and young adulthood [3]. Untreated mental health problems and disorders in adolescents and young adults are strong predictors of poor vocational achievements, problematic interpersonal and family functioning, as well as reduced life expectancy due to associated medical conditions, such as diabetes, heart diseases and stroke, respiratory conditions, and suicide [4][5][6][7]. However, despite the great burden of illness incurred by these conditions, research shows, worldwide, between 70 %-80 % of young people and adults do not receive the mental health care they need [8][9][10]. A recent systematic review [11] of perceived barriers and facilitators for mental health help-seeking indicated that perceived stigma and embarrassment, problems in symptom identification and a preference for self-reliance were the most important intra-personal barriers to mental health help-seeking.

Mental health literacy
Mental health literacy is a significant determinant of mental health and has the potential to improve both individual and population health [12][13][14]. Evidence shows that improved knowledge about mental health and mental disorders, better awareness of how to seek help and treatment, and reduced stigma against mental illness at individual, community and institutional levels may promote early identification of mental disorders, improve mental health outcomes and increase the use of health services [15][16][17].
We conceptualize mental health literacy to include 4 domains: 1) understanding how to obtain and maintain good mental health; 2) understanding mental disorders and their treatments; 3) decreasing stigma against mental illness; and 4) enhancing help-seeking efficacy [13,18]. And therefore, mental health literacy addresses 3 interrelated cocepts: knowledge (knowledge of mental illness and positive mental health), attitudes and help-seeking efficacy. This definition is consistent with the current construct of health literacy defined and promoted by the WHO as an empowerment tool for people to participate in their health care [19].
We located five reviews on the effectiveness of mental health literacy interventions [12,18,[20][21][22]. In addition there were three literature reviews describing stigma [23][24][25] and knowledge measures. However, there has been a lack of comprehensive understanding of current available mental health literacy measures. Thus, there exists a need to conduct a study to help better understand strengths and weaknesses of existing measures and to help shape future development of measures. We conducted a scoping review, a systematic approach to map the literature in an area of interest and to accumulate and synthesize evidence available, to bridge the gap. This current scoping review was guided by Arksey and O'Malley's work (2005) [26], proposing four purposes: 1. to examine the extent, range and nature of research activity; 2. to determine the value of undertaking a full systematic review; 3. to summarize and disseminate research findings; and 4. to identify research gaps in the existing literature.
We analyzed available mental health literacy measures that focus on four common mental disorders with onset before or during adolescence and young adulthood: Schizophrenia, Depression, Anxiety Disorders, and Attention Deficit Hyperactivity Disorder (ADHD).

Methods
We used our definition of mental health literacy [13,18] that is composed of 4 constructs addressing three outcomes: mental health knowledge (including knowledge about positive mental health (construct 1) and knowledge about mental illness and treatments (construct 2)), stigma/ attitudes towards mental illness, and help-seeking, to define our search scope.

Search strategy
One of the authors of this review and a health librarian designed the search strategies together. We searched PubMed, PsycINFO, Embase, CINAHL, Cochrane Library, and ERIC between 2013 and 2014, and re-ran the search in 2015. We applied four sets of search terms to identify domains of mental health literacy as outlined in Additional file 1.

Inclusion criteria
We included quantitative studies that used, developed, or investigated measurement properties of mental health literacy measures evaluating any one, or combinations of the mental health literacy outcomes: knowledge, stigma/ attitudes towards mental disorders, and help-seeking. Study designs included any type of quantitative studies: randomized controlled trials (RCTs), cluster RCTs, quasi-experimental studies; cohort studies; crosssectional/survey studies, and controlled-before-and-after studies (pre/post tests). Only studies published in English were eligible and non-English publications were excluded at the screening stage. Year of publication and study participants, including their age, were not restricted.

Exclusion criteria
Studies were not eligible if they addressed mental health literacy but did not mention or describe the measure applied in the study. Studies of smoking prevention/cessation and other substance use prevention programs were not included. Studies of suicide prevention interventions that did not address related mental disorders, such as Depression were not eligible. Qualitative studies were excluded.

Data extraction and study selection (Charting)
Two reviewers used the search strategy, and independently searched pre-identified databases. We first screened out irrelevant studies which mostly focused on stigma against HIV/AIDS, cognitive behavioral therapies, substance abuse/smoking, resilience scales, and clinical treatment related studies by reviewing titles and abstracts. We then imported the remaining studies, into RefWorks 2.0 database management software (2001) [27]. Duplicates were removed. We then screened titles and abstracts again and briefly scanned the full text to exclude studies not evaluating target outcomes. All studies that passed this exclusion process were included in the third stage of review for relevance by scanning title, abstracts and the full text for relevancy. At the next stage, we reviewed fulltext articles for all the final included studies. Additionally, we added original studies which were referenced in included studies that cited their psychometric properties. We also checked the reference list of included studies for additional studies.
We applied "charting" techniques to conduct data extraction. For scoping reviews, "charting" is a data extraction technique to synthesize and interpret data by "sifting, charting and sorting materials according to key issues and themes" (page 26) [26,28]. The key themes we followed in the "charting" of our data are the three outcome measures: knowledge, attitudes, and help-seeking, which was the base of the data categorization. We also charted data by year of publication, study location, study type, outcome measures, and types of psychometrics examined (e.g., reliability, validity, and responsiveness/sensitivity to change). The detailed charting process for this review is depicted in Fig. 1.
A data extraction form, developed in advance, was used for data extraction. We categorized studies into four types based on the extent of how psychometric properties were investigated and reported in the study: validation studies with evaluating psychometrics (any type) and/or responsiveness/sensitivity to change as the major purpose of the study (coded as P); studies evaluating effectiveness of interventions or survey studies evaluating psychometrics (any type) and/or responsiveness/sensitivity to change of the outcome measures (coded as I/P or S/P); studies just reporting but not evaluating psychometrics and/or responsiveness/sensitivity to change of the applied tool (coded as I/? or S/?); and studies mentioning the measurement tool applied but not reporting psychometrics (coded as I or S), including studies that quoted psychometrics from other studies, but did not evaluate it in the current study. We then sorted and defined the data by measures on knowledge, attitudes/stigma towards mental illness, and help seeking respectively, listed authors who first applied the tool, and calculated the number of psychometrics studies for each outcome measurement. In addition, we collated all psychometrics studies in separate tables. Figure 1 illustrates this process.
Once this charting process was completed, we reviewed all included studies, developed and populated tables, and created charts and figures according to the above described typology in an Excel spreadsheet. To help ensure consistency in interpretation and validity of the final results, one of the reviewers read and charted all included studies. Then the second reviewer checked all tables and compared and discussed the results with the first reviewer and they came to a consensus on the interpretation of the results. One methodology expert and two content experts were invited to help make the final decision when consensus was not reached between the two reviewers. Figure 2 presents the flow chart of the screening process and final included studies. A total of 401 studies 1 were identified that met study criteria, including 113 studies   (Table 3) [35,101,[143][144][145][146][147][148][149][150][151][152][153][154][155][156][157][158][159]. These 3 tables summarized characteristics of validated studies, however we only listed authors who developed or first applied the measures although we included and summarized study results from other authors. Of these 130 studies, 5 studies also evaluated and reported responsiveness/sensitivity to change. Total number of studies for each sub-category may not necessarily match the total number of included studies because some studies tested more than one measurement tool in one study.

Results
Study characteristics, such as study participants, locations, publication dates, and tool outcomes are reported in Figs. 3, 4, 5 and 6. Studies were conducted in 32 countries, with the United States of America most commonly, followed by Australia and Canada. Study participants were mainly post-secondary students, especially students in psychology or related professions, followed by the general public, and mental health service users (e.g., patients and their families). Most of the studies (n = 337) were published after the year 2000.
The 69 knowledge measures evaluated general knowledge about mental health, knowledge on specific disorders such as depression, schizophrenia/psychosis, ADHD, and       [29,31]. These 14 measures evaluated general mental health knowledge (6 measures), depression (4 measures), schizophrenia (2 measures), ADHD (1 measure), and anxiety disorders (1 measure) ( Table 1). The rest of the twelve measures only reported but didn't evaluate psychometrics (internal consistency) and therefore we didn't include them in Table 1. Most knowledge measures applied self-report multiple choice answers (true, false, I don't know/not sure), or vignettes with open-ended/closed questions (e.g. [172]), or used Likert-scale statements as self-evaluation formats.
The 111 focus of the stigma/attitudes measures included: 1. stigma against mental illness or the mentally ill, such as social distance (the degree to which people are willing to accept the mentally ill in regular social life), personal stigma (participants' personal attitudes toward people with mental illness) and perceived stigma (participants' beliefs about others' attitudes about mental illness); 2. self-stigma; 3. experienced stigma by mental   (Fig. 6). Social distance measures investigated issues such as a person's willingness to engage the mentally ill in the workplace and the community (e.g., employment, renting, being neighbors, marriage) [46,69,106,108,124,173]. Similarly, measures evaluating stigmatizing experiences by the mentally ill focused on challenges people with mental illness experience in family and social life [52,75,92,99,126,130].
Measures evaluating personal and perceived stigma covered areas such as authoritarianism, benevolence, mental hygiene ideology, social restrictions to the mentally ill, and etiology [59,134]. Other measures evaluated components such as stigma related to illness prevalence, consequences, dangerousness/threat, treatment and recovery of mental illness, or the social/family life, social responsibilities, human rights, intelligence [36,74,79,81,105,111,116,135]. In addition, there were personal and perceived stigma measures focusing on emotional/rejection responses, willingness to help, and disclosure concerns [44,48,57,61,86,98,124].
Self-stigma measures mostly evaluated cognition such as self-esteem, self-confidence, self-satisfaction/concurrence, self-blame; negative emotions such as low pride of oneself, shame, embarrassment, sense of inadequacy, inferiority to others, helpless, pressure; and behaviors such as withdrawal, fear of seeking help, and secrecy [47,63,83,92,113,121,136].
Measures examining stigma against treatment/helpseeking/mental health care/medical model/psychiatry addressed perspectives and emotions. For example, some measures evaluated stigma towards help seeking (e.g., help-seeking as personal weakness; people seeking help being less likeable, disturbed, posed risks to others, and should hide the fact of seeking help) [101,137]. Other tools [55,122,132] investigated stigma toward psychiatry, for example, skepticism towards psychiatry; and stereotypes of psychiatrists, psychiatric hospitals, patients, and psychiatric treatments. Some tools measured emotional responses (e.g., fear, discomfort and embarrassment) to psychological services and mental health care [89,144].  Eighty one (73 %) articles on stigma tools reported on some psychometrics. Sixty five measures had evidence of reliability (e.g., Crobach's α; item-total correlations; KR-20; test-retest reliability; inter-rater reliability), validity (e.g, construct; concurrent; discriminant; convergent; predicative), or responsiveness/sensitivity to change (Table 2). Sixteen measures demonstrated only internal consistency, but none included discussions on how this was measured. Of these 81 measures, 48 evaluated stigma against the mental illness/ the mentally ill in general; 11 were selfstigma measures; 6 evaluated personally experienced stigma; and 12 evaluated stigma against mental health treatment (psychological and pharmacological), psychiatry, help-seeking, or mental health care facilities. One tool did not specify what it measured.

Help-seeking measures
Of the 35 help-seeking related measures, the most widely used are: Attitudes towards Help-Seeking Scale (later modified as Attitudes toward Seeking Professional Psychological Help Scale) (ATSPPH) [143,147]; the mental health literacy questionnaire (MHLQ) that contains items on beliefs towards treatments [160]; General Help Seeking Questionnaire (GHSQ) [145]; and Intention of Seeking Counseling Inventory (ISCI) [174].
These help-seeking measures evaluating help-seeking intentions; beliefs or attitudes towards seeking psychological help for mental health problems or illness; beliefs towards mental health help or treatment in general; actual help-seeking behaviors; help-seeking efficacy (e.g. knowledge about where and how to find help, and who to find help from); self-reported ability to help others; or multiple components such as help-seeking intentions, help-seeking efficacy, and barriers for help-seeking (Fig. 6).
Unlike measures of stigma against help-seeking described above, measures evaluating attitudes towards psychological help-seeking mostly addressed: recognition of need for psychological help; interpersonal openness; confidence in and trustworthiness of mental health practitioners [143]. Measures evaluating beliefs toward treatment mostly evaluated the perceived helpfulness, effectiveness or safety of various interventions [150,175], or the myths of treatment [176]. One measure [177] added social norm items on perceived attitudes of others (e.g., friends, employer) on depression intervention.
Measures evaluating help-seeking intentions examined willingness, or preferences to seek help from different sources (e.g., friends, families, professionals, religion, or spiritual healers [151,172,[178][179][180][181][182]. One measure [172] further evaluated 3 extra dimensions of help-seeking intentions: talking to the listed sources; comfort level of talking to these resources; and helpfulness of these resources. Another tool measured intention levels for various emotional/ behavioral challenges among college students [144]. Two measures didn't specify how intentions were measured [161,183]. Measures addressing help-seeking behaviors evaluated whether help-seeking was sought, and if so, what type of help was sought (formal vs. informal) for both stressful events and mental illness [178,181,184,185].
Ten measures had some psychometric evaluation such as internal consistency, reliability, factor analysis, construct validity, and criterion validity [143,145,147,[149][150][151][152][153][154][156][157][158][159]. Details of the psychometrics of these 10 measures are presented in Table 3. The 10 measures with psychometrics addressed attitudes or beliefs towards help-seeking or treatments, and intentions for helpseeking (Table 3). Two measures reported the internal consistency of the tool [172,181], but did not discuss how this were measured, and therefore were not included in the table. No psychometric properties were reported on measures of help-seeking behaviors.

Discussion
We identified a number of significant issues for consideration. These are: 1) representativeness of study samples; 2) geographic weighting: 3) adequacy of measurement of mental health literacy (knowledge, stigma, and help-seeking).

Representative samples
Almost half of the studies (n = 185) were conducted among adolescents and young adults, particularly with post-secondary students (n = 117) (Fig. 3) mostly from health related professions, such as psychology, social work, and nursing. This raises concerns about the generalizability of findings as participants are not representative of the general population.
Even within the context of postsecondary education, much less attention (only 9 studies) has been paid to the mental health literacy of educators, who are important role models and youth influencers in addressing mental health literacy [186]. Further research into mental health literacy should take these important factors into account.

Geographic weighting
Research on the measurement of mental health literacy started as early as in late 1950'S but did not bloom until after 2000 (n = 336; 84 %) (Fig. 5). Most studies (Fig. 4  either adapted existing measures, or used the conceptual framework from developed countries to create their measures, however, very few discussed the process of translation or the method of cultural adaptation. Therefore, the impact of important contextual factors, such as culture, ethnicity, geographic locations, education and health system, on mental health literacy and its measurement is currently unknown.

Adequacy of measurement
Our analysis suggests that, out of three outcomes of mental health literacy (knowledge, attitudes and help-seeking), most measures evaluated stigma (n = 111), followed by measures that evaluated knowledge (n = 69), and a smaller number of help-seeking (n = 33). Only a relatively small number of measures were validated in any way. Secondly, widely used measures are often not validated. For example, the WPA mental health knowledge questionnaire was applied in 9 studies but no research has been identified to analyze its psychometric properties except for internal consistency.
Given the high proportions of un-validated measures being applied, it was difficult to determine the value of the study results and not possible to conduct cross-study comparisons of different interventions. There is a pressing need to validate these measures before their application.
With the measures that have been validated, there has been no research identified that appraised the quality of psychometric studies, and therefore, we were not able to recommend which measures are better than others. Further, given that the measures included in this review vary in their content, purposes and quality (measurement properties), more advanced research, such as systematic reviews is needed to locate evidence-based measures for use. Consensus-based Standards for the Selection of Health Measurement Instruments (COSMIN) [187] has been developed to serve this purpose and could be adapted for use in the comparative evaluation of mental health literacy measures.
Further, our review did not identify any measures addressing knowledge of positive/good mental health. Future measures should investigate knowledge on how to obtain and maintain good health as this now is recognized as an important component of mental health literacy.

Knowledge measures
Our findings indicate that the diagnostic vignette approach is widely used as a measure of mental health knowledge. However, a recent study in which diagnostic vignettes were compared against non-diagnostic vignettes showed an inability of participants to discriminate across "normal" and "ill" categories [188]. Further study to establish the validity of the diagnostic vignette evaluation approach as a measure of mental health knowledge is needed.
The myths and facts approach to measure knowledge has covered a wide range of aspects of mental health. However, we are unable to determine if there are different and developmentally appropriate knowledge components addressed at different points of the life-span among the current available measures.

Stigma measures
The plethora of stigma measures, developed from numerous different ideological models (e.g., labeling theory [189]; attribution framework model [61]; cognitive behavioral model [190]; and social stigma model [191], has made evaluation of their validity in addressing stigma/attitudes challenging. The challenge has been to both validate each of the specific models and to determine which model may provide a better explanatory prediction for stigma or attitudes in different groups of people. Further, only a few measures have targeted people's emotional responses (n = 8) towards mental illness. This is an important area because stigma is associated with selfexperience of unpleasant feelings about mental illness and this may influence how people interact with those with mental illness [9]. Only very recently has research measured the stigma experience of people with mental illness (n = 28 studies). This may provide a more comprehensive picture of how society treats people with mental illness. This may help to provide more concrete and useful information on how stigma interventions should be developed and delivered at both individual and community level.
Despite the challenges discussed above, this review has mapped out how stigma measures were developed and what they intended to measure, and this information may provide researchers and practitioners some guidance on which path to take either in designing their measures, or applying/ adapting existing measures, or developing related interventions or programs in the future.

Help-seeking measures
Help-seeking behaviors are challenging to measure as they are influenced by many factors, such as knowledge about the behaviors, attitudes and beliefs towards the behaviors, social norms, and intentions to perform the given behavior [192]. Most help-seeking measures in this review have focused on attitudes towards help-seeking/treatment (n = 20) and intentions to seek help (n = 11), and very few measures (n = 4) directly measured actual help-seeking behaviors. Further, all 4 help-seeking behavior measures had no psychometric validation.
As Ajzen and Fishbein [191] pointed out, behaviors also may be influenced by self-expressed behavioral control which requires a person to have the skills, capacities, resources, and other important capacities needed to perform the behavior. However, we have not identified any measures to address these factors except for one tool measuring help-seeking efficacy (e.g. knowledge about where and how to find help, and who to find help from) [178].

Limitations
We did not conduct a systematic review of the literature on available mental health literacy measures and therefore we are unable to come to conclusions about the quality of the studies applying the measures. We excluded non-English studies (n = 21 at the title and abstract screening stage) and may have missed important measures in other languages. We did not check the grey literature that includes non peer-reviewed publication or documents/reports produced on all levels of governments and academics, and therefore may have missed some eligible studies. We may also have mistakenly excluded some measures at the first screening stage of reviewing titles and abstracts where measures were not mentioned.
Additionally, although we tried to categorize and interpret measures within the category we attributed them to, some measures may contain items relevant to other categories, however we were unable to distinguish them with available information we have.

Conclusions
Our review provides a compendium of available mental health literacy measurement measures for researchers and practitioners who are interested in applying existing measures or developing new measures that of particular relevance to their work. Because of how we selected eligible studies, our review further automatically forms a comprehensive dataset of current mental health literacy interventions for stakeholders to consider for their use. This review also identifies the many gaps in the field, such as the unbalanced application of knowledge and helpseeking evaluation measures compared to the stigma/ attitudes measures, the yet-to-be validated measures in each outcome category, and the lack of measures that measure all components of mental health literacy concurrently. This gap identification could potentially guide future research work in the field. Further, we have conducted a thorough summary and synthesis of the psychometrics properties of included measures, and clarified the need to further investigate the quality of the psychometrics studies. At this stage, most of the measures were created without consultation with the intended participants such as students, teachers, patients or health providers. Future work should focus on joint collaboration across disciplines, between investigators and stakeholders and across more varied demographic and geographic groups.

Endnotes
1 References of studies that applied mental health literacy measures but did not evaluate the psychometrics of the applied measure were included in the supplementary document unless they were specifically cited in the manuscript.

Additional file
Additional file 1: Supplementary files contain an example of search strategies in PubMed, and supplementary references of studies that applied mental health literacy measures but did not provide related psychometrics information. (ZIP 97 kb)

Competing interests
None of the authors have any competing interests.
Authors' contributions YW defined the study scope; developed the study structure; collected and analyzed data; drafted and finalized the manuscript. PM conceived of the study; advised on study scope and structure; reviewed and advised on data analysis; and contributed significantly to manuscript revisions and final writing. JH contributed significantly to the development of research methodology, and participated in the manuscript revision and final writing. SK conceived of the study, participated in the study design and manuscript writing and final revision. All authors read and approved the final manuscript.

Authors' information
Ms. Yifeng Wei (PhD candidate) is the school mental health lead of the Sun Life Financial Chair in Adolescent Mental Health team at the IWK Health Centre, Halifax, Nova Scotia. Her research interest on school mental health, especially on promoting mental health literacy of students and teachers alike in the secondary school setting, has enabled her to complete this current scoping review. Dr. Patrick J. McGrath is a clinical psychologist, is Vice President Research, Innovation and Knowledge Translation for the IWK Health Centre and the Nova Scotia Health Authority and Canada Research Chair and Professor of Psychology, Pediatrics, and Psychiatry at Dalhousie University. His research on pain in child health has spanned a wide range of studies on measurement and psychosocial interventions in many different types of pain in infants, children and youth. Dr. Jill Hayden is the associate professor of Community Health and Epidemiology at Dalhousie University. Her research expertise includes systematic review methods, prognostic research and musculoskeletal health. For many years, she has contributed to the Cochrane Collaboration. Her research is broadly related to improving the quality and use of research evidence for more effective health care decision-making and improved patient outcomes. Dr. Stan Kutcher is the Sun Life Financial Chair in Adolescent Mental Health at the IWK Health Centre and professor of Psychiatry at Dalhousie University. Dr Kutcher is the leading scientist to develop, promote, and research mental health literacy interventions in Canadian secondary schools.