Skip to main content

Protocol for a randomised controlled trial evaluating the effectiveness of a CBT-based smartphone application for improving mental health outcomes in adolescents: the MobiliseMe study

Abstract

Background

Depression is a leading cause of disability in adolescents, however few receive evidence-based treatment. Despite having the potential to overcome barriers to treatment uptake and adherence, there are very few CBT-based smartphone apps for adolescents. To address this gap, we developed ClearlyMe®, a self-guided CBT smartphone app for adolescent depression and anxiety. ClearlyMe® consists of 37 brief lessons containing core CBT elements, accessed either individually or as part of a ‘collection’. Here, we describe the protocol for a randomised controlled trial aiming to evaluate the effect of ClearlyMe® on depressive symptoms and secondary outcomes, including engagement, anxiety and wellbeing, when delivered with and without guided support compared to an attention matched control.

Methods

We aim to recruit 489 adolescents aged 12-17 years with mild to moderately-severe depressive symptoms. Participants will be screened for inclusion, complete the baseline assessment and are then randomly allocated to receive ClearlyMe® (self-directed use), ClearlyMe® with guided SMS support (guided use) or digital psychoeducation (attention-matched control). Depressive symptoms and secondary outcomes will be assessed at 6-weeks (primary endpoint) and 4-months post-baseline (secondary endpoint). Engagement, conceptualised as uptake, adherence and completion, will also be assessed 6-weeks post-baseline. Mixed-effects linear modelling will be used to conduct intention-to-treat analyses to determine whether reductions in depressive symptoms and secondary outcomes are greater for conditions receiving ClearlyMe® relative to control at 6-weeks and 4-months post-baseline and greater for intervention adherers relative to non-adherers. To minimise risk, participants will be encouraged to use the Get Help section of the app and can also opt to receive a call from the team clinical psychologist at baseline, and at the 6-week and 4-month post-baseline assessments when reporting suicidal ideation.

Discussion

This is the first clinical trial examining a CBT smartphone app specifically designed for adolescent depression. It will provide empirical evidence on the effects of ClearlyMe® on depressive symptoms when used with and without guided support.

Trial registration

Australian New Zealand Clinical Trials Registry (ACTRN12622000131752).

Universal trial number

U1111-1271-8519.

Peer Review reports

Background

Depression is a leading cause of disability in adolescents and can compromise social functioning, academic performance and emotional development [1, 2]. As up to 25% of adolescents worldwide, at any given time, experience elevated symptoms of depression [3], timely access to effective interventions is critical to prevent chronic, lifelong illness [4]. Cognitive behavioural therapy (CBT) is a structured, skills-based psychotherapy typically delivered by a trained clinician over several sessions [5, 6]. It is the gold-standard psychological treatment for depression [7,8,9,10] and is recommended as a first-line treatment in children and adolescents aged 5-18 years [11]. However, many adolescents do not seek professional help due to a range of barriers [12,13,14,15]. Among those who do seek treatment, many do not receive CBT [16, 17]. As a result, treatment outcomes for depressed adolescents remain poor. The limited access and uptake of CBT is a major barrier that needs to be rectified.

Digital CBT interventions have been developed to overcome clinician shortages and other barries impeding access to treatment. These interventions been shown to be effective for lowering symptoms of depression and anxiety symptoms among adolescents [18,19,20]. Several recent meta-analyses examining the effectiveness of computerised, web-based, and smartphone app-CBT interventions in adolescents have shown there are small to medium effects in favour of digital interventions over no-intervention controls (e.g., wait list controls) in reducing depressive symptoms at post-treatment, however, these interventions did not appear to provide superior effects to therapeutically active comparators (e.g., group or individual CBT) [18,19,20,21]. Individuals’ engagement with digital interventions is also suboptimal, with low levels of program completion compromising users’ exposure to critical elements of treatment, such as cognitive restructuring [20, 22]. For example, reported completion rates in trials of digital CBT average 57%, with a range of 0 to 100%, demonstrating substantial variability between interventions [18]. If improved engagement and completion of digital CBT interventions increases exposure and development of critical CBT skills, as demonstrated in in-person CBT [23, 24], it may increase the benefits of digital delivery, however this requires empirical validation.

Low engagement with digital CBT among adolescents does not appear to be due to their unwillingness to use digital technologies to address mental health concerns. Uptake of commercially developed mental health smartphone apps is substantial, with millions of downloads and thousands of active monthly users [25]. Despite widespread uptake, many commercially developed apps lack both proof of effectiveness and critical CBT elements [26]. Among evaluated mental health smartphone apps, two recent reviews collectively identified only three smartphone apps for adolescents that contained elements of CBT, with none specifically targeting adolescent depression [27, 28]. Young people have reported that many of the existing computerised CBT interventions are less visually appealing than commercially developed apps, with unrefined usability and content that is not aligned with their needs [20]. This may explain why comparisons of studies show smartphone mental health apps outperform computerised and web-based interventions in terms of engagement, with an average retention rate of 80% [28] compared to 57% [18], although there is substantial variability in completion rates between apps. Given high rates of ownership and uninhibited access to internet and multimedia functionality, smartphones offer a promising means of delivering CBT that is yet to be adequately exploited.

While engagement with existing evidence-based digital mental health interventions is poor, human support (also referred to as guidance) has been associated with increased adherence to digital mental health interventions [29] and improved outcomes [18, 20, 21], however also see [19, 30]. Human support can take various forms. It may be provided by therapists or non-therapists either in-person, via telephone, chat sessions or email [18, 21]. Support may consist of encouragement to use the intervention, motivational interviewing, feedback on activity completion or therapeutic support to implement strategies [18, 21]. There is some evidence to suggest young people prefer text-based support over telephone calls [31, 32]. However, the positive impact of this specific form of support on outcomes, adolescent engagement in digital CBT, or its effect on factors associated with increased likelihood of using a mental health app, such as perceived need for help, and beliefs about app effectiveness and usability [33], is unclear [19, 32, 34]. Specifically, little is understood about the superiority of smartphone CBT with text-based guided support in terms of benefit or in promoting engagement and adherence relative to unsupported use [35]. Developing this understanding through robust research may inform how digital CBT could be offered to adolescents to optimise engagement [27, 28].

To overcome the gaps in current treatment availability, the Black Dog Institute developed ClearlyMe® - a free, self-directed CBT smartphone app that provides therapeutic content and symptom management strategies for mild to moderately severe symptoms of depression and anxiety in adolescents. Adolescents helped develop the app through extensive co-design, described in detail in Li et al. [36]. The app contains 37 brief (up to 10 minutes in duration), non-sequential ‘lessons’ consisting of psychoeducation, cognitive restructuring (thought challenging and behavioural experiments), emotion awareness and acceptance, activity scheduling, behavioural activation, goal setting, problem solving, exposure, relaxation, mindfulness, and values identification. The lessons encourage participants to practise the psychological skills between periods of app use and return to the app to reflect or complete further content as needed. Users are encouraged to complete the lessons via nine curated ‘collections’, which are a structured program of lessons grouped together to target a specific set of symptoms. Each collection varies in length, taking approximately 20 minutes to complete. Users can also complete the individual lessons from a ‘show all’ list whereby they self-select lessons considered to be relevant to their individual needs. In this list, lessons are categorised into three groups depending on their target: ‘emotions’, ‘thoughts’, ‘behaviours’. The app also includes a MoodCheck (i.e., rating mood on a scale from awful to great to track changes in mood over time), MindHacks (i.e., quick strategies that help in the moment), and Stories (i.e., short videos of young peoples’ experience managing mental health symptoms and positive help-seeking experience) to provide users with additional pathways to access therapeutic content. For these features, specific lessons are recommended after the feature is accessed. ClearlyMe® also includes in-app reminders, ‘saving’ and ‘favourite’ functions to support users to return to the app to reengage in content. The app also includes a ‘Get Help’ section that contains information of when and where to access additional mental health support services. ClearlyMe® is the first self-guided smartphone app providing a comprehensive CBT intervention developed specifically for adolescent depression. A rigorous clinical treatment trial among adolescents with mild to moderately-severe depressive symptoms is now needed to determine whether ClearlyMe® is effective for improving depressive symptoms and for engaging young people in depression treatment delivered via their smartphone.

Trial objectives

This protocol is reported following SPIRIT guidelines to facilitate the reporting of the trial results using CONSORT guidelines [37]. It describes the methodology for the MobiliseMe study. The primary aim of the MobiliseMe study is to evaluate the effectiveness of the ClearlyMe® app for reducing depressive symptoms in adolescents. Using a three-arm parallel-group randomised controlled trial (RCT), the MobiliseMe study will compare self-directed use of the ClearlyMe® app with guided use alongside an attention-matched control condition to determine the effectiveness of the app for improving self-reported depressive symptoms in adolescents with mild to moderately-severe depressive symptoms after 6-weeks of use (primary endpoint) and at 4-month follow-up (secondary endpoint). This study will also evaluate the secondary impacts of the ClearlyMe® app on anxiety symptoms, psychological distress, emotional wellbeing, quality of life, rumination, emotion regulation and CBT skill acquisition. Participants’ engagement with the ClearlyMe® app (i.e., app uptake, treatment adherence and completions) and its influence on outcomes will also be explored. Creating and reporting multiple measures of engagement facilitates an understanding of the multiple ways participants may engage with the intervention, allows comparison between interventions and follows recent recommendations [38].

Hypotheses

The primary hypothesis is that relative to the attention control condition, participants who receive the ClearlyMe® app (self-directed or guided) will have greater reductions in self-reported depressive symptoms between baseline and 6-weeks post-baseline (primary endpoint), as well as baseline and 4-month follow-up (secondary endpoint). A secondary hypothesis is that relative to the control condition, participants who receive the ClearlyMe® app (self-directed or guided) will report greater improvements in anxiety symptoms, psychological distress, emotional wellbeing, quality of life, rumination, emotion regulation and CBT skill acquisition between baseline and 6-weeks post-baseline, as well as baseline and 4-month follow-up. It is hypothesised that participants who receive guided support to use ClearlyMe® will report significantly higher levels of engagement than those who do not, however, no hypotheses have been made regarding the effect of guided support on outcomes given mixed findings in the literature e.g. [19, 21, 30, 35]. Exploratory analyses will examine whether the intervention was more effective for certain subgroups of the sample (e.g., based on age, symptom severity at baseline, greater perceived need for care, and higher openness to digital mental health) and whether the effects of the intervention were driven by other variables, such as rumination, emotion regulation, CBT skill acquisition, digital therapeutic alliance, and measures of app engagement.

Methods/design

Trial design

This superiority trial will utilise a three-arm, parallel group randomised controlled trial design with an equal allocation ratio. Outcome measures will be assessed at baseline, post-intervention (primary endpoint, measured at 6-weeks post-baseline) and follow-up (secondary endpoint, measured at 4-months post-baseline). A 4-month follow-up was selected to ensure participants in the control condition were provided with the intervention after a reasonable duration and to reduce participant attrition. The University of New South Wales is the sponsor of this clinical trial and ethics approval was given by the University of New South Wales Human Research Ethics Committee (HC#210889). This trial was registered with the Australian New Zealand Clinical Trials Registry on the 27th of January 2022 (ACTRN12622000131752) and has been allocated the Universal Trial Number U1111-1271-8519.

Setting

This trial will be conducted entirely online, using the bespoke Black Dog Institute Research Engine, purpose built from which to conduct research trials. Data will be collected from individuals residing in Australia.

Participants

Eligible adolescents are those aged 12 to 17 years old; located in Australia; who own or have access to a smartphone; have access to the Internet, an active email address and mobile phone number; comfortable with reading English at Grade 7-8 level; and who can obtain their parent or guardian’s consent. Eligible adolescents must also report mild to moderately-severe depressive symptoms as determined by a total score ranging between 5 and 19 on the Patient Health Questionnaire for Adolescents (PHQ-A) at screening. Ineligible adolescents are those who are currently receiving or about to start psychological treatment from a mental health professional for symptoms of depression or anxiety; are currently taking or about to start a course of prescribed medication for symptoms of depression or anxiety; report severe suicidal ideation as determined by a score of ≥2 on item 9 of the PHQ-A; respond ‘yes’ to items asking whether within the past month they have had serious thoughts or intentions about ending their life, or have made an attempt to end their life; nil or severe depressive symptoms as determined by a total score of ≤4 or ≥ 20 on the PHQ-A; or fail to satisfy any of the inclusion criteria at screening. Uptake of other depression interventions during the trial is permitted and will be assessed at primary and secondary endpoints.

Interventions

Participants allocated to the intervention conditions (i.e., self-directed use of ClearlyMe® and guided use of ClearlyMe®) will be instructed to complete the ClearlyMe® app content by undertaking at least one collection per week (approximately 20 minutes in duration) for 6 weeks; however, participants can complete the app content in any order and according to their preferences. Participants will also be instructed to use the ‘Get Help’ section of the ClearlyMe® app if they feel they require extra mental health support during the intervention period, although this is not monitored by the research team. To promote compliance, participants allocated to this condition will be sent weekly SMS reminders (total of 6) to use the app. The ClearlyMe® app also includes an in-built ‘revisit the app’ reminder, which notifies participants to use the app after 7 days of inactivity. Participants who allow notifications during the app’s onboarding will receive these reminders but can deactivate it at any time via their mobile phone settings. Participants who fail to download the ClearlyMe® app within 7 days of randomisation will be sent one email and one SMS reminder to download the app. Participants who do not download the app will remain in the trial and be included in the intention to treat analyses.

Participants in the Guided use of ClearlyMe® condition will also receive a weekly guided support session delivered outside of school hours via Short Message Service (SMS) by a member of the research team using a standard script and decision flow chart (see Additional File 1). Research Assistants (RAs) will be trained to conduct these sessions and will be supervised by an experienced clinical psychologist. These sessions will consist primarily of technical support with some low intensity motivational coaching to encourage app use. The chat sessions are designed to take place in one sitting (i.e., approximately 20 minutes) but given the nature of SMS communication, will occur until the script is completed. Duration and script compliance will be recorded. Chat sessions will be prescheduled to take place between 3:30 pm and 6 pm on Mondays to Fridays. For non-responsive participants, the research team will instigate one additional contact attempt, made within 2 days of the initial attempt, however, the weekly chat schedule will remain unchanged, regardless of when a participant responds. Participants in this condition will be provided an additional $25 reimbursement for phone credit upon allocation to the guided support condition.

Control condition

This study will utilise an active attention-matched control condition consisting of digital psychoeducation flyers. Attention-matched control conditions account for nonspecific intervention effects and are frequently used in other studies within the digital health field [39]. Psychoeducation is a known and acceptable intervention for young people that has demonstrated small positive effects on mental health without containing active CBT elements [40,41,42]. It is also information frequently consumed by young people during autonomous digital help seeking [36]. Digital psychoeducation in the current study has been designed to be comparable to the intervention conditions in terms of weekly dosage of content, activities, visual appeal, and duration to complete. These factors ensure digital psychoeducation satisfies both ethical and methodological requirements and has the potential to reduce control condition dropout [43]. Participants allocated to this condition will receive six weekly psychoeducation flyers, delivered via SMS that contains a URL to a static Portable Document Format (PDF). Participants are required to click the URL to access the flyer directly on their mobile device. The psychoeducation consists of six topics (taking approximately 20 minutes to read) containing information on mental health problems, links to credible Australian mental health organisations, and suggested activities for self-care. The content was created by the Black Dog Institute and used in other mental health studies among adolescents [41, 42]. Participants in this condition will not receive any additional contact from the research team, unless technical support (requested via email) is required. Engagement with the psychoeducation flyers will be determined by recording the number of times an individual participant clicks the flyer link, including time and date of access. Post-trial, participants in this condition will receive one email that includes a collated PDF containing all the psycho-education material in one file for future use. Participants will be instructed to use the ‘Get Help’ section of the study site if they feel they require extra mental health support during the study period, although this will not be monitored by the research team. All participants in the control condition will be provided with 6-weeks access to ClearlyMe® once their participation in the trial has concluded.

Procedure and participant timeline

Table 1 outlines the schedule of enrolment, interventions and assessments and Fig. 1 the study flow. Interested adolescents will be directed to the study webpage and asked to undergo a short online screener to determine their eligibility. Participants who do not pass screening will be provided with mental health service information. Only eligible participants will be invited to review the online Participant Information and Consent Form (PICF; see here) and proceed to the baseline assessment. Once completed, participants will be asked to register to the study by creating an account on the Black Dog Institute Research Engine. Participants are required to provide their full name, email, mobile phone number, date of birth, and to create a password. Registered participants will then be invited to view the online Parent Participant Information and Consent Form (P-PICF), to be completed by their parent or guardian within 1 week. Participants will receive three reminders to submit the P-PICF. Upon completion of the parental consent form, parents will be automatically emailed a copy of the P-PICF. If an invalid parent email address is entered, the research team will be notified via the study inbox and will contact the participant via email to confirm their parent’s details. Once the P-PICF has been submitted, participants will be instructed to complete the baseline assessment within 7 days. Participants who do not provide full consent or complete their baseline assessment within the allocated time will be automatically withdrawn from the study.

Table 1 Schedule of enrolment, interventions, and assessments
Fig. 1
figure 1

Study flow

Upon completion of the baseline assessment, participants will be randomised to one of the three study conditions and receive the relevant intervention instructions via email and SMS. At 6-weeks post-baseline and at 4-months follow-up, all randomised participants will be invited via email and SMS to complete the study assessments. Participants will have 7 days to complete each assessment and will receive two reminders to do so. All study assessments will take approximately 20 minutes to complete and can be undertaken on any Internet-enabled device. Participants will be reimbursed 10AUD (electronic gift voucher) for each study assessment completed (30AUD total). In addition, participants receiving guided use of ClearlyMe® will receive 25AUD upon group allocation to cover the cost of sending and receiving SMSs. Discontinuation of the allocated intervention will occur if the participant withdraws from the trial or in the event that they are no longer able to engage in the intervention due to changed circumstances.

Outcomes

Depressive symptoms

The primary outcome of this study is self-reported depressive symptoms as measured by the Patient Health Questionnaire-9 Adolescent version (PHQ-A) [44]. This 9-item self-report measure assesses the severity and frequency of depressive symptoms in the previous 14 days, with items rated on a 4-point Likert scale ranging from not at all (0) to nearly every day (3). Items are summed to produce a total score (range: 0 to 27) with higher scores indicating more severe depressive symptoms. Total scores will be used as the outcome measure, and can be classified into the following symptom severity categories: nil to minimal (0-4), mild (5-9), moderate (10-14), moderately severe (15-19), and severe (20-27). The PHQ-A is an adapted version of the validated PHQ-9 questionnaire [44, 45], has good psychometric properties for assessing depressive symptomology in adolescent populations [46,47,48,49] and has been endorsed for research and clinical evaluation with adolescents [44, 50, 51].

Generalized anxiety symptoms

This will be measured by the Generalized Anxiety Disorder-7 scale (GAD-7) [52]. This 7-item self-report measure assesses the severity and frequency of anxiety symptoms in the previous 14 days, with items rated on a 4-point Likert scale ranging from not at all (0) to nearly every day (3). Items are summed to produce a total score (range: 0 to 21) with higher scores indicating more severe anxiety symptoms. Total scores will be used as the outcome measure, and can be classified into the following symptom severity categories: nil to minimal (0-4), mild (5-9), moderate (10-14), moderately severe (15-19), and severe (20-27). The GAD-7 has good psychometric properties for assessing anxiety symptoms in adolescent populations [53, 54].

Psychological distress

This will be measured by the Distress Questionnaire-5 (DQ-5) [55]. This 5-item self-report measure assesses the frequency of psychological distress in the previous 30 days, with items are rated on a 5-point Likert scale, ranging from never (1) to always (5). Items are summed to produce a total score (range: 5 to 25) with higher scores indicating more severe psychological distress. The scale has established psychometric properties [55, 56] and has been used in adolescent populations [41, 57,58,59,60].

Emotional well-being

This will be measured by the Short Warwick-Edinburgh Mental Wellbeing Scale (SWEMWBS) [61, 62]. This 7-item self-report measures assesses individuals’ wellbeing in the previous 2 weeks, with items rated on a 5-point Likert scale, ranging from none of the time (1) to all of the time (5). Items are summed to produce a total score (range: 7 to 35) with higher scores indicating greater well-being. It has adequate psychometric properties to measure emotional wellbeing [63] in adolescent populations [64,65,66].

Quality of life

This will be measured by the Child Health Utility 9D (CHU-9D) [67]. This 9-item self-report measure assesses adolescents’ health related quality of life in nine domains of functioning including: worry, sadness, pain, tiredness, annoyance, schoolwork/homework, sleep, daily routine and ability to join activities. Each domain is rated on a 5-point scale, with each level representing increasing levels of severity within each domain e.g., I don’t feel worried today (1) to I feel very worried today (5). Items are summed to produce a total score (range: 9 to 45). The scale has good psychometric properties among adolescents [68,69,70] and can estimate quality adjusted life years for use in future cost-effectiveness evaluations of ClearlyMe® [71].

Rumination

This will be measured by the Ruminative Responses Scale – short version (RRS) [72]. This 10-item self-report measure assesses individuals’ tendency to engage in rumination. Individuals are asked to rate each item in response to the prefacing statement “what you generally do, not what you think you should do when feel down, sad or depressed”. The scale is composed of two subscales; reflection and brooding, with 5-items relating to each factor. Each item is rated on a 4-point scale ranging from almost never (1) to almost always (4). Items are summed to produce a total score (range: 10 to 40). Higher scores reflect higher levels of ruminative responses. The measure has good psychometric properties in adolescent populations [73,74,75].

Emotion regulation

This will be measured by the Emotion Regulation Questionnaire for Children and Adolescents (ERQ-CA) [76]. This a 10-item measure assessing individuals’ propensity to use cognitive reappraisal (6-items) and expressive suppression (4-items) as emotion regulation strategies. Items are rated on a 5-point scale, ranging from Strongly disagree (1) to Strongly agree (5). Items are summed on each subscale to produce total scores; cognitive reappraisal ranging from 6 to 30 and expressive suppression ranging from 4 to 20. Higher scores on each subscale reflect greater use of the corresponding emotion regulation skill. The measure has shown good psychometric properties for the assessment of emotion regulation strategies in children and adolescents [76].

CBT skill acquisition

This will be measured by the Cognitive Behavioural Therapy Skills Questionnaire (CBTSQ) [77]. The CBTSQ is a 16-item scale assesses the frequency of individuals’ use of cognitive behavioural skills. The measure is composed of two subscales measuring cognitive restructuring skills (9-items) and behavioural activation strategies (7-items). Items are rated on a 5-point scale, ranging from I don’t do this (1) to I always do this (5). Items are summed on each subscale to produce a total score for cognitive restructuring (range: 9 to 45) and for behavioural activation (range: 7 to 35). Higher scores on the subscales indicate greater use of CBT skills. The CBTSQ has good psychometric properties [77] and has been used in adolescent populations [78].

Demographics

Participants will be asked their gender identity (female, male, non-binary, different identity, free response option), whether they identify as Aboriginal or Torres Strait Islander (yes, no, prefer not to say), whether they identify as Lesbian, Gay, Bisexual, Transgender, Queer and/or Intersex (yes, no, I’d prefer not to say), their Australian state or territory location (New South Wales [NSW], Queensland [QLD], Victoria [VIC], Tasmania [TAS], South Australia [SA], Western Australia [WA], Northern Territory [NT], Australian Capital Territory [ACT]), location description (metropolitan, regional, or rural/remote), their date of birth, school grade (Years 7, 8, 9, 10, 11, 12, not currently in school) and the type of mobile device they use (iOS [e.g., Apple], Android [e.g., Samsung], other). Participants will also be asked how they heard about the study (e.g., social media, BDI website, word of mouth, other) and their motivations for participating (e.g., need for mental health care, desire to contribute to a broader social good, parents/carers encouragement, friend participating, financially motivated, desire to help research, interest in learning about mental health or other).

Recent mental health care

At baseline participants will be asked to report whether they have ever been diagnosed with anxiety or depression by a health professional (yes - depression only, yes - anxiety only, yes -both depression and anxiety, no, I don’t know, I’d rather not say), whether they have ever received psychological treatment for a mental health problem or mental illness from a health professional (yes, no, I’d rather not say) and whether they have ever taken prescribed medication for a mental health problem or mental illness like depression or anxiety (yes, no, I’d rather not say). At the primary and secondary endpoints participants will be asked to report whether throughout the study they were diagnosed with depression or anxiety by a health professional (yes - depression only, yes - anxiety only, yes -both depression and anxiety, no, I don’t know, I’d rather not say), whether they received psychological treatment from a health professional (yes, no, I’d rather not say), starting taking prescribed medication for a mental health problem or mental illness (yes, no, I’d rather not say) or felt like they needed help for a mental health issue like depression or anxiety (yes, no, I’d rather not say).

Previous use of mobile apps for mental health

Participants will be asked to report whether they have ever used any mobile app to help with their emotional wellbeing or mental health (yes, no, I’d rather not say), and if yes, whether they found it helpful (yes, no). Participants will also be asked how much they think their emotional wellbeing could be improved by using a mental health smartphone app on a 5-point scale ranging from not at all (1) to extremely (5).

Perceived need and help-seeking barriers

This will be measured by the Perceived Need for Care Questionnaire (PNCQ). The PNCQ is a four-item measure designed to determine an individuals’ perceived need for mental health care. Perceived need is assessed across four domains of adolescent mental health care: counselling, medication, information, and skill training. Each domain is assigned one of four levels of perceived need (need fully met, need partially met, need unmet, or no need) using the information provided regarding whether sufficient care was received. If perceived need is not fully met, participants select the type of help needed (e.g., counselling, medication, information, and skill training) and the barriers (e.g., self-reliance, stigma, accessibility, financial reasons) kept them from receiving adequate care. Barriers are categorised as attitudinal or structural (e.g., “Couldn’t get an appointment when needed [structural]” and “Wanted to work out the problem on my own [attitudinal; self-reliance]”) [15]. The instrument has good psychometric properties [79] and has been adapted for use in large representative samples of adolescents [80, 81].

Digital therapeutic Alliance (digital working Alliance inventory)

This will be measured by the Digital Working Alliance Inventory (DWAI) [82]. The DWAI is a 6-item measure adapted from the Working Alliance Inventory- short version to assess the therapeutic alliance between an individual and digital intervention (WAI-SF) [83, 84]. The measure assesses three core domains: goals (e.g., I trusted the app to guide me towards my personal goals), tasks (e.g., I believed the app tasks will help me to address my problem) and bond (e.g., The app encouraged me to accomplish tasks and make progress). Each item is rated on a 7-point scale ranging from strongly disagree (1) to strongly agree (7). Items are summed to produce a total score (range: 6 to 42). Higher scores reflect higher levels of digital working alliance with the program. The original scale has good psychometric properties [83, 84], and preliminary evidence of the DWAI’s psychometric properties is positive, however it is yet to undergo rigorous psychometric evaluation [85].

Digital program satisfaction

This will be assessed using an 11-item measure adapted from previous digital mental health research [59]. The measure is designed to assess participants’ satisfaction, experience and perceived helpfulness of the intervention received. For the first eight items on the scale, participants are asked to agree or disagree with a set of statements that assess aspects of their satisfaction and experience with the intervention. The measure also includes a helpfulness item, where participants are asked to rate the overall helpfulness of the intervention on a 5-point scale ranging from extremely unhelpful (1) to extremely helpful (5). The final two free response questions examine how ClearlyMe® or the psychoeducation flyers were helpful (e.g., In what ways did ClearlyMe help you?) and provide participants with the opportunity to suggest improvements (e.g., What would make ClearlyMe better?).

SMS chat feedback

The SMS Chat Feedback questionnaire is an 8-item measure that was adapted from [86] by the study authors to gain feedback on the SMS chat support provided to participants in the guided support condition. The first item on the scale asks participants whether they participated in the SMS chat support (yes, no). Participants who respond ‘no’ are asked what stopped them participating (free response). Participants who respond ‘yes’ are asked seven additional questions. Five are rated on a 5-point scale and assess how much participants liked the SMS chats (disliked a lot [1] to liked a lot [5]), perceived helpfulness for encouraging use of ClearlyMe® (extremely unhelpful [1] to extremely helpful [5]), convenience of the timing (extremely inconvenient [1] to extremely convenient [5]), and their level of satisfaction with the frequency and length of the SMS chats (not at all satisfied [1] to extremely satisfied [5]). Participants will also be asked two free response questions about what they liked or disliked about the SMS chats and how the SMS chats could be improved.

Digital program barriers

The Digital Program Barriers questionnaire is a 14-item measure was adapted from previous digital mental-health research conducted by researchers of this study [59]. This measure is designed to examine any barriers or difficulties that participants experienced accessing or using the ClearlyMe® app or the psychoeducation flyers. Each item requires participants to select “yes” or “no” in response to a statement describing a barrier, including technical issues, accessibility and individual factors.

Engagement – ClearlyMe®

The current study utilises and will report multiple measures of engagement, consisting of ‘uptake’, ‘adherence’, and ‘completions’. In the current study, ‘uptake’ is defined as the proportion of participants (%) who take up the app, measured by app downloads. ‘Adherence’ is defined as the proportion (%) of participants who followed the instructions for app use. This will be measured in two ways: 1) A categorical variable (adherers vs. non-adherers) in which participants who complete at least one collection each week for the entire 6 weeks will be deemed ‘adherers’; 2) A numerical variable in which participants will earn 1 point for each week that they adhered to the instructions (i.e., completed at least one collection) resulting in a total adherence score (range: 0 to 6). ‘Completions’ is used to describe participants’ exposure to the app content, measured in three ways: 1) the number of lessons completed as a proportion of the total number of lesson, 2) the proportion (%) who completed more than half of the app content (i.e, 19 of 37 lessons), 3) the proportion (%) that completed all of the app content (i.e., all 37 lessons). Additional exploratory data will be collected to describe participants’ use of the ClearlyMe® app: collections completed (out of 9), the total number of times app features are accessed (e.g., Stories, View all, Get help now, Mood check, Saved), time spent in app (minutes), the collections and lessons liked/disliked, the collections and lessons saved, and individual responses to the lesson activities. Different measures of engagement have been employed to move beyond the pervasive conceptualisation of engagement as the ‘more use, the better’ in acknowledgement that there are multiple ways participants may engage with the intervention [38, 87, 88]. Recommendations have been followed [88] to include measures frequently used in the literature (e.g., proportion of lessons completed), intervention specific measures (e.g., frequency of feature access) and a measure of adherence defined as the extent to which actual use aligns with instructions for use. This allows a deeper analysis of the most beneficial way to use ClearlyMe® and supports comparisons with other studies given the heterogeneous reporting of engagement [36, 38, 88].

Engagement – control condition

For participants allocated to the control condition, the number of times each psychoeducation flyer is accessed via the URLs in the weekly SMSs will be recorded for each participant and include time and date of access.

Sample size

The total sample size required for detecting change in the primary outcome at the primary endpoint was calculated to be 489. This was based α = 0.05, power = 0.8, small to medium within-group effect sizes for psychoeducation (d = 0.10) [40, 41], self-directed CBT (d = 0.35) [89], and guided CBT (d = 0.65) [18, 19] on depressive outcomes and a conservative attrition rate of 20% between baseline and primary endpoint. This sample size will also be sufficient to detect small between-group effect sizes between the intervention conditions and the attention control condition and between the self-guided and supported use intervention conditions.

Recruitment

Trial recruitment began on the 11th of May 2022, subsequent to submission of the trial protocol for peer-review. The first trial participant was enrolled on the 12th of May 2022. This study will utilise an online recruitment strategy. Study advertisements will be published on the Black Dog Institute website and social media channels (Facebook, Twitter, and Instagram) and provided to schools and organisations to share with their communities. Paid advertisements will be placed on social media sites (Twitter, Facebook, Instagram, Snapchat and Google). Online recruitment is an effective avenue to recruit generalised mental health samples [59, 90]. The research team will also contact relevant Australian mental health organisations and services to request distribution of study advertisements on their organisation’s communication channels (e.g., website, social media, newsletters, mailing list, in-person clinics). Study advertisements will also be emailed to individuals who have consented to being contacted about research studies conducted by the Institute. All study adverts will direct interested participants to the study website, where they will review the study information, and may choose to undertake screening, and provide consent using the provided links. Direct contact between the participant and study team is not required during recruitment and study enrolment. To ensure participant safety during recruitment, excluded participants are provided with contact information for Australian mental health and crisis support services and encouraged to seek help and speak to a trusted adult. The same information is provided to included participants immediately following consent, again at the end of the baseline survey and in the study welcome email. They are also encouraged to contact the research team if they have questions or are experiencing distress.

Randomisation and blinding

Randomisation will be carried out according to the International Council for Harmonisation guidelines [91]. Randomisation to one of the three trial arms will be conducted immediately after completion of the baseline assessment using a computerised randomisation procedure within the Black Dog Institute Research Engine. A stratified randomisation approach with a block size of 6 (1:1 ratio) will be used to ensure balance across the conditions in terms of age (12 to 14 years vs. 15 to 17 years), and symptom severity (mild [as determined by a total score of 9 or less on the PHQ-A] vs moderate to moderately-severe [as determined by a total score of 10 or more on the PHQ-A]). Allocation will be fully automatic, with no input from the research team. Participants will not be directly informed of their condition allocation; however, they may be able to deduce this based on differences in the study activities. For the primary outcomes analyses, the statistician will be blinded to participants’ allocations. The statistician will be unblinded when examining intervention completion rates due to differences in the total number of modules/lessons across each condition as well as when conducting exploratory moderator analyses. Key members of the research team not involved in data analysis, including research assistants providing guided support, will be unblinded to participants’ allocation as they will require access to the Black Dog Institute Research Engine to contact participants if they experience adverse events and to provide support to participants allocated to the guided use condition.

Data collection, management and statistical methods

All participant data will be collected and stored on the Black Dog Institute Research Engine, hosted on secure servers commissioned by the University of New South Wales at GovDC Data Centres located in Sydney, Australia. The data is stored in a SQL Server 2016 database which is backed up daily. The Research Engine automatically generates a unique participant identification code, to enable the protection of participant confidentiality. For analyses, all study assessment data will be exported, via a Microsoft Excel file, from the Research Engine to SPSS Version 26. These files will be stored on UNSW OneDrive and deidentified by approved members of the research team. All identifiable information, including participant name, email address, mobile phone number, IP addresses, and free response data, and group allocation will be removed from the outcomes analysis file. This dataset will then be transferred to the trial statistician for analyses using a password protected OneDrive file. All identifiable study assessment data will only be accessible to the research team listed on the project, while aggregate deidentified data will be retained for future evidence synthesis.

Primary analyses will be conducted to determine the effect of the interventions on depressive symptoms at the primary and secondary endpoints. Analyses will be undertaken on an intention-to-treat basis, including all participants randomised, regardless of adherence or intervention received. The effectiveness of the trial interventions will be established by change on the PHQ-A between baseline and 6-weeks post-baseline (primary endpoint) and baseline and 4-month follow-up (secondary endpoint), based on the interaction between time and condition, using mixed-effects repeated measures linear modelling with an unadjusted p value of 0.05. The mixed-effect linear modelling will account for all available data, under the missing at random assumption. Effect sizes will be calculated based on differences in observed change scores between baseline and 6-weeks post-baseline, using standard deviations of the change scores pooled across conditions. Attrition analyses will be conducted to determine whether missing data is associated with any baseline demographics (age, gender), mental health status (symptom severity, past or current treatment) or other descriptive variables. Any baseline variables identified as substantially imbalanced between groups will be added to the models on an exploratory basis to confirm the robustness of the findings to this imbalance. Where distributional assumptions cannot be satisfied, other modelling may be used to confirm the robustness of the findings. Similar models will be used for secondary outcomes. To analyse the effects of adherence on outcomes, the mixed-effects repeated measures linear modelling will be repeated for each intervention condition separately using the categorical adherence measure as a between group factor (adherers v non-adherers v control). The categorial adherence measure has been selected for this analysis to determine whether instructions for use optimise benefit as intended or require adjustment. These results will be published in the primary outcomes paper. Exploratory analyses will examine evidence for moderation and mediation, that is, whether the intervention was more effective for certain subgroups of the sample (e.g., based on gender, baseline symptom severity, perceived need for help and openness to digital interventions) and whether the effects on outcomes were influenced by other variables, including rumination, emotion regulation, CBT skill acquisition, digital therapeutic alliance, and measures of app engagement. These results may be published in separate papers to appropriately describe the methods and analyses. A deidentified data set will be made available upon reasonable request at the discretion of the principal investigator.

Monitoring

The current trial will utilise a Trial Management Group to supervise the overall conduct and safety of the trial in accordance with the National Health and Medical Research Council (NHMRC) guidelines on Data Safety Monitoring [92]. Details about this group are published on the Australian New Zealand Clinical Trials Registry (see here). As this trial includes symptomatic adolescents, each study assessment will include a list of mental health services and support resources suitable for adolescents. Participants who report an Adverse Event (AE), which includes severe suicide ideation (indicated by a score of ≥2 on item-9 of the PHQ-A) reported at baseline or the primary or secondary end-points and severe depressive symptoms (indicated by a total score ≥ 20 on the PHQ-9-A) reported at the secondary endpoint, will be offered a follow-up call from a Clinical Psychologist who will conduct a risk assessment over the phone. If at risk, the participant will be advised to contact an appropriate service. Parents will be contacted if the risk of harm is deemed imminent or the participant requests it. All adverse events and serious adverse events will be formally recorded and reported to the Trial Management Group in a monthly report and to the University of New South Wales Human Research Ethics Committee by the Clinical Trial Manager. No interim analyses on primary or secondary outcomes are planned, unless it is warranted by safety reports or upon the request of the Trial Management Group. The Trial Management Group may recommend pausing or terminating the trial if they have concerns for participant safety, based on (but not limited to) a higher than anticipated rate of one or more of the primary endpoints. At any time, the University of New South Wales can audit trial conduct and this process will be independent from the investigators and the sponsor.

Dissemination

A summary of the research results will be published on the Black Dog Institute website and emailed to all participants and their parents upon completion of the data analysis. Trial outcomes will also be prepared for publication in relevant peer-reviewed journals by the research team and presented at academic conferences. In accordance with the National Health and Medical Research Council open access policy, the researchers will endeavour to publish all papers from this research in open access journals. In all reports, participants will not be individually identifiable. Numerical data will be presented at the aggregate level. Any qualitative data reported will use the non-identifiable code allocated to it.

Discussion

This protocol outlines the MobiliseMe study, a clinical trial which aims to examine the effectiveness of a new CBT-based smartphone app (ClearlyMe®) for reducing depressive symptoms and other mental health outcomes, including anxiety and wellbeing, in adolescents aged 12-17 years. ClearlyMe® was designed to address current gaps in treatment provision for depressed adolescents by creating a smartphone app containing a full course of CBT with qualities, such as visual appeal, usability and relevance, equivalent to popular commercially developed apps. The provision of CBT via a means that is accessible, youth friendly and retains users is critical to ensure effective early intervention, reduce dysfunction and prevent potential chronic disability across the lifespan [22, 27, 28]. If found to be effective, ClearlyMe® has the potential to provide an accessible, low cost, and scalable treatment to improve depressive symptoms in young people. This study addresses several gaps in the literature. It is the first evaluation of a CBT-based smartphone app targeting depression, designed with, and specifically for adolescents. It is also the first trial to compare the outcomes between users receiving guided support and those who do not, and to explore the factors that promote engagement and effectiveness. Outcomes of this trial will advance knowledge of how to support young people experiencing mental health symptoms using digital technologies and inform innovations in the field.

Strengths

The design of the MobiliseMe trial is robust. First, the inclusion of an active-attention matched control condition will allow an evaluation of the impact of ClearlyMe® over and above the support young people commonly receive when help-seeking independently, that is, information about mental symptoms and treatment options [93]. Second, a comparison condition that involves using ClearlyMe® with SMS guidance is well-matched to the needs of young people [31, 32] and will provide much needed knowledge on the influence of guidance on engagement and effectiveness. Third, the sample size (N = 489) of the trial will ensure adequate power to detect intervention effects. Finally, a thorough consideration of ‘engagement’ and transparent reporting of how constructs such as uptake, adherence and completion have been operationalised, measured and analysed addresses a well-documented limitation in the field [22, 67, 88, 94].

Limitations

Due to its scale, this study will not conduct clinical interviews to yield a formal diagnosis of a mental disorder. However, measures with well documented psychometric properties that provide clinically relevant classifications of adolescent depressive symptomology and severity, which are frequently used in the literature for this purpose, have been selected [51]. Engagement with, and effectiveness of ClearlyMe®, will be examined in the controlled environment of a research trial providing little indication of how the app will be used in less controlled environments, such as in the community. Additional research examining the use of ClearlyMe® will be required to determine whether adolescents use the intervention differently in real-world settings [95]. The 4-month follow-up assessment will not allow for conclusions to be drawn about the long-term sustainability of the effects of ClearlyMe®. The incorporation of longer follow-up periods in future studies will be required. Instructions on how to use ClearlyMe® (i.e., at least 1 collection each week) were determined by predicting use that would produce optimal therapeutic benefit based on clinical experience. While this may not prove to be the optimal way to use ClearlyMe®, this trial will allow an examination of patterns of use associated with benefit. Finally, while the purpose of this first evaluation is to determine whether ClearlyMe® is effective and safe, subsequent evaluations could include an assessment of its cost effectiveness.

Notwithstanding these limitations, outcomes of this trial will provide valuable insights into the potential of a new CBT smartphone app to fill a gap in current treatment provision and factors driving and associated with therapeutic benefit.

Availability of data and materials

Data sharing is not applicable to this article as no datasets were generated or analysed during the current study. A detailed study design and analysis plan were preregistered, including access to material, see here.

Abbreviations

ACT:

Australian Capital Territory

AE:

Adverse Event

BDI:

The Black Dog Institute

CHU-9D:

Child Health Utility 9D

CBT:

Cognitive Behavioural Therapy

CBTSQ:

Cognitive Behavioural Therapy Skills Questionnaire

DWAI:

Digital Working Alliance Inventory

DQ-5:

Distress Questiionaire-5

ERQ-CA:

Emotion Regulation Questionnaire for Children and Adolescents

GAD-7:

Generalized Anxiety Disorder-7 sc

NHMRC:

National Health and Medical Research Council

NSW:

New South Wales

NT:

Northern Territory

PHQ-A:

Patient Health Questionnaire for Adolescents

PNCQ:

Perceived Need for Care Questionnaire

QLD:

Queensland

RCT:

Randomised Controlled Trial

RRS:

Ruminative Responses Sca

SA:

South Australia

SMS:

Short Message Service

SWEMWBS:

Short Warwick-Edinburgh Mental Wellbeing Scale

TAS:

Tasmania

TMG:

Trial Management Group

UNSW:

University of New South Wales

VIC:

Victoria

WA:

Western Australia

References

  1. Hughes EK, Gullone E, Watson SD. Emotional functioning in children and adolescents with elevated depressive symptoms. J Psychopathol Behav Assess. 2011;33(3):335–45.

    Article  Google Scholar 

  2. Verboom CE, Sijtsema JJ, Verhulst FC, Penninx BWJH, Ormel J. Longitudinal associations between depressive problems, academic performance, and social functioning in adolescent boys and girls. Dev Psychol. 2014;50(1):247–57.

    Article  PubMed  Google Scholar 

  3. Racine N, McArthur BA, Cooke JE, Eirich R, Zhu J, Madigan S. Global prevalence of depressive and anxiety symptoms in children and adolescents during COVID-19: a Meta-analysis. JAMA Pediatr. 2021;175:1142–50.

    Article  PubMed  Google Scholar 

  4. Davey CG, McGorry PD. Early intervention for depression in young people: a blind spot in mental health care. Lancet Psychiatry. 2019;6(3):267–72.

    Article  PubMed  Google Scholar 

  5. Barlow DH, Farchione TJ, Fairholme CP, Ellard KK, Boisseau CL, Allen LB, et al. Unified protocol for transdiagnostic treatment of emotional disorders: therapist guide. Oxford: Oxford University Press; 2010.

  6. Zinbarg RE, Craske MG, Barlow DH. Mastery of your anxiety and worry (MAW): therapist guide. Oxford: Oxford University Press; 2006.

  7. David D, Cristea I. The new great psychotherapy debate: Scientific integrated psychotherapy vs plurality Why cognitive-behavior therapy is the gold standard in psychotherapy and a platform for scientific integrated psychotherapy. J Evid Based Psychother. 2018;18(2):1.

    Article  Google Scholar 

  8. Higa-McMillan CK, Francis SE, Rith-Najarian L, Chorpita BF. Evidence base update: 50 years of research on treatment for child and adolescent anxiety. J Clin Child Adolesc Psychol. 2016;45(2):91–113.

    Article  PubMed  Google Scholar 

  9. Watts SE, Turnell A, Kladnitski N, Newby JM, Andrews G. Treatment-as-usual (TAU) is anything but usual: a meta-analysis of CBT versus TAU for anxiety and depression. J Affect Disord. 2015;175:152–67.

    Article  PubMed  Google Scholar 

  10. Weersing VR, Jeffreys M, Do M-CT, Schwartz KT, Bolano C. Evidence base update of psychosocial treatments for child and adolescent depression. J Clin Child Adolesc Psychol. 2017;46(1):11–43.

    Article  PubMed  Google Scholar 

  11. Wise J. Depression in children: offer digital CBT as first line treatment, says NICE. BMJ. 2019;364:l364.

    Article  PubMed  Google Scholar 

  12. Aguirre Velasco A, Cruz ISS, Billings J, Jimenez M, Rowe S. What are the barriers, facilitators and interventions targeting help-seeking behaviours for common mental health problems in adolescents? A systematic review. BMC Psychiatry. 2020;20(1):293.

    Article  PubMed  PubMed Central  Google Scholar 

  13. Cadigan JM, Lee CM, Larimer ME. Young adult mental health: a prospective examination of service utilization, perceived unmet service needs, attitudes, and barriers to service use. Prev Sci. 2019;20(3):366–76.

    Article  PubMed  PubMed Central  Google Scholar 

  14. Gulliver A, Griffiths KM, Christensen H. Perceived barriers and facilitators to mental health help-seeking in young people: a systematic review. BMC Psychiatry. 2010;10(1):113.

    Article  PubMed  PubMed Central  Google Scholar 

  15. Mojtabai R, Olfson M, Sampson NA, Jin R, Druss B, Wang PS, et al. Barriers to mental health treatment: results from the National Comorbidity Survey Replication. Psychol Med. 2011;41(8):1751–61.

    Article  CAS  PubMed  Google Scholar 

  16. Gandhi E, Ogradey-Lee M, Jones A, Hudson JL. Receipt of evidence-based care for children and adolescents with anxiety in Australia. Aust N Z J Psychiatry. 2022;56(11):1463–76. https://doi.org/10.1177/00048674211068780.

    Article  PubMed  Google Scholar 

  17. Sawyer MG, Reece CE, Sawyer ACP, Hiscock H, Lawrence D. Adequacy of treatment for child and adolescent mental disorders in Australia: a national study. Aust N Z J Psychiatry. 2018;53(4):326–35.

    Article  PubMed  Google Scholar 

  18. Christ C, Schouten MJE, Blankers M, van Schaik DJF, Beekman ATF, Wisman MA, et al. Internet and computer-based cognitive behavioral therapy for anxiety and depression in adolescents and young adults: systematic review and Meta-analysis. J Med Internet Res. 2020;22(9):e17831.

    Article  PubMed  PubMed Central  Google Scholar 

  19. Ebert DD, Zarski A-C, Christensen H, Stikkelbroek Y, Cuijpers P, Berking M, et al. Internet and computer-based cognitive behavioral therapy for anxiety and depression in youth: a Meta-analysis of randomized controlled outcome trials. PLoS One. 2015;10(3):e0119895.

    Article  PubMed  PubMed Central  Google Scholar 

  20. Garrido S, Millington C, Cheers D, Boydell K, Schubert E, Meade T, et al. What works and what Doesn’t work? A systematic review of digital mental health interventions for depression and anxiety in young people. Front Psychiatry. 2019;10:759.

    Article  PubMed  PubMed Central  Google Scholar 

  21. Grist R, Croker A, Denne M, Stallard P. Technology delivered interventions for depression and anxiety in children and adolescents: a systematic review and Meta-analysis. Clin Child Fam Psychol Rev. 2019;22(2):147–71.

    Article  PubMed  Google Scholar 

  22. Hollis C, Falconer CJ, Martin JL, Whittington C, Stockton S, Glazebrook C, et al. Annual research review: digital health interventions for children and young people with mental health problems–a systematic and meta-review. J Child Psychol Psychiatry. 2017;58(4):474–503.

    Article  PubMed  Google Scholar 

  23. Glenn D, Golinelli D, Rose RD, Roy-Byrne P, Stein MB, Sullivan G, et al. Who gets the most out of cognitive behavioral therapy for anxiety disorders? The role of treatment dose and patient engagement. J Consult Clin Psychol. 2013;81(4):639–49.

    Article  PubMed  PubMed Central  Google Scholar 

  24. Lee P, Zehgeer A, Ginsburg GS, McCracken J, Keeton C, Kendall PC, et al. Child and adolescent adherence with cognitive behavioral therapy for anxiety: predictors and associations with outcomes. J Clin Child Adolesc Psychol. 2019;48(sup1):S215–s26.

    Article  PubMed  Google Scholar 

  25. Carlo AD, Hosseini Ghomi R, Renn BN, Strong MA, Areán PA. Assessment of real-world use of behavioral health Mobile applications by a novel stickiness metric. JAMA Netw Open. 2020;3(8):e2011978.

    Article  PubMed  PubMed Central  Google Scholar 

  26. Wasil AR, Venturo-Conerly KE, Shingleton RM, Weisz JR. A review of popular smartphone apps for depression and anxiety: assessing the inclusion of evidence-based content. Behav Res Ther. 2019;123:103498.

    Article  PubMed  Google Scholar 

  27. Buttazzoni A, Brar K, Minaker L. Smartphone-based interventions and internalizing disorders in youth: systematic review and Meta-analysis. J Med Internet Res. 2021;23(1):e16490.

    Article  PubMed  PubMed Central  Google Scholar 

  28. Leech T, Dorstyn D, Taylor A, Li W. Mental health apps for adolescents and young adults: a systematic review of randomised controlled trials. Child Youth Serv Rev. 2021;127:106073.

    Article  Google Scholar 

  29. Beatty L, Binnion C. A systematic review of predictors of, and reasons for, adherence to online psychological interventions. Int J Behav Med. 2016;23(6):776–94.

    Article  PubMed  Google Scholar 

  30. Harrer M, Adam SH, Baumeister H, Cuijpers P, Karyotaki E, Auerbach RP, et al. Internet interventions for mental health in university students: a systematic review and meta-analysis. Int J Methods Psychiatr Res. 2019;28(2):e1759.

    Article  PubMed  Google Scholar 

  31. Batchelor S, Stoyanov S, Pirkis J, Kõlves K. Use of kids helpline by children and young people in Australia during the COVID-19 pandemic. J Adolesc Health. 2021;68(6):1067–74.

    Article  PubMed  Google Scholar 

  32. Navarro P, Bambling M, Sheffield J, Edirippulige S. Exploring young People’s perceptions of the effectiveness of text-based online counseling: mixed methods pilot study. JMIR Ment Health. 2019;6(7):e13152.

    Article  PubMed  PubMed Central  Google Scholar 

  33. O'Dea B, Achilles MR, Werner-Seidler A, Batterham PJ, Calear AL, Perry Y, et al. Adolescents’ perspectives on a Mobile app for relationships: cross-sectional survey. JMIR Mhealth Uhealth. 2018;6(3):e56.

    Article  PubMed  PubMed Central  Google Scholar 

  34. Pennant ME, Loucas CE, Whittington C, Creswell C, Fonagy P, Fuggle P, et al. Computerised therapies for anxiety and depression in children and young people: a systematic review and meta-analysis. Behav Res Ther. 2015;67:1–18.

    Article  PubMed  Google Scholar 

  35. Bernstein EE, Weingarden H, Wolfe EC, Hall MD, Snorrason I, Wilhelm S. Human support in app-based cognitive behavioral therapies for emotional disorders: scoping review. J Med Internet Res. 2022;24(4):e33307.

    Article  PubMed  PubMed Central  Google Scholar 

  36. Li SH, Achilles MR, Spanos S, Habak S, Werner-Seidler A, B OD. A cognitive behavioural therapy smartphone app for adolescent depression and anxiety: co-design of ClearlyMe. Cogn Behav Ther. 2022;00:1 of 32.

    Google Scholar 

  37. Chan A-W, Tetzlaff JM, Altman DG, et al. SPIRIT 2013 statement: defining standard protocol items for clinical trials. Ann Intern Med. 2013;158:200–7.

    Article  PubMed  PubMed Central  Google Scholar 

  38. Hanano M, Rith-Najarian L, Boyd M, Chavira D. Measuring adherence within a self-guided online intervention for depression and anxiety: secondary analyses of a randomized controlled trial. JMIR Ment Health. 2022;9(3):e30754.

    Article  PubMed  PubMed Central  Google Scholar 

  39. Han J, McGillivray L, Wong QJ, Werner-Seidler A, Wong I, Calear A, et al. A Mobile health intervention (LifeBuoy app) to help young people manage suicidal thoughts: protocol for a mixed-methods randomized controlled trial. JMIR Res Protoc. 2020;9(10):e23655.

    Article  PubMed  PubMed Central  Google Scholar 

  40. Donker T, Griffiths KM, Cuijpers P, Christensen H. Psychoeducation for depression, anxiety and psychological distress: a meta-analysis. BMC Med. 2009;7(1):79.

    Article  PubMed  PubMed Central  Google Scholar 

  41. O'Dea B, Subotic-Kerry M, King C, Mackinnon AJ, Achilles MR, Anderson M, et al. A cluster randomised controlled trial of a web-based youth mental health service in Australian schools. Lancet Reg Health West Pac. 2021;12:100178.

    Article  PubMed  PubMed Central  Google Scholar 

  42. O’Dea B, King C, Subotic-Kerry M, Achilles MR, Cockayne N, Christensen H. Smooth sailing: a pilot study of an online, school-based, mental health Service for Depression and Anxiety. Front Psychiatry. 2019;10:574.

    Article  PubMed  PubMed Central  Google Scholar 

  43. Aycock DM, Hayat MJ, Helvig A, Dunbar SB, Clark PC. Essential considerations in developing attention control groups in behavioral research. Res Nurs Health. 2018;41(3):320–8.

    Article  PubMed  Google Scholar 

  44. Johnson JG, Harris ES, Spitzer RL, Williams JBW. The patient health questionnaire for adolescents: validation of an instrument for the assessment of mental disorders among adolescent primary care patients. J Adolesc Health. 2002;30(3):196–204.

    Article  PubMed  Google Scholar 

  45. Kroenke K, Spitzer RL, Williams JB. The PHQ-9: validity of a brief depression severity measure. J Gen Intern Med. 2001;16(9):606–13.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  46. Allgaier A-K, Pietsch K, Frühe B, Sigl-Glöckner J, Schulte-Körne G. Screening for depression in adolescents: Validity of the Patient Health Questionnaire in pediatric care. Depress Anxiety. 2012;29(10):906–13.

    Article  PubMed  Google Scholar 

  47. Ganguly S, Samanta M, Roy P, Chatterjee S, Kaplan DW, Basu B. Patient health Questionnaire-9 as an effective tool for screening of depression among Indian adolescents. J Adolesc Health. 2013;52(5):546–51.

    Article  PubMed  Google Scholar 

  48. Leung DYP, Mak YW, Leung SF, Chiang VCL, Loke AY. Measurement invariances of the PHQ-9 across gender and age groups in Chinese adolescents. Asia Pac Psychiatry. 2020;12(3):e12381.

    Article  PubMed  PubMed Central  Google Scholar 

  49. Tsai FJ, Huang YH, Liu HC, Huang KY, Huang YH, Liu SI. Patient health questionnaire for school-based depression screening among Chinese adolescents. Pediatrics. 2014;133(2):e402–9.

    Article  PubMed  Google Scholar 

  50. American Psychiatric Association. Online Assessment Measures n.d. Available from: https://www.psychiatry.org/psychiatrists/practice/dsm/educational-resources/assessment-measures.

  51. Burdzovic Andreas J, Brunborg GS. Depressive symptomatology among Norwegian adolescent boys and girls: the patient health Questionnaire-9 (PHQ-9) psychometric properties and correlates. Front Psychol. 2017;8:887.

    Article  PubMed  PubMed Central  Google Scholar 

  52. Spitzer RL, Kroenke K, Williams JB, Löwe B. A brief measure for assessing generalized anxiety disorder: the GAD-7. Arch Intern Med. 2006;166(10):1092–7.

    Article  PubMed  Google Scholar 

  53. Sun J, Liang K, Chi X, Chen S. Psychometric properties of the generalized anxiety disorder Scale-7 item (GAD-7) in a large sample of Chinese adolescents. Healthcare (Basel). 2021;9(12):1709.

    Article  PubMed  Google Scholar 

  54. Tiirikainen K, Haravuori H, Ranta K, Kaltiala-Heino R, Marttunen M. Psychometric properties of the 7-item generalized anxiety disorder scale (GAD-7) in a large representative sample of Finnish adolescents. Psychiatry Res. 2019;272:30–5.

    Article  PubMed  Google Scholar 

  55. Batterham PJ, Sunderland M, Carragher N, Calear AL, Mackinnon AJ, Slade T. The distress Questionnaire-5: population screener for psychological distress was more accurate than the K6/K10. J Clin Epidemiol. 2016;71:35–42.

    Article  PubMed  Google Scholar 

  56. Batterham PJ, Sunderland M, Slade T, Calear AL, Carragher N. Assessing distress in the community: psychometric properties and crosswalk comparison of eight measures of psychological distress. Psychol Med. 2018;48(8):1316–24.

    Article  CAS  PubMed  Google Scholar 

  57. Calear AL, Morse AR, Batterham PJ, Forbes O, Banfield M. Silence is deadly: a controlled trial of a public health intervention to promote help-seeking in adolescent males. Suicide Life Threat Behav. 2021;51(2):274–88.

    Article  PubMed  Google Scholar 

  58. McGillivray L, Rheinberger D, Wang J, Burnett A, Torok M. Non-disclosing youth: a cross sectional study to understand why young people do not disclose suicidal thoughts to their mental health professional. BMC Psychiatry. 2022;22(1):3.

    Article  PubMed  PubMed Central  Google Scholar 

  59. O’Dea B, Han J, Batterham PJ, Achilles MR, Calear AL, Werner-Seidler A, et al. A randomised controlled trial of a relationship-focussed mobile phone application for improving adolescents’ mental health. J Child Psychol Psychiatry. 2020;61(8):899–913.

    Article  PubMed  PubMed Central  Google Scholar 

  60. Werner-Seidler A, Huckvale K, Larsen ME, Calear AL, Maston K, Johnston L, et al. A trial protocol for the effectiveness of digital interventions for preventing depression in adolescents: the future proofing study. Trials. 2020;21(1):2.

    Article  PubMed  PubMed Central  Google Scholar 

  61. Stewart-Brown S, Tennant A, Tennant R, Platt S, Parkinson J, Weich S. Internal construct validity of the Warwick-Edinburgh mental well-being scale (WEMWBS): a Rasch analysis using data from the Scottish health education population survey. Health Qual Life Outcomes. 2009;7(1):1–8.

    Article  Google Scholar 

  62. Tennant R, Hiller L, Fishwick R, Platt S, Joseph S, Weich S, et al. The Warwick-Edinburgh mental well-being scale (WEMWBS): development and UK validation. Health Qual Life Outcomes. 2007;5(1):1–13.

    Article  Google Scholar 

  63. Haver A, Akerjordet K, Caputi P, Furunes T, Magee C. Measuring mental well-being: A validation of the short Warwick–Edinburgh mental well-being scale in Norwegian and Swedish. Scand J Public Health. 2015;43(7):721–7.

    Article  PubMed  Google Scholar 

  64. Hunter SC, Houghton S, Wood L. Positive mental well-being in australian adolescents: evaluating the Warwick-Edinburgh mental well-being scale. Aust Educ Dev Psychol. 2015;32(2):93–104.

    Article  Google Scholar 

  65. McKay MT, Andretta JR. Evidence for the psychometric validity, internal consistency and measurement invariance of Warwick Edinburgh mental well-being scale scores in Scottish and Irish adolescents. Psychiatry Res. 2017;255:382–6.

    Article  PubMed  Google Scholar 

  66. Ringdal R, Bradley Eilertsen M-E, Bjørnsen HN, Espnes GA, Moksnes UK. Validation of two versions of the Warwick-Edinburgh mental well-being scale among Norwegian adolescents. Scand J Public Health. 2018;46(7):718–25.

    Article  PubMed  Google Scholar 

  67. Stevens K. Developing a descriptive system for a new preference-based measure of health-related quality of life for children. Qual Life Res. 2009;18(8):1105–13.

    Article  PubMed  Google Scholar 

  68. Ratcliffe J, Couzner L, Flynn T, Sawyer M, Stevens K, Brazier J, et al. Valuing child health utility 9D health states with a young adolescent sample: a feasibility study to compare best-worst scaling discrete-choice experiment, standard gamble and time trade-off methods. Appl Health Econ Health Policy. 2011;9(1):15–27.

    Article  PubMed  Google Scholar 

  69. Ratcliffe J, Stevens K, Flynn T, Brazier J, Sawyer M. An assessment of the construct validity of the CHU9D in the Australian adolescent general population. Qual Life Res. 2012;21(4):717–25.

    Article  PubMed  Google Scholar 

  70. Stevens K, Ratcliffe J. Measuring and valuing health benefits for economic evaluation in adolescence: an assessment of the practicality and validity of the child health utility 9D in the Australian adolescent population. Value Health. 2012;15(8):1092–9.

    Article  PubMed  Google Scholar 

  71. Furber G, Segal L. The validity of the child health utility instrument (CHU9D) as a routine outcome measure for use in child and adolescent mental health services. Health Qual Life Outcomes. 2015;13:22.

    Article  PubMed  PubMed Central  Google Scholar 

  72. Treynor W, Gonzalez R, Nolen-Hoeksema S. Rumination reconsidered: a psychometric analysis. Cogn Ther Res. 2003;27(3):247–59.

    Article  Google Scholar 

  73. Extremera N, Fernández-Berrocal P. Validity and reliability of Spanish versions of the ruminative responses scale-short form and the distraction responses scale in a sample of Spanish high school and college students. Psychol Rep. 2006;98(1):141–50.

    Article  PubMed  Google Scholar 

  74. He J, Liu Y, Cheng C, Fang S, Wang X, Yao S. Psychometric properties of the Chinese version of the 10-item ruminative response scale among undergraduates and depressive patients. Front Psychiatry. 2021;12:626859.

    Article  PubMed  PubMed Central  Google Scholar 

  75. Xavier A, Cunha M, Pinto-Gouveia J. Rumination in adolescence: the distinctive impact of brooding and reflection on psychopathology. Span J Psychol. 2016;19:E37.

    Article  PubMed  Google Scholar 

  76. Gullone E, Taffe J. The emotion regulation questionnaire for children and adolescents (ERQ–CA): a psychometric evaluation. Psychol Assess. 2012;24(2):409–17.

    Article  PubMed  Google Scholar 

  77. Jacob KL, Christopher MS, Neuhaus EC. Development and validation of the cognitive-behavioral therapy skills questionnaire. Behav Modif. 2011;35(6):595–618.

    Article  PubMed  Google Scholar 

  78. Hetrick SE, Yuen HP, Bailey E, Cox GR, Templer K, Rice SM, et al. Internet-based cognitive behavioural therapy for young people with suicide-related behaviour (reframe-IT): a randomised controlled trial. Evid Based Ment Health. 2017;20(3):76–82.

    Article  PubMed  Google Scholar 

  79. Meadows G, Harvey C, Fossey E, Burgess P. Assessing perceived need for mental health care in a community survey: development of the perceived need for care questionnaire (PNCQ). Soc Psychiatry Psychiatr Epidemiol. 2000;35(9):427–35.

    Article  CAS  PubMed  Google Scholar 

  80. Johnson SE, Lawrence D, Sawyer M, Zubrick SR. Mental disorders in Australian 4-to 17-year olds: parent-reported need for help. Aust N Z J Psychiatry. 2018;52(2):149–62.

    Article  PubMed  Google Scholar 

  81. Schnyder N, Lawrence D, Panczak R, Sawyer MG, Whiteford HA, Burgess PM, et al. Perceived need and barriers to adolescent mental health care: agreement between adolescents and their parents. Epidemiol Psychiatr Sci. 2020;29:e60.

    Article  Google Scholar 

  82. Henson P, Wisniewski H, Hollis C, Keshavan M, Torous J. Digital mental health apps and the therapeutic alliance: initial review. BJPsych Open. 2019;5(1):e15.

    Article  PubMed  PubMed Central  Google Scholar 

  83. Hatcher RL, Gillaspy JA. Development and validation of a revised short version of the working Alliance inventory. Psychother Res. 2006;16(1):12–25.

    Article  Google Scholar 

  84. Horvath AO, Greenberg LS. Development and validation of the working Alliance inventory. J Couns Psychol. 1989;36(2):223.

    Article  Google Scholar 

  85. Goldberg SB, Baldwin SA, Riordan KM, Torous J, Dahl CJ, Davidson RJ, et al. Alliance with an unguided smartphone app: validation of the digital working Alliance inventory. Assessment. 2021. https://doi.org/10.1177/10731911211015310.

  86. Czyz EK, Arango A, Healy N, King CA, Walton M. Augmenting safety planning with text messaging support for adolescents at elevated suicide risk: development and acceptability study. JMIR Ment Health. 2020;7(5):e17345.

    Article  PubMed  PubMed Central  Google Scholar 

  87. Baumel A. Therapeutic activities as a link between program usage and clinical outcomes in digital mental health interventions: a proposed research framework. J Tech Behav Sci. 2022;7:234–9.

  88. Yardley L, Spring BJ, Riper H, Morrison LG, Crane DH, Curtis K, et al. Understanding and promoting effective engagement with digital behavior change interventions. Am J Prev Med. 2016;51(5):833–42.

    Article  PubMed  Google Scholar 

  89. Karyotaki E, Efthimiou O, Miguel C, Bermpohl FMG, Furukawa TA, Cuijpers P, et al. Internet-based cognitive behavioral therapy for depression: a systematic review and individual patient data network Meta-analysis. JAMA Psychiatry. 2021;78(4):361–71.

    Article  PubMed  Google Scholar 

  90. Batterham PJ. Recruitment of mental health survey participants using internet advertising: content, characteristics and cost effectiveness. Int J Methods Psychiatr Res. 2014;23(2):184–91.

    Article  PubMed  PubMed Central  Google Scholar 

  91. Lewis JA. Statistical principles for clinical trials (ICH E9): an introductory note on an international guideline. Stat Med. 1999;18(15):1903–42.

    Article  CAS  PubMed  Google Scholar 

  92. National Health and Medical Research Council (NHMRC). Data safety monitoring Boards (DSMBs). 2018.

    Google Scholar 

  93. Pretorius C, Chambers D, Coyle D. Young People’s online help-seeking and mental health difficulties: systematic narrative review. J Med Internet Res. 2019;21(11):e13873.

    Article  PubMed  PubMed Central  Google Scholar 

  94. Perski O, Blandford A, West R, Michie S. Conceptualising engagement with digital behaviour change interventions: a systematic review using principles from critical interpretive synthesis. Transl Behav Med. 2017;7(2):254–67.

    Article  PubMed  Google Scholar 

  95. Fleming T, Bavin L, Lucassen M, Stasiak K, Hopkins S, Merry S. Beyond the trial: systematic review of real-world uptake and engagement with digital self-help interventions for depression, low mood, or anxiety. J Med Internet Res. 2018;20(6):e199.

    Article  PubMed  PubMed Central  Google Scholar 

Download references

Acknowledgements

The authors would like to acknowledge the individuals who contributed to the design and development of the ClearlyMe® app. Thank you to the young people, parents and mental health professionals and the team of psychologists, researchers, user experience designers and software developers at the BDI.

Funding

This work has been supported by the Goodman Foundation; National Health and Medical Research Council (NHMRC) Investigator Grants (AW-S, grant number GNT1197074, JN, grant number: GNT2008839); and a National Health and Medical Research Council (NHMRC) Medical Research Future Fund (MRFF) Fellowship (BOD, grant number 1197249). Clinical trial sponsor is UNSW Sydney. Sponsors and funders were not involved in any aspect of the study design, analysis, interpretation or manuscript preparation.

Author information

Authors and Affiliations

Authors

Contributions

BOD conceived the project and prepared the initial proposal for funding application. BOD, SL, MS-K and MRA contributed to the intervention and study design. JMN, HC, PB, AW-S and AM provided methodological expertise in the study design. BOD, MRA and SHL prepared the study protocol. All authors read and approved the final manuscript.

Corresponding author

Correspondence to S. H. Li.

Ethics declarations

Ethics approval and consent to participate

This research will be carried out in accordance with the Declaration of Helsinki and has been approved by the UNSW Human Research Ethics Committee (HREC; HC210889). The trial is registered with the Australian New Zealand Clinical Trials Registry (ACTRN12622000131752). Protocol modifications will be submitted for approval through UNSW HREC before implementation and updated on the ANZCTR. Informed written consent will be obtained from adolescent participants and their parent or guardian to be eligible to participate in this study.

Consent for publication

Not applicable.

Competing interests

The authors declare they have no competing interests.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Li, S.H., Achilles, M.R., Subotic-Kerry, M. et al. Protocol for a randomised controlled trial evaluating the effectiveness of a CBT-based smartphone application for improving mental health outcomes in adolescents: the MobiliseMe study. BMC Psychiatry 22, 746 (2022). https://doi.org/10.1186/s12888-022-04383-3

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1186/s12888-022-04383-3

Keywords