Eligibility criteria
Participants were required to meet the following criteria: Australian resident aged 18 to 75 years; own an internet-enabled mobile phone; have access to a desk-top computer with internet capability; have a valid email address; and report symptoms of mild-to-moderate depression, anxiety and/or stress, defined as a total score of 27–63 inclusive on the Depression Anxiety and Stress Scales (DASS; [17]). Consistent with our public health focus on mild-to-moderate symptoms, people were excluded if they scored 64 or more on the DASS (severe symptomatology), answered positively to questions asking about suicidal thoughts, intent and/or previous suicide attempts, or met criteria for psychotic symptoms, as measured by the Psychosis Screening Questionnaire (PSQ; [18]).
Sample & setting
Based on data from our proof of concept study [14], our targeting of mild-to-moderate symptoms, and the fact that myCompass is a self-help intervention without therapist support, we estimated a between-group effect size of d = 0.2. Power calculations established that for 80% power and an alpha of .05, 400 people per group would be required to detect a difference if one was present. To allow for sample attrition, we set a sample size target of n = 650 per group. Participants were recruited nationally and online between October 2011 and March 2012 via social media (Facebook and Twitter), websites of the Black Dog Institute (BDI), corporate and government organisations, advertisements placed on national radio and in print media, and the BDI volunteer research register. Due to time and funding restrictions, it was not possible to extend recruitment activities beyond this period. Written informed consent was provided by all participants. The RCT was approved by the Human Research Ethics Committee at the University of New South Wales, Sydney, Australia (HREC 100019), and registered as Australian New Zealand Clinical Trials Registry ACTRN 12610000625077.
Design
A mixed factorial repeated measures design was employed, with full randomisation to three conditions.
Randomisation
A research assistant not involved in the RCT randomised participants after baseline using computerised random numbers. Allocation was either to the myCompass, AC or WL condition. Participants received advice of their group assignment by email.
Interventions
myCompass is a fully-automated, self-help, public health intervention, that is tailored to the user and has no therapist input. Real-time self-monitoring of symptoms (e.g., problem moods, thoughts and behaviours) via mobile phone and/or computer is a key therapeutic feature. Users can self-monitor three symptoms of their choice at any one time, selected from a list of 20, or three that are recommended to them by the program (e.g., confidence, worry, irritability, motivation, diet, and medication use). Each symptom is rated on a 10-point scale (e.g., “how confident do you feel right now”, “how worried do you feel right now”, “how satisfied are you that you have taken your prescribed medication today”). At the time of rating, users also provide contextual information about where they are, what they are doing and who they are with, using a series of drop-down menus. Users can schedule short message service (SMS) or email reminders to facilitate self-monitoring (frequency of reminders determined by the user); receive and print graphical feedback about their monitoring, including contextual information, on their phone or computer (to monitor change and assist identification of triggers); and elect to receive helpful facts, mental health-care tips or motivational statements by SMS or email.
Evidence-based and interactive psychological modules that users can complete via the internet on their computers are another key element of myCompass. The program contains 12 skill-building modules derived from CBT, Interpersonal Psychotherapy, Problem-solving Therapy and Positive Psychology that cover topics such as Managing Fear and Anxiety, Tackling Unhelpful Thinking, Managing Loss and Major Life Change, and Solving Problems. Each module comprises three 10-minute sessions and includes activities for users to complete on the computer. Modules also recommend home practice tasks for completion between the weekly sessions to promote skill generalisation. Users can complete the modules of their choice, or those recommended to them by the myCompass program.
Tailoring of the myCompass program self-monitoring and module recommendations occurs via users’ responses to a profile questionnaire completed at registration. This questionnaire asks users to rate the extent to which they experience a range of symptoms of depression, anxiety, and stress (e.g., worry, irritability, difficulties with motivation and concentration). Targeting the three highest rated symptoms, in-built algorithms generate personalised feedback to the user about the self-monitoring dimensions and psychological modules that may be of greatest benefit (i.e., each dimension and module is weighted according to its clinical relevance for each symptom, and those with the highest weightings are recommended).
User privacy is managed by a password protected log-on, and by ensuring that user generated data (i.e., self-monitoring ratings) are not stored on the users’ phone but instead transferred via the internet using secure sockets layer protocols (which encrypt transmitted data by rendering it unreadable to anyone other than the intended recipient) and by storing the data in secure servers. Registering to use myCompass is free and users are not billed for the SMSs they receive.
Participants randomised to the myCompass intervention were emailed instructions for accessing and logging onto the program. They were advised that they had full access to the program for seven weeks, and were encouraged to use the program ad libitum during this time. However, it was recommended that they complete a minimum of two modules and monitor daily at least three moods or behaviours during the intervention phase. Access to the program was withdrawn at the end of seven weeks.
Attention control participants received a control mental health program matched to the active intervention on duration and mode of delivery. Each week for seven weeks, they received a fact sheet containing information about depression, anxiety or stress sent to their email address. The information was designed to be read on computer in approximately 10 minutes, and to be credible but void of management advice or treatment strategies. They also received on their mobile phones weekly SMS messages containing brief factual statements about depression, anxiety and stress. The mobile phone statements were also therapeutically inactive, but chosen to ensure that the control program had face validity. Messages varied in length from 160 to 275 characters so as to match, as far as possible, the minimum amount of SMS content that myCompass participants received each week, namely one self-monitoring reminder (160 characters) plus an optional motivational statement, mental health care tip or fact (average length 95 characters).
Waitlist participants did not receive emails or SMSs during the intervention phase, but received full access to the myCompass program at the end of the seven weeks.
By including both the AC group and the WL group, we were able to control for non-specific effects of the intervention. We were also able to test whether any effects seen during the intervention phase were replicated when the WL control group subsequently completed the myCompass program.
Procedure
Data collection took place between October 2011 and October 2012. All study consent, screening and questionnaire data were collected online using online survey software. Individuals not meeting screening criteria received automated feedback explaining the reason/s for their ineligibility, and referring them to other online resources of the BDI. Individuals with symptoms in the severe range were also advised to seek face-to-face support from a health professional and were provided the contact details of crisis support services.
Eligible participants completed a baseline questionnaire prior to randomisation, a post-intervention questionnaire administered at eight weeks, and a follow-up questionnaire administered 12 weeks later for participants in the myCompass and AC groups, and 19 weeks later for the WL group. At each assessment point, participants accessed the appropriate study questionnaire via a link sent to them in an email message.
Primary outcome
The Depression, Anxiety and Stress Scales measure (DASS-21; [17]) is a widely used self-report measure of depression, anxiety and stress with high internal consistency, acceptable test-retest reliability [19], and yields reliable and valid data when used in an online format [20]. Respondents are asked to indicate the frequency with which they experienced symptoms of depression, anxiety and stress over the previous week. Total scores range from 0 to 126 and subscale scores range from 0 to 42, with higher scores indicating greater symptom severity.
Secondary outcomes
The Work and Social Adjustment Scale (WSAS; [21]) assesses the degree to which mental health problems interfere with day-to-day functioning in five domains: work, social leisure activities, private leisure activities, home-management, and personal relationships [21]. The measure provides an assessment of the experiential impact of mental health symptoms from the sufferer’s point of view. Scores range from 0 to 40, with higher scores indicating poorer adjustment. Meyer et al. [22] provide data supporting the psychometric adequacy of the WSAS when administered in an online format.
Demographic and clinical information collected within the baseline questionnaire included age, gender, marital status, educational level, employment status, frequency of mobile phone and internet use, frequency of depressive and anxiety episodes, and age of first episode. At each assessment point, health service utilisation for mental health reasons (e.g., visits to general practitioner, mental health professionals and alternative health practitioners) and use of antidepressant and anxiolytic medication were also assessed. Satisfaction with the program was measured at post-intervention for participants in the myCompass and AC groups using a study-specific measure assessing users’ perceptions of program usability, content, flexibility and functionality.
Adherence, defined as the extent to which participants engaged with the intervention [23], was examined for the myCompass group with respect to three indices, namely, frequency of logins, frequency of self-monitoring, and number of modules attempted.
Statistical analyses
Statistical analyses were completed with SPSS 20.0 software. Characteristics of the three groups at baseline were compared using chi-square tests for categorical variables and analysis of variance (ANOVA) for continuous variables. Chi-square and ANOVA were also used to compare baseline characteristics of participants who did (‘non-dropouts’) and did not (‘dropouts’) return completed questionnaires at post-intervention and follow-up, to explore possible biases in attrition.
Effects of the myCompass intervention on study outcomes were evaluated using intention-to-treat (ITT) analyses that included data from all participants who completed the baseline assessment and any follow-up assessment. Strategies for dealing with missing data in longitudinal studies vary, so we adopted two recommended techniques for analysing incomplete datasets, namely, mixed models repeated measures (MMRM) and multiple imputation (MI; for a review see [24]). The two analyses yielded similar outcomes and, as MMRM is the dominant strategy adopted in studies of web-based interventions, the findings derived from MMRM are presented here.
In MMRM, no participant is removed from the analysis because all available data is used to obtain parameter estimates. In the present study, restricted maximum likelihood (REML) was used to estimate model parameters and error degrees of freedom were calculated using Satterthwaite’s approximation [25]. Analyses assumed a compound symmetric structure, in line with Fairclough’s recommendation that the covariance structure be restricted in situations where attrition is high [26].
Post-intervention analyses involved a series of 3 groups (myCompass, AC and WL) by 2 times (pre-intervention, post-intervention) repeated measures models. Because WL participants were exposed to the myCompass intervention during the follow-up period, repeated measures models analysing follow-up data comprised 2 groups (myCompass and AC) by 3 times (pre, post and follow-up). For each analysis, scores at baseline on the outcome variable of interest was entered as a covariate. Significant group by time interaction effects were examined using sets of Bonferroni adjusted interaction contrasts to compare between-group differences in mean change on the outcome measure from baseline to post-intervention (i.e., intervention period) and from post-intervention to follow-up (i.e., follow-up phase), as appropriate. All effects were tested at the p < 0.05 level, with adjustment according to the number of contrasts in each set.
Cohen’s d was calculated following the procedure outlined in Ruwaard et al. [27] and using both observed and estimated marginal means. For all outcome measures, within-and between-group differences were standardised to Cohen’s d using the pooled standard deviation of the observed scores obtained at baseline.
To determine the replicability of treatment effects resulting from the intervention, primary and secondary outcome data from WL participants who used the myCompass program at the end of their waiting period were examined at post-intervention and follow-up using paired samples t-tests and Cohen’s d.
