All eligible patients who registered for help at a mental health service were offered online treatment before the onset of face-to-face treatment. Participants who were willing to use the five week online treatment could start with the treatment the same day they signed up for the study. The other participants started with five weeks of waiting. They were asked by phone why they declined to offer of online treatment. After five weeks, both groups received the regular face-to-face treatment.
The internet course was not offered to every patient but only to those 18 years or older, with elevated depressive symptoms (scoring ≥14 on the BDI-II), anxiety symptoms (scoring > 8 on the HADS) and/or work-related stress (scoring ≥ 2.2 on high emotional exhaustion in combination with ≤ 2.2 depersonalization or ≤ 3.66 personal accomplishment on the MBI). It was not offered to patients with suicidal thoughts, without Internet access, or without sufficient understanding of the Dutch language. All new eligible patients of the mental health care center were informed about the Internet course and the study, by e-mail and/or by phone. All participating patients gave written informed consent to use the routine outcome monitoring data for this study. The enrollment took place between October 2010 and April 2011.
The online treatment program that was used in this study was a five week Problem Solving Treatment (PST) [8, 12]. It has proved to be effective in reducing symptoms of depression and anxiety [8, 9]. Also, the generic character of the online PST makes it suitable to address different kinds of symptoms. This is useful because of the high prevalence of comorbidity in psychiatric disorders [3, 13] and can therefore fit well into a stepped-care model in specialized or primary mental health care institutions .
The online treatment has three steps. First, participants have to write down things that really matter to them. Second, they have to describe their current problems and subdivide these into unimportant problems (i.e. unrelated to the things that matter to them), solvable problems, and unsolvable problems (e.g. the death of a loved one). For each category a different technique is suggested to solve or cope with the problems. The core of the program consists of solving the solvable problems. This is done in six steps: (1) writing a full description of the problem, (2) generating multiple solutions, (3) selecting the best solution, (4) making a plan for carrying out the solution, (5) actually carrying out the solution and (6) evaluating whether the solution has solved the problem. During the last step participants create a plan for the future in which they write down how they will try to achieve the things that really matter to them. The course takes five weeks and consists of one lesson a week. The website of the course contains instructions, exercises, examples of people applying the principles of PST. All participants received support in the form of feedback from a trained Master student in Clinical Psychology. The feedback was given within three working days by email and consisted of guiding the participants through the course and more personalized feedback (e.g. psycho education on how to cope with depression, anxiety and/or stress). The time spent on feedback was approximately 40 minutes per lesson.
All participants were asked to fill out the questionnaires at baseline, five weeks later (or directly after the online treatment) and twelve weeks after baseline. A week prior to an assessment point, participants received an email alert. The self report questionnaires were administered online.
Depressive symptoms were measured with the 21-item Beck Depression Inventory Second Edition (BDI-II) , a widely used multiple-choice self-report inventory which detects, assesses, and monitors changes in the severity of depressive symptoms as listed in the American Psychiatric Association’s Diagnostic and Statistical Manual of Mental Disorders Fourth Edition (DSM-IV; 1994). It consists of questions with answers being scored on a 0–3 scale, about how the person felt the last week. The total score varies between 0 and 63 with higher scores indicating more depression. Scores below 14 indicate absence of depressive symptoms. We used this cut-off score as an indication for recovery from depressive symptoms. The validity of the BDI-II has been tested in different populations . In our study the Cronbach’s alpha was .86.
For measuring anxiety symptoms the 7-item anxiety subscale of the Dutch version of the Hospital Anxiety and Depression Scale (HADS) was used . Item-responses are on a 0 to 3 scale. The total score varies from 0 to 21, with higher scores indicating greater degrees of symptom severity. The cut-off score of 8 is an inclusion criterion for this study. The Dutch version of the HADS showed good homogeneity and reliability . In our study the Cronbach’s alpha was .55 for the anxiety subscale.
The Dutch version of the Maslach Burnout Inventory (MBI) was used to measure work related stress. It consists of 15 items and assesses three components: emotional exhaustion (EE), depersonalisation (D), and personal accomplishment (C). The items are written in the form of statements about personal feelings of attitudes and are answered on 7-point scale (ranging from 0, “never” to 6 “very often”). Higher scores on the two subscales emotional exhausting and depersonalisation indicate more experienced work-related stress, while lower scores on personal accomplishment corresponds to higher degrees of work-related stress. A score >2.2 on the subscale emotional exhaustion in combination with a score >2.2 on the subscale depersonalisation or a score <3.66 on personal accomplishment is an indication for a burnout syndrome. Its validity and reliability has been tested in different populations [17, 18]. In our study the Cronbach’s alphas for the subscales emotional exhausting, depersonalisation and personal accomplishment were .91, .85 and .87.
Quality of life
We used the sixth item of the EuroQol questionnaire (EQ-5D) to assess quality of life . Participants are asked to rate their current health from 0 (worst imaginable) to 100 (best imaginable).
Statistical analyses were conducted using SPSS 18 for Windows. Baseline differences in demographics and clinical characteristics between the two groups were investigated using Chi-square and independent samples t-tests. We examined the effects between the two groups on the three outcome measures (depression, anxiety and work related stress) separately. First, we used ANOVA and follow-up t-tests to evaluate differences between the two groups for each assessment point. Second, we used independent samples t-tests to investigate the difference in mean scores between the two groups. Third, we calculated between group effect sizes (Cohen’s d) by subtracting the post-test mean score of the WL group from the post-test mean score of the PST group, divided by the pooled standard deviation. Effect sizes larger than d = 0.8 were considered to be large, d = 0.5–0.8 moderate, and d = 0.2–0.5 small . Fourth, clinical significant change was determined with norms for the outcome measure. We calculated differences between the two groups in the percentage of patients who had recovered. We considered participants to have recovered when they scored <14 on the BDI-II, < 8 on the HADS-A and < 2.2 on emotional exhaustion in combination with a score < 2.2 on depersonalisation or > 3.66 on personal accomplishment on the MBI. Improvement was determined following the suggestions of Jacobson and Truax  calculating a reliable change index (RC).
All analyses were performed on the intention-to-treat sample. Pre-test data were available for all participants. Missing values of post-test non-responders were imputated by multiple regression analyses using available baseline data, demographics as well as data on baseline severity, from all participants.
Ethical approval and registration was not required for this study according to Dutch law.