Study protocol | Open | Open Peer Review | Published:
The GET READY relapse prevention programme for anxiety and depression: a mixed-methods study protocol
BMC Psychiatryvolume 19, Article number: 64 (2019)
Since anxiety and depressive disorders often recur, self-management competencies are crucial for improving the long-term course of anxiety and depressive disorders. However, few relapse prevention programmes are available that focus on improving self-management. E-health combined with personal contact with a mental health professional in general practice might be a promising approach for relapse prevention. In this protocol, the GET READY (Guided E-healTh for RElapse prevention in Anxiety and Depression) study will be described in which a relapse prevention programme is developed, implemented and evaluated. The aim of the study is to determine patients’ usage of the programme and the associated course of their symptoms, to examine barriers and facilitators of implementation, and to assess patients’ satisfaction with the programme.
Participants are discharged from mental healthcare services, and are in complete or partial remission. They receive access to an E-health platform, combined with regular contact with a mental health professional in general practices. Online questionnaires will be completed at baseline and after 3, 6 and 9 months. Also, semi-structured qualitative individual interviews and focus group interviews will be conducted with patients and mental health professionals.
This mixed-methods observational cohort study will provide insights into the use of a relapse prevention programme in relation to the occurrence of symptoms, as well as in its implementation and evaluation. Using the results of this study, the relapse prevention programme can be adapted in accordance with the needs of patients and mental health professionals. If this programme is shown to be acceptable, a randomized controlled trial may be conducted to test its efficacy.
Retrospectively registered in the Netherlands Trial Register (NTR7574; 25 October 2018).
The course of anxiety and depressive disorders is often unfavourable, with chronic [1, 2] or intermittent episodes of anxiety and/or depression  and a high risk of relapse. Percentages of 22–58 for anxiety [1, 3,4,5] and 27–77 for depression [6,7,8,9] have been reported as relapse rates with higher rates in longer follow-up studies [1, 6]. Furthermore, patients often achieve partial remission instead of full remission, and have residual symptoms . The risk of relapse is higher for patients with a partial remission when compared to patients with full remission . Moreover, even if remission is achieved, patients with residual symptoms have a three times higher risk of relapse than patients without residual symptoms . Since anxiety and depression are often recurrent and patients remain vulnerable, these disorders should be approached as chronic diseases. Chronic care models, previously developed for other chronic diseases such as diabetes , can be applied. One essential element of chronic care models is the support of self-management . Lorig  defined self-management as “learning and practicing skills necessary to carry on an active and emotionally satisfying life in the face of a chronic condition” (p.11). Self-management for anxiety and depression focuses on recovery and stabilisation of symptoms, prevention of relapse, and improving functioning and quality of life [16, 17]. People with recurrent anxiety and depression should be supported in performing self-management strategies.
One of the major challenges in the management of anxiety and depression symptoms is the prevention of relapse . In their meta-analysis, Biesheuvel-Leliefeld et al.  concluded that relapse prevention strategies such as cognitive therapy or mindfulness-based cognitive therapy are effective in reducing relapse of depression. However, few relapse prevention programmes are available for this patient group of depressed patients. In an empirical study from the same authors, the efficacy of a self-help preventive cognitive therapy combined with weekly telephone guidance in preventing depression was examined . They demonstrated that this was significantly more effective than care as usual. To date, little is known about the efficacy of relapse prevention strategies for patients with anxiety disorders . The fact that depression and anxiety are often comorbid, and relapse into another disorder frequently occurs (from anxiety to depression and vice versa), highlights the need to target both anxiety and depression in a single relapse prevention programme.
Over the past two decades, it has been suggested that primary healthcare facilities should play a vital role in the long-term treatment of patients with anxiety and depression [22, 23]. In the Netherlands, primary mental healthcare is usually provided by a mental health professional (MHP), working in a general practice and reporting to the general practitioner (GP). This MHP could be a (community) mental health nurse, a social worker or a (junior) psychologist. Since general practices are always located close to where patients live, and the services are freely available, the MHPs are easily approachable. They can play a pivotal role in the monitoring of symptoms and the support of self-management, and therefore in the prevention of relapse. However, many MHPs are unfamiliar with relapse prevention interventions, including the use of supporting tools to monitor symptoms . To encourage general practices to offer structured relapse prevention, while also supplying supporting tools for MHPs, the use of E-health could offer a solution. E-health tools can be easily tailored to the needs and preferences of patients, which is important for the tools’ acceptability and effectiveness. Most (92%) of the MHPs in the Netherlands are already familiar with E-health . If the use of E-health is embedded in personal contact with a MHP, it is likely to be more effective .
We therefore developed a relapse prevention programme that can be offered in general practices by MHPs to patients with (partially) remitted anxiety or depression. This programme aims to support self-management skills and provides tools for monitoring symptoms. The E-health modules can be individually tailored to the needs of patients, and is combined with regular contact with the MHP. The aim of the present study is to implement and evaluate this guided self-help online relapse prevention programme for patients who are completely or partially in remission from anxiety and/or depressive disorders, and who previously received treatment in mental healthcare services. This study will provide insight into: 1) the extent to which patients make use of the relapse prevention programme; 2) the factors that influence the use of the programme; 3) the association between usage intensity and course of symptoms; 4) barriers and facilitators in implementation of the programme; and 5) how patients evaluate the programme.
The methods section is divided into three parts: 1) the development of the relapse prevention programme; 2) the content of the relapse prevention programme; and 3) the study design.
Development of the relapse prevention programme
The programme was developed using input from different studies. Muntingh et al.  examined preferences of patients regarding relapse prevention and revealed that patients prefer a relapse prevention programme that is effective but not too time-consuming, taking up a maximum of 1 h a week. Using a personal relapse prevention plan, in combination with regular contact with a professional (about once every 3 months) was the most preferred relapse prevention strategy. The use of a relapse prevention plan as a practical aid for the early recognition and management of potential triggers and signs that indicate relapse is in accordance with the NICE guidelines . According to several studies, relapse prevention should be flexible and tailored to the patient’s individual situation and preferences in order to increase acceptability [29, 30]. Our online programme therefore consists of a personal relapse prevention plan and flexible E-health modules aiming at the promotion of self-management skills. The MHP can individually tailor the programme in conjunction with the patient.
The initial format and underpinning of the relapse prevention programme was discussed with academic experts, a panel of eight patients, and four MHPs. The content of the E-health modules was written by two expert psychologists, based on the principles of cognitive behavioural therapy (CBT). This therapy is effective in the treatment of anxiety and depression , and in preventing relapse in depression . Some optional modules were added at the request of patients, such as healthy food and physical exercise. These modules contain psychoeducation and the ability to plan healthy behaviour, with the aim of increasing physical and mental health. Preliminary versions of the E-health modules were reviewed by the members of the research team and E-health developers. Also, the patient panel reviewed the modules and provided feedback. This feedback was thoroughly discussed and processed by the researchers. For example: more lengthy text parts were placed in ‘read more’ menus, an overview page was created, and the possibility to print text was added. Finally, the adjusted content was released to the online platform.
Content of relapse prevention programme
The online programme consists of three basic components and 12 optional modules (see Fig. 1). The three basic components are: ‘Relapse psychoeducation’, ‘Relapse prevention plan’, and ‘Mood & anxiety diary’. As soon as the patient completes the module ‘Relapse psychoeducation’, the optional follow-up psychoeducation modules ‘Depression’, ‘Anxiety’ and ‘Medication’ will appear. Following the module ‘Relapse prevention plan’, the patient can choose which other optional modules he or she wishes to complete, based on own preferences and goals, thus offering the opportunity for the patient to customise treatment. The dotted lines in Fig. 1 indicate that certain modules refer automatically to other optional follow-up modules.
The modules include short videos on what to expect in the module, written information, exercises, and clinical examples of fictional patients. In all modules except the psychoeducation modules, patients have the possibility to ask for feedback from the MHP. For a more detailed overview of the contents of the E-health programme, see Table 1.
It takes about 30–60 min to complete each module. In each module, patients draft plans related to the specific content of that module, and are encouraged to continue practising, by offering the possibility to print out the plans.
Patients receive reminders via email regarding the completion of modules in the E-health platform. They also receive a newsletter every 6 weeks to keep them involved in the programme by providing information about numbers of included patients, experiences of other patients and interesting articles or facts about anxiety and depression. The MHP has access to the patient’s account, and can check whether the patient has been using the diary to monitor symptoms of anxiety and depression. Also, the MHP can monitor the patient’s individual use of the different modules. In turn, MHPs can provide feedback on the completed modules and start a conversation in the E-health platform with the patient.
Role of MHP
The MHP and the patient will have at least one face-to-face contact during the nine-month period of the study, and are encouraged to meet each other every 3 months. During the first contact, the patient and MHP will start drawing up the relapse prevention plan (if not yet available) and decide on the frequency and number of contacts. In the follow-up contacts, the outcomes of the ‘mood and anxiety diary’, the use of the available modules and possible questions will be discussed. The time required for the first contact is 45 min, and 20 min for the follow-up contacts. The MHP can actively support the patient in using the online programme and provide online feedback at the request of the patient.
In this mixed-methods observational cohort study, the relapse prevention programme we developed will be implemented and evaluated. This programme is targeted at patients who have been discharged from mental healthcare services, and are in complete or partial remission from an anxiety or depressive disorder. This relapse prevention programme is called ‘GET READY’ and offers access to an E-health platform, combined with regular contact with a MHP for a period of 9 months. General practices will be included in this study. Eligible patients completed mental health treatment for anxiety or depression and are in (partial) remission. Patients will complete online questionnaires at baseline, after 3, 6 and 9 months. In addition, individual interviews and focus group interviews will be held with patients as well as MHPs to evaluate the programme and its implementation.
Definition of terms
Throughout the literature, several terms are used regarding relapse and remission. Based on the definitions described by Frank et al. , we will make use of three terms: the term ‘relapse’ refers to a return of full symptomatology in concordance with the DSM-IV criteria for a depressive or anxiety disorder . The term ‘full remission’ indicates that no more than minimal symptoms are present and DSM-IV criteria for a disorder have not been fulfilled. In addition, ‘partial remission’ also indicates that DSM-IV criteria for a disorder have not been fulfilled, but that more than minimal symptoms are present.
This study will be performed in two settings:
General practices throughout the Netherlands, with the participation of approximately 50 MHPs. Patients follow the programme at home via an E-health platform, accompanied by face-to-face contact with the MHPs.
Ambulatory mental healthcare services with the participation of eligible patients whose GP is not willing to participate in this study. A trained MHP will deliver the relapse prevention programme using the same protocol as in the participating general practices.
Recruitment of MHPs
We aim to recruit 50 MHPs throughout the Netherlands. Recruitment will be done via telephone, letters, advertisements on websites for MHPs, and via the professional networks of the researchers. For the MHPs working in a general practice, the GP agrees that the MHP participates in this study. Informed consent will be obtained from the MHPs before inclusion.
Recruitment of participants
Patients will be recruited via general practices and mental healthcare services. Patients are eligible to participate when they have been treated for an anxiety disorder and/or a depressive disorder in mental healthcare services and are in full or partial remission, according to their clinician. To confirm the clinician’s judgement regarding remission status, the Inventory of Depressive Symptomatology (IDS-SR) and the Beck Anxiety Inventory (BAI) will be administered at baseline [34, 35]. Patients with scores > 39 on the IDS-SR and > 30 on the BAI are excluded from the study, since these scores indicate severe symptoms [36, 37]. Inclusion criteria are: patients have completed their treatment for anxiety and/or depression within the last 2 years, have a score on the Global Assessment of Functioning scale (GAF) of 50 or higher, are at least 18 years old, and have sufficient command of the Dutch language. Patients are excluded if they participate in another structured psychological intervention, when they do not have access to the internet, or when the severity of a comorbid psychiatric disorder requires specialised treatment.
Recruitment via general practices
Each MHP is requested to include all patients that completed mental health treatment for anxiety and depression and meet the inclusion criteria. MHPs will be asked to identify eligible patients through their patient files. MHPs invite potentially eligible patients for a consultation to discuss participation in the study, to provide information about the study and to check whether patients meet the inclusion criteria. If interest is shown in the study, the researcher contacts the patient and sends the baseline questionnaire. When patients have completed the baseline questionnaire, they receive access to the E-health platform. Eight days after sending the invitation for login, the researchers check if the patient has logged in. If not, they contact the patient to offer technical or practical support.
Recruitment via mental healthcare services
Patients who completed their treatment and are recruited via mental healthcare services, will be contacted by the researchers directly. These patients will be supported during the GET READY programme by a MHP working in the ambulatory mental healthcare.
Implementing the relapse prevention programme
Training of the MHP
Each MHP participates in a four-hour training course, focusing on the background and relevance of relapse prevention in anxiety and depressive disorders and potential effective intervention strategies regarding relapse prevention. Next, the content and use of the E-health platform is explained. Finally, information is provided about the study protocol. This training course will be given by either a psychologist or psychiatrist (both part of the research team), together with the first author of this paper.
Support for the MHP
After completing the training course, MHPs will receive an information package, including a protocol in which the following topics are described: recruitment of patients, the content of the face-to-face contact with patients, instructions on how to complete the case registration forms, and a guide on the E-health platform, containing screenshots. The package also includes invitation and information letters for patients.
The researchers offer monthly individual consultation via phone to the MHPs to support them in recruiting patients, discuss possible issues, and answer additional questions. Every month the MHPs receive a newsletter to update them on the study and motivate them to continue including patients.
Data collection – measurements
Patients are asked to complete four online questionnaires: at baseline (T0), after 3 months (T3), after 6 months (T6), and after 9 months (T9). It takes approximately 20–30 min to complete the questionnaires. Patients receive email invitations and if they do not complete the questionnaire, weekly reminders are sent.
Next, patients are requested to complete the ‘mood & anxiety diary’ every week to rate their level of anxiety and depression. They can complete the diary on their smartphone or on a computer.
After each face-to-face contact, the MHP completes a case registration form. This form contains information on the duration and content of the face-to-face contact, clinical status description, and whether additional appointments were made.
Data collection – outcome measures
Use of relapse prevention programme
Data will be collected regarding the use of the E-health platform data (number of logins, number of completed/uncompleted modules, and number of diary entries) and the frequency of contact with the MHPs, registered via the case registration forms.
Demographics and clinical variables
Sociodemographic characteristics and clinical variables of patients will be assessed at baseline, see Table 2. In addition, patients are asked to estimate their risk of relapse, and to indicate the expected effect of the programme.
Self-management strategies will be measured using the ‘Self-management in depression and anxiety’ questionnaire. This questionnaire was developed during a study on self-management interventions for chronic anxiety and depression . It consists of 45 items describing strategies that patients use to cope with anxiety and depression. Each item is rated on a 5-point Likert scale ranging from 1 (not at all) to 5 (a lot). The total score can be calculated by summing the individual scores.
Anxiety and depression scores
Anxiety severity will be measured by the Beck Anxiety Inventory (BAI), a 21-item tool, in which each item is rated on a 0 to 3 point scale, resulting in a total score between 0 to 63, with ≥30 indicating severe anxiety . This tool is reported to have a good reliability  and validity .
The Anxiety Sensitivity Index (ASI) is a 16-item questionnaire for measuring anxiety sensitivity. Each item is rated on a 5-point Likert scale with scores from 0 (barely) to 4 (very much), with a total score ranging from 0 (no anxiety sensitivity) to 64 (severe anxiety sensitivity) . The ASI is a reliable measure . Anxiety sensitivity has been found to be a predictor for relapse in patients with remitted anxiety disorders .
The Inventory of Depressive Symptomatology – self-report (IDS-SR) is a 30-item questionnaire for measuring severity of depressive symptoms . Each item is rated on a 0 to 3-point scale, and by summing 28 of the 30 items the total score ranges from 0 to 84, with ≥39 indicating severe depressive symptoms. The questionnaire has highly acceptable psychometric properties .
Anxiety and mood data are also collected in the weekly diary. Mood is measured by asking ‘how would you rate your mood today?, from 1 (very sad) to 10 (very happy)’. Anxiety is measured by asking ‘how would you rate the intensity of anxiety today?, from 1 (very relaxed) to 10 (very anxious/tense)’.
General functioning and disability will be measured using the World Health Organization Disability Assessment Schedule 2.0 (WHODAS 2.0). This 36-item tool measures six domains: cognition, mobility, self-care, getting along, life activities and participation. Each item is rated on a 5-point Likert scale with scores from 0 (no difficulty) to 4 (extreme difficulty or cannot do). By using the ‘item-response theory’-based scoring, the scores will range from 0 (no disability) to 100 (full disability) . The WHODAS 2.0 is reported to have strong psychometric properties .
Healthcare and medication use will be measured using an adjusted version of the Trimbos/iMTA Questionnaire for Costs Associated with Psychiatric Illness (TiC-P). Only psychotropic medication will be registered. Health care use is measured as the number of contacts with different healthcare providers. The TiC-P is a feasible and reliable instrument for measuring healthcare use and costs .
Satisfaction with the relapse prevention programme
Satisfaction with the relapse prevention programme will be assessed using the ‘Satisfaction with treatment measure’. This 14-item measure provides insight into overall satisfaction, how helpful and useful patients valued the programme, perceived support and effect . Also patients will be asked to rate each online module. The original scale was translated into Dutch by the authors, with permission of the author.
Quantitative data analysis
For each of the quantitative research questions, a separate analysis will be performed using log data from the E-health platform, data from the case registration forms that have been completed by the MHPs, and data from the four questionnaires:
To what extent do patients use a tailored relapse prevention programme?
The use of the tailored relapse prevention programme will be described, focusing on different aspects of its use, such as the number of login sessions, the amount of time spent online, the number of completed modules, the number of diary entries, the number of sessions and the number of online and face-to-face conversations with the MHP. Descriptive statistics will be used to provide insight into these aspects of the use of the relapse prevention programme.
What factors influence the use of the programme?
The use of the tailored relapse prevention programme may differ across patients, and may depend on age, education level and clinical situation. We will explore possible differences in usage between groups of patients systematically, considering sociodemographic factors, clinical symptoms and self-management strategies.
What is the association between usage intensity and course of symptoms?
Descriptive statistics will be used to provide insight into course of symptoms for ‘high-use’ participants and ‘low-use’ participants. In order to analyse the association between usage intensity and course of symptoms, multiple regression analyses will be performed. We will first analyse the change in anxiety severity from baseline to follow-up (9 months) with the following independent variables: intensity of E-health use, number of diary entries and number of contacts with the MHP. We will repeat these analyses for the depression outcomes. In addition, analyses will be performed using ‘deterioration: yes or no’ as outcome variable. Deterioration is defined as an increase of 1 standard deviation on the IDS-SR and/or on the BAI. The standard deviation will be calculated using data from the NESDA study , the study from Kok et al. , and the present study. This analysis will be performed using a time-lag model, in which the outcome will be assessed using determinants from an earlier measurement (deterioration at timet predicted by usage intensity at timet-1), since we assume that usage intensity might influence whether deterioration occurs at a later point in time.
Because of the observational nature of the data, it will be difficult to draw conclusions about the effects of the relapse prevention programme. However, we will conduct additional explorative analyses to estimate the association between usage intensity and course of symptoms. All data analysis will be performed using SPSS statistical analysis software.
In order to determine the sample size, we assumed a moderate effect (r = 0.24 that corresponds to Cohen’s d = 0.5) of the intensity of programme use (X) on the change in symptoms of anxiety and depression (Y). Since this is a non-randomized study, we will use a set of covariates W to correct for selectivity in the uptake of the programme. In applying the G*Power 3 software  to determine the sample size, we assume that the effect of X corresponds to a 6% of explained variance of Y (equivalent to the moderate effect, since R2 = r2 = 0.242 = 0.06) above the covariates W and assume a 6% reduction of the total variance in Y of the residual variance due to the use of covariates of W, leading to a partial R2 = 0.06 or, equivalently, an effect size of f2 = 0.0638. Furthermore, setting α = 0.05 and the power of 1 – β = 0.80, the sample size calculation shows that 126 patients are needed. Patients will be considered as drop-out when they refuse to complete questionnaires. With an estimated attrition of 20% (i.e. 80% complete at least one follow-up questionnaire), we aim to include 158 patients.
Data collection – measurements
The individual interviews and focus group interviews are conducted to assess implementation and satisfaction with the programme.
Paired interviews will be conducted separately with patients and their MHPs in order to gain insight into both perspectives on one case. These paired interviews provide valuable insights regarding similarities and differences in experiences of MHPs and patients [51, 52]. Purposive sampling will be used to select patients and realise sufficient variation in our sample with respect to gender, age, severity of symptoms, and use of the programme . Patients will be invited by email and telephone to participate in a semi-structured interview to evaluate their experience with the relapse prevention programme. If willing to participate, their MHP will be invited as well. The interviews will be conducted by two researchers, who will prepare the interviews by performing two test interviews. A senior researcher will provide supervision and participate in the analysis. We aim to conduct 12–15 interviews with patients and 12–15 interviews with MHPs. The final number of interviews depends on when data saturation is achieved.
A topic guide is developed to support the interview process [see Additional file 1]. This guide was drafted by the first author and reviewed by two experts, and inspired by the Consolidated Framework for Implementation Research (CFIR) . Main topics are: experiences with the relapse prevention programme, use of the relapse prevention programme, useful and less useful aspects of the programme, and suggestions to improve the programme. Input from the E-health platform will be used, such as completed and uncompleted modules, use of the diary, and number of online conversations with the MHP. The topic guide will be evaluated and updated after conducting four interviews.
The MHPs of patients who participate in the interviews will also be invited by email and telephone to participate in an interview. Besides evaluating the programme (research question 5), implementation barriers and facilitators will be discussed in these interviews (research question 4), see Additional file 2 for the topic guide.
These interviews will be conducted in a setting selected by participants and will take approximately 45 min.
Focus group interviews
After completing the individual interviews, two focus group interviews will be conducted, one with 8–10 patients and one with 8–10 MHPs. The goal of the focus group interviews is to present, test and discuss preliminary findings and conclusions from the individual interviews. A focus group interview is appropriate for this situation, since the group interaction provides insight into topics of agreement and favourable and unfavourable topics, and experiences with the programme can be shared . A moderator will lead the group discussion, and an assistant moderator will take notes, observe and keep track of the time . The input from the individual interviews will be used in these focus group interviews, to discuss desirable changes to the E-health programme, in order to further improve its quality and usability. In order to obtain new perspectives on the evaluative data from the individual interviews, one half of the focus group participants will not have participated in a previous individual interview. The other half of the focus group members will be purposively selected from the patients and MHPs who previously participated in the individual interviews, where the selection is based on the diverging perspectives on using the relapse prevention programme. These focus group interviews will be conducted in a mental healthcare facility and will take approximately 90 min.
Qualitative data analysis
For each of the qualitative research questions, a separate analysis will be performed:
What are barriers and facilitators in the implementation of the programme?
The barriers and facilitators in the implementation of the programme will be assessed through the semi-structured interviews and focus group interviews with the MHPs. These interviews will be audio recorded, and transcribed verbatim. Data will be sorted using the CFIR , which indicates five domains of implementation (intervention characteristics, outer setting, inner setting, characteristics of individuals and process). Data collection will alternate with data analysis and new topics will be discussed in the following interviews. Two researchers will independently openly code the first three interviews, compare the codes and draft a coding tree. The coding tree will be complemented through the following interviews. This data will be analysed using thematic analysis , which means that the interviews will be coded for themes, these codes will then be sorted and analyses will be performed using software programme MaxQDA. When reporting the data, quotations will be used to illustrate the findings.
During this process, multiple researchers will be involved in order to increase the validity and reliability . From the beginning of data collection until the end of data analysis, detailed field notes will be documented, containing specific situations, reflections, and ideas and thoughts of the researchers . In addition, a summary will be made of each interview, containing the most important findings.
How do patients evaluate the program?
The evaluation of the programme will be assessed by the semi-structured interviews and focus group interviews with the patients. In addition, in the last follow-up questionnaire (T9), patients will be asked about their satisfaction with the programme and to rate every module they completed on a scale from 0 to 10. The data from the interviews and focus group interviews will be analysed in the same way as described above (research question 4). Data from T9 will be analysed using descriptive statistics.
Providing relapse prevention for anxiety and depression is important, since these disorders are often chronic and recurrent. In this study we will implement and evaluate a newly developed relapse prevention programme, specifically targeted at patients that are (partially) remitted from an anxiety disorder or depression. The mixed-methods approach in this study will provide valuable insights into how patients use the programme, what influences the use of the programme, how usage intensity influences symptoms and how patients evaluate the programme. In addition, information on implementation will be provided which may be relevant for broad implementation of the programme. The findings of this study can be used to further refine and adapt the programme as preferred by patients and MHPs.
A strength of this study is that the relapse prevention programme has been developed based on patient preferences and in close collaboration with patients. Additionally, this programme is not only web-based, but also supplemented by contact with MHPs. Earlier research suggests that E-health can be especially effective when accompanied by therapist contact . By using log data to establish the usage of the programme, the actual use and usage behaviour can be determined . Furthermore, by using mixed methods, a complete view on the use of the programme and its evaluation can be obtained .
This design is not appropriate to examine efficacy, since only within-group effect sizes can be determined, which might be considered a limitation of this study. However, the aim of this study was not to examine the efficacy of the programme, but its acceptability. If the programme is acceptable, a randomized controlled trial should be conducted to determine the efficacy of the programme.
Anxiety Sensitivity Index
Beck Anxiety Inventory
Cognitive behavioural therapy
Consolidated framework for implementation research
Diagnostic and statistical manual of mental disorders
- GET READY:
Guided E-healTh for RElapse prevention in Anxiety and Depression
Inventory of Depressive Symptomatology – self-report
Mental health professional
Trimbos/iMTA Questionnaire for Costs Associated with Psychiatric Illness
World Health Organization Disability Assessment Schedule
Bruce SE, Yonkers KA, Otto MW, Eisen JL, Weisberg RB, Pagano M, et al. Influence of psychiatric comorbidity on recovery and recurrence in generalized anxiety disorder, social phobia, and panic disorder: a 12-year prospective study. Am J Psychiatry. 2005;162:1179–87. https://doi.org/10.1176/appi.ajp.162.6.1179.
Verduijn J, Verhoeven JE, Milaneschi Y, Schoevers RA, van Hemert AM, Beekman ATF, et al. Reconsidering the prognosis of major depressive disorder across diagnostic boundaries: Full recovery is the exception rather than the rule. BMC Med. 2017;15:1–9. https://doi.org/10.1186/s12916-017-0972-8
Scholten WD, Batelaan NM, van Balkom AJ, Penninx BW, Smit JH, van Oppen P. Recurrence of anxiety disorders and its predictors. J Affect Disord. 2013;147:180–5.
Penninx BWJH, Nolen WA, Lamers F, Zitman FG, Smit JH, Spinhoven P, et al. Two-year course of depressive and anxiety disorders: results from the Netherlands study of depression and anxiety (NESDA). J Affect Disord. 2011;133:76–85. https://doi.org/10.1016/j.jad.2011.03.027.
Taylor JH, Jakubovski E, Bloch MH. Predictors of anxiety recurrence in the coordinated anxiety learning and management (CALM) trial. J Psychiatr Res. 2015;65:154–65. https://doi.org/10.1016/j.jpsychires.2015.03.020.
Conradi HJ, Bos EH, Kamphuis JH, de Jonge P. The ten-year course of depression in primary care and long-term effects of psychoeducation, psychiatric consultation and cognitive behavioral therapy. J Affect Disord. 2017;217:174–82. https://doi.org/10.1016/j.jad.2017.03.064.
Gopinath S, Katon WJ, Russo JE, Ludman EJ. Clinical factors associated with relapse in primary care patients with chronic or recurrent depression. J Affect Disord. 2007;101:57–63.
Hardeveld F, Spijker J, De Graaf R, Hendriks SM, Licht CMM, Nolen WA, et al. Recurrence of major depressive disorder across different treatment settings: results from the NESDA study. J Affect Disord. 2013;147:225–31. https://doi.org/10.1016/j.jad.2012.11.008.
Yiend J, Paykel E, Merritt R, Lester K, Doll H, Burns T. Long term outcome of primary care depression. J Affect Disord. 2009;118:79–86. https://doi.org/10.1016/j.jad.2009.01.026.
Van Londen L, Molenaar RP, Goekoop JG, Zwinderman AH, Rooijmans HG. Three- to 5-year prospective follow-up of outcome in major depression. Psychol Med. 1998;28:731–5.
Pintor L, Gastó C, Navarro V, Torres X, Fañanas L. Relapse of major depression after complete and partial remission during a 2-year follow-up. J Affect Disord. 2003;73:237–44.
Paykel ES, Ramana R, Cooper Z, Hayhurst H, Kerr J, Barocka A. Residual symptoms after partial remission: an important outcome in depression. Psychol Med. 1995;25:1171–80.
Wagner EH, Austin BT, Davis C, Hindmarsh M, Schaefer J, Bonomi A. Improving chronic illness care: translating evidence into action. Health Aff. 2001;20:64–78.
Von Korff M, Gruman J, Schaefer J, Curry S, Wagner E. Collaborative Management of Chronic Illness. Ann Intern Med. 1997;127:1097–102.
Lorig K. Self-management of chronic illness: a model for the future. Gener XVII. 1993;3:11–4.
van Balkom A, Oosterbaan D, Visser S, van Vliet I. Handboek Angststoornissen. Utrecht: De Tijdstroom uitgeverij BV; 2011.
Schene A, Sabbe B, Spinhoven P, Ruhé H. Handboek Depressieve stoornissen. 2nd editio. Utrecht: De Tijdstroom uitgeverij BV; 2016.
Howell CA. Preventing depression relapse: a primary care approach. Prim Care Ment Heal. 2004;2:151–156.
Biesheuvel-Leliefeld KEM, Kok GD, Bockting CLH, Cuijpers P, Hollon SD, Van Marwijk HWJ, et al. Effectiveness of psychological interventions in preventing recurrence of depressive disorder: meta-analysis and meta-regression. J Affect Disord. 2015;174:400–10. https://doi.org/10.1016/j.jad.2014.12.016.
Biesheuvel-Leliefeld KEM, Dijkstra-Kersten SMA, Van Schaik DJF, Van Marwijk HWJ, Smit F, Van Der Horst HE, et al. Effectiveness of supported self-help in recurrent depression: a randomized controlled trial in primary care. Psychother Psychosom. 2017;86:220–30.
Scholten WD, Batelaan NM, Van Oppen P, Smit JH, Van Balkom AJLM. Discontinuation of antidepressants in remitted anxiety disorder patients: the need for strategies to prevent relapse. Psychother Psychosom. 2013;82:399–400.
Von Korff M, Goldberg D. Improving outcomes in depression. BMJ. 2001;323:948–9.
Katon W, Rutter C, Ludman EJ, Von Korff M, Lin E, Simon G, et al. A randomized trial of relapse prevention of depression in primary care. Arch Gen Psychiatry. 2001;58:241–7. https://doi.org/10.1001/archpsyc.58.3.241.
Hermens MLM, Muntingh A, Franx G, Van Splunteren PT, Nuyen J. Stepped care for depression is easy to recommend, but harder to implement: results of an explorative study within primary care in the Netherlands. BMC Fam Pract. 2014;15:5. https://doi.org/10.1186/1471-2296-15-5.
Krijgsman J, Swinkels I, van Lettow B, de Jong J, Out K, Friele R, et al. Meer dan techniek, eHealth-monitor 2016: Den Haag & Utrecht; 2016.
Baumeister H, Reichler L, Munzinger M, Lin J. The impact of guidance on internet-based mental health interventions - a systematic review. Internet Interv. 2014;1:205–15. https://doi.org/10.1016/j.invent.2014.08.003.
Muntingh A, Hoogendoorn A, Van Schaik D, Van Straten A, Stolk E, Van Balkom A, et al. Patient preferences for a guided self-help programme to prevent relapse in anxiety or depression: A discrete choice experiment. In press.
The British Psychological Society and The Royal College of Psychiatrists. The Nice Guideline on the Treatment and Depression the Treatment and Management of Depression (updated edition). 2010. https://www.ncbi.nlm.nih.gov/books/NBK63748/.
Boggs JM, Beck A, Felder JN, Dimidjian S, Metcalf CA, Segal ZV. Web-based intervention in mindfulness meditation for reducing residual depressive symptoms and relapse prophylaxis: a qualitative study. J Med Internet Res. 2014;16:1–12. https://doi.org/10.2196/jmir.3129.
Notley C, Christopher R, Hodgekins J, Byrne R, French P, Fowler D. Participant views on involvement in a trial of social recovery cognitive-behavioural therapy. Br J Psychiatry. 2015;206:122–7. https://doi.org/10.1192/bjp.bp.114.146472.
Butler AC, Chapman JE, Forman EM, Beck AT. The empirical status of cognitive-behavioral therapy: a review of meta-analyses. Clin Psychol Rev. 2006;26:17–31.
Frank E, Prien RF, Jarrett RB, Keller MB, Kupfer DJ, Lavori PW, et al. Conceptualization and rationale for consensus definitions of terms in major depressive disorder. Remission, recovery, relapse, and recurrence. Arch Gen Psychiatry. 1991;48:851–5.
American Psychiatric Association. DSM-IV Diagnostic and Statistical Manual of Mental Disorders. 1994.
Beck AT, Epstein N, Brown G, Steer RA. An inventory for measuring clinical anxiety: psychometric properties. J Consult Clin Psychol. 1988;56:893–7. https://doi.org/10.1037/0022-006X.56.6.893.
Rush AJ, Gullion CM, Basco MR, Jarrett RB, Trivedi MH. The inventory of depressive symptomatology (IDS): psychometric properties. Psychol Med. 1996;26:477–86.
Steer RA, Beck AT. Beck anxiety inventory manual. San Antonio: The Psychological Corporation Harcourt Brace & Company; 1993.
Inventory of Depressive Symptomatology (IDS) and Quick Inventory of Depressive Symptomatology (QIDS) - Interpretation. http://www.ids-qids.org/interpretation.html. Accessed 22 May 2018.
Zoun MHH, Koekkoek B, Sinnema H, Muntingh ADT, van Balkom AJLM, Schene AH, et al. Effectiveness and cost-effectiveness of a self-management training for patients with chronic and treatment resistant anxiety or depressive disorders: design of a multicenter randomized controlled trial. BMC Psychiatry. 2016;16:216. https://doi.org/10.1186/s12888-016-0927-1.
De Ayala RJ, Vonderharr-Carlson DJ, Kim D. Assessing the reliability of the Beck anxiety inventory scores. Educ Psychol Meas. 2005;65:836–50.
Fydrich T, Dowdall D, Chambless DL. Reliability and validity of the beck anxiety inventory. J Anxiety Disord. 1992;6:55–61.
Reiss S, Peterson RA, Gursky DM, McNally RJ. Anxiety sensitivity, anxiety frequency and the predictions of fearfulness. Behav Res Ther. 1986;24:1–8. https://doi.org/10.1016/0005-7967(86)90143-9.
Rodriguez BF, Bruce SE, Pagano ME, Spencer MA, Keller MB. Factor structure and stability of the anxiety sensitivity index in a longitudinal study of anxiety disorder patients. Behav Res Ther. 2004;42:79–91.
Trivedi MH, Rush AJ, Ibrahim HM, Carmody TJ, Biggs MM, Suppes T, et al. The inventory of depressive symptomatology, clinician rating (IDS-C) and self-report (IDS-SR), and the quick inventory of depressive symptomatology, clinician rating (QIDS-C) and self-report (QIDS-SR). Psychol Med. 2004;34:73–82.
WHO. WHO Disability Assessment Schedule 2.0 (WHODAS 2.0). 2018. https://www.who.int/classifications/icf/WHODAS2.0_36itemsSELF.pdf. Accessed 14 Aug 2018.
Konecky B, Meyer EC, Marx BP, Kimbrel NA, Morissette SB. Using the WHODAS 2.0 to assess functional disability associated with DSM-5 mental disorders. Am J Psychiatry. 2014;171:818–20. https://doi.org/10.1176/appi.ajp.2014.14050587.
Bouwmans C, De Jong K, Timman R, Zijlstra-Vlasveld M, Van der Feltz-Cornelis C, Tan SS, et al. Feasibility, reliability and validity of a questionnaire on healthcare consumption and productivity loss in patients with a psychiatric disorder (TiC-P). BMC Health Serv Res. 2013;13:217. https://doi.org/10.1186/1472-6963-13-217.
Richards D, Murphy T, Viganó N, Timulak L, Doherty G, Sharry J, et al. Acceptability, satisfaction and perceived efficacy of “space from depression” an internet-delivered treatment for depression. Internet Interv. 2016;5:12–22.
Penninx BWJH, Beekman ATF, Smit JH, Zitman FG, Nolen WA, Spinhoven P, et al. The Netherlands study of depression and anxiety (NESDA): rationale, objectives and methods. Int J Methods Psychiatr Res. 2008;17:121–40.
Kok G, Burger H, Riper H, Cuijpers P, Dekker J, Van Marwijk H, et al. The three-month effect of mobile internet-based cognitive therapy on the course of depressive symptoms in remitted recurrently depressed patients: results of a randomized controlled trial. Psychother Psychosom. 2015;84:90–9.
Faul F, Erdfelder E, Lang A-G, Buchner A. G*power 3: a flexible statistical power analysis program for the social, behavioral, and biomedical sciences. Behav Res Methods. 2007;39:175–91.
Bosman RC, Huijbregts KM, Verhaak PF, Ruhé HG, van Marwijk HW, van Balkom AJ, et al. Long-term antidepressant use: a qualitative study on perspectives of patients and GPs in primary care. Br J Gen Pract. 2016;66 https://bjgp.org/content/66/651/e708. Accessed 14 Sep 2017.
Dikkers MF, Westerman MJ, Rubinstein SM, Van Tulder MW, Anema JR. Why neck pain patients are not referred to manual therapy: a qualitative study among Dutch primary care stakeholders. PLoS One. 2016;11:1–18. https://doi.org/10.1371/journal.pone.0157465.
Coyne IT. Sampling in qualitative research. Purposeful and theoretical sampling; merging or clear boundaries? J Adv Nurs. 1997;26:623–30. https://doi.org/10.1046/j.1365-2648.1997.t01-25-00999.x.
Damschroder LJ, Aron DC, Keith RE, Kirsh SR, Alexander JA, Lowery JC. Fostering implementation of health services research findings into practice: a consolidated framework for advancing implementation science. Implement Sci. 2009;4:1–15.
Duggleby W. What about focus group interaction data? Qual Health Res. 2005;15:832–40.
Boeije H. Analyseren in kwalitatief onderzoek. 2nd ed. Amsterdam: Boom Lemma; 2014.
Braun V, Clarke V. Using thematic analysis in psychology. Qual Res Psychol. 2006;3:77–101. https://doi.org/10.1191/1478088706qp063oa.
Richards D, Richardson T. Computer-based psychological treatments for depression: a systematic review and meta-analysis. Clin Psychol Rev. 2012;32:329–42. https://doi.org/10.1016/j.cpr.2012.02.004.
Sieverink F, Kelders S, Poel M, van Gemert-Pijnen L. Opening the black box of electronic health: collecting, analyzing, and interpreting log data. JMIR Res Protoc. 2017;6:e156. https://doi.org/10.2196/resprot.6452.
This study is funded by SIA-RAAK: The Taskforce for Applied Research, part of the Netherlands Organisation for Scientific Research (NWO). The funder had no role in study design, data collection and analysis, interpretation of data, and preparation of the manuscript.
Availability of data and materials
Upon reasonable request, the datasets used and/or analysed during the current study will be available from the corresponding author.
Ethics approval and consent to participate
The Medical Ethics Committee of the VU University Medical Centre confirmed that the “Medical Research Involving Human Subjects ACT” (WMO) does not apply to this study (registration number 2016.280), and gave their permission for the execution of the study. The Scientific Research Committee of GGZ inGeest Mental Health Organization approved the research proposal (CWO 2016–023). Informed consent will be obtained from participants via an online form for the quantitative data, and a separate written informed consent form for the qualitative interviews.
Consent for publication
The authors declare that they have no competing interests.
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.