- Study protocol
- Open Access
- Open Peer Review
Effectiveness and cost-effectiveness of web-based treatment for phobic outpatients on a waiting list for psychotherapy: protocol of a randomised controlled trial
BMC Psychiatryvolume 12, Article number: 131 (2012)
Phobic disorders are highly prevalent and constitute a considerable burden for patients and society. As patients wait for face-to-face psychotherapy for phobic disorders in outpatient clinics, this time can be used for guided self-help interventions. The aim of this study is to investigate a five week internet-based guided self-help programme of exposure therapy in terms of clinical effectiveness and impact on speed of recovery in psychiatric outpatients, as well as the cost-effectiveness of this pre-treatment waiting list intervention.
A randomised controlled trial will be conducted among 244 Dutch adult patients recruited from waiting lists of outpatient clinics for face-to-face psychotherapy for phobic disorders. Patients suffering from at least one DSM-IV classified phobic disorder (social phobia, agoraphobia or specific phobia) are randomly allocated (at a 1:1 ratio) to either a five-week internet-based guided self-help program followed by face-to-face psychotherapy, or a control group followed by face-to-face psychotherapy. Waiting list status and duration are unchanged and actual need for further treatment is evaluated prior to face-to-face psychotherapy. Clinical and economic self-assessment measurements take place at baseline, post-test (five weeks after baseline) and at 3, 6, 9 and 12 months after baseline.
Offering pre-treatment internet-based guided self-help efficiently uses time otherwise lost on a waiting list and may increase patient satisfaction. Patients are expected to need fewer face-to-face sessions, reducing total treatment cost and increasing speed of recovery. Internet-delivered treatment for phobias may be a valuable addition to psychotherapy as demand for outpatient treatment increases while budgets decrease.
Netherlands Trial Register NTR2233
Anxiety disorders have a pooled lifetime prevalence of 19.8% . This makes these disorders the most prevalent mental disorders for women, and the second most prevalent for men. Specific phobias are the most common form of anxiety disorders for both genders, with a total 12-month prevalence of 7.1%, followed by social phobia (4.8%) and agoraphobia without panic disorder (1.2%). Phobias are characterised by an excessive fear of situations or objects and are often treated by gradual exposure to the fear-inducing stimulus and/or cognitive behavioural therapy.
The economic burden of these conditions is considerable , as is the negative impact on quality of life and psychosocial functioning , especially if the patient is often confronted with the fear-inducing situation or object in life. In clinical populations this burden may have long-term effects on psychosocial functioning and well-being, as research suggests that often more than a decade passes between onset of symptoms and first therapy attendance . The prevalence and burden of phobic disorders call for effective and cost-effective treatment options and the Internet offers an accessible and widespread platform to disseminate low-threshold treatment for phobias.
Research has shown that guided self-help is effective for anxiety disorders  and that it can be as effective as face-to-face (FTF) psychotherapy . Self-help programs adapted for the Internet have shown similar results [7, 8], e.g. for social anxiety disorder , generalised anxiety disorder  and severe health anxiety . The use of internet interventions for treating anxiety disorders has been well-established and deemed useful and acceptable in primary care and the general population . A recent small scale study found a large effect size of internet-based treatment for panic disorder in a psychiatric outpatient setting  and internet-based cognitive behavioural therapy (CBT) for adolescent anxiety appeared to be as efficacious as clinic-based FTF psychotherapy . Apart from these studies, little is known about the clinical effectiveness or cost-effectiveness of internet-based CBT in psychiatric outpatient settings. Few studies have examined internet-based treatment of phobias in particular and these tended to focus on social phobia in the general population or primary care patients.
Waiting lists of varying lengths are ubiquitous in psychiatric outpatient clinics. As demand for psychotherapy often exceeds supply, patients cannot be treated immediately. Time otherwise ‘lost’ while a patient is on a waiting list could be spent more efficiently when a waiting list guided self-help intervention is offered that comprises elements common in FTF psychotherapy for phobias, e.g. psychoeducation, goal setting and exposure exercises. Offering patients a self-help intervention while on a waiting list for FTF treatment can be advantageous to both outpatient clinics and patients for several reasons.
Firstly, following the stepped-care model, the intervention may be sufficient treatment for some phobic disorders, cancelling the need for further psychotherapy and preventing inappropriate use of outpatient resources.
Secondly, being placed on a waiting list may be disappointing to patients who have been suffering from a phobia for a long time. This disappointment may lead to pre-treatment attrition, i.e., patients not starting FTF psychotherapy or 'no-show'. This has been shown to be relatively high in FTF settings, up to 30% in social phobia . Offering the intervention during the waiting period offers a mutual benefit of keeping the patient actively engaged in his or her treatment and thus possibly reducing this pre-treatment attrition.
Thirdly, depending on the uptake and effects of the guided self-help intervention, fewer FTF psychotherapy sessions might suffice when the waiting period has ended and FTF treatment is scheduled, thus lowering the pressure on psychiatric outpatient clinics. This can also lead to lower per-patient treatment costs.
As healthcare budgets stagnate or decline and demand for treatment for phobias increases, a larger number of patients will need to be treated at lower costs per patient.
Aims and hypotheses
The objective of this study is to establish the clinical and cost-effectiveness of an internet-based brief guided self-help intervention for phobic patients in outpatient clinics. We expect that patients are more satisfied as they can start treatment while on a waiting list and that outpatient clinics can reduce the number of costly FTF psychotherapy sessions due to the skills and knowledge already acquired by the patient in the guided self-help intervention. Furthermore, we expect that patients in the intervention group will recover more quickly than in the control group.
This study is a randomised controlled trial alongside an economic evaluation. Patients on a waiting list for FTF psychotherapy are randomised to either the intervention group, who will receive access to the internet-based guided self-help program, or the control group who will receive a self-help book for motivational purposes without additional guidance. Both groups will receive FTF psychotherapy at the outpatient clinic following the guided self-help or unguided motivational book.
The study protocol, information brochure, questionnaires and informed consent form were approved by the Medical Ethics Committee of the VU University Medical Centre (registration number 2010/77).
Inclusion and exclusion criteria
All patients aged 18 and over on the waiting list of participating outpatient clinics and willing to enrol in an internet-based self-help program are eligible for this study. Inclusion criteria are: suffering from a DSM-IV diagnosis of agoraphobia, social phobia or specific phobia as determined by a trained interviewer using a diagnostic interview (the Composite International Diagnostic Interview (CIDI) ). Patients are allowed to continue their psychotropic medication if they stay on a stable regimen throughout the intervention. Exclusion criteria are: no access to the internet, inadequate self-reported computer literacy, inadequate proficiency in Dutch, suicidal ideation or increased risk of suicide and bipolar or psychotic disorder. Comorbid disorders other than bipolar or psychotic disorders are allowed.
Patients are recruited at several outpatient clinics. For practical reasons, only clinics with a high number of monthly enrolments were selected. Generally speaking, patients are referred to the outpatient clinic by their general practitioner (GP). At the clinic the patient is briefly screened and placed on a waiting list. After some time an initial meeting with a therapist takes place, during which the patient’s treatment needs and preferences are determined in detail. Based on this, a treatment suited to the individual patient is selected, e.g. group CBT therapy for social anxiety disorders or individual exposure-based therapy for specific phobias. The researchers are not involved in this decision process. Depending on therapist workload, treatment modality and other factors, there is considerable variability in the length of the interval between referral to the clinic and first therapeutic contact. This interval typically ranges from five - twenty weeks.
For this study, patients are recruited when placed on the post-screening waiting list of outpatient clinics. Patients agreeing to be contacted by the researchers are approached by the researchers and a telephone diagnostic interview (CIDI) is conducted by a research assistant to determine clinical status. If eligible, the patients are administered a telephone baseline assessment and randomised. Patients are sent an information kit containing a study brochure, informed consent form and stamped return envelope. The first therapeutic contact (i.e., first meeting with therapist to determine patient’s status and needs) is not postponed if the patient is enrolled in this study. For a detailed representation of participant flow in this study, see Figure 1.
After baseline assessment patients are randomised by an external researcher using a computer-generated randomisation chart which is stratified by clinic. Block randomisation is used with eight allocations per block, using a 1:1 ratio. Researchers are blind to the randomisation procedure and to treatment group allocation until randomisation is complete and unequivocal. Due to the nature of the intervention, patients cannot be blinded to group allocation as the two treatment options need to be revealed prior to randomisation.
Intervention: internet-based guided self-help exposure therapy
The intervention is adapted for the internet from a self-help book on phobias , which is based on exposure therapy, the de facto standard treatment in phobic anxiety disorders. The internet treatment is offered at no cost to the patient, takes five weeks to complete and is based on psycho-education and exposure therapy with optional though schedules. Patients select fear-inducing situations or stimuli and construct a ‘fear hierarchy’ to increasingly expose themselves to these situations or stimuli in gradual, ascending steps. Background information and psychoeducation on phobias is given, as well as information on relapse prevention, recommendations and gathering support from friends. A number of example patients are included to illustrate the principles and concepts of the intervention. Support consists of online coaching messages, delivered through a closed, secured message system on the intervention website. This is offered by master’s level students of clinical psychology who have received training in online coaching. Research has shown that relatively inexperienced persons can provide effective online coaching on a level equal to an experience clinician, if they are adequately trained, supervised and the treatment is sufficiently standardized and manualized [18, 19]. The patient completes exposure exercises as homework assignments and reports on his or her accomplishments each week. The coach monitors the fear hierarchy and evaluates whether the patient’s exposure exercise planning is feasible, relevant and useful and replies with a support message with praise, recommendations and information relevant to the patient’s homework accomplishments or experiences.
The intervention is tunnelled, i.e., no new material is available to the patient until the patient has reported on that week’s achievements and the coach has provided feedback on these achievements. If applicable, the coach sends a standardized reminder message if the patient did not use the website that week. Material from previous weeks remains accessible to the patient. To allow for any technical difficulties, patients are allowed to overrun the five-week intervention by one week.
All coaching will be supervised by an experienced psychotherapist (MdN), who is trained in in-vivo exposure therapy. We will keep close contact with the institutions and therapists and carefully monitor each patient, sending the therapist an informative reminder message of the patient’s participation in the study immediately before first therapeutic contact. This is to encourage the therapist to assess the patient’s condition first and, if deemed necessary, to adjust therapeutic approach and aims to suit the patient’s prior knowledge and previous exposure accomplishments attained during the intervention.
The control group will receive a self-help book  free of charge, without additional support. This self-help book contains the same information as the intervention website, i.e., psycho-education and exposure therapy, but has more optional exercises such as thought schedules. It was deemed unethical to enrol patients into the trial and subsequently allocate them to a control condition without offering at least some compensation for their time and effort. The book is sent as-is, without further instructions or expectations, as unguided bibliotherapy. The patients will have no contact with the research team with regards to the self-help book during or after the self-help period.
Patients in both the intervention group and the control group are scheduled to receive follow-up FTF psychotherapy by licensed health care psychologists at the end of the waiting list but patients can choose to decline further FTF treatment.
In total, 6 assessments will take place. The baseline assessment takes place before randomisation, with follow-up assessments post-intervention (at five weeks), and at 3, 6, 9 and 12 months. Baseline measures are collected by telephone to ensure pre-treatment assessments are present for all patients and follow-up measures are administered as online questionnaires.
Primary outcome: presence of phobia and avoidance behaviour
The primary outcome measure is the Fear Questionnaire (FQ) . This instrument measures severity of fear and avoidance of phobic stimuli. The FQ is a self-rated questionnaire adapted for use on the internet and consists of 15 items that measure total phobia and five items that measure anxiety and depression. Patients are asked to rate how often they avoid different situations on a scale of 0 (“I do not avoid this”) to 8 (“I always avoid this”).
Due to the large number of possible specific phobias, the primary phobia for which the patient seeks treatment is filled out in a single item rather than chosen from a list.
A total phobia score ranges from 15–120 and is calculated from three subscales; agoraphobia, blood/injection phobia and social phobia. A higher score indicates greater avoidance of phobic stimuli. A single item measures global phobic distress, or the extent to which phobic symptoms limit the patient’s daily activities, on a scale of 0 (no phobia) to 8 (extremely limiting or disruptive). The psychometric validity of the FQ has been established for both the original [21, 22] and the Dutch version .
The Beck Anxiety Inventory (BAI)  is a 21-item self-report questionnaire which focuses on physiological manifestations of anxiety, e.g. sweating, trembling and increased heart rate. Items can be scored from 0–3 for a total sum score range of 0–63, where higher scores indicate higher levels of self-reported manifestations of anxiety. The BAI has been validated for patients with agoraphobia  and other anxiety disorders .
The Centre for Epidemiological Studies Depression Scale (CES-D)  is administered as a self-rated questionnaire on the internet. The CES-D consists of 20 items which are scored from 0–3, the total score can range from 0–60, where a higher score indicates a higher level of depressive symptoms. A Dutch version of the CES-D has been validated in an internet-administrated form .
Quality of life
Health-related quality of life is measured using an internet-based version of the EuroQol (EQ-5D). The EQ-5D appears to have sufficient validity [29, 30]. Patients report whether they have ‘no problems’, ‘some problems’ or ‘extreme problems’ on each of five domains: mobility, self-care, daily activities, pain and discomfort and anxiety/depression. The combination of items results in 243 distinct health states. This results in a utility score for each health state that ranges from death (0) to perfect health (1). The Dutch tariff will be used to value these health states .
Satisfaction with the internet intervention or the self-help book is evaluated using the Client Satisfaction Questionnaire (CSQ-8)  which has been validated for use in a Dutch population . The CSQ-8 has been adapted to a self-rated internet-administered questionnaire. A total of 8 items can be scored from 1–4, yielding a total range score of 8–32, where a higher score indicates a higher level of satisfaction with received care.
Several items are added to assess the patient’s satisfaction with the intervention website and coach or the self-help book and, if applicable, why the patient decided to stop using the book or the website.
The CAGE questionnaire  is a brief screening measure for alcohol dependence. It has been validated as being able to establish heavy drinking and/or alcohol dependence . Although heavy alcohol use is not among the exclusion criteria, we monitor problematic alcohol use in the light of the complex relationship between anxiety disorders and alcohol consumption .
Health care utilization and work absenteeism
The iMTA Questionnaire on Costs Associated with Psychiatric Illness (TiC-P)  is a structured questionnaire to measure costs from a societal perspective. Cost categories that are included are use of medication, health services and informal care (direct costs), and work absenteeism or and/or reduced work productivity (indirect costs). The friction cost method will be applied to estimate costs of work absenteeism. Dutch standard costs will be used to value direct and indirect costs .
Data on FTF psychotherapy after intervention or self-help book
Data on received psychotherapy at the outpatient clinic will be retrieved post-hoc from the institutions’ electronic medical records for both groups. Data on the modality of psychotherapy (e.g. individual CBT, group therapy) and the number of sessions will be recorded.
Internet-specific data and sundry expenses
Internet usage specifics such as number of logins, total duration of session, number of modules completed will be collected on a per-patient basis. Total costs for development, maintenance, patient screening, coaching and sundry expenses are calculated on a per-patient basis (Table 1).
To obtain 90% statistical power with a 2-sided α equal to 0.05 and assuming a mean standardised effect size (Cohen’s d) of 0.7 in the intervention group and 0.2 in the control group, 85 patients are needed for each trial arm to establish the clinical effect of the internet intervention compared to the waiting list condition (170 patients in total). We assume a dropout rate of 30% during one-year follow-up, thus, 244 patients should be included.
Results will be reported on both intention-to-treat and per-protocol basis. Missing values will be imputed using multiple imputation, as recent research suggests that this is a reliable method for dealing with missing values within datasets .
The mean difference in costs between the control group and intervention group is calculated, using bootstrapping to estimate 95% confidence intervals. Incremental cost-effectiveness ratios (ICERs) will then be calculated by dividing the difference in costs between the two groups by the difference in clinical effects. Bootstrapping will be used to estimate the uncertainty surrounding the ICERs, which will be graphically represented on cost-effectiveness planes. Cost-effectiveness acceptability curves and net monetary benefits will also be calculated. Cost-effectiveness acceptability curves show the probability that the intervention is cost-effective in comparison with usual care for a range of different ceiling ratios, thereby showing decision uncertainty .
This study will evaluate the clinical and cost-effectiveness of offering internet-based guided self-help for phobic outpatients. Results of internet therapy for phobias in the general population are encouraging and implementation in outpatient clinics may be a valuable addition to existing treatment options. Also, offering of internet therapy while a patient is waiting for FTF psychotherapy makes efficient use of this time and may have large benefits for both the patient and the outpatient clinic in terms of clinical effectiveness, cost-effectiveness and patient satisfaction.
This study faces a number of challenges. Firstly, it is imperative that the patients’ therapists assess the needs of their patients prior to actually starting FTF psychotherapy. If the therapist starts the patient on a standard treatment (e.g. a fixed number of FTF psychotherapy sessions as prescribed by treatment protocol) regardless of the skills and knowledge acquired during the internet intervention, then the addition of the internet intervention would be ineffectual in lowering treatment costs.
Secondly, there is considerable variability in the time patients spend on a waiting list. This varies both between and within institutions and is influenced by several factors such as seasonal changes and peaks or drops in patient referrals or staff workloads. The direct implication is that there is some variability in the time between the end of the guided self-help intervention and the start of FTF psychotherapy. Patients are allowed to re-visit the intervention website throughout the remainder of the waiting list period. However, they will no longer be able to communicate with a coach. Since patients in both the intervention group and the control group are on the same waiting list, there is no reason to assume that there is a difference in time spent on the waiting list between both groups.
Thirdly, attrition from and lack of adherence to internet-based interventions are debated topics in internet-based interventions. A recent meta-analysis found a mean dropout rate of 31% for internet-based interventions . These dropout figures are often seen as major drawbacks for the implementation of internet-based interventions. However, it should be noted that a meta-analysis of 123 studies found a mean non-usage attrition in psychotherapy of 47% . Two main categories of attrition are patients not or only partially using the intervention (non-usage attrition) and patients not completing follow-up assessments (dropout attrition ). Recently, a systematic review of FTF treatments for social phobia found no link between pre-treatment variables and non-usage attrition . All aforementioned studies on attrition have focussed mainly on the general population instead of a clinical (outpatient) population. To prevent non-usage and dropout attrition, several e-mail and telephone reminders are scheduled throughout this trial. Also, we assume dropout in this outpatient population to be lower than that of internet interventions that recruit from the general population.
To our knowledge, this is the first large-scale pragmatic cost-effectiveness and effectiveness study of an internet-based intervention for anxiety in outpatient clinics. Considering the increase in the addition of e-mental health interventions to the treatment packages offered at psychiatric outpatient clinics, the findings of this study will provide valuable information on the additional value of online treatment of anxiety disorders in general and phobias in particular.
Bijl RV, Ravelli A, van Zessen G: Prevalence of psychiatric disorder in the general population: results of The Netherlands Mental Health Survey and Incidence Study (NEMESIS). Soc Psychiatr Psychiatr Epidemiol. 1998, 33: 587-595. 10.1007/s001270050098.
Konnopka A, Leichsenring F, Leibing E, König H-H: Cost-of-illness studies and cost-effectiveness analyses in anxiety disorders: a systematic review. J Affect Disord. 2009, 114: 14-31. 10.1016/j.jad.2008.07.014.
Mendlowicz MV, Stein MB: Quality of life in individuals with anxiety disorders. Am J Psychiatry. 2000, 157: 669-682. 10.1176/appi.ajp.157.5.669.
Dingemans A, van Vliet I, Couvée J, Westenberg H: Characteristics of patients with social phobia and their treatment in specialized clinics for anxiety disorders in the Netherlands. J Affect Disord. 2001, 65: 123-129. 10.1016/S0165-0327(00)00238-X.
Hirai M, Clum GA: A meta-analytic study of self-help interventions for anxiety problems. Behav Ther. 2006, 37: 99-111. 10.1016/j.beth.2005.05.002.
Cuijpers P, Donker T, van Straten A, Li J, Andersson G: Is guided self-help as effective as face-to-face psychotherapy for depression and anxiety disorders? A systematic review and meta-analysis of comparative outcome studies. Psychol Med. 2010, 40: 1943-1957. 10.1017/S0033291710000772.
Griffiths KM, Farrer L, Christensen H: The efficacy of internet interventions for depression and anxiety disorders: a review of randomised controlled trials. Med J Aust. 2010, 192: S4-S11.
Cuijpers P, Marks IM, van Straten A, Cavanagh K, Gega L, Andersson G: Computer-aided psychotherapy for anxiety disorders: a meta-analytic review. Cogn Behav Ther. 2009, 38: 66-82. 10.1080/16506070802694776.
Hedman E, Andersson G, Ljótsson B, Andersson E, Rück C, Mörtberg E, Lindefors N: Internet-based cognitive behavior therapy vs. cognitive behavioral group therapy for social anxiety disorder: a randomized controlled non-inferiority trial. PLoS One. 2011, 6: e18001-10.1371/journal.pone.0018001.
Paxling B, Almlöv J, Dahlin M, Carlbring P, Breitholtz E, Eriksson T, Andersson G: Guided internet-delivered cognitive behavior therapy for generalized anxiety disorder: a randomized controlled trial. Cogn Behav Ther. 2011, 40: 159-173. 10.1080/16506073.2011.576699.
Hedman E, Andersson G, Andersson E, Ljótsson B, Rück C, Asmundson GJG, Lindefors N: Internet-based cognitive-behavioural therapy for severe health anxiety: randomised controlled trial. Br J Psychiatry. 2011, 198: 230-236. 10.1192/bjp.bp.110.086843.
Andrews G, Cuijpers P, Craske MG, McEvoy P, Titov N: Computer therapy for the anxiety and depressive disorders is effective, acceptable and practical health care: a meta-analysis. PLoS One. 2010, 5: e13196-10.1371/journal.pone.0013196.
Bergström J, Andersson G, Karlsson A, Andréewitch S, Rück C, Carlbring P, Lindefors N: An open study of the effectiveness of Internet treatment for panic disorder delivered in a psychiatric setting. Nord J Psychiatry. 2009, 63: 44-50. 10.1080/08039480802191132.
Spence SH, Donovan CL, March S, Gamble A, Anderson RE, Prosser S, Kenardy J: A randomized controlled trial of online versus clinic-based CBT for adolescent anxiety. J Consult Clin Psychol. 2011, 79: 629-642.
Issakidis C, Andrews G: Pretreatment attrition and dropout in an outpatient clinic for anxiety disorders. Acta Psychiatr Scand. 2004, 109: 426-433. 10.1111/j.1600-0047.2004.00264.x.
World Health Organization: Composite International Diagnostic Interview (CIDI). 1990, Geneva: World Health Organization
de Neef M, Cuijpers P: Fobieën. 2007, Amsterdam: Boom Uitgeverij
van Oppen P, van Balkom AJLM, Smit JH, Schuurmans J, van Dyck R, Emmelkamp PMG: Does the therapy manual or the therapist matter most in treatment of obsessive-compulsive disorder? A randomized controlled trial of exposure with response or ritual prevention in 118 patients. J Clin Psychiatry. 2010, 71: 1158-1167. 10.4088/JCP.08m04990blu.
Almlöv J, Carlbring P, Källqvist K, Paxling B, Cuijpers P, Andersson G: Therapist Effects in Guided Internet-Delivered CBT for Anxiety Disorders. Behav Cogn Psychother. 2011, 71: 311-322.
Marks IM, Mathews AM: Brief standard self-rating for phobic patients. Behav Res Ther. 1979, 17: 263-267. 10.1016/0005-7967(79)90041-X.
Mavissakalian M: The Fear Questionnaire: a validity study. Behav Res Ther. 1986, 24: 83-85. 10.1016/0005-7967(86)90154-3.
Cox BJ, Swinson RP, Shaw BF: Value of the Fear Questionnaire in differentiating agoraphobia and social phobia. Br J Psychiatry. 1991, 159: 842-845. 10.1192/bjp.159.6.842.
Van Zuuren FJ: The fear questionnaire. Some data on validity, reliability and layout. Br J Psychiatry. 1988, 153: 659-662. 10.1192/bjp.153.5.659.
Beck AT, Steer RA: Beck Anxiety Inventory Manual. 1993, San Antonio, TXL: Psychological Corperation
de Beurs E, Wilson Ka, Chambless DL, Goldstein a J, Feske U: Convergent and divergent validity of the Beck Anxiety Inventory for patients with panic disorder and agoraphobia. Depress Anxiety. 1997, 6: 140-146. 10.1002/(SICI)1520-6394(1997)6:4<140::AID-DA2>3.0.CO;2-G.
Leyfer OT, Ruberg JL, Woodruff-Borden J: Examination of the utility of the Beck Anxiety Inventory and its factors as a screener for anxiety disorders. J Anxiety Disord. 2006, 20: 444-458. 10.1016/j.janxdis.2005.05.004.
Radloff LS: The CES-D Scale: A Self-Report Depression Scale for Research in the General Population. Appl Psychol Meas. 1977, 1: 385-401. 10.1177/014662167700100306.
Donker T, van Straten A, Marks IM, Cuijpers P: Brief self-rated screening for depression on the Internet. J Affect Disord. 2010, 122: 253-259. 10.1016/j.jad.2009.07.013.
van Agt HM, Essink-Bot ML, Krabbe PF, Bonsel GJ: Test-retest reliability of health state valuations collected with the EuroQol questionnaire. Soc Sci Med. 1994, 39: 1537-1544. 10.1016/0277-9536(94)90005-1.
Brooks R: EuroQol: the current state of play. Health Policy. 1996, 37: 53-72. 10.1016/0168-8510(96)00822-6.
Lamers LM, Stalmeier PF, McDonnell J, Krabbe PF, van Busschbach JJ: [Measuring the quality of life in economic evaluations: the Dutch EQ-5D tariff]. Ned Tijdschr Geneeskd. 2005, 149: 1574-1578.
Nguyen TD, Attkisson CC, Stegner BL: Assessment of patient satisfaction: development and refinement of a service evaluation questionnaire. Eval Program Plann. 1983, 6: 299-313. 10.1016/0149-7189(83)90010-1.
De Brey H: A cross-national validation of the Client Satisfaction Questionnaire: The Dutch experience. Eval Program Plann. 1983, 6: 395-400. 10.1016/0149-7189(83)90018-6.
Mayfield D, McLeod G, Hall P: The CAGE questionnaire: validation of a new alcoholism screening instrument. Am J Psychiatry. 1974, 131: 1121-1123.
Bradley KA, Bush KR, McDonell MB, Malone T, Fihn SD: Screening for problem drinking: Comparison of CAGE and AUDIT. J Gen Intern Med. 1998, 13: 379-388. 10.1046/j.1525-1497.1998.00118.x.
Kushner MG, Abrams K, Borchardt C: The relationship between anxiety disorders and alcohol use disorders: a review of major perspectives and findings. Clin Psychol Rev. 2000, 20: 149-171. 10.1016/S0272-7358(99)00027-6.
Hakkaart-van Roijen L: Technology. Trimbos/iMTA questionnaire for Costs associated with Psychiatric Illness (TiC-P). 2002
Hakkaart-van Roijen L, Tan S, Bouwmans CAM: Handleiding voor kostenonderzoek: Methoden en standaard kostprijzen voor economische evaluaties in de gezondheidszorg. Geactualiseerde versie 2010. 2011, Diemen: [Dutch manual for costing in economic evaluations]
Blankers M, Koeter MWJ, Schippers GM: Missing data approaches in eHealth research: simulation study and a tutorial for nonmathematically inclined researchers. J Med Internet Res. 2010, 12: e54-10.2196/jmir.1448.
Fenwick E, O’Brien BJ, Briggs A: Cost-effectiveness acceptability curves–facts, fallacies and frequently asked questions. Health Econ. 2004, 13: 405-415. 10.1002/hec.903.
Melville KM, Casey LM, Kavanagh DJ: Dropout from Internet-based treatment for psychological disorders. Br J Clin Psychol. 2010, 49: 455-471. 10.1348/014466509X472138.
Wierzbicki M, Pekarik G: A meta-analysis of psychotherapy dropout. Prof Psychol Res Pract. 1993, 24: 190-195.
Eysenbach G: The law of attrition. J Med Internet Res. 2005, 7: e11-10.2196/jmir.7.1.e11.
Eskildsen A, Hougaard E, Rosenberg NK: Pre-treatment patient variables as predictors of drop-out and treatment outcome in cognitive behavioural therapy for social phobia: A systematic review. Nord J Psychiatry. 2010, 64: 94-105. 10.3109/08039480903426929.
The pre-publication history for this paper can be accessed here:http://www.biomedcentral.com/1471-244X/12/131/prepub
This study is funded by ZonMw (project number 80-82310-97-10015).
The author(s) declare that they have no competing interests.
PC and MdN authored the intervention. PC and AvS drafted the study protocol. RK authored this article. MdN provides professional supervision for students supporting the patients. AvS, JB, PC, MdN and AB provided feedback and suggestions for this manuscript. All authors read and approved the final manuscript.
Authors’ original submitted files for images
Below are the links to the authors’ original submitted files for images.