This article has Open Peer Review reports available.
The working alliance in a randomized controlled trial comparing online with face-to-face cognitive-behavioral therapy for depression
© Preschl et al; licensee BioMed Central Ltd. 2011
Received: 2 June 2011
Accepted: 6 December 2011
Published: 6 December 2011
Although numerous efficacy studies in recent years have found internet-based interventions for depression to be effective, there has been scant consideration of therapeutic process factors in the online setting. In face-to face therapy, the quality of the working alliance explains variance in treatment outcome. However, little is yet known about the impact of the working alliance in internet-based interventions, particularly as compared with face-to-face therapy.
This study explored the working alliance between client and therapist in the middle and at the end of a cognitive-behavioral intervention for depression. The participants were randomized to an internet-based treatment group (n = 25) or face-to-face group (n = 28). Both groups received the same cognitive behavioral therapy over an 8-week timeframe. Participants completed the Beck Depression Inventory (BDI) post-treatment and the Working Alliance Inventory at mid- and post- treatment. Therapists completed the therapist version of the Working Alliance Inventory at post-treatment.
With the exception of therapists' ratings of the tasks subscale, which were significantly higher in the online group, the two groups' ratings of the working alliance did not differ significantly. Further, significant correlations were found between clients' ratings of the working alliance and therapy outcome at post-treatment in the online group and at both mid- and post-treatment in the face-to-face group. Correlation analysis revealed that the working alliance ratings did not significantly predict the BDI residual gain score in either group.
Contrary to what might have been expected, the working alliance in the online group was comparable to that in the face-to-face group. However, the results showed no significant relations between the BDI residual gain score and the working alliance ratings in either group.
In the past decade, accumulating research has demonstrated that internet-based interventions can have beneficial effects on psychological health . There is particular interest in the use of new communications technologies for the treatment of depression. Adult depression has a high prevalence in the general population; it is associated with significant impairments in health and functional status, as well as with high economic costs . Effective and cost-efficient treatment approaches that reach large populations are therefore needed.
Internet-based interventions for depression can be delivered in different forms, from self-help treatments delivered without therapist guidance to mainly text-based interventions with high therapist involvement [3, 4]. However, research indicates that the treatment outcomes of internet-based interventions are related to amount of therapist involvement. In their meta-analysis of internet-based interventions for depression, Andersson and Cuijpers  found a strong influence of therapist support on treatment outcome. Computerized interventions with therapist support showed a mean between-group effect size of d = .61, which is comparable with face-to-face treatment for depression, whereas interventions with little or no therapist contact had a significantly smaller treatment effect size of d = 0.25. This pattern of results replicates the findings of a previously published meta-analysis . Moreover, studies on entirely self-guided programs have shown not only reduced treatment effects, but also substantial attrition rates of up to 41% [7–11]. Analyses have also revealed a significant correlation between the amount of therapist time in minutes per participant and the between-group effect sizes of internet-based interventions . Based on the findings of their Swedish studies, Andersson and colleagues have suggested that it can be sufficient for the therapist to spend about 100 minutes per patient over a 10-week program giving comments on patients' homework and providing feedback . The latest studies indicate that increasing therapist contact time beyond a certain threshold may not facilitate further treatment gains . In his review, Titov  concluded that highly standardized internet-based interventions with low-intensity therapist support can achieve excellent clinical outcomes. Overall, these studies on internet-based interventions for depression thus suggest that a minimum of human therapeutic contact is needed to reduce attrition rates and to alleviate symptoms of depression.
Despite the growing interest in the influence of therapist support (e.g., therapist time spent per patient) in internet-based interventions, there has been little research on therapeutic process factors and predictors of treatment outcome in online settings. It therefore remains unclear whether the factors and therapeutic processes that are responsible for symptom reduction in face-to-face therapy operate in the same way in online therapeutic settings. We expect more factors to be involved than the mere amount of time that the therapist spends giving feedback to patients.
One of the therapeutic process factors associated with treatment outcome is the working alliance between therapist and patient. Numerous empirical studies have demonstrated the importance of the working alliance--that is, the relationship or collaboration between therapist and patient--for therapeutic outcomes in conventional treatment settings . It has also been noted that clients' assessments of the therapeutic alliance are more predictive than are therapists' or observers' ratings. Krupnick and colleagues  demonstrated that the therapeutic alliance significantly influenced symptoms of depression as outcome measures. They found significant predictive effects for patient ratings, but not for therapist ratings. In view of these findings, the therapeutic alliance has traditionally been seen as a key element adding to the treatment success of face-to-face psychotherapy . Against this background, the fact that internet-based interventions involve less therapeutic contact--not only in terms of time, but also through their restriction to purely text-based and computer-mediated communication--may be a cause for concern. However, there has to date been little empirical research on the impact of the working alliance in online settings as compared with face-to face therapeutic settings.
Cook and colleagues  were among the first to evaluate the online working alliance. They compared results from an online sample (N = 15) with normative data from a representative sample in face-to-face therapy (N = 25). The online group showed higher means on the composite score and the goals subscale of the Working Alliance Inventory . The goals subscale reflects the agreement between therapist and client on what is to be achieved in the therapy. However, these preliminary results should be interpreted carefully: the sample size was small and patients were not randomly allocated to the conditions. In the same vein, Reynolds and colleagues  reported preliminary results (N = 16 therapists, N = 17 clients) on the therapeutic alliance as assessed by the Agnew Relationship Measure  in an online setting, which they compared with existing data from a face-to-face group. The clients in the online study presented with depression, stress, anxiety, or childhood abuse. Like Cook and Doyle , the authors reported similar therapeutic alliance ratings for both conditions, with the online groups showing higher means on the confidence subscale. In a randomized controlled study, Knaevelsrud and Maercker  compared the therapeutic alliance in a total of 96 PTSD patients assigned at random to an internet-based treatment or a waiting list control group. The treatment involved 10 writing assignments, on which therapists gave detailed feedback. The authors reported relatively low drop-out rates (16%) and relatively high scores for the therapeutic alliance (Working Alliance Inventory, patient ratings: M = 6.3, therapist ratings: M = 5.8). These results were again comparable with face-to-face therapy, indicating that a strong therapeutic relationship could be established even in an online setting with no direct personal contact. Further, the composite scores of both the therapists' and the clients' ratings of the therapeutic alliance late in treatment were moderately but not significantly correlated with treatment outcome .
Beside these studies of internet-supported therapeutic interventions with therapist support based on computer-mediated communication without the use of a specific self-help program, Klein and colleagues  and Kiropoulos and colleagues  have reported positive results on the therapeutic alliance in therapist-assisted internet programs. In a randomized controlled trial, Kiropoulos and colleagues compared a 12-week internet-based cognitive behavioral therapy (CBT) for panic disorder and agoraphobia provided via the online program Panic Online with face-to-face CBT (N = 86). The program combines standardized instructions and information with e-mail contact with a therapist. Patients in the internet-based group had significantly less therapist contact than those in the face-to-face group. Nevertheless, both groups rated the intervention as similarly satisfying (Treatment Satisfaction Questionnaire-Modified, TSQ; ) and credible (Treatment Credibility Scale, TCS-M; ). However, participants in the face-to-face group enjoyed communication with their therapist more than did those in the internet-based group, and their therapists reported higher compliance to treatment (Therapist Alliance Questionnaire, TAQ; modified version of the Helping Alliance Questionnaire, HAQ; ). In an open trial, Klein and colleagues investigated a therapist-assisted internet CBT for PTSD provided via the interactive CBT program PTSD Online. These authors reported 194.5 min of therapist time spent across the 10-week intervention. Nevertheless, the participants (N = 22) gave high therapeutic alliance ratings (87.5%) on the Therapeutic Alliance Questionnaire, TAQ.
Based on these findings, we conducted a randomized controlled study investigating the therapeutic alliance in online (computer-mediated communication without the use of a specific self-help program) and face-to-face CBT treatment settings for depression. To our knowledge, this is the first randomized controlled trial for depression to compare the therapeutic alliance between patient and therapist in the two settings in an experimental design. To maximize comparability, all patients received the same treatment manual over the same timeframe. The treatment manual was based on a German CBT treatment manual for depression  with an added life-review intervention module . The first objective of this study was to examine whether the therapeutic alliance was comparable in the online group and the face-to-face group. Second, we investigated whether the therapeutic alliance predicted depression as outcome in the online and/or face-to-face condition. Third, we examined the therapeutic alliance from the therapists' perspective as a predictor of treatment outcome in both conditions.
A randomized controlled trial comparing an internet-based with a face-to face CBT intervention for depression was conducted at the University of Zurich . Both treatment groups received the same cognitive behavioral therapy over an 8-week timeframe, at the end of which participants completed the Beck Depression Inventory and the Working Alliance Inventory. Assessments were conducted at baseline and post-treatment.
Age, M (SD) (in years)
F(.91) = .25, ns
χ 2 (1), p .01
χ 2 (3) = .58, ns
χ 2 (2) = .21, ns
Vocational-track sec. school
track sec. school
χ 2 (1) = .14, ns
Previous face-to-face psychotherapy
χ 2 (1) = .54, ns
Visits to medical doctors in last 12 months, M (SD)
F(.002) = .30, ns
Where did you hear about the study?
Caregiver (psychologist, primary care physician)
The average BDI baseline score was M = 22.5 (S.D = 6) for the online group and M = 23.6 (SD = 7.9) for the face-to-face group. The BDI baseline scores of the two groups did not differ significantly, t(50) = -0.567; p > .05. Information on post-treatment BDI scores and associated test statistics are reported elsewhere . Preliminary results for the primary outcome (depression) revealed no differences between the online and the face-to-face condition.
A web page was created for the study, presenting general information about CBT and its effects in treating depression, and giving an outline of the study. Participants indicated their interest in the study by contacting the intake coordinator via the e-mail address indicated on the website (for further information, see ). The intake coordinator sent a reply e-mail with a patient information sheet and the inclusion and exclusion criteria. Participants who indicated that they met and were comfortable with the requirements entered an online screening procedure, data from which were later used as pretest measures. After confidentiality issues had been addressed, eligible applicants returned a signed informed consent form--which informed them about potential risks and benefits of study participation--by fax or post. The treatment commenced 3 to 4 days after the patients had returned their informed consent form. The intake coordinator told participants that they could withdraw from the study at any time. Further, participants received 24-hour contact numbers for emergency situations or crises. They were also encouraged to call or e-mail the therapist or intake coordinator at any time during their participation in the study in case of distress or crisis. Participants were randomly assigned to one of the two conditions as they were included in the study. Applicants excluded from the study were informed about other available forms of treatment.
All measures were self-reports administered via an online diagnostic assessment. Fidy  found no significant differences between paper-and-pencil and online administration of the German versions of the BDI and the Beck Suicide Ideation Scale (BSIS), which were also used in the present study. Outcome measures were administered at baseline and post-treatment. The working alliance (patients' ratings) was also assessed at mid-treatment after 4 weeks.
Depression was assessed with the German version  of the Beck Depression Inventory-II (BDI; ), which comprises 21 multiple-choice items assessing specific symptoms of depression. The BDI has shown high reliability across diverse populations. The internal consistency in the current sample was α = .91.
The quality of the working alliance was assessed by the German version  of the Working Alliance Inventory (WAI ). Respondents were asked to rate each statement on a 7-point Likert scale ranging from 1 (never) to 7 (always). In this study, both the client and the therapist version of the 12-item WAI-S  were administered at post-treatment. The WAI covers three aspects of the working alliance: bond (degree of mutual trust, acceptance, and confidence between client and therapist; client: α = .84; therapist: α = .84), tasks (agreement on therapeutic tasks; client: α = .88; therapist: α = .77), and goals (agreement on therapeutic goals; client: α = .87; therapist: α = .84). The internal consistencies for the composite scores in our sample were high (client: α = .94, therapist: α = .93).
Applicants were excluded if they met any of the following criteria: currently receiving treatment elsewhere, substance abuse or dependence, on antidepressant medication for less than 4 weeks, age below 18 years, not fluent in German. Further exclusion criteria were high risk of suicide, psychotic symptoms, post-traumatic stress disorder, anxiety, phobia, bipolar disorder, and low depression symptom severity.
Symptom severity was assessed by the German version of the Beck Depression Inventory . Patients were excluded if their BDI score was below 12.
Suicide ideation was assessed with the Beck Suicide Ideation Scale , a 21-item inventory developed to measure the intensity and chronicity of suicide ideation in adults. The first 5 items make up a brief subscale measuring the presence of suicidal thoughts, either recently (in the last 6 months) or ever in one's life.
Risk of psychosis
Risk of psychosis was measured using the Dutch Screening Device for Psychotic Disorder , a seven-item inventory that is a good predictor of psychotic episodes. Because no data are yet available from a German norm group, the Dutch norm data were used.
Anxiety was assessed using the Anxiety subscale of the German version of the Symptom Checklist by Derogatis . This 10-item subscale covers various symptoms of anxiety, including cognitive and somatic correlates of anxiety.
The German version of the Symptom Checklist by Derogatis  was also used to measure phobia. The Phobia subscale contains seven items assessing severity of phobic symptoms.
The Post-traumatic Stress Scale 10 , a short screening instrument tapping DSM-III symptoms of post-traumatic stress disorder including symptoms of hyperarousal, was used to measure symptoms of post-traumatic stress.
Six female psychologists and psychotherapists participated in this study. All psychologists were trained in psychotherapy and CBT for depression specifically for this study. The therapists were given special training in therapeutic writing for the online treatment and received regular supervision (face-to-face and online), with therapists in both groups receiving the same amount of supervision. All but one of the therapists were involved in both treatment conditions. Therapists were not randomly allocated to patients.
Both treatment conditions were of equal length (8 weeks) and followed an evidence-based short-term CBT treatment manual for depression . This German manual is based on the cognitive theory of depression by Beck and colleagues . The program involved the following modules: introduction, behavioral analysis, planning of activities, daily structure, cognitive restructuring, promotion of social competence, and relapse prevention. A life-review module was added to the standard CBT treatment manual . The aims of life review are to revisit and reattribute past experiences and to activate positive memories and individual resources in order to achieve a balance between positive and negative memories. In the present context, this method was essentially used to activate individual resources (e.g., to identify coping strategies that had helped participants to cope with unresolved past experiences or depressive episodes).
Patients in both groups were given the same psychoeducation and received the treatment modules in the same chronological order. Psychoeducation played an important role in the therapeutic approach. At the beginning of each new treatment module, the patient was informed about the meaning and background of each treatment technique, the significance of the homework set, and the meaning of certain symptoms or reactions.
Patients in the face-to-face condition attended one-hour weekly treatment sessions for 8 weeks with their allocated psychologist in the Department of Psychopathology and Clinical Intervention at the University of Zurich. They were also given weekly homework assignments (e.g., daily structure diaries, negative thoughts log).
For the online condition, the CBT treatment manual for depression  was adapted for use as an internet-based intervention, based on the principles applied in a number of previous studies [3, 41–43]. To this end, a highly structured treatment manual was developed. The treatment consisted of structured writing and homework assignments (e.g., behavioral analysis of depressive symptoms, activity diaries, cognitive restructuring worksheets) based on the CBT approach and on the written disclosure procedure developed by Pennebaker and colleagues [3, 44]. Each writing assignment lasted 45 minutes and took place at regular, scheduled times. Within one working day, the therapist provided individual written feedback along with instructions on the next writing assignment. Model responses for the therapists were available, but they also had the option to provide their own commentary or supportive feedback on their patients' texts. Patients were given two writing assignments in each week of the 8-week treatment period. The therapist time involved in responding to texts ranged from 20 to 50 minutes per text, depending on the therapist's experience with internet-based therapies.
SPSS 17.0 for Windows was used for all analyses. In preliminary analyses, we compared the online and face-to-face group at baseline using t and chi-square tests. T tests were then used to compare the therapeutic alliance in the two intervention groups. In addition, bivariate and partial correlations (Pearson) were calculated to examine the relationship between the working alliance and therapy outcome.
Treatment outcome was assessed as (a) the BDI score at post-treatment (BDI-post) and (b) the BDI residual gain score (the difference between the z-transformed BDI scores at post-treatment and baseline multiplied by the correlation between the two scores ). The therapeutic alliance was assessed in terms of the composite score on the WAI and the scores on the three subscales (bond, tasks, goals) of the clients' (WAI-C) and the therapists' (WAI-T) ratings.
To quantify the magnitude of differences between the two groups (online versus face-to-face), we used Cohen's d as a measure of effect size. Cohen  distinguished between small (d = .20), medium (d = .50) and large (d = .80) effect sizes.
Since we did at no time obtain data concerning therapeutic alliance from drop outs we could not conduct intention-to-treat analysis.
Quality of the working alliance in the treatment groups
Working alliance in the two intervention groups: means, standard deviations, t test comparisons, and effect sizes.
Online (N= 25)
Face-to-Face (N= 28)
Working alliance and therapy outcome
Correlations of the WAI scores with the BDI score at post-treatment and the BDI residual gain score in the online and face-to-face groups.
BDI residual gain score
The aim of this study was to investigate the quality of the therapeutic alliance between patient and therapist in an online and face-to-face CBT for depression. To our knowledge, this was the first randomized controlled trial in this context. First, we examined whether the therapeutic alliance was comparable in both groups. Our results showed that the online and the face-to-face group differed significantly in only one subscale: therapists' ratings of the tasks subscale were significantly higher in the online group. This finding is in line with previous studies reporting that a strong working alliance, comparable to that formed in face-to-face settings, can also be established in online settings. The WAI mean scores in our study ranged from 5.39 to 6.22 (of a maximum of 7). These findings are comparable to data presented by Knaevelsrud and Maercker , who reported mean scores ranging from 5.6 to 6.4 in Table 3 of their article. Furthermore, authors using other scales or other versions of the WAI have also provided evidence for a comparably strong working alliance in online settings as in face-to-face therapy. Cook and Doyle , for example, reported results for an online sample to be comparable with normative data from a representative sample in face-to-face therapy. Most of the participants in their sample presented with relationship issues, depression, anxiety, or grief. However, because of the small sample size and the non-randomized allocation of patients, these preliminary results should be interpreted with caution. In the same vein, Reynolds and colleagues  reported ratings of the therapeutic alliance in an online setting to be similar to existing data from a face-to-face group. The participants in their study presented with depression, stress, anxiety, or childhood abuse. We were able to replicate the findings from both of these studies in a randomized controlled setting with a sample of depressive adults. The higher therapist ratings of the tasks subscale in the online group in our study may be attributable to the clear presentation and structuring of the tasks in the online mode, and to the opportunity to focus carefully on elaborated tasks. This fact may have positively influenced the agreement between clients and therapists on the therapeutic tasks.
Further, the drop-out rate in our study was relatively low. Seven (22%) participants in the online group and two (7%) participants in the face-to-face group discontinued the treatment. In general, drop-out rates in internet-based interventions are known to be problematic . However, the drop-out rates reported for studies involving internet-based interventions for depression over the last five years differ widely. For instance, Titov and colleagues  reported that 11% of participants in a clinician-assisted internet-delivered CBT for depression did not complete post treatment questionnaires. In contrast, Spek and colleagues  reported a drop-out rate of 66% for the intervention group of an internet-based CBT intervention study for subthreshold depression (individuals who did not complete post-test, did not start the intervention, or withdrew). In our sample, the attrition rates in the online group (22%) versus the face-to face group (7%) differed significantly, χ 2 (1) = 4.737, p .05. This may indicate that the more anonymous online therapeutic relationship is less stable than the face-to-face relationship. It is easier for patients in online treatment settings to stop therapeutic communication by simply "disappearing." A study of online romantic relationships revealed that avoidance behavior and discontinuity are more likely in online relationships than in face-to face relationships .
Furthermore, we were interested in whether the therapeutic alliance predicted depression as outcome in the online or the face-to-face group. In both groups, only the clients' ratings of the working alliance were associated with depression at post-treatment (specifically, the composite score and tasks subscale in the face-to-face group at mid-treatment and, at post-treatment, the tasks subscale in the online group and the composite score and the tasks and goals subscales in the face-to-face group). It is worth noting that the correlations reported here are statistically significant, but only moderately high, ranging from r = -.42 to r = .52. These results are in line with findings on face-to-face psychotherapy. In a review article, Martin and colleagues  reported a moderate but consistent relationship between the therapeutic alliance and outcomes of face-to-face psychotherapy. However, in the online group, only the working alliance at post-treatment was significantly associated with depression at post-treatment. This result replicates the findings of Knaevelsrud and Maercker , who found no significant relationship between the working alliance at mid-treatment and PTSD change scores. Further, our data showed no significant relations between the BDI residual gain score and the working alliance in either group at mid- or post-treatment. Knaevelsrud and Maercker  discussed the importance of investigating the working alliance at several stages of the therapeutic process to elucidate the relationship between working alliance and outcome. The authors suggested that the working alliance might be more an "additional indirect measure of outcome" than a predictor of treatment outcome.
The limitations of our study include the assessment of the working alliance and depression. As participants were first contacted online and later allocated at random to the online or the face-to-face group, all measures were administered as self-rated questionnaires in an online setting. Although this procedure has proven valid and reliable in various previous studies [18, 20, 22–25], a structured clinical interview would have allowed a better quality of diagnosis of depression and the therapeutic relationship.
A further limitation is that we are unable to present follow-up data at the present time. Collection of follow-up data (after 3, 6 and 12 months) is still ongoing. Therefore, it remains an open question whether the working alliance at post-treatment predicts outcomes at follow-up.
Furthermore, the sample used in this study was small, relatively well educated and more than half of the participants already had experience of psychotherapy. Future studies should enroll larger and more heterogeneous samples. Another limitation of the study is the generalizability of our results. Due to our strict exclusion criteria regarding co-morbidity, suicide ideation, and psychosis, a number of applicants were excluded from the study. Our findings may therefore not be comparable with more naturalistic designs. Further research is needed to focus specifically on patients with co-morbidities.
In conclusion, an internet-based intervention has the potential to facilitate a working alliance that is comparable to that formed in face-to-face settings, though not as influential with respect to symptom reduction. This is the first randomized controlled trial to evaluate the therapeutic alliance between patient and therapist in online and face-to-face treatment settings for depression in an experimental design. Our study contributes to a better understanding of the working alliance in internet-supported therapeutic interventions, replicating previous findings [18, 20, 22–25] showing that a strong working alliance can be established in an online setting, comparable to that established in face-to-face settings.
The authors would like to thank Andrea Horn, Jenni Keel, Luigina Di Lorenzo, and Regula Usteri, who served as therapists in the study. This study was co-funded by the Werner Selo Foundation.
- Barak A, Hen L, Boniel-Nissim M: A comprehensive review and a meta-analysis of the effectiveness of internet-based psychotherapeutic interventions. J Tech Human Services. 2008, 26: 109-160. 10.1080/15228830802094429.View ArticleGoogle Scholar
- Andrews G, Henderson S, Hall W: Prevalence, comorbidity, disability and service utilisation. Overview of the Australian National Mental Health Survey. Br J Psychiatry. 2001, 178: 145-153. 10.1192/bjp.178.2.145.View ArticlePubMedGoogle Scholar
- Ruwaard J, Schrieken B, Schrijver M, Broeksteeg J, Dekker J, Vermeulen H, Lange A: Standardized web-based cognitive behavioural therapy of mild to moderate depression: a randomized controlled trial with a long-term follow-up. Cogn Behav Ther. 2009, 38: 206-221. 10.1080/16506070802408086.View ArticlePubMedGoogle Scholar
- Andersson G: Internet-based cognitive-behavioral self-help for depression. Expert Rev Neurother. 2006, 6: 1637-1642. 10.1586/14737126.96.36.1997.View ArticlePubMedGoogle Scholar
- Andersson G, Cuijpers P: Internet-based and other computerized psychological treatments for adult depression: a meta-analysis. Cogn Behav Ther. 2009, 38: 196-205. 10.1080/16506070903318960.View ArticlePubMedGoogle Scholar
- Spek V, Cuijpers P, Nyklicek I, Riper H, Keyzer J, Pop V: Internet-based cognitive behaviour therapy for symptoms of depression and anxiety: a meta-analysis. Psychol Med. 2007, 37: 319-328. 10.1017/S0033291706008944.View ArticlePubMedGoogle Scholar
- Kaltenthaler E, Parry G, Beverley C, Ferriter M: Computerised cognitive-behavioural therapy for depression: systematic review. Br J Psychiatry. 2008, 193: 181-184. 10.1192/bjp.bp.106.025981.View ArticlePubMedGoogle Scholar
- Clarke G, Eubanks D, Reid E, Kelleter C, O'Connor E, DeBar LL, Lynch F, Nunley S, Gullion C: Overcoming Depression on the Internet (ODIN) (2): a randomized trial of a self-help depression skills program with reminders. J Med Internet Res. 2005, 7: e16-10.2196/jmir.7.2.e16.View ArticlePubMedPubMed CentralGoogle Scholar
- Clarke G, Reid E, Eubanks D, O'Connor E, DeBar LL, Kelleter C, Lynch F, Nunley S: Overcoming Depression on the Internet (ODIN): a randomized controlled trial of an Internet depression skills intervention program. J Med Internet Res. 2002, 4: e14-10.2196/jmir.4.3.e14.View ArticlePubMedPubMed CentralGoogle Scholar
- Christensen H, Griffiths KM, Mackinnon AJ, Brittliffe K: Online randomized controlled trial of brief and full cognitive behaviour therapy for depression. Psychol Med. 2006, 36: 1737-1746. 10.1017/S0033291706008695.View ArticlePubMedGoogle Scholar
- Christensen H, Griffiths K, Groves C, Korten A: Free range users and one hit wonders: community users of an Internet-based cognitive behaviour therapy program. Aust N Z J Psychiatry. 2006, 40: 59-62. 10.1080/j.1440-1614.2006.01743.x.View ArticlePubMedGoogle Scholar
- Palmqvist B, Carlbring P, Andersson G: Internet-delivered treatments with or without therapist input: does the therapist factor have implications for efficacy and cost?. Expert Rev Pharmacoecon Outcomes Res. 2007, 7: 291-297. 10.1586/14737188.8.131.521.View ArticlePubMedGoogle Scholar
- Andersson G, Carlbring P, Berger T, Almlov J, Cuijpers P: What makes internet therapy work?. Cogn Behav Ther. 2009, 38: 55-60. 10.1080/16506070902916400.View ArticlePubMedGoogle Scholar
- Vernmark K, Lenndin J, Bjärehed J, Carlsson M, Karlsson J, Öberg J, Carlbring P, Eriksson T, Andersson G: Internet administered guided self-help versus individualized e-mail therapy: a randomized trial of two versions of CBT for major depression. Behav Res Ther. 2010, 48: 368-376. 10.1016/j.brat.2010.01.005.View ArticlePubMedGoogle Scholar
- Titov N: Internet-delivered psychotherapy for depression in adults. Curr Opin Psychiatry. 2011, 24: 18-23. 10.1097/YCO.0b013e32833ed18f.View ArticlePubMedGoogle Scholar
- Horvath AO, Symonds BD: Relation between working alliance and outcome in psychotherapy: a metaanalysis. J Couns Psychol. 1991, 38: 139-149.View ArticleGoogle Scholar
- Krupnick JL, Sotsky SM, Simmens S, Moyer J, Elkin I, Watkins J, Pilkonis PA: The role of the therapeutic alliance in psychotherapy and pharmacotherapy outcome: findings in the national institute of mental health treatment of depression collaborative research program. J Consult Clin Psych. 1996, 64: 532-539.View ArticleGoogle Scholar
- Cook JE, Doyle C: Working alliance in online therapy as compared to face-to-face therapy: preliminary results. Cyberpsychol Behav. 2002, 5: 95-105. 10.1089/109493102753770480.View ArticlePubMedGoogle Scholar
- Horvath AO, Greenberg LS: Development and validation of the Working Alliance Inventory. J Couns Psychol. 1989, 36: 223-233.View ArticleGoogle Scholar
- Reynolds DJ, Stiles WB, Grohol JM: An investigation of session impact and alliance in Internet based psychotherapy: preliminary results. Counsel Psychother Res. 2006, 6: 164-168. 10.1080/14733140600853617.View ArticleGoogle Scholar
- Agnew-Davies R, Stiles WB, Hardy GE, Barkham M, Shapiro DA: Alliance structure assessed by the Agnew Relationship Measure (ARM). Br J Clin Psychol. 1998, 37 (pt 2): 155-172.View ArticlePubMedGoogle Scholar
- Knaevelsrud C, Maercker A: Internet-based treatment for PTSD reduces distress and facilitates the development of a strong therapeutic alliance: a randomized controlled clinical trial. BMC Psychiatry. 2007, 7: 13-10.1186/1471-244X-7-13.View ArticlePubMedPubMed CentralGoogle Scholar
- Knaevelsrud C, Maercker A: Does the quality of the working alliance predict treatment outcome in online psychotherapy for traumatized patients?. J Med Internet Res. 2006, 8: e31-10.2196/jmir.8.4.e31.View ArticlePubMedPubMed CentralGoogle Scholar
- Klein B, Mitchell J, Abbott J, Shandley K, Austin D, Gilson K, Kiropoulos L, Cannard G, Redman T: A therapist-assisted cognitive behavior therapy internet intervention for posttraumatic stress disorder: pre-, post- and 3-month follow-up results from an open trial. J Anxiety Disord. 2010, 24: 635-644. 10.1016/j.janxdis.2010.04.005.View ArticlePubMedGoogle Scholar
- Kiropoulos LA, Klein B, Austin DW, Gilson K, Pier C, Mitchell J, Ciechomski L: Is internet-based CBT for panic disorder and agoraphobia as effective as face-to-face CBT?. J Anxiety Disord. 2008, 22: 1273-1284. 10.1016/j.janxdis.2008.01.008.View ArticlePubMedGoogle Scholar
- Cox BJ, Fergus KD, Swinson RP: Patient satisfaction with behavioural treatments for panic disorder with agoraphobia. J Anxiety Disord. 1994, 8: 193-206. 10.1016/0887-6185(94)90001-9.View ArticleGoogle Scholar
- Borkovec TD, Nau SD: Credibility of analogue therapy rationales. Journal of Behav Ther Exp Psy. 1972, 3: 257-260. 10.1016/0005-7916(72)90045-6.View ArticleGoogle Scholar
- Luborsky L: The helping alliance questionnaire: patient version. Assessing outcome in clinical practice. Edited by: Ogles, B, Lambert M, Masters K. 1985, Sydney, Allyn and Bacon, 150-151.Google Scholar
- Hautzinger M: Cognitive Behaviour Therapy of Depression. 2003, Weinheim, Beltz, 6Google Scholar
- Maercker A: Lebensrückblicksinterventionen als wirksame Therapietechniken [Life-review interventions as effective psychotherapeutic techniques]. Ärztliche Psychotherapie. 2009, 4: 10-16.Google Scholar
- Wagner B, Maercker A: Online vs. face-to-face cognitive-behavioural therapy for depression: a randomized controlled trial. Manuscript in preparation. 2011Google Scholar
- Beck AT, Steer RA, Brown GK: Manual for the Beck Depression Inventory-II. 1996, San Antonio, Psychological CorporationGoogle Scholar
- Fidy R: Psychologische Suizidalitäts-Diagnostik im Internet [Internet diagnosis of suicidal ideation]. Unpublished Lizenziat thesis. 2008Google Scholar
- Hautzinger M, Keller F, Kühner C: Das Beck Depressionsinventar II. Deutsche Bearbeitung und Handbuch zum BDI II [The Beck Depression Inventory II. German version and manual for the BDI II]. 2006, Frankfurt am Main, Harcourt Test ServicesGoogle Scholar
- Horvath AO, Greenberg LS, eds: The Development of the Working Alliance Inventory. 1986, New York, Guilford PressGoogle Scholar
- Tracey TJ, Kokotovic AM: Factor structure of the Working Alliance Inventory. Psychol Assessment. 1989, 1: 207-210.View ArticleGoogle Scholar
- Beck AT, Brown GK, Steer RA: Psychometric characteristics of the Scale for Suicide Ideation with psychiatric outpatients. Behaviour Research and Therapy. 1997, 35: 1039-1046. 10.1016/S0005-7967(97)00073-9.View ArticlePubMedGoogle Scholar
- Lange A, Schrieken B, Blankers M, Van de Ven JP, Slot M: Constructie en validatie van de Gewaarwordingenlijst (GL) een hulpmiddel bij het signaleren van een verhoogde kans op psychosen [Construction and Validation of the Screening Device for Psychotic Disorder]. Directieve Therapie. 2000, 20: 162-173.Google Scholar
- Franke G: SCL-90-R. Die Symptom-Checkliste von Derogatis. [SCL-90-R. The Symptom-Checklist from Derogatis]. 1995, Göttingen, Beltz Test GmbHGoogle Scholar
- Maercker A: Posttraumatic Stress Scale-10 (PTSS-10) - German version adapted after Schüffel & Schade (unpublished). 1998, DresdenGoogle Scholar
- Lange A, Rietdijk D, Hudcovicova M, van de Ven JP, Schrieken S, Emmelkamp PMG: Interapy: a controlled randomized trial of the standardized treatment of posttraumatic stress through the internet. J Consult Clin Psychol. 2003, 71: 901-909.View ArticlePubMedGoogle Scholar
- Wagner B, Knaevelsrud C, Maercker A: Internet-based cognitive-behavioral therapy for complicated grief: a randomized controlled trial. Death Stud. 2006, 30: 429-453. 10.1080/07481180600614385.View ArticlePubMedGoogle Scholar
- Ruwaard J, Lange A, Bouwman M, Broeksteeg J, Schrieken B: E-mailed standardized cognitive behavioural treatment of work-related stress: a randomized controlled trial. Cogn Behav Ther. 2007, 36: 179-192. 10.1080/16506070701381863.View ArticlePubMedGoogle Scholar
- Berry DS, Pennebaker JW: Nonverbal and verbal emotional expression and health. Psychother Psychosom. 1993, 59: 11-19. 10.1159/000288640.View ArticlePubMedGoogle Scholar
- Heinecke K, Weise C, Rief W: Chronischer Tinnitus: Für wen ist eine ambulante Kurzzeitpsychotherapie indiziert? [Chronic tinnitus: which kind of patients benefit from an outpatient psychotherapy?]. Psychother Psychosom Med Psychol. 2010, 60: 271-278. 10.1055/s-0029-1225334.View ArticlePubMedGoogle Scholar
- Cohen J: A power primer. Psychol Bull. 1992, 112: 155-159.View ArticlePubMedGoogle Scholar
- Titov N, Andrews G, Davies M, McIntyre K, Robinson E, Solley K: Internet treatment for depression: a randomized controlled trial comparing clinician vs. technician assistance. PLoS One. 2010, 5: e10939-10.1371/journal.pone.0010939.View ArticlePubMedPubMed CentralGoogle Scholar
- Spek V, Nyklicek I, Smits N, Cuijpers P, Riper H, Keyzer J, Pop V: Internet-based cognitive behavioural therapy for subthreshold depression in people over 50 years old: a randomized controlled clinical trial. Psychol Med. 2007, 37: 1797-1806.PubMedGoogle Scholar
- Merkle E, Richardson R: Digital dating and virtual relating: Conceptualizing computer mediated romantic relationships. Family Relations. 2000, 49: 187-192. 10.1111/j.1741-3729.2000.00187.x.View ArticleGoogle Scholar
- Martin DJ, Garske JP, Davis MK: Relation of the therapeutic alliance with outcome and other variables: a meta-analytic review. J Consult Clin Psych. 2000, 68: 438-450.View ArticleGoogle Scholar
- The pre-publication history for this paper can be accessed here:http://www.biomedcentral.com/1471-244X/11/189/prepub
This article is published under license to BioMed Central Ltd. This is an Open Access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.