Internet-based treatment for Romanian adults with panic disorder: protocol of a randomized controlled trial comparing a Skype-guided with an unguided self-help intervention (the PAXPD study)
© Ciuca et al. 2016
Received: 22 May 2015
Accepted: 6 January 2016
Published: 14 January 2016
Efficacy of self-help internet-based cognitive behavior therapy (ICBT) for anxiety disorders has been confirmed in several randomized controlled trials. However, the amount and type of therapist guidance needed in ICBT are still under debate. Previous studies have shown divergent results regarding the role of therapist guidance and its impact on treatment outcome. This issue is central to the development of ICBT programs and needs to be addressed directly. The present study aims to compare the benefits of regular therapist guidance via online real-time audio-video communication (i.e. Skype) to no therapist guidance during a 12-week Romanian self-help ICBT program for Panic Disorder. Both treatments are compared to a waiting-list control group.
A parallel group randomized controlled trial is proposed. The participants, 192 Romanian adults fulfilling diagnostic criteria for panic disorder according to a diagnostic interview, conducted via secured Skype or telephone, are randomly assigned to one of the three conditions: independent use of the internet-based self-help program PAXonline, the same self-help treatment with regular therapist support via secured Skype, and waiting-list control group. The primary outcomes are severity of self-report panic symptoms (PDSS-SR) and diagnostic status (assessors are blind to group assignment), at the end of the intervention (12 weeks) and at follow-up (months 3 and 6). The secondary measures address symptoms of comorbid anxiety disorders, depression, quality of life, adherence and satisfaction with ICBT. Additional measures of socio-demographic characteristics, personality traits, treatment expectancies, catastrophic cognitions, body vigilance and working alliance are considered as potential moderators and/ or mediators of treatment outcome.
To the best of our knowledge, the present study is the first effort to investigate the efficacy of a self-help internet-based intervention with therapist guidance via real-time video communication. A direct comparison between therapist guided versus unguided self-directed intervention for panic disorder will also be addressed for the first time. Findings from this study will inform researchers and practitioners about the added value of online video-therapy guidance sessions and the type of patients who may benefit the most from guided and unguided ICBT for Panic disorder.
ACTRN12614000547640 (Australian New Zealand Clinical Trials Registry). Registered 22/05/2014.
Hundreds of millions of people are affected by mental health disorders worldwide and failing to adequately address this issue brings significant suffering, disability and economic loss. The lifetime prevalence for mental health disorders in general is above 46 %, with the highest prevalence recorded for anxiety disorders, given that almost 1 out of 3 people (28.8 %) suffer from an anxiety disorder during their lifetime . Every year over 38.2 % (164.7 million people) of the total EU population suffer from at least one mental disorder and the resulting economic costs rise at 3–4 % of EU Gross Domestic Product (GDP) .
One third of medical consultations made by general practitioners are in fact provided for mental health problems [3–5]. A report released by WHO (World Health Organization) and WONCA (World Organization of Family Doctors) in 2008 stated that “up to 60 % of people attending primary care clinics have a diagnosable mental disorder” (page VII) . There is an alarming underdetection of mental health problems – for example, detection rates of anxiety and mood disorders in general practice reach only 15 and 36 %, respectively [7, 8]. Moreover, collaboration between GPs and mental health services is either poor or non-existent.
Despite the potential to successfully treat mental disorders, only a small minority receive even the most basic treatment. Studies have shown that 60 % of those in need receive no treatment at all, and only 32.7 % of those who receive treatment get it at minimal quality standards . Furthermore, there are numerous individual and systemic barriers to accessing mental health services , thus the treatment is provided only with a significant delay from the onset of the disorder (after 9–23 years for anxiety disorders and 6–8 years for mood disorders) . In some countries, the situation is even worse. In Romania, where the current study is conducted, 76.4 % of the individuals with mental health disorders do not have access to any form of treatment; only about 11.5 % receive psychological or psychiatric help .
Computer and Internet-based cognitive behavioral treatments (ICBT) represent recent attempts to innovate the psychotherapeutic process, matching both the existent demands of the market, and the new attitudes of those who seek help in the digital era. Research on internet-based treatments has grown rapidly in the past decade, gathering more than 100 controlled studies that tested various intervention programs for anxiety disorders, mood disorders, or other psychiatric conditions [13–15]. Overall, the outcomes of these studies have been positive and very convincing. Several studies reported no significant differences in treatment outcomes when comparing internet-based interventions to traditional face-to-face therapy [16–21]. For a review see Andersson et al. . Because half of all lifetime cases of mental disorders start by age 14 years and the median age of onset for anxiety is 11 years , it should be mentioned that ICBT shows promising results not just for adults, but also for treating anxiety and depression in youth [23, 24]. Anxiety disorders seem to be one of the most approachable and treatable types of mental disorder through online therapy [25–31]. Panic disorder (PD) with or without agoraphobia is one of the conditions for which ICBT programs have been developed since the late 1990s and there are several empirical studies supporting their efficacy and effectiveness [13, 17, 19, 32–42]. In Romania, only one internet-based treatment has been tested so far, the Swedish program Sofie . The study yielded positive outcomes for the treatment of social anxiety disorder, but the program is currently not in use.
The most promising interventions designed and tested so far are guided self-help interventions. A guided ICBT intervention is a web-based self-help program combined with minimal, but regular therapist support. In guided programs, asynchronous communication (emails or text messages) is most commonly used and, on average, patient guidance does not take more than 10 min per week . So far, research is very limited regarding web-based treatment programs that provide guidance through synchronous (real time) audio-video communication (e.g. video chat programs such as Skype). On the other hand, there is growing evidence that the efficacy of CBT delivered via videoconference is equivalent to face-to-face treatment for mood and anxiety disorders [45–50]. This type of communication closely matches face-to-face therapy and enables access to important face-to-face cues such as verbal tone, facial expressions, and body language. Through synchronous guidance, the therapists may be able to offer more individually tailored feedback and support according to each patient’s personal needs. Furthermore, this type of contact may facilitate higher engagement, feelings of accountability and social support, and reduce the risk of misunderstandings, compared to emails and instant messaging . All of the above could also facilitate the development of a better therapeutic alliance , a factor that has consistently proven to be a key predictor for therapeutic change in face-to-face therapy [53, 54]. So far, there is mixed evidence regarding the importance of the therapeutic alliance in guided ICBT . Even if contact with the therapist is only minimal, patients seem to develop a strong alliance, but this makes no difference for the outcome, at least for self-help interventions with rather low-intensity guidance . However, the therapeutic alliance in internet-based treatments that include video conferencing is yet to be investigated . Of course, there are also disadvantages for using real-time audio-video communication. For instance, compared to asynchronous written exchanges, patients do not have the opportunity to revisit and reread the dialogues with their therapists . The patients are still able to revisit and reflect upon the information enclosed in the treatment modules, but they cannot review their guidance sessions.
It is still not clear enough whether guided online treatments work better than unguided treatments [18, 31, 56–59], however there is a strong tendency for the superiority of guided treatment, especially for depression [14, 60]. So far, the literature suggests a superiority of guided versus unguided self-help treatments in terms of efficacy, adherence to treatment and drop-out rates [14, 61–63], but the conclusions are based on few studies and mainly limited to depression and social anxiety disorder, thus restricting the generalizability of the findings .
There is good quality evidence that ICBT treatments, especially guided interventions, are both efficacious and effective, either compared with waiting-list or with face to face interventions, but there is no intervention that is suitable for everyone . In order to be able to make recommendations for a certain person, with certain characteristics, we need to know which variables can predict or mediate/ moderate the outcomes. Unfortunately, the studies conducted so far showed that there are few stable predictors across studies and diagnoses . For example, treatment adherence was considered as a potential predictor variable, but there are inconsistencies, with some studies finding an association between adherence and clinical outcomes [66–68] and others not [65, 69, 70]. Another example is the severity of the disorder at baseline. In one study, a high baseline severity predicted better outcomes , while another study showed the contrary, low levels of panic symptom severity predicting better outcomes . Another interesting result concerns the role of comorbidity. A recent study showed that having a comorbid anxiety disorder was associated with a better treatment outcome among patients with panic disorder but not patients with social anxiety disorder , while others found no significant association [65, 73]. Considering these mixed results, we agree with Andersson who concluded “We do not know much about predictors of treatment outcome and the mechanisms involved in generating good outcome. Thus, more studies on moderators and mediators are needed” (, p.106).
Objectives and research questions
The aim of the present trial is to investigate the clinical efficacy of an internet-based self-help treatment program for panic disorder with therapist guidance via secured Skype in Romanian adults. The program (PAXonline for panic disorder) has been developed de novo by the authors, based on empirically validated cognitive-behavioral models of anxiety disorders [74–76], and it is currently the only available psychological treatment of this kind in Romania. A secondary objective is to compare the guided self-help intervention to a similar but unguided intervention program, considering clinical outcomes, adherence to treatment and drop-out. Additional exploratory research will be conducted to identify potential predictors, moderators and mediators of treatment outcome, such as patients’ expectations, patient-therapist working alliance, panic disorders severity and comorbidity status, and other relevant factors that were chosen based on previous research findings.
The study design is reported in line with the CONSORT 2010 statement (Consolidated Standards of Reporting Trials)  and the SPIRIT 2013 Statement (Standard Protocol Items: Recommendations for Interventional Trials) .
Web-based self-help program for panic disorder with intensive guidance (PAX-guided): participants in this group will complete the PAXonline panic disorder treatment program with support from a therapist, which include weekly emails and regular 15-30 min Skype sessions. In total, there will be 10 Skype sessions available.
Web-based self-help program (PAX-unguided): participants in this group will complete the PAXonline panic disorder program in a self-guided format, i.e. without any support from a therapist.
Waiting list control group. This group will not receive any treatment for 12 weeks. At the end of the waiting period, the participants receive any of the two treatment programs, depending on their choice.
Trial registration data set as recommend by World Health Organization (WHO)
Primary registry and trial identifying number
Australian New Zealand Clinical Trial Registry (ANZCTR), ACTRN12614000547640
Date of registration in primary registry
Secondary identifying numbers
Nil, U1111-1156-7294, 366360, 31697
Source(s) of monetary or material support
Babes-Bolyai University, Faculty of Psychology; Sciex - Scientific Exchange Programme NMS.CH; The Romanian Association for Online Counselling and Psychotherapy (ACPOR)
Babes-Bolyai University, Faculty of Psychology; 37, Republicii Street, Cluj-Napoca, 400015, Cluj
Sciex - Scientific Exchange Programme NMS.CH; CRUS Sciex-MPC, P.O. Box 607, CH-3000 Berne 9
ACPOR- Cluj-Napoca, Romania
Contact for public queries
Babes-Bolyai University, School of Psychology and Educational Sciences, Department of Psychology
Mr Mircea Miclea, Mr Liviu G. Crisan (firstname.lastname@example.org; email@example.com)
Contact for scientific queries
Ms Amalia Ciuca (firstname.lastname@example.org; email@example.com) Babes-Bolyai University, School of Psychology and Educational Sciences;
PAXonline: A Randomized Controlled Trial Assessing the Efficacy of an Internet-Based Cognitive Behavior Intervention for Panic Disorder
PAXonline: A Randomized Controlled Trial Comparing the Efficacy of an Internet-Based Cognitive Behavior Intervention, delivered with or without assistance from a therapist, to waiting-list in Romanian adults with Panic Disorder
Countries of recruitment
Romania, Spain, Italy
Health condition(s) or problem(s) studied
Mental health, anxiety disorders
Active comparators: Internet cognitive-behavioral treatment for panic disorder offered with or without assistance from a psychotherapist
Placebo comparator: waiting list group.
Key inclusion and exclusion criteria
Inclusion criteria: diagnostic of Panic Disorder (confirmed by an experienced clinician through semi-structured clinical interview); age within the range of 18–65 years; access to a computer with internet connection; written informed consent provided; no participation in psychological treatment for panic disorder in the last 3 months.
Exclusion criteria: severe comorbidities (e.g. bipolar disorders, psychotic disorders, substance abuse); mental retardation; suicidal ideation or behaviors; benzodiazepines treatment
Allocation: randomized; Intervention model: parallel assignment Masking: blind outcomes assessment
Primary purpose: treatment
Date of first enrolment
28 May 2014
Target sample size
120 (the target sample size was changed to 192, but an official change was not made in the registry)
Symptoms and severity of panic disorder (PDSS-SR); The Agoraphobic Cognitions Questionnaire; The Body Sensations Questionnaire
Key secondary outcomes
Symptoms and severity of depression - Patient Health Questionnaire-9 (PHQ-9); The Work and Social Adjustment Scale; the Working Alliance Inventory Short Revised (WAI-SR); Psychiatric Diagnostic and Screening Questionnaire (PDSQ); Panic Attack Cognition Questionnaire (PACQ); SS-5 - a 5-item shortened version of the Medical Outcomes Study Social Support Scale (MOS-SSS); Credibility/ Expectancy Questionnaire (CEQ); Body vigilance scale (BVS); Dependent personality disorder traits (Scale from OMNI-IV - Personality Disorder Inventory); System Usability Scale (SUS).
Native Romanian speakers with Panic Disorder (n = 192) will be recruited via media (TV news, internet advertisements, and social media), promotional advertising displayed in emergency rooms, and direct recommendations made by a network of general practitioners and psychotherapists.
Inclusion and exclusion criteria
Participants are included in the study if the following conditions are met: (1) their symptomatology fulfills the diagnostic criteria for panic disorder according to DSM-IV-TR (APA, 2000); (2) their age is within the range of 18–65 years; (3) they have access to a computer with internet connection; (4) they are native Romanian speakers; (5) they provide a written informed consent.
Participants are excluded if any of the following criteria is met: (1) they are currently enrolled in a different psychotherapeutic program or have received psychotherapy in the previous 3 months; (2) they present severe comorbidities, other than a different anxiety disorder or mild depression (e.g. bipolar disorders, psychotic disorders, substance abuse); (3) they suffer from mental retardation; (4) their symptoms are aggravated by a severe medical problem (e.g. ventricular tachycardia, heart attack, stroke, pulmonary fibrosis, hyperthyroidism, epilepsy); (5) they present suicidal ideation or behaviors.
Medication use is permitted, but only if the dosage was constant in the previous month and the same dosage must remain constant during the trial. Benzodiazepines are not allowed, according to existing recommendations [79–81]. All eligible participants must fill out and return a detailed informed consent form before starting the trial.
Individuals who struggle with panic attacks can freely access the study website which is publicly available at: http://studiu.paxonline.ro. The website provides information about panic disorder and treatment options, a description of the present clinical trial, and information about enrollment. The screening process for eligible participants is described below.
The participants start the screening process on the study website by filling out several online forms, anonymously. Firstly, participants must provide an online informed consent, then provide age and gender information. Next, they fill out three key questionnaires: (1) a screening for the exclusion criteria; (2) a scale for panic disorder diagnosis and severity (PDSS-SR); and (3) a screening for psychiatric comorbidities (PDSQ). Only participants who do not meet any exclusion criteria, who have a PDSS-SR score of at least 6 (see  for PDSS-SR cutoffs), and have completed the entire PDSQ screening are invited to provide an email address for further contact. Eligible participants are contacted by email to schedule a semi-structured clinical interview via secured Skype or telephone. The interviews are conducted by a trained clinical psychologist, who seeks to verify and confirm the diagnosis of a panic disorder and the compliance with all the inclusion/ exclusion criteria. At this point, participants who remain eligible after the interview receive an email from the principal investigator with a detailed informed consent and a link to fill out the online pre-treatment questionnaires. Participants are required to give their express consent to be included in the study. Specifically, they reply to the contact email confirming that they have read the informed consent and they accept to voluntarily participate in the study. They must also fill out their name and contact details at the end of the document. Participants who consent to this procedure are then randomly allocated to one of the three study groups.
After randomization, each participant receives an email containing instructions for the next phase. The guided participants receive a document with the recommended pathway for the entire course of treatment, along with instructions on how to communicate with their psychotherapist in the platform. They have a few days to explore the platform and read the first recommended module before scheduling their first Skype session with their psychotherapist. The first session is an introductory session in which the therapist explains the platform’s architecture and further collaboration between therapist and participant. All the Skype sessions will be audio recorded by the psychotherapists. To ensure treatment integrity and adherence to protocol, an impartial licensed clinical psychologist will randomly check 3 sessions from the work done with each patient.
We are using a secured Skype application for the guidance sessions. Skype is using a Peer-to-peer (“P2P”) technology, meaning that individuals communicate directly through their computers rather than through a third party host. All Skype-to-Skype voice, video, file transfers and instant messages are encrypted. Skype uses the AES (Advanced Encryption Standard), which is used by the US Government to protect sensitive information, and Skype also uses the strong 256-bit encryption. User public keys are certified by the Skype server at login using 1536 or 2048-bit RSA certificates .
The unguided participants receive a document with the recommended pathway for the entire course of treatment and are informed that they may ask for help with technical problems. This means they may ask for help if they are unable to access the platform, have problems with seeing movies, fill in documents or download resources. Although they receive strong recommendations to work through the modules in a certain order, there are no restrictions imposed and patients have access to all modules and resources, from the very beginning.
Measurements and time of assessment
Time of assessment
Semi-structured clinical interview for DSM-IV axis I disorders
DSM-IV Axis I disorders
Pre and post intervention (12 weeks)
Primary outcome measure
Panic disorder severity scale - self report
Severity of panic symptoms
Pre-treatment, every two weeks after intervention commencement, mid-treatment, post-treatment, and at follow-up: months 1, 3, 6 and 12
Secondary outcome measures
The agoraphobic cognitions questionnaire
Pre-treatment, mid-treatment, post-treatment, and at follow-up: months 1, 3, 6 and 12
The body sensations questionnaire
Fear of body sensations
Pre-treatment, mid-treatment, post-treatment, and at follow-up: months 1, 3, 6 and 12
Patient health questionnaire-9
Symptoms and severity of depression
Pre-treatment, mid-treatment, post-treatment, and at follow-up: months 1, 3, 6 and 12
The work and social adjustment scale
Pre-treatment, mid-treatment, post-treatment, and at follow-up: months 1, 3, 6 and 12
Psychiatric diagnostic and screening questionnaire
Axis 1 disorders
Pre-treatment, post-treatment, and at follow-up: months 6 and 12
Panic attack cognition questionnaire
Pre-treatment, mid-treatment, post-treatment, and at follow-ups
Body vigilance scale
Attentional focus on body sensations
Pre-treatment, mid-treatment, post-treatment, and at follow-ups
Perceived stress scale – 10
Perceived stress levels
Pre-treatment, mid-treatment, post-treatment, and at follow-ups
Working alliance inventory short form
Administered after first and fourth session
Credibility/ expectancy questionnaire
Expectancy for change and treatment credibility
Administered at 2 and 6 weeks during intervention
System usability scale
Usability and learnability
Administered at 2 and 6 weeks during intervention
a 5-item shortened version of the Medical Outcomes Study Social Support Scale (MOS-SSS)
Pre-treatment, mid-treatment, post-treatment, and at follow-ups
OMNI-IV personality disorder inventory
Personal autonomy questionnaire
Panic disorder knowledge
Pre and post-treatment
Drop-out reasons questionnaire
After dropping out
Patient feedback questionnaire
The waiting list control group does not receive treatment for 12 weeks. Throughout this waiting period, the participants in this group are required to fill out all the scheduled questionnaires - at pre-randomization, after 6 weeks, and at the end of the 12 weeks period. Afterwards, they can choose to follow the PAXonline treatment for panic disorder either independently or with guidance from a therapist. For an overview of measurements and time of assessment please see Table 2. After the last follow-up evaluation, additional treatment (4 weeks using the intervention program and one Skype-session) is available upon request for all participants in the study.
Paxonline panic disorder treatment
Introductory module. Anxiety, from normality to pathology
Psychoeducation concerning fear, anxiety, fight or flight response, the brain circuits involved in the fear response, etiology of anxiety and treatment directions. Normalizes patients’ reactions and sets correct expectancies.
Module 1. Understand what’s happening with you
Introductory information on panic attacks, panic symptoms and how panic disorder develops. Study cases and video illustrations. Patients begin to register PD related behaviors and thoughts using online worksheets.
Module 2. Understand what you have to do
Introducing treatment strategies, the treatment plan and boosting confidence and motivation for change.
Module 3. Reduce hyperactivation through breathing retraining
Information on the effects of hyperventilation, breathing regulation and abdominal breathing exercises.
Modules 4 and 5. Reduce hyperactivation through autogenic training
Inducing relaxation by using autogenic training exercises. Videos and audio files are provided for practice.
Module 6. Reduce hyperactivation through physical exercises
Psychological and biological benefits of physical exercises are presented; how to use them regularly.
Module 7. Optimizing attentional functioning
Attention biases in anxiety and panic disorder. Retraining attention away from the body and panic-like thoughts.
Module 8. Changing maladaptive conscious cognitions
The importance of catastrophic thoughts and attitudes. ICAR technique for cognitive restructuring:
Identify catastrophic cognitions
Challenge each dysfunctional cognition
Find the Alternative cognition
Repeat the procedure
Module 9. Changing maladaptive unconscious cognitions
Cognitive restructuring techniques for unconscious beliefs. Identify personal values and live by them.
Module 10. Avoidance reduction through interoceptive exposure
Explain the mechanism of avoidance and prepare for exposure exercises. Reduce fear of bodily sensations through interoceptive exposure.
Module 11. Avoidance reduction through exteroceptive exposure
Reduce avoidance behaviors through gradual exposure to feared situations.
Module 13. Positive emotions development
Increase positive emotions in daily life through cognitive and behavioral exercises drawn from positive psychology practices.
Module 14. Relapse prevention
Resume the intervention strategies, set the correct expectancies and make a plan to continue improvement and deal with relapse situations.
Module 12. You learn how to solve problems (optional)
Learn to organize problems according to importance and urgency. Implement problem-solving technique.
Module 15. Reducing depressive symptoms associated with panic – behavioral activation (optional)
Depression symptoms co-morbid to panic disorder. Behavioral activation techniques.
Module 16. Reducing depressive symptoms associated with panic – cognitive restructuring (optional)
ICAR technique for cognitive restructuring of dysfunctional cognitions that induce and maintain depression.
Both treatment groups use the same intervention, but the first group also receives guidance from a licensed psychotherapist. The participants in this group have regular 15–30 min Skype sessions (the length depends on the complexity of the modules and the needs of each patient) with their psychotherapist. The first Skype session is introductory and takes place in the first week of the treatment, after the participant completed the module “Anxiety, from normality to pathology”. From then on, a new session is scheduled almost every week, but only after the participant has completed the corresponding modules from the recommended timetable. The sessions are scheduled via an encrypted asynchronous communication system available on the treatment platform. The participants can also use this system between two Skype sessions if they need further clarifications from their therapist.
In total, there are 10 Skype sessions programmed. During these sessions, held over secured Skype, the psychotherapist checks if the participant has completed and understood each module, answers questions, and helps the participant carry out the recommended exercises. The main purpose of each online session is to make sure that the patient has everything he needs in order to put the recommended exercises into practice regularly. That means, according to a behavioral model of change , the psychotherapist try to set the necessary antecedents by providing procedural knowledge (‘I know what I have to do and how to do it”), self-efficacy (“I believe I can do it”) and motivation (“I’m convinced that it is worth doing it”). The agenda for Skype sessions usually involves: review of the week and how the participant feels, checking the homework assigned in previous sessions, discussing the last module, demonstrating a certain exercise or practicing together with the participant and, at the end, planning homework that should be carried out until the next session.
For example, in the first and the second session the psychotherapist works with the patient to develop a case conceptualization in order to understand the origin of the problem and the mechanisms by which it is maintained. They then discuss the treatment plan so that the patient understands how the intervention will answer his/her needs. In the session following the third module, the psychotherapist briefly explains the hyperventilation process, describes and demonstrates the breathing control technique to the patient and then they practice it once together. In the end, the psychotherapist helps the patient find a time and place to practice the technique twice a day. After the interoceptive exposure module, the psychotherapist explains the rationale of the technique once more, encourages the patient and does a few exercises from the list together with the patient (e.g. breathing for 2 min through a straw, spin in a chair for 20–30 s, cover the head with a scarf or a jacket etc.).
The modules and the entire therapeutic environment of PAXonline are carefully designed in order to facilitate the psychotherapeutic process (e.g. personalization of intervention, organizing information and knowledge in learning objects and use of other effective chunking strategies, enhancing learnability by using multimedia content, use of three layers of information, providing immediate feedback to queries, teach-back exercises, providing a reward after each module completion, offering access to a personal portfolio and a virtual library etc.) [10, 85, 86]. The program also contains certain techniques aimed at improving adherence and compliance with the treatment, e.g. rewards after module completion (short movies that induce positive emotions), the “meditation of the day”, which is sent via e-mail every 5 days and contains a link to a short movie or animation that is meant to be inspirational and/ or motivational. The participants’ adherence is carefully monitored by measuring overall usage time, the modules and homework assignments completed, the time spent on each module and each module component.
Three licensed psychotherapists with formal training in cognitive-behavioral therapy and a minimum of 3 years of clinical experience work with the participants allocated in the first treatment group. They have also participated in a 3 months blended course (3 face to face meetings plus online learning) prior to this study and were instructed in using the PAXonline platform and delivering CBT interventions online. The guidance will be delivered according to a standard treatment protocol, but it will also be personalized according to each participant’s need. During the study, the team meets once a month for discussions and the main researcher is available for contact at any time.
The selected instruments have already been well validated and are frequently used in CBT clinical trials for panic disorder in particular, as well as for other anxiety and mood disorders. Measures are taken at baseline (pre-treatment), 6 weeks after the intervention started (mid-treatment), directly after the intervention (post-treatment, 12 weeks after baseline), 1, 3, 6 and 12 months after the intervention ends. All self report measures are administered through the internet (secured links are used and all the data are encrypted) and the diagnostic interviews are conducted through secured Skype or by phone. The questionnaires, which were not yet validated for Romanian population, were previously translated in Romanian and underwent a rigorous back-translation process to ensure a good adaptation.
The primary outcomes are the diagnostic status of the participants at post-assessment and symptoms of panic disorder as assessed with the PDSS-SR.
Panic Disorder Severity Scale – Self Report (PDSS-SR). PDSS is originally a face-to-face interview and was adapted to a self-report questionnaire by Houck et al. . The scale contains seven items that measure the severity of seven dimensions of panic disorder and associated symptoms: 1) frequency of panic attacks; 2) distress during panic attacks; 3) anticipatory anxiety (worry about future panic attacks); 4) agoraphobic fear and avoidance; 5) interoceptive fear and avoidance (i.e., apprehension and avoidance of bodily sensations); 6) impairment of or interference with work functioning; and 7) impairment of or interference with social functioning. The PDSS-SR generates a total score ranging from 0 to 28, with a higher score indicating more severe panic symptoms. The questionnaire has good psychometric properties (Cronbach’s alpha = 0.92, test-retest reliability is 0.83) and is sensitive to changes following treatment . A cut-off score of six may discriminate between the presence and absence of current DSM-IV panic disorder and a cut-off score of fourteen may discriminate between mild and severe panic disorder .
Panic Disorder is assessed using the Romanian adapted version of Psychiatric Diagnostic Screening Questionnaire (PDSQ) [88, 89], which comprises a self-report screening scale, followed by a semi-structured interview delivered by a clinician. The PDSQ scale has good psychometric properties. In the Romanian validation study the mean of the alpha coefficients was .85 and test–retest reliability was above .80 for nine subscales, with a mean test–retest of .85 . A diagnosis of Panic Disorder is considered if the participant’s score reaches a value of at least 4 on the Panic scale. The same cutoff is applied for agoraphobia. Among Romanian psychiatric patients, these values offered the best balance between sensitivity and specificity: .90/.72 for panic disorder, and .95/.85 for agoraphobia .
Following the screening procedures, all eligible participants are interviewed by one of three experienced licensed clinical psychologists, based on the semi-structured interviews included in the PDSQ. This interview has been developed based on DSM-IV diagnosis criteria. In addition to Panic and Agoraphobia diagnoses, any other mental disorder that reached PDSQ screening cutoff point is assessed during the interview. In order to increase inter-raters agreement on assessment protocol, the assessors also participated in a two days training before study commencement. The diagnostic interviews are conducted before and after treatment by clinicians blind to the treatment group. At post-treatment diagnostic interview we are going to count missing data as equivalent to still meeting diagnostic criteria for panic disorder. This is the procedure that is recommended according to the intention-to-treat paradigm and was used in several other studies (e.g. Hedman et al. ).
Secondary outcome measures include quality of life, stress levels, depressive symptoms, catastrophic cognitions, body vigilance, and fear of bodily sensations.
The Patient Health Questionnaire (PHQ-9) is a widely used measure for depression. PHQ-9 contains nine items and covers nine criteria listed in the Diagnostic and Statistical Manual of Mental Disorders, 4th Edition (DSM-IV), in regard to depression , requiring respondents to rate the frequency of present difficulties during the past 2 weeks. Scores indicate presence and severity of depression, with a maximum score of 27 and a minimum score of 0. Scores of 5, 10, 15, and 20 indicate mild, moderate, moderately severe, and severe depression, respectively. The internal reliability of the English version of the PHQ-9 in a clinical population was in the range of 0.86–0.89 , which indicates good reliability. The test-retest reliability was also good, 0.84, and the correlation with interview results is very high, 0.84. Furthermore, sensitivity to change — an essential characteristic of measures used to monitor response to treatment — has been repeatedly established .
Mundt, Marks, Shear, and Greist  developed the Work and Social Adjustment Scale (WSAS). The WSAS is a 5-item simple, reliable and valid measure and we chose it in order to evaluate the self-reported functional impairment produced by anxiety problems. The scale items encompass different domains of functioning and include the following: ability to work, home management, social leisure, private leisure, and ability to form and maintain close relationships. Each item is rated on a 9-point Likert-type scale, ranging from 0 (no impairment) to 8 (very severe impairment). The maximum total score is 40, with higher scores representing greater impairment. The WSAS has demonstrated good internal consistency (range between 0.70 and 0.94) and test-retest reliability (0.73) and is sensitive to patients’ perceptions of disorder severity .
Body Vigilance Scale (BVS)  . We chose this scale in order to detect any changes in attention vigilance toward body sensations, a problem targeted in our treatment program. The 4 items of BVS measure the degree of attentional focus on, and perceived sensitivity towards changes in bodily sensations, and the amount of time spent scanning for bodily sensations. Research suggests that the BVS has adequate internal consistency in clinical and nonclinical populations and can be used to assess changes in bodily attention during cognitive–behavioral treatment for panic disorder [92–94].
Fear of body sensations
Body Sensations Questionnaire (BSQ) . BSQ was chosen in order to measure the “fear of anxiety” concept, an aspect that it is targeted in our intervention program. This is a 17 item self-report questionnaire and is used to assess the degree to which sensations associated with autonomic arousal (e.g., “feeling short of breath,” “heart palpitations”) are experienced as frightening. Each item is rated on a five-point scale ranging from 1 (not at all) to 5 (extremely). The BSQ has good internal consistency (α = 0.89) and high test–retest reliability (r = 0.79) . BSQ is also sensitive to changes following treatment, so it is considered to be a good choice for measuring treatment outcomes .
Catastrophic cognitions relevant to panic attacks and agoraphobia
Panic Attack Cognition Questionnaire (PACQ). The PACQ  is a 25-item scale assessing the extent to which catastrophic cognitions dominated subject’s thoughts before, during, or after panic attacks. Participants are asked to rate each thought on a 4-point scale indicating their level of preoccupation with that thought from 0 = not at all to 3 = totally dominates my thoughts. The total PACQ score is the sum of the 25 item scores. This scale was composed from items generated from the DSM-III-R description of panic attacks, from ideation reported in the anxiety literature, and from interviews with patients. The fifth edition of DSM is currently in use, but analyzing the DSM versions, we could conclude that the panic disorder diagnostic criteria have not changed much from DSM-III-R to DSM-5 (APA, 2013). There is still the distinction between autonomic nervous system symptoms (11 symptoms) and catastrophic cognitions (fear of dying, fear of losing control or going crazy). In the DSM-5 panic disorder and agoraphobia are two separated diagnoses, but other than that there are no major changes, at least not concerning the catastrophic cognitions, which are assessed through PACQ. The PACQ has good internal consistency as calculated by Cronbach alpha (.88). It has also been shown to successfully differentiate between individuals with and without panic attacks .
Agoraphobic Cognitions Questionnaire (ACQ) . The ACQ is a 14 item self-report questionnaire, which assesses the frequency of frightening or maladaptive thoughts about the consequences of panic and anxiety. Each item is rated on a five-point scale ranging from 1 (thought never occurs) to 5 (thought always occurs when I am nervous). The ACQ has shown good internal consistency (α = 0.82) and test–retest reliability (r = 0.79) . ACQ was also proven to be sensitive to changes following treatment . Both measures, PACQ and ACQ, were chosen because they measure catastrophic cognitions relevant to panic disorder and agoraphobia, a very important aspect that is directly targeted in our treatment.
Perceived Stress Scale – 10 (PSS-10) is a short and easy to use questionnaire which proved acceptable psychometric properties and is recommended for the assessment of perceived stress, both in practice and research . PSS-10  measures the degree to which one perceives aspects of one’s life as uncontrollable, unpredictable, and overloading. Participants are asked to respond to each question on a 5-point Likert scale ranging from 0 (never) to 4 (very often), indicating how often they have felt or thought a certain way within the past month. Scores range from 0 to 40, with higher composite scores indicating greater perceived stress. The PSS-10 has demonstrated good internal consistency (0.89) and also good divergent and convergent validity [98, 100].
Treatment credibility and patient expectancies
The Credibility/Expectancy Questionnaire (CEQ) of Devilly and Borkovec is used to measure expectancy for change and treatment credibility. It comprises six questions, four on “thinking” and two on “feeling” and they are rated on a Likert-type scale from 1 to 9. The scale has shown good internal consistency (expectancy, α = .79–.90; credibility, α = .81–.86) and test–retest reliability (expectancy, 0.82; credibility, 0 .75) . CEQ is administered twice; after the first and the fourth Skype session for the guided participants, and at 2 and 6 weeks during intervention for the unguided participants.
Working Alliance Inventory-Short Form (WAI-S) [102, 103]. The WAI-S is a 12-item client self-report measure to assess the working relationship and therapeutic bond between client and therapist. This measure is commonly used in CBT treatment outcome studies’, including those in which treatment is delivered remotely. The 3 subscales are: agreement on goals for treatment, agreement on therapeutic tasks, and perception of therapeutic bond. Ratings are given on a 7 - point Likert scale, with higher scores reflecting a stronger working alliance. Busseri and Tyler  reported that the WAI - S has good internal reliability for the composite score (alpha = .91) and the subscales (alpha = .73 to .86). WAI-S is administered twice, after the first and the fourth Skype session, according to existing recommendations .
Personality traits (dependent personality and personal autonomy)
OMNI-IV . Derived from the 375-item OMNI Personality Inventory, the 210-item OMNI-IV measures abnormal personality traits and assesses personality disorders. It is composed of 10 Personality Disorder scales based on the DSM-IV Axis II personality disorder criteria. For our study we use the Dependent Personality Subscale, which has 15 items. Previous studies showed good internal consistency, α = .80–.82 and also test-retest reliability, 0.80–0.84 . In the Romanian sample used to validate the scale internal consistency varied between 0.79 and 0.88 and test-retest reliability was 0.67 .
Personal Autonomy Questionnaire (PAQ) . The PAQ is a 36-item questionnaire designed to measure four dimensions of personal autonomy: cognitive autonomy, behavioral autonomy, emotional autonomy and value autonomy. Ratings are given on a 5-point Likert scale, with higher scores reflecting a stronger personal autonomy (reverse score items: 1, 4, 12, 13, 14, 15, 19, 29, 34, 35 and 36). PAQ has good internal consistency, α = .80 and also test-retest reliability, 0.86. .
Perceived social support
The SS-5 is an abbreviated version of the Medical Outcomes Study Social Support Scale (MOS-SSS) . This is a 5 item short and reliable measure for assessing perceived social support, a putative predictor or moderator of treatment outcome. Each item is scored on a 5-point Likert scale and the items are summed for a total score ranging from 5–25. The SS-5 proved to have adequate test-retest reliability (0.92) and internal consistency (α = .88). Equivalence between paper and web-based administration has also been demonstrated .
The System Usability Scale (SUS) is one of the oldest questionnaires used to evaluate usability of a system and was designed in 1986 by Broke. The SUS consists of 10 items scored on a 5-point Likert scale (1-5), with odd-numbered items worded positively and even-numbered items worded negatively. The scale has been used in hundreds of studies and the psychometric qualities are adequate (α = .88). For many years the scale was considered unidimensional, but recent studies have shown that it measures not just usability, but also learnability of a new system .
To evaluate the sociodemographic characteristics of the participants, we used a 7 questions survey referring to: environment (urban or rural), ethnicity, marital status, education, occupation, income levels and computer skills (1 to 10).
Psycho-Education Questionnaire (PEQ). In order to evaluate the information participants have about panic disorder and its treatment, we used a brief knowledge test that consisted of an 8-item questionnaire. Seven of the questions used are multiple choice questions and the last one is a self-evaluation question concerning the level of knowledge about PD and its treatment (from 1 – “I don’t know anything”, to 10,” I know almost everything”).
Adherence to treatment and homework compliance
Adherence to treatment and completion rates will be assessed. Number of modules completed, time spent in total, time spent on each module, and usage of the support will be analyzed to identify potential mediating effects.
Frequency of use of anxiety reduction strategies
In order to assess how frequent the anxiety reduction techniques were used we have developed a 10 items questionnaire. Eight of these questions are scored on a 5-point Likert scale (0-never and 4-daily or almost daily) and the last two are open ended questions: “Which techniques helped you the most?” and “Which are the techniques that you plan on using after the end of the intervention?”
The Patient Feedback Questionnaire (PFQ). We have developed a 14-items questionnaire to evaluate the patients’ satisfaction with the platform and some other aspects (the quality and utility of different components of the platform). Other items refer to giving improvement suggestions, what they like most, if they would use it again or recommend it to a friend with anxiety problems.
In order to find the reasons or motives participants had for dropping out of the study, we have developed an 18-item questionnaire. Each item is considered to be a possible motive (e.g. lack of motivation, lack of time, different expectations, exacerbation of symptoms, interface is too complicated, the treatment pace is too slow or too fast, finding another treatment, getting better, important life events etc.).
Participants who return the informed consent and meet all the required criteria are randomly allocated to one of the three conditions (1:1:1 allocation ratio). The randomization process is done by a software that was developed to implement a minimization algorithm , that assures a balanced randomization between groups with respect to certain predefined prognostic (stratification) factors. In this study, two stratification factors have been considered: SEVERITY (3 levels according to PDSS-SR screening scores; level 1: 6 to 13 points; level 2: 14 to 20 points; level 3: 21 to 28 point) and CHRONICITY (2 levels; more or less than 6 months since the beginning of PD) of panic disorder. The minimization method has been shown to outperform simple randomization in achieving balanced groups  and its use in clinical trials has been previously recommended as a better option than other randomization methods (e.g. [113–115]). Before each allocation, the algorithm computes an imbalance score for the three available treatments, taking stratification factors levels into account. The treatment with the lowest imbalance score is then given preference when assigning treatments, but the allocation probability varies for each patient, depending on the actual level of imbalance . This method is preferred because it avoids the deterministic allocation of pure minimization . The allocation is done by an independent researcher and is concealed, i.e. patients and researchers have no knowledge and no control over the allocation of participants when they randomize a participant with the computer program.
Taking the characteristics of our research into consideration, it is impossible to keep patients or psychotherapists blind to the study procedures and intervention. All participants are provided with detailed information about the aims and the methodology of the study. They can request more information about the study and they have the right to terminate participation at any time. The personnel in charge with the clinical interviews, on the other hand, will be blind to the treatment group allocation. We specifically instruct the participants not to mention group allocation at the post-treatment interview, but also test whether blinding was successful. The clinicians who conduct the diagnostic interviews are required to describe in their report if the participants disclosed their study group, directly or indirectly.
The effect sizes reported between guided and unguided interventions are quite heterogeneous with small to medium Cohen’s d size effects between 0.27  and .42 . Taking into consideration that, in most part, the guidance in previous studies was in fact low-intensity guidance, and that more support from a therapist yields larger effects [21, 62, 63], we expect an effect size of 0.5 (Cohen’s d) between guided and unguided treatment for PD. Based on this effect size, a power of 0.80, and an alpha level of 0.05, we would need 51 subjects in each condition. This recommended sample size has been increased to accommodate an attrition rate of 20 %. Thus, the study aims to achieve a sample of 192 participants.
All statistical analyses will be performed using the IBM SPSS Statistics version 20. The main analyses will be conducted on the intention-to-treat samples (all randomised participants, regardless of protocol adherence). Following current standards [27, 28], a linear mixed effects models approach with full information maximum likelihood estimation will be used. This approach has been recommended since it uses all available data and can handle missing data appropriately [118, 119]. The approach is based on the assumption that data are missing at random, and does not assume that missing data remain stable, as in the last-observation-carried-forward (LOCF) approach [119, 120]. Significance testing of dichotomous data such as diagnostic status will be conducted with chi-square tests. Calculations of within- and between-groups effect sizes (Cohen’s d) will be based on the pooled standard deviations. Regression analyses will be used to identify predictors of treatment outcome. Moderation and mediation analyses will be conducted with multiple regression models, using the PROCESS macro for SPSS , a computational technique that can compute both simple and complex moderation and mediation models. Effect sizes for significant effects will be indicated by Cohen’s f.
Consistent with CONSORT  and SPIRIT  recommendations we mention our intention to perform secondary analyses for minimal treatment users, defined as participants that complete at least 5 of the recommended modules, have at least 3 of the available guidance sessions (this is available only for the Skype-guided group) and use the intervention program for at least 80 min. This definition is based on the specific of our intervention program and the criteria used in other studies [27, 119].
The prevalence of mental health problems is very high and bears considerable suffering and burden worldwide. However, only a small percentage of the affected individuals reach out and receive treatment for their problems. There are many barriers at different levels, yet a viable solution to increase access to quality treatment resides in the possibilities provided by internet-based cognitive behavioral treatments. Although several ICBT programs have been tested with strong positive results and even integrated into regular health care, there are countries where validated internet-based interventions are yet to be developed. The present trial aims both to validate the overall efficacy of the first Romanian ICBT program for panic disorder, and to investigate aspects that are novel to ICBT, such as brief therapist guidance sessions via secured Skype.
There are still divergent results regarding the effect of therapist guidance in ICBT [18, 31, 56–58], and the efficacy of Skype-based (or equivalent) therapy sessions in ICBT have not been previously investigated. Thus, a central focus of our trial is to test the additive effect of therapeutic guidance via secured Skype in self-help ICBT programs. Specifically, the current study design allows us to assess the effect of the same self-help intervention modules in two treatment conditions: therapist guided via Skype vs. unguided self-directed intervention.
There is a general consensus that internet-based therapy can be efficient for many psychiatric conditions, but not all the patients benefit from it . Thus, there is an increasing interest to describe the characteristics of the patients who would benefit the most from this type of interventions. Taking into consideration this issue, additional exploratory analyses of potential predictors, moderators or mediators will be made. Patients’ treatment expectations, personality traits (e.g. personal autonomy), and usability of the online treatment program could prove to be key factors of treatment adherence and outcome, especially in the unguided self-help treatment group.
Several possible limitations should be noted. Firstly, the study sample size is large enough to reach our main research objectives, but will not permit us to perform detailed subgroup analyses. However, we expect to identify candidate predictors of treatment outcome that could be tested directly in future studies with larger samples. Secondly, the control group, for ethical reasons, will receive treatment in the second stage of the trial, thus the control group will not be available at follow-up. Furthermore, most of the recruitment will be made from the community, through advertisements in the media and on the study website, which could lead to a self-selected sample. Regarding the latter limitation, several findings from recent studies suggest that sampling bias may not affect the generalizability of internet-based trials. For example, Titov et al.  revealed that patients undergoing web-based treatments resemble national samples and present disorders as severe as those undergoing traditional psychotherapy. Another example is a recent study by Donkin et al.  on sampling bias in an internet-based clinical trial for depression. The authors revealed that out of 35 potentially biasing factors, only 4 were associated with consenting likelihood, accounting for limited variance in explaining participation, and none were associated with the primary outcomes. Future research will be required, nonetheless, to investigate whether the results from the PAXonline trial can be generalized to samples in regular mental health care settings.
In Romania, the need for easily accessible treatment programs for mental disorders is very high. 76.4 % of the individuals with mental health problems do not have access to any form of treatment, and only approximately 11.5 % receive psychological or psychiatric treatment . Internet-based treatments can offer a viable alternative solution to the current situation, considering that internet access has increased rapidly over the past years in Romania (in 2004 only 6 % households had access to the internet, and presently the number has increased to 61 % ). However, although ICBT advantages are easily recognized, both patients and psychologists in Romania have a reserved attitude towards using internet-based psychological intervention . By developing rigorous clinical trials on the efficiency of Romanian ICBT programs, we can increase the uptake of these solutions, and thus have a significant positive impact on the quality of life, both at an individual and national level.
Trial start date: May 22, 2014.
Currently recruiting (Ncurrent = 99 as of December 15, 2015).
The authors would like to acknowledge Roxana Costache for running pre-randomization blind diagnostic interviews and for offering guidance to participants; Angela Kozma for offering guidance to participants and Claudia Gageanu for running pre and post-treatment blind diagnostic interviews. We want to also thank to COGNITROM Ltd for giving us the permission to use their web-based software and make all the necessary modifications for research purposes.
This study is partially funded by a Scientific Exchange Programme NMS.CH Fellowship granted to the first author (Sciex Fellowship 13.224 – Sciex-N-9, contract no. 37435). The Babeș-Bolyai University, School of Psychology and Educational Sciences, Department of Psychology and The Romanian Association for Online Counseling and Psychotherapy provided material and logistic support for implementing the study. The work of TB is funded by a personal grant from the Swiss National Science Foundation (SNSF, PP00P1_144824). The funding body played no role in the design of the study or the writing of the manuscript; they have no authority over study-related activities: collection, management, analysis, and interpretation of data; the writing of the report; and the decision to submit the report for publication.
Open AccessThis article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated.
- Kessler RC, Berglund P, Demler O, Jin R, Merikangas KR, Walters EE. Lifetime prevalence and age-of-onset distributions of DSM-IV disorders in the National Comorbidity Survey Replication. Arch Gen Psychiatry. 2005;62(6):593–602.PubMedView ArticleGoogle Scholar
- Wittchen HU, Jacobi F, Rehm J, Gustavsson A, Svensson M, Jonsson B, et al. The size and burden of mental disorders and other disorders of the brain in Europe 2010. Eur Neuropsychopharmacol. 2011;21(9):655–79.PubMedView ArticleGoogle Scholar
- Strain JJ. Needs for psychiatry in the general hospital. Hosp Community Psychiatry. 1982;33(12):996–1001.PubMedGoogle Scholar
- Linzer M, Spitzer R, Kroenke K, Williams JB, Hahn S, Brody D, et al. Gender, quality of life, and mental disorders in primary care: results from the PRIME-MD 1000 study. Am J Med. 1996;101(5):526–33.PubMedView ArticleGoogle Scholar
- Shah A. The burden of psychiatric disorder in primary care. Int Rev Psychiatry. 1992;4(3-4):243–50.View ArticleGoogle Scholar
- News from WHO and Wonca: new report calls for mental health to be better integrated into primary care. Mental Health Fam Med. 2008; 5(2):125-125.Google Scholar
- White RJ, McGarry E, Jim. Cognitive behavioural computer therapy for the anxiety disorders: a pilot study. J Ment Health. 2000;9(5):505–16.View ArticleGoogle Scholar
- Üstün TB, Sartorius N. Mental illness in general health care: an international study. Chichester (UK):John Wiley & Sons, 1995.Google Scholar
- Wang PS, Lane M, Olfson M, Pincus HA, Wells KB, Kessler RC. Twelve-month use of mental health services in the united states - results from the national comorbidity survey replication. Arch Gen Psychiatry. 2005;62(6):629–40.PubMedView ArticleGoogle Scholar
- Miclea M, Miclea Ş, Ciuca AM, Budau O. Computer-mediated psychotherapy. Present and prospects. A developer perspective. Cogni Brain Behav. 2010;14:185–208.Google Scholar
- Wang PS, Berglund P, Olfson M, Pincus HA, Wells KB, Kessler RC. Failure and delay in initial treatment contact after first onset of mental disorders in the national comorbidity survey replication. Arch Gen Psychiatry. 2005;62(6):603–13.PubMedView ArticleGoogle Scholar
- Florescu SCM, Popovici G, Galaon M, Ladea M, Petukhova M. The Romanian mental health study: main aspects of lifetime prevalence and service use of DSM-IV disorders [Romanian]. Manag Health. 2009;13(3):22–30.View ArticleGoogle Scholar
- Hedman E, Ljotsson B, Lindefors N. Cognitive behavior therapy via the internet: a systematic review of applications, clinical efficacy and cost-effectiveness. Expert Rev Pharmacoecon Outcomes Res. 2012;12(6):745–64.PubMedView ArticleGoogle Scholar
- Richards D, Richardson T. Computer-based psychological treatments for depression: a systematic review and meta-analysis. Clin Psychol Rev. 2012;32(4):329–42.PubMedView ArticleGoogle Scholar
- Mewton L, Smith J, Rossouw P, Andrews G. Current perspectives on internet-delivered cognitive behavioral therapy for adults with anxiety and related disorders. Psychol Res Behav Manag. 2014;7:37–46.PubMedPubMed CentralView ArticleGoogle Scholar
- Andrews G, Davies M, Titov N. Effectiveness randomized controlled trial of face to face versus internet cognitive behaviour therapy for social phobia. Aust N Z J Psychiatry. 2011;45(4):337–40.PubMedView ArticleGoogle Scholar
- Bergstrom J, Andersson G, Ljotsson B, Ruck C, Andreewitch S, Karlsson A, et al. Internet-versus group-administered cognitive behaviour therapy for panic disorder in a psychiatric setting: a randomised trial. BMC Psychiatry. 2010;10.Google Scholar
- Botella C, Gallego MJ, Garcia-Palacios A, Guillen V, Banos RM, Quero S, et al. An internet-based self-help treatment for fear of public speaking: a controlled trial. Cyberpsych Beh Soc N. 2010;13(4):407–21.View ArticleGoogle Scholar
- Carlbring P, Nilsson-Ihrfelt E, Waara J, Kollenstam C, Buhrman M, Kaldo V, et al. Treatment of panic disorder: live therapy vs. self-help via the Internet. Behav Res Ther. 2005;43(10):1321–33.PubMedView ArticleGoogle Scholar
- Hedman E, Andersson G, Ljotsson B, Andersson E, Ruck C, Mortberg E, et al. Internet-based cognitive behavior therapy vs. cognitive behavioral group therapy for social anxiety disorder: a randomized controlled non-inferiority trial. PLoS One. 2011;6(3):e18001.PubMedPubMed CentralView ArticleGoogle Scholar
- Wagner B, Horn AB, Maercker A. Internet-based versus face-to-face cognitive-behavioral intervention for depression: a randomized controlled non-inferiority trial. J Affect Disorders. 2014;152:113–21.PubMedView ArticleGoogle Scholar
- Andersson G, Cuijpers P, Carlbring P, Riper H, Hedman E. Guided Internet-based vs. face-to-face cognitive behavior therapy for psychiatric and somatic disorders: a systematic review and meta-analysis. World Psychiatry. 2014;13(3):288–95.PubMedPubMed CentralView ArticleGoogle Scholar
- Ebert DD, Zarski A-C, Christensen H, Stikkelbroek Y, Cuijpers P, Berking M, et al. Internet and computer-based cognitive behavioral therapy for anxiety and depression in youth: a meta-analysis of randomized controlled outcome trials. PLoS One. 2015;10(3), e0119895.PubMedPubMed CentralView ArticleGoogle Scholar
- Silfvernagel K, Gren-Landell M, Emanuelsson M, Carlbring P, Andersson G. Individually tailored internet-based cognitive behavior therapy for adolescents with anxiety disorders: a pilot effectiveness study. Internet Interv. 2015;2(3):297–302.View ArticleGoogle Scholar
- Titov N, Andrews G, Johnston L, Robinson E, Spence J. Transdiagnostic internet treatment for anxiety disorders: a randomized controlled trial. Behav Res Ther. 2010;48(9):890–9.PubMedView ArticleGoogle Scholar
- Berger T, Boettcher J, Caspar F. Internet-based guided self-help for several anxiety disorders: a randomized controlled trial comparing a tailored with a standardized disorder-specific approach. Psychotherapy. 2014;51(2):207–19.PubMedView ArticleGoogle Scholar
- Ivarsson D, Blom M, Hesser H, Carlbring P, Enderby P, Nordberg R, et al. Guided internet-delivered cognitive behavior therapy for post-traumatic stress disorder: a randomized controlled trial. Internet Interv. 2014;1(1):33–40.View ArticleGoogle Scholar
- Andersson G, Paxling B, Roch-Norlund P, Ostman G, Norgren A, Almlov J, et al. Internet-based psychodynamic versus cognitive behavioral guided self-help for generalized anxiety disorder: a randomized controlled trial. Psychother Psychosom. 2012;81(6):344–55.PubMedView ArticleGoogle Scholar
- Andersson E, Enander J, Andren P, Hedman E, Ljotsson B, Hursti T, et al. Internet-based cognitive behaviour therapy for obsessive-compulsive disorder: a randomized controlled trial. Psychol Med. 2012;42(10):2193–203.PubMedPubMed CentralView ArticleGoogle Scholar
- Boettcher J, Carlbring P, Renneberg B, Berger T. Internet-based interventions for social anxiety disorder - an overview. Verhaltenstherapie. 2013;23(3):160–8.View ArticleGoogle Scholar
- Berger T, Caspar F, Richardson R, Kneubühler B, Sutter D, Andersson G. Internet-based treatment of social phobia: a randomized controlled trial comparing unguided with two types of guided self-help. Behav Res Ther. 2011;49(3):158–69.PubMedView ArticleGoogle Scholar
- Carlbring P, Bohman S, Brunt S, Buhrman M, Westling BE, Ekselius L, et al. Remote treatment of panic disorder: a randomized trial of internet-based cognitive behavior therapy supplemented with telephone calls. 2006.Google Scholar
- Richards JC, Klein B, Austin DW. Internet cognitive behavioural therapy for panic disorder: does the inclusion of stress management information improve end-state functioning? Clin Psychol. 2006;10(1):2–15.View ArticleGoogle Scholar
- Klein B, Richards JC, Austin DW. Efficacy of internet therapy for panic disorder. J Behav Ther Exp Psy. 2006;37(3):213–38.View ArticleGoogle Scholar
- Farvolden P, Denisoff E, Selby P, Bagby RM, Rudy L. Usage and longitudinal effectiveness of a Web-based self-help cognitive behavioral therapy program for panic disorder. J Med Internet Res. 2005;7(1), e7.PubMedPubMed CentralView ArticleGoogle Scholar
- Kiropoulos LA, Klein B, Austin DW, Gilson K, Pier C, Mitchell J, et al. Is internet-based CBT for panic disorder and agoraphobia as effective as face-to-face CBT? J Anxiety Disord. 2008;22(8):1273–84.PubMedView ArticleGoogle Scholar
- Shandley K, Austin DW, Klein B, Pier C, Schattner P, Pierce D, et al. Therapist-assisted, internet-based treatment for panic disorder: can general practitioners achieve comparable patient outcomes to psychologists? J Med Internet Res. 2008;10(2), e14.PubMedPubMed CentralView ArticleGoogle Scholar
- Wims E, Titov N, Andrews G, Choi I. Clinician-assisted internet-based treatment is effective for panic: a randomized controlled trial. Aust Nz J Psychiatry. 2010;44(7):599–607.View ArticleGoogle Scholar
- Nordgreen T, Standal B, Mannes H, Haug T, Sivertsen B, Carlbring P, et al. Guided self-help via internet for panic disorder: dissemination across countries. Comput Hum Behav. 2010;26(4):592–6.View ArticleGoogle Scholar
- Ruwaard J, Broeksteeg J, Schrieken B, Emmelkamp P, Lange A. Web-based therapist-assisted cognitive behavioral treatment of panic symptoms: a randomized controlled trial with a three-year follow-up. J Anxiety Disord. 2010;24(4):387–96.PubMedView ArticleGoogle Scholar
- Ruwaard J, Lange A, Schrieken B, Dolan CV, Emmelkamp P. The effectiveness of online cognitive behavioral treatment in routine clinical practice. PLoS One. 2012;7(7), e40089.PubMedPubMed CentralView ArticleGoogle Scholar
- Hedman E, Ljotsson B, Ruck C, Bergstrom J, Andersson G, Kaldo V, et al. Effectiveness of Internet-based cognitive behaviour therapy for panic disorder in routine psychiatric care. Acta Psychiatr Scand. 2013;128(6):457–67.PubMedView ArticleGoogle Scholar
- Tulbure BT, Szentagotai A, David O, Stefan S, Månsson KNT, David D, et al. Internet supported intervention for social anxiety disorder in Romania: a randomized controlled trial. PLoS One. 2015;10(5), e0123997.PubMedPubMed CentralView ArticleGoogle Scholar
- Andersson G. The internet and CBT: a clinical guide. CRC Press, Taylor & Francis; 2015.Google Scholar
- Stubbings DR, Rees CS, Roberts LD, Kane RT. Comparing in-person to videoconference-based cognitive behavioral therapy for mood and anxiety disorders: randomized controlled trial. J Med Internet Res. 2013;15(11), e258.PubMedPubMed CentralView ArticleGoogle Scholar
- Bouchard S, Paquin B, Payeur R, Allard M, Rivard V, Fournier T, et al. Delivering cognitive-behavior therapy for panic disorder with agoraphobia in videoconference. Telemed J E-health. 2004;10(1):13–25.PubMedView ArticleGoogle Scholar
- Germain V, Marchand A, Bouchard S, Drouin MS, Guay S. Effectiveness of cognitive behavioural therapy administered by videoconference for posttraumatic stress disorder. Cogn Behav Ther. 2009;38(1):42–53.Google Scholar
- Théberge-Lapointe N, Marchand A, Langlois F, Gosselin P, Watts S. Efficacy of a cognitive-behavioural therapy administered by videoconference for generalized anxiety disorder. Revue Européenne de Psychologie Appliquée/European Review of Applied Psychology. 2015;65(1):9–17.View ArticleGoogle Scholar
- Yuen EK, Herbert JD, Forman EM, Goetter EM, Juarascio AS, Rabin S, et al. Acceptance based behavior therapy for social anxiety disorder through videoconferencing. J Anxiety Disord. 2013;27(4):389–97.PubMedView ArticleGoogle Scholar
- Hilty DM, Ferrer DC, Parish MB, Johnston B, Callahan EJ, Yellowlees PM. The effectiveness of telemental health: a 2013 review. Telemed J E-health. 2013;19(6):444–54.PubMedPubMed CentralView ArticleGoogle Scholar
- Abbott J-AM, Klein B, Ciechomski L. Best practices in online therapy. J Technol Human Serv. 2008;26(2-4):360–75.View ArticleGoogle Scholar
- Simpson SG, Reid CL. Therapeutic alliance in videoconferencing psychotherapy: a review. Aust J Rural Health. 2014;22(6):280–99.PubMedView ArticleGoogle Scholar
- Horvath AO, Del Re AC, Fluckiger C, Symonds D. Alliance in individual psychotherapy. Psychotherapy. 2011;48(1):9–16.PubMedView ArticleGoogle Scholar
- Martin DJ, Garske JP, Davis MK. Relation of the therapeutic alliance with outcome and other variables: a meta-analytic review. J Consult Clin Psych. 2000;68(3):438–50.View ArticleGoogle Scholar
- Sucala M, Schnur JB, Constantino MJ, Miller SJ, Brackman EH, Montgomery GH. The therapeutic relationship in E-therapy for mental health: a systematic review. J Med Internet Res. 2012;14(4):175–87.View ArticleGoogle Scholar
- Berger T, Hammerli K, Gubser N, Andersson G, Caspar F. Internet-based treatment of depression: a randomized controlled trial comparing guided with unguided self-help. Cogn Behav Ther. 2011;40(4):251–66.PubMedView ArticleGoogle Scholar
- Furmark T, Carlbring P, Hedman E, Sonnenstein A, Clevberger P, Bohman B, et al. Guided and unguided self-help for social anxiety disorder: randomised controlled trial. Br J Psychiatr. 2009;195(5):440–7.View ArticleGoogle Scholar
- Titov N, Andrews G, Choi I, Schwencke G, Johnston L. Randomized controlled trial of web-based treatment of social phobia without clinician guidance. Aust Nz J Psychiatr. 2009;43(10):913–9.View ArticleGoogle Scholar
- Olthuis JV, Watt MC, Bailey K, Hayden JA, Stewart SH. Therapist-supported Internet cognitive behavioural therapy for anxiety disorders in adults. Cochrane Database Syst Rev. 2015;3:Cd011565.PubMedGoogle Scholar
- Spek V, Cuijpers P, Nykicek I, Riper H, Keyzer J, Pop V. Internet-based cognitive behaviour therapy for symptoms of depression and anxiety: a meta-analysis. Psychol Med. 2007;37(3):319–28.PubMedView ArticleGoogle Scholar
- Andersson G, Titov N. Advantages and limitations of Internet-based interventions for common mental disorders. World Psychiatr. 2014;13(1):4–11.View ArticleGoogle Scholar
- Johansson R, Andersson G. Internet-based psychological treatments for depression. 2012.Google Scholar
- Palmqvist B, Carlbring P, Andersson G. Internet-delivered treatments with or without therapist input: does the therapist factor have implications for efficacy and cost? 2007.Google Scholar
- Baumeister H, Reichler L, Munzinger M, Lin J. The impact of guidance on internet-based mental health interventions—a systematic review. Internet Interv. 2014;1(4):205–15.View ArticleGoogle Scholar
- El Alaoui S, Hedman E, Ljótsson B, Bergström J, Andersson E, Rück C, et al. Predictors and moderators of internet- and group-based cognitive behaviour therapy for panic disorder. PLoS One. 2013;8(11), e79024.PubMedPubMed CentralView ArticleGoogle Scholar
- Andersson G, BergstrÖM JAN, HollÄNdare F, Carlbring PER, Kaldo V, Ekselius L. Internet-based self-help for depression: randomised controlled trial. Br J Psychiatry. 2005;187(5):456–61.PubMedView ArticleGoogle Scholar
- Meyer B, Berger T, Caspar F, Beevers CG, Andersson G, Weiss M. Effectiveness of a novel integrative online treatment for depression (Deprexis): randomized controlled trial. J Med Internet Res. 2009;11(2), e15.PubMedPubMed CentralView ArticleGoogle Scholar
- Hilvert-Bruce Z, Rossouw PJ, Wong N, Sunderland M, Andrews G. Adherence as a determinant of effectiveness of internet cognitive behavioural therapy for anxiety and depressive disorders. Behav Res Ther. 2012;50(7-8):463–8.PubMedView ArticleGoogle Scholar
- Farrer L, Griffiths K, Christensen H, Mackinnon A, Batterham P. Predictors of adherence and outcome in internet-based cognitive behavior therapy delivered in a telephone counseling setting. Cogn Ther Res. 2014;38(3):358–67.View ArticleGoogle Scholar
- Hoifodt RS, Mittner M, Lillevoll K, Katla SK, Kolstrup N, Eisemann M, et al. Predictors of response to Web-based cognitive behavioral therapy with high-intensity face-to-face therapist guidance for depression: a Bayesian analysis. J Med Internet Res. 2015;17(9), e197.PubMedPubMed CentralView ArticleGoogle Scholar
- Warmerdam L, Van Straten A, Twisk J, Cuijpers P. Predicting outcome of internet-based treatment for depressive symptoms. Psychother Res. 2013;23(5):559–67.PubMedView ArticleGoogle Scholar
- Haug T, Nordgreen T, Öst L-G, Kvale G, Tangen T, Andersson G, et al. Stepped care versus face-to–face cognitive behavior therapy for panic disorder and social anxiety disorder: predictors and moderators of outcome. Behav Res Ther. 2015;71:76–89.PubMedView ArticleGoogle Scholar
- Vernmark K, Lenndin J, Bjarehed J, Carlsson M, Karlsson J, Oberg J, et al. Internet administered guided self-help versus individualized e-mail therapy: a randomized trial of two versions of CBT for major depression. Behav Res Ther. 2010;48(5):368–76.PubMedView ArticleGoogle Scholar
- Clark DA, Beck AT. Cognitive therapy of anxiety disorders: science and practice. Guilford Press; 2011.Google Scholar
- Leahy RL, Holland SJ. Treatment plans and interventions for depression and anxiety disorders. New York; London: Guilford Press; 2000.Google Scholar
- Andrews G. The treatment of anxiety disorders : clinician’s guide and patient manuals. 2nd ed. Cambridge: Cambridge University Press; 2003.Google Scholar
- Schulz KF, Altman DG, Moher D. CONSORT 2010 statement: updated guidelines for reporting parallel group randomized trials. Ann Intern Med. 2010;152(11):726–32.PubMedView ArticleGoogle Scholar
- Chan AW, Tetzlaff JM, Altman DG, Laupacis A, Gotzsche PC, Krleza-Jeric K, et al. SPIRIT 2013 statement: defining standard protocol items for clinical trials. Ann Intern Med. 2013;158(3):200–7.PubMedView ArticleGoogle Scholar
- Kumar S, Malone D. Panic disorder. BMJ Clin Evid. 2008;2008:1010.PubMedPubMed CentralGoogle Scholar
- Otto MW, McHugh RK, Kantak KM. Combined pharmacotherapy and cognitive-behavioral therapy for anxiety disorders: medication effects, glucocorticoids, and attenuated treatment outcomes. Clin Psychol. 2010;17(2):91–103.Google Scholar
- Excellence NIfHaC. Generalised anxiety disorder and panic disorder (with or without agoraphobia) in adults: management in primary, secondary and community care. In: NICE clinical guideline 113. 2011.Google Scholar
- Furukawa TA, Shear MK, Barlow DH, Gorman JM, Woods SW, Money R, et al. Evidence-based guidelines for interpretation of the panic disorder severity scale. Depress Anxiety. 2009;26(10):922–9.PubMedPubMed CentralView ArticleGoogle Scholar
- Skype Security Center [http://www.skype.com]. Accessed 2 October 2015.
- David D, Holdevici I, Szamoskozi S, Baban A. Psihoterapie si hipnoterapie cognitiv-comportamentala (Cognitive-behavioral psychotherapy and hypnotherapy). Cluj-Napoca: Risoprint; 1998.Google Scholar
- Miclea M, Miclea Ş, Ciuca A. Computer-supported psychotherapy should pay attention to e-learning. Cognitie, Creier, Comportament/Cogn Brain Behav. 2008;12(1):131.Google Scholar
- Miclea M, Ciuca A, Miclea S. How to produce e-content for e-mental health solutions. Basic guidelines. Cogn Brain Behav. 2009;13:1–9.Google Scholar
- Houck PR, Spiegel DA, Shear MK, Rucci P. Reliability of the self-report version of the panic disorder severity scale. Depress Anxiety. 2002;15(4):183–5.PubMedView ArticleGoogle Scholar
- Zimmerman M, Mattia JI. The psychiatric diagnostic screening questionnaire: development, reliability and validity. Compr Psychiatr. 2001;42(3):175–89.View ArticleGoogle Scholar
- Ciuca A, Albu M, Perţe A, Budău O, Miclea Ş, Miclea M. The Romanian adaptation of the psychiatric diagnostic screening questionnaire (PDSQ). Cogn Brain Behav. 2011;XV:69–93.Google Scholar
- Kroenke K, Spitzer RL, Williams JBW, Lowe B. The patient health questionnaire somatic, anxiety, and depressive symptom scales: a systematic review. Gen Hosp Psychiatr. 2010;32(4):345–59.View ArticleGoogle Scholar
- Mundt JC, Marks IM, Shear MK, Greist JH. The work and social adjustment scale: a simple measure of impairment in functioning. Br J Psychiatr. 2002;180:461–4.View ArticleGoogle Scholar
- Schmidt NB, Lerew DR, Trakowski JH. Body vigilance in panic disorder: evaluating attention to bodily perturbations. J Consult Clin Psych. 1997;65(2):214–20.View ArticleGoogle Scholar
- Barlow DH. Anxiety and its disorders : the nature and treatment of anxiety and panic. 2nd ed. New York: Guilford Press; 2002.Google Scholar
- Olatunji BO, Deacon BJ, Abramowitz JS, Valentiner DP. Body vigilance in nonclinical and anxiety disorder samples: structure, correlates, and prediction of health concerns. Behav Ther. 2007;38(4):392–401.PubMedView ArticleGoogle Scholar
- Chambless DL, Caputo GC, Bright P, Gallagher R. Assessment of fear of fear in agoraphobics - the body sensations questionnaire and the agoraphobic cognitions questionnaire. J Consult Clin Psych. 1984;52(6):1090–7.View ArticleGoogle Scholar
- Arrindell WA. The fear of fear concept: stability, retest artefact and predictive power. Behav Res Ther. 1993;31(2):139–48.PubMedView ArticleGoogle Scholar
- Clum GA, Broyles S, Borden J, Watkins PL. Validity and reliability of the panic attack symptoms and cognitions questionnaires. J Psychopathol Behav. 1990;12(3):233–45.View ArticleGoogle Scholar
- Lee EH. Review of the psychometric evidence of the perceived stress scale. Asian Nurs Res. 2012;6(4):121–7.View ArticleGoogle Scholar
- Cohen S, Williamson G. Perceived stress in a probability sample of the United States. In: Oskamp SSS, editor. The social psychology of health: Claremont symposium on applied social psychology. Newbury Park: Sage; 1988.Google Scholar
- Roberti JW, Harrington LN, Storch EA. Further psychometric support for the 10‐item version of the perceived stress scale. J Coll Couns. 2006;9(2):135–47.View ArticleGoogle Scholar
- Devilly GJ, Borkovec TD. Psychometric properties of the credibility/expectancy questionnaire. J Behav Ther Exp Psy. 2000;31(2):73–86.View ArticleGoogle Scholar
- Horvath AO, Greenberg LS. Development and validation of the working alliance inventory. J Couns Psychol. 1989;36(2):223.View ArticleGoogle Scholar
- Tracey TJ, Kokotovic AM. Factor structure of the working alliance inventory. Psychol Assess. 1989;1(3):207.View ArticleGoogle Scholar
- Busseri MA, Tyler JD. Interchangeability of the working alliance inventory and working alliance inventory, short form. Psychol Assess. 2003;15(2):193.PubMedView ArticleGoogle Scholar
- Fluckiger C, Del Re AC, Wampold BE, Symonds D, Horvath AO. How central is the alliance in psychotherapy? A multilevel longitudinal meta-analysis. J Couns Psychol. 2012;59(1):10–7.PubMedView ArticleGoogle Scholar
- Loranger A. OMNI Personality inventories. Professional manual. Lutz: Psychological Assessment Resources; 2001.Google Scholar
- Ciuca A, Costache R, Albu M. Adaptation and standardization of OMNI-IV on the Romanian population. In: OMNI-IV Professional Manual. edn. Cluj-Napoca: ASCR; in press.Google Scholar
- Albu M. Un nou instrument pentru evaluarea autonomiei personale la adolescenţi/ A new instrument for personal autonomy evaluation in adolescents. Anuarul Institutului de Istorie George Bariţ din Cluj-Napoca Seria Humanistica. 2007;V:99–114.Google Scholar
- Sherbourne CD, Stewart AL. The Mos social support survey. Soc Sci Med. 1991;32(6):705–14.PubMedView ArticleGoogle Scholar
- McCarrier KP, Bushnell DM, Martin ML, Paczkowski R, Nelson DR, Buesching D. Validation and psychometric evaluation of a 5-item measure of perceived social support. Value Health. 2011;14(3):A148.View ArticleGoogle Scholar
- Lewis JR, Sauro J. The factor structure of the system usability scale. Lect Notes Comput Sci. 2009;5619:94–103.View ArticleGoogle Scholar
- Pocock SJ, Simon R. Sequential treatment assignment with balancing for prognostic factors in the controlled clinical trial. Biometrics. 1975;31(1):103–15.PubMedView ArticleGoogle Scholar
- Scott NW, McPherson GC, Ramsay CR, Campbell MK. The method of minimization for allocation to clinical trials: a review. Control Clin Trials. 2002;23(6):662–74.PubMedView ArticleGoogle Scholar
- Treasure T, MacRae KD. Minimisation: the platinum standard for trials? Randomisation doesn’t guarantee similarity of groups; minimisation does. Br Med J. 1998;317(7155):362–3.View ArticleGoogle Scholar
- Hagino A, Hamada C, Yoshimura I, Ohashi Y, Sakamoto J, Nakazato H. Statistical comparison of random allocation methods in cancer clinical trials. Control Clin Trials. 2004;25(6):572–84.PubMedView ArticleGoogle Scholar
- Hofmeijer J, Anema PC, van der Tweel I. New algorithm for treatment allocation reduced selection bias and loss of power in small trials. J Clin Epidemiol. 2008;61(2):119–24.PubMedView ArticleGoogle Scholar
- Pond GR. Statistical issues in the use of dynamic allocation methods for balancing baseline covariates. Br J Cancer. 2011;104(11):1711–5.PubMedPubMed CentralView ArticleGoogle Scholar
- Gueorguieva R, Krystal JH. Move over ANOVA: progress in analyzing repeated-measures data and its reflection in papers published in the Archives of General Psychiatry. Arch Gen Psychiatry. 2004;61(3):310–7.PubMedView ArticleGoogle Scholar
- Meyer B, Bierbrodt J, Schröder J, Berger T, Beevers CG, Weiss M, et al. Effects of an internet intervention (Deprexis) on severe depression symptoms: randomized controlled trial. Internet Interv. 2015;2(1):48–59.View ArticleGoogle Scholar
- Blankers M, Koeter MWJ, Schippers GM. Missing data approaches in eHealth research: simulation study and a tutorial for nonmathematically inclined researchers. J Med Internet Res. 2010;12(5), e54.PubMedPubMed CentralView ArticleGoogle Scholar
- Hayes AF. Introduction to mediation, moderation, and conditional process analysis : a regression-based approach. 4th ed. New-York: The Guilford Press; 2013.Google Scholar
- Titov N, Andrews G, Kemp A, Robinson E: Characteristics of Adults with Anxiety or Depression Treated at an Internet Clinic: Comparison with a National Survey and an Outpatient Clinic. PloS one 2010, 5(5):e10885.Google Scholar
- Donkin L, Hickie IB, Christensen H, Naismith SL, Neal B, Cockayne NL, et al. Sampling bias in an internet treatment trial for depression. Translat Psychiatry. 2012;2(10), e174.Google Scholar
- Eurostat. Level of internet access – households [http://ec.europa.eu/eurostat/tgm/table.do?tab=table&init=1&language=en&pcode=tin00134&%20plugin=1]. Accessed 20 April 2015.
- Costache R, Ciuca A, Miclea M. Psychologists’ and potential patients’ attitudes towards computer-mediated psychotherapy. In: 35th STAR Conference. Cluj-Napoca, Romania; ASCR Press, 2014. ISBN: 978-606-8244-87-7.Google Scholar