Skip to main content
Download PDF

Systemic therapy in children and adolescents with mental disorders: a systematic review and meta-analysis

BMC Psychiatry volume 24, Article number: 125 (2024) Cite this article

Abstract

Background

Systemic therapy (ST) is a psychotherapeutic intervention in complex human systems (both psychological and interpersonal). Cognitive behavioural therapy (CBT) is an established treatment for children and adolescents with mental disorders. As methodologically rigorous systematic reviews on ST in this population are lacking, we conducted a systematic review and meta-analysis to compare the benefit and harm of ST (and ST as an add-on to CBT) with CBT in children and adolescents with mental disorders.

Methods

We searched MEDLINE, Embase, PsycINFO and other sources for randomised controlled trials in 14 mental disorder classes for the above comparisons in respect of effects on patient-relevant outcomes (search date: 7/2022). Where possible, meta-analyses were performed and results were graded into 3 different evidence categories: “proof”, “indication”, or “hint” (or none of these categories). PRISMA standards were followed.

Results

Fifteen studies in 5 mental disorder classes with usable data were identified. 2079 patients (mean age: 10 to 19 years) were analysed. 12/15 studies and 29/30 outcomes showed a high risk of bias. In 2 classes, statistically significant and clinically relevant effects in favour of ST were found, supporting the conclusion of a hint of greater benefit of ST for mental and behavioural disorders due to psychoactive substance use and of ST as an add-on to CBT for obsessive-compulsive disorders. In 2 other classes (eating disorders; hyperkinetic disorders), there was no evidence of greater benefit or harm of ST. For affective disorders, a statistically significant effect to the disadvantage of ST was found for 1 outcome, supporting the conclusion of a hint of lesser benefit of ST.

Conclusions

Our results show a hint of greater benefit of ST (or ST as an add-on to CBT) compared with CBT for 2 mental disorder classes in children and adolescents (mental and behavioural disorders due to psychoactive substance use, obsessive compulsive disorders). Given the importance of CBT as a control intervention, ST can therefore be considered a beneficial treatment option for children and adolescents with certain mental disorders. Limitations include an overall high risk of bias of studies and outcomes and a lack of data for several disorders.

Peer Review reports

Background

Systemic therapy (ST) or systems-oriented therapy can be defined as a psychotherapeutic intervention in complex human systems (both psychological and interpersonal) [1]. ST focuses on relationships and interactions, and include a contextual view of the problem with circular models of pathogenesis. ST therefore often involves working with patients and their families, as well as significant others such as teachers and friends, without being restricted to a particular setting [2].

Prior to the publication of 3 systematic reviews in 2013 and 2017 [2,3,4], there were no evidence syntheses on ST for a wide range of mental disorders in children and adolescents. The 2 reviews by von Sydow et al. and Retzlaff et al. [2, 3] compared ST with other or no psychotherapeutic interventions across a wide range of mental disorders and associated conditions (e.g. juvenile delinquency). Positive effects of ST were shown in individual studies for different disorders, but no meta-analyses were conducted to support these findings. Riedinger et al. [4] compared ST with other or no psychotherapeutic interventions for mental disorders and conducted meta-analyses showing positive effects of ST for some disorders. However, they did not analyse individual outcomes and did not compare specific interventions with each other. They concluded that “more research is needed before more general conclusions about the effects of ST can be drawn” [4].

In Germany, decisions on the reimbursement of health care services by the statutory health insurance (SHI) funds are made by the highest decision-making body in the health care system, the Federal Joint Committee (Gemeinsamer Bundesausschuss, G-BA) [5]. The G-BA regularly commissions the German health technology assessment (HTA) agency, the Institute for Quality and Efficiency in Health Care (Institut für Qualität und Wirtschaftlichkeit im Gesundheitswesen, IQWiG) [6], to provide the scientific basis for these decisions in the form of HTA reports.

Psychoanalysis, psychodynamic therapy, and cognitive behavioural therapy (CBT) have been available as SHI outpatient services for children, adolescents and adults for decades [7]; ST for adults followed in 2020 after an IQWiG HTA report [8].

In 2023, IQWiG published another HTA report to inform the (still pending) decision on whether outpatient ST should also be reimbursed for children and adolescents with mental disorders [9]. The full HTA report assessed the benefit and harm of ST compared with several control interventions. This article presents the comparison between ST and CBT, which was chosen as the control intervention for presentation because it is the most common form of psychotherapy offered to children and adolescents in Germany in the outpatient sector [10] and because previous research suggests that it is superior to placebo (e.g. [11,12,13]). CBT therefore sets a relatively high bar for demonstrating benefit, so comparisons that include CBT as a control intervention are particularly informative.

The aim of this systematic review was to compare the benefit and harm of ST with CBT (and of ST as an add-on to CBT with CBT alone) in children and adolescents with mental disorders.

Methods

General information

IQWiG’s general methodological approach is described in its methods paper [14].

This systematic review was part of the German-language HTA report mentioned above [9]. The (German-language) protocol [15] (not registered in a protocol database) was published on the IQWiG website before the actual HTA was conducted; the HTA report [9] is also published there. This protocol also applies to the present systematic review. The HTA report includes data on a wide range of mental disorders. This review only considers mental disorders for which data on ST versus CBT (or ST as an add-on to CBT versus CBT alone) are available. Only completed studies were used, so ethical approval and patient consent were not required. We adhered to the PRISMA statement [16, 17] throughout the manuscript. Our description of methods broadly follows that in previous journal articles on IQWiG reviews ([18, 19], Supplementary file 1: Additional file 1).

Study eligibility

We included both published and previously unpublished randomised controlled trials (RCTs) on ST versus CBT (or ST as an add-on to CBT versus CBT alone) in children and adolescents with mental disorders and investigating at least one predefined patient-relevant outcome [9]. In this context, the term “patient-relevant” refers to “how a patient feels, functions or survives” [20] and includes the categories of mortality, morbidity, and health-related quality of life [14].

Eligible studies included patients with any mental disorder listed in one of the established diagnostic classification systems such as the International Statistical Classification of Diseases and Related Health Problems 10th Revision (ICD-10, [21]), the Diagnostic and Statistical Manual of Mental Disorders 5th Edition (DSM-5, [22]) or any of their previous versions. There was no lower age limit for patients; the upper age limit was 21 years.

The studies had to examine psychotherapeutic interventions classified in the literature as systemic therapy [23,24,25,26,27]. The inclusion of a specific intervention was decided on the basis of its name or description in the study publication or other sources of information. If it was not possible to assign the experimental intervention directly to a specific systemic approach mentioned in the literature, its description was evaluated independently by 2 reviewers (MvPP and MM or MvPP and SG) to decide on whether or not the intervention could be classified as systemic. They discussed their classification with an additional internal reviewer (e.g., DHS) or the external reviewer (ROA), a psychiatrist for children and adolescents and systemic therapist. Disagreements were resolved by consensus. The setting of an intervention (e.g., family or group setting) was irrelevant for classification. Systemic-integrative approaches (containing both systemic and not clearly systemic components) were also considered. To be included in the review, they either had to be described in the literature as a systemic or systemic-integrative approach or it had to be clear from the description that the approach was predominantly systemic.

The control intervention was CBT. If the control intervention included both CBT and non-CBT components, the former had to predominate.

Co-interventions had to be similar between groups. CBT could also be a co-intervention; in this case, ST was investigated as an add-on to CBT compared with CBT alone.

Two comparisons were therefore examined:

  1. (1)

    ST versus CBT

  2. (2)

    ST as an add-on to CBT versus CBT alone

The detailed inclusion criteria are listed in Table 1.

Table 1 Inclusion criteriaa
Full size table

Search strategy and study selection

The following bibliographic databases and study registries were searched by an experienced information specialist: MEDLINE (1946 to 2022), Embase (1974 to 2022), PsycINFO (1806 to 2022), the Cochrane Central Register of Controlled Trials, ClinicalTrials.gov, and the International Clinical Trials Registry Platform (ICTRP) Search Portal. The peer-reviewed search strategy included a combination of subject headings and free texts, with terms such as “systemic therapy” and “family therapy” (see Supplementary file 1: Additional file 2 for the full search strategy). The last search was conducted on 12 July 2022.

In addition, the reference lists of relevant systematic reviews and HTA reports published between 2012 and 2021 were screened to identify further studies. Moreover, persons and parties who had submitted comments on the preliminary version of IQWiG’s HTA report [30] were asked to provide any additional relevant studies. Finally, documents submitted to the G-BA during the public hearing were also reviewed.

After removing duplicates, 2 reviewers independently screened the titles and abstracts of the retrieved citations to identify potentially eligible publications. The full texts of these articles were independently assessed by the same reviewers. Non-German or non-English full texts that appeared to be relevant based on the information in the abstract were translated. All documents retrieved from non-bibliographical sources were also checked for eligibility or relevant study information. Disagreements were resolved by consensus. Where necessary, authors were contacted to decide on the final inclusion or exclusion of studies.

Data extraction

Data extraction and risk-of-bias assessment were always conducted by one reviewer and checked by another; disagreements were resolved by consensus. Details of the studies were extracted using standardised tables. We extracted information on:

  1. (1)

    Study characteristics, including the study design, length of follow-up, sample size, location, and period in which the study had been conducted

  2. (2)

    Characteristics of study participants, including inclusion and exclusion criteria, age, sex, diagnoses of mental disorders

  3. (3)

    Characteristics of the experimental and control interventions

  4. (4)

    Outcomes and type of outcome measures. Given the large number of potential outcomes to be assessed, some of the outcomes that were considered less important were excluded from the assessment before the respective results were examined.

  5. (5)

    Risk-of-bias items (see below).

Where necessary, authors were contacted to provide missing data or to clarify issues. For more details on the methods applied and the handling of missing data/dropouts, intention-to-treat (ITT) analyses, and scale assessments, please see previous IQWiG publications [9, 14].

Risk‑of‑bias assessment

Using IQWiG’s methods paper [14] including Cochrane methods [31], the risk of bias at the study and outcome level was rated as high or low using the following items at the study level: generation of a randomisation sequence, allocation concealment, blinding of patients and health care professionals, reporting of all relevant outcomes irrespective of results, and other aspects. If the generation of a randomisation sequence or allocation concealment was judged to be inadequate, the other items at the study level were not assessed, because a high risk of bias at the study level was already apparent. A high risk of bias at the study level resulted in a high risk of bias at the outcome level. If the risk of bias at the study level was low, the following outcome-specific items were assessed: blinding of outcome assessors, use of the ITT principle, reporting of individual outcomes independent of results, and other aspects.

In a further step, we assessed the certainty of the study results and graded it as moderate or high, depending on the results of the risk-of-bias assessment.

Grading of results

Using IQWiG’s methods [14], the results for each outcome were graded into 3 different evidence categories: “proof”, “indication”, or “hint” (or none of these categories) of greater benefit of the experimental intervention. In short, proof of greater benefit of the experimental intervention is inferred if a meta-analysis of at least 2 studies with a high certainty of results shows a statistically significant effect in favour of the experimental intervention. An indication of greater benefit is inferred if a single study with a high certainty of results shows a statistically significant effect in favour of the experimental intervention, or a meta-analysis of studies with a moderate certainty of results shows a statistically significant effect in favour of the experimental intervention. A hint of greater benefit is inferred if a single study with a moderate certainty of results shows a statistically significant effect in favour of the experimental intervention. No evidence (i.e., no proof, indication or hint) of greater benefit is inferred if there are no statistically significant differences between the experimental and control interventions, if the results are inconclusive, or if no suitable data are available. If studies with both a low and a high risk of bias are available for a given outcome, the studies with a low risk of bias are primarily used to derive evidence (i.e., proof, indication or hint) of greater benefit of ST. The above approach is also used to determine harm.

Based on each outcome assessment and using the same evidence categories, an assessment of the benefit and harm of ST was performed across outcomes, taking into account the clinical relevance of the outcomes and the strength of the evidence (in particular, effect sizes and consistency across effects for a given outcome).

Data analysis

In the full HTA report [9], the studies included were grouped into classes that covered similar mental disorders. The grouping was based on a G-BA guideline that defines the criteria that a psychotherapeutic approach must fulfil in order to be offered as an SHI service [7]. This approach largely corresponds to the classification of mental disorders commonly used in established international diagnostic classification systems. The benefit of ST was assessed separately for the 14 different classes of mental disorders (plus the category “unspecified mental disorders”) included in the G-BA guideline.

If results for several different analysis points were available, the key analysis points were chosen (e.g., at baseline, mid-study, end-of-study, and follow-up). Odds ratios (OR) were calculated to compare dichotomous outcomes. Mean differences (MD) or Hedges’ g were calculated to compare continuous outcomes. In most cases, Hedges’ g was used to adjust for the different scales used to measure outcomes. A value of Hedges’ g of 0.2 (or − 0.2) was used as a clinical irrelevance threshold for continuous outcomes [32]. 95% confidence intervals (CIs) were reported for all effect estimates. To be assigned to one of the evidence categories “proof”, “indication” or “hint”, effects on continuous outcomes had to be not only statistically significant, but also considered clinically relevant.

Where possible and appropriate, data were pooled using meta-analyses. An overall effect was calculated using the Knapp and Hartung method with the Paule-Mandel heterogeneity estimator [33]. If 4 or fewer studies were available, a fixed-effect model was used to combine the study results. If relevant statistical heterogeneity [34] was present (Cochran’s Q test; p < 0.05), no overall effect estimate was calculated and, if possible, a 95% prediction interval [35] was calculated instead. The results of the meta-analysis were presented in forest plots. A p-value of < 0.05 was considered statistically significant.

Subgroup analyses were performed for age, sex, and type of systemic approach (systemic approaches only versus approaches combining systemic components with other components) if there were at least 10 patients with usable data and, in the case of binary data, at least 10 events per subgroup.

SAS software (version 9.4) was used for the data analysis.

Results

Information retrieval and study selection

The selection of studies is shown in Fig. 1. A total of 15 RCTs (Table 2) with usable data were included ([36,37,38,39,40,41,42,43,44,45,46,47,48,49,50,51,52,53,54,55,56,57,58,59,60,61,62,63,64,65,66,67,68,69,70,71,72,73,74,75,76,77,78,79,80,81,82,83,84,85,86,87,88,89,90,91,92,93,94,95,96,97,98,99,100,101,102,103,104,105,106,107,108,109,110,111,112,113,114,115,116,117,118,119,120,121,122,123]; details of the study pool are provided in Supplementary file 1: Additional file 3).

Fig. 1
figure 1

PRISMA flow chart and study selection

Full size image
Table 2 Study pool of included RCTs
Full size table

General study characteristics

The 15 studies with 2079 eligible randomised patients (range 11 to 600 per study) were conducted in the United States (n = 11) and Europe (n = 4) and published between 1997 and 2020. There were 972 patients in the experimental intervention group (range: 5 to 212 per study) and 1107 patients in the control group (range: 6 to 238 per study). About 55% of all patients were male, with 1 study (Nyman-Carlsson et al. [102,103,104]) including only women. The mean age of patients in the studies was between 10 and 19 years. More details on the study characteristics are provided in Supplementary file 1: Additional file 4.

Risk-of-bias assessment and certainty of results

Twelve studies had a high risk of bias (Supplementary file 1: Additional file 5). Only Schmidt et al. [112,113,114,115], Boyer et al. [36,37,38,39], and INCANT [63,64,65,66,67,68,69,70,71,72,73,74,75,76,77,78,79,80,81,82,83,84,85] had a low risk of bias. At the outcome level, only 1 out of 30 outcomes had a low risk of bias and therefore a high certainty of results: “substance use detected by laboratory tests” (No. 27 in Supplementary file 1: Additional file 5) in INCANT [63,64,65,66,67,68,69,70,71,72,73,74,75,76,77,78,79,80,81,82,83,84,85]. All other 29 outcomes had a high risk of bias at the outcome level and therefore a moderate certainty of results.

Main results

A total of 30 patient-relevant outcomes were identified in 5 classes of mental disorders with usable data (Fig. 2). Some outcomes either yielded heterogeneous results that did not allow the pooling of data or the identification of clear directions of effect, or significant results exceeded clinical irrelevance thresholds (± 0.2) and were therefore considered irrelevant (i.e. 95% CI covers − 0.2 or 0.2). No studies with usable data on the experimental and control intervention (ST vs. CBT; ST as an add-on to CBT vs. CBT) included could be identified for unspecified mental disorders or 9 further classes of mental disorders specified in the G-BA guideline [7], namely 1) conduct disorders, 2) pervasive developmental disorders, 3) somatoform disorders and dissociative disorders (conversion disorders), 4) reaction to severe stress, and adjustment disorders, 5) non-organic sleep disorders, 6) sexual dysfunction, 7) personality disorders and conduct disorders, 8) mental illnesses as a result of severe chronic diseases, and 9) schizophrenic and affective psychotic disorders. Table 3 shows the main results for the comparison of ST and CBT, i.e. those that support the conclusion that the data provide evidence (i.e. proof, indication or hint) of greater or lesser benefit of ST for a given outcome. Table 4 shows the main results for the comparison of ST as an add-on to CBT and CBT alone. More details are provided in the following sections and in Supplementary file 1: Additional file 6, including 18 forest plots.

Fig. 2
figure 2

All conclusions on benefit for all classes of mental disorders (with usable data) and individual outcomes. [Legend: *: Comparison 1: ST versus CBT; **: Comparison 2: ST as an add-on to CBT versus CBT alone; [↘]: hint of lesser benefit of ST compared with CBT (based on a single study); [↗]: hint of greater benefit of ST (or ST as an add-on to CBT) compared with CBT (based on a single study); [ ↔]: no evidence (i.e. no proof, indication or hint) of greater benefit or harm of ST (or ST as an add-on to CBT) (based on a single study); [⇑]: indication of greater benefit of ST (supported by meta-analysis); [⇔]: no evidence of greater benefit or harm of ST (homogeneous results between studies); [⇑⇓]: no evidence of greater benefit or harm of ST (heterogeneous results between studies); OCD: obsessive-compulsive disorder; prim. diagn.: primary diagnosis; ST: systemic therapy; CBT: cognitive behavioural therapy. See Supplementary file 1: Additional file 6 for full definitions of outcomes

Full size image
Table 3 Statistically significant effects for outcomes for which the data provided evidence (proof, indication or hint) of greater or lesser benefit of ST compared with CBT (from [9])
Full size table
Table 4 Statistically significant effects for outcomes for which the data provided evidence (proof, indication or hint) of greater or lesser benefit of ST as an add-on to CBT compared with CBT alone (from [9])
Full size table

Comparison 1: ST versus CBT

For the comparison of ST and CBT, data were available for 5 classes of mental disorders:

Class I: Affective disorders

One relevant study was identified (Brent et al. [40,41,42,43,44,45,46,47,48,49,50,51,52,53]); 2 of the 3 study arms (72 patients) compared ST with CBT in patients with major depression. The mean age of the patients was 16 years (range 13 to 18 years). The length of follow-up ranged from 12 weeks to 28 months.

Data were reported for 4 outcomes (Nos. 1 to 4 in Supplementary file 1: Additional file A6.1). For the dichotomous outcome “depressive symptoms” (Beck Depression Inventory [BDI] < 9), there was a statistically significant effect to the disadvantage of ST (OR = 0.35; 95% CI: [0.13; 0.97], p = 0.045; Table 3). No statistically significant effects were found for the other outcomes or operationalisations. However, most of the respective point estimates showed effects to the disadvantage of ST. A meta-analysis was not performed because only one study was available.

For depressive symptoms, the data provided a hint of lesser benefit of ST compared with CBT, whereas for the other outcomes there was no evidence (i.e. no proof, indication or hint) of greater or lesser benefit of ST; however, the point estimates largely indicated a disadvantage of ST.

Overall, the results support the conclusion that there is a hint of lesser benefit of ST compared with CBT for affective disorders.

Class II: Anxiety disorders

One study including 124 patients was identified (Lebowitz et al. [86]). The study investigated ST in patients with an anxiety disorder and compared ST with CBT. The mean age of patients was 10 years (range 7 to 14 years). The length of follow-up was 12 weeks.

The study reported data on 4 outcomes (Nos. 5 to 8 in Supplementary file 1: Additional file A6.2). No statistically significant or clinically relevant effects were found. A meta-analysis was not performed because only one study was available.

The data provided no evidence (i.e. no proof, indication or hint) of greater or lesser benefit of ST for any of the 4 outcomes.

Overall, the results support the conclusion that there is no evidence of greater benefit or harm of ST for anxiety disorders.

Class III: Eating disorders

Three relevant studies were identified comparing ST with CBT in patients with anorexia nervosa (Nyman-Carlsson et al. [102,103,104]), bulimia nervosa (2 out of 3 study arms in Le Grange et al. [87,88,89,90,91]), and bulimia nervosa and an eating disorder not otherwise specified (Schmidt et al. [112,113,114,115]). The sample sizes of the relevant populations ranged from 78 to 110 patients per study (total: 273 patients). The mean age of the patients was between 16 and 19 years (range across all 3 studies: 12 to 24 years). The length of follow-up ranged from 8 weeks to 36 months.

The 3 studies reported data on 6 outcomes (Nos. 12 to 17 in Supplementary file 1: Additional files A6.4, A6.5).

Results for hospitalisation were reported in 2 studies (Le Grange et al. [87,88,89,90,91] and Nyman-Carlsson et al. [102,103,104]). No pooled effect estimate was calculated due to relevant statistical heterogeneity. As only Le Grange et al. showed a statistically significant effect, it was not possible to conclude an effect on hospitalisation (see Section 3.1.4 in [14]).

Two studies (Le Grange et al. [87,88,89,90,91] and Schmidt et al. [112,113,114,115]) provided data on binge eating and compensatory behaviours, including 10 different operationalisations. The respective effects were either not statistically significant or statistically significant, but in opposite directions. Therefore, no clear conclusion about a benefit of ST could be drawn for this outcome.

No statistically significant effects were found for the other outcomes. No pooled effect estimates were calculated because of relevant statistical heterogeneity or because only results from 1 study were available for each outcome, operationalisation or analysis point.

The data provided no evidence (i.e. no proof, indication or hint) of greater or lesser benefit of ST for any of the 6 outcomes.

Overall, the results support the conclusion that there is no evidence of greater benefit or harm of ST for eating disorders.

Class IV: Hyperkinetic disorders

One study including 159 patients investigated ST in patients with attention-deficit/hyperactivity disorder (Boyer et al. [36,37,38,39]). The mean age of the patients was 14 years (range 12 to 17 years). ST and CBT were provided as an add-on to motivational interviewing, psychoeducation and, in some cases, medication. The length of follow-up ranged from about 2 to 5 months.

The study reported data on 4 outcomes (Nos. 18 to 21 in Supplementary file 1: Additional file A6.6). No statistically significant or clinically relevant effects were found. A meta-analysis was not performed because only one study was available.

The data provided no evidence (i.e. no proof, indication or hint) of greater or lesser benefit of ST for any of the 4 outcomes.

Overall, the results support the conclusion that there is no evidence of greater benefit or harm of ST for hyperkinetic disorders.

Class V: Mental and behavioural disorders due to psychoactive substance use

Seven studies investigated ST in patients with cannabis abuse or dependence (CYT [54,55,56,57,58,59,60], Dakof et al. [61, 62], INCANT [63,64,65,66,67,68,69,70,71,72,73,74,75,76,77,78,79,80,81,82,83,84,85], Liddle et al. 2008 [92,93,94,95,96,97,98,99]), cannabis use disorder (Liddle et al. 2018 [100, 101]), substance abuse (Waldron et al. [122, 123]) or abuse or dependence of alcohol or other substances (Slesnick et al. [117,118,119,120,121]). The sample sizes of the relevant populations ranged from 61 to 450 patients (total: 1378 patients). The mean age of the patients was 15 to 16 years (range across all included studies: 12 to 18 years). The length of follow-up ranged from 2 to 42 months. All studies compared ST with CBT. Three studies (CYT, Slesnick et al., and Waldron et al.) included additional study arms without CBT. In 6 studies, CBT was supplemented with measures to increase motivation, mostly motivational interviewing (CYT, Dakof et al., INCANT, Liddle et al. 2018, Slesnick et al., and Waldron et al.). In 1 study, CBT included components of dialectal behaviour therapy (Liddle et al. 2008). In 1 study (INCANT), all patients received CBT; almost half of these patients also received psychodynamic therapy. The control intervention in Slesnick et al. and one of the control interventions in CYT was the community reinforcement approach. This was a behavioural therapy intervention based on operant conditioning and included contingency management techniques, functional analysis, and skills training. Due to the high degree of overlap between the content of this approach and that of CBT, these control interventions were equated with CBT and included in the review.

The 7 studies reported data on 9 outcomes (Nos. 22 to 30 in Supplementary file 1: Additional files A6.7, A6.8). For 3 outcomes, there were statistically significant and clinically relevant effects in favour of ST (“cannabis use disorder”: OR = 1.68, 95% CI: [1.15; 2.44], p = 0.007; “symptoms of cannabis use disorder”: Cohen’s d = 1.27, 95% CI: [0.51; 2.03], p < 0.001; MD =  − 0.60, 95% CI [− 0.99; − 0.21] p = 0.003; and “use of cannabis”: MD =  − 27.14, 95% CI: [− 44.24; − 10.04], p = 0.002; MD =  − 8.22, 95% CI: [− 14.40; − 2.03], p = 0.009; MD =  − 8.30, 95% CI: [− 14.83; − 1.77], p = 0.013; p < 0.025; Table 3). For use of cannabis, we pooled data from 2 studies for 2 analysis points. For the different operationalisations of “use of substances for which criteria for a substance use disorder are not met”, effects were either not statistically significant or statistically significant, but in opposite directions. Therefore, no clear conclusion could be drawn for this outcome. For the other outcomes or operationalisations, the effects were either not statistically significant or not clinically relevant.

For externalising problems, internalising problems, substance use problem severity, and use of cannabis, the effects were pooled for several operationalisations. For the other outcomes, no pooled effect estimates were calculated because of relevant statistical heterogeneity or because results were available from only one study.

For cannabis use disorder, symptoms of cannabis use disorder, and use of cannabis, the data provided a hint of greater benefit of ST, whereas for the other outcomes there was no evidence (i.e. no proof, indication or hint) of greater or lesser benefit.

All effects in favour of ST were found in 2 studies (INCANT and Waldron et al.). In Waldron et al., the control intervention was explicitly described as individual CBT. In INCANT, all patients received CBT and only about half of these patients also received psychodynamic therapy. As all patients in Waldron et al. and the majority of patients in INCANT received CBT, the effects shown could be attributed with sufficient certainty to CBT. The results therefore support the conclusion of a hint of greater benefit of ST compared with CBT for mental and behavioural disorders due to psychoactive substance use.

Comparison 2: ST as an add-on to CBT versus CBT alone

For the comparison of ST as an add-on to CBT and CBT alone, data were available only for anxiety disorders and obsessive-compulsive disorders (OCD):

Class II: Anxiety disorders and obsessive-compulsive disorders

Two relevant studies including 73 patients (range: 11 to 62) were identified. They investigated ST in patients with an anxiety disorder (Siqueland et al. [116]) or an OCD (Peris et al. [105,106,107,108]). The mean age of patients was between 13 to 15 years (range across both studies 8 to 18 years). In the 2 studies, ST as an add-on to CBT was compared with CBT alone. The length of follow-up ranged from 14 weeks to 13 months.

The 2 studies reported data on 5 outcomes (Nos. 5 to 11 in Supplementary file 1: Additional file A6.3). There was a statistically significant effect in favour of ST as an add-on to CBT for 3 outcomes: “OCD symptoms” measured with the Children's Yale-Brown Obsessive Compulsive Scale (CY-BOCS; score ≤ 14): OR = 3.81, 95% CI [1.29; 11.20], p = 0.01; “overall functioning” measured with the Child Obsessive Compulsive Impact Scale-Revised (COIS-R): Hedges’ g =  − 0.75, 95% CI: [− 1.27; − 0.23], p = 0.004; and “overall improvement in clinical condition” measured with the Clinical Global Impression – Improvement (CGI-I; score ≤ 2): OR = 3.15, 95% CI: [1.10; 8.99], p = 0.03 (Table 4). While OCD symptoms and overall improvement in clinical condition were dichotomous outcomes, overall functioning was a continuous outcome, with the effect classified as clinically relevant because the upper limit of the 95% CI was below the irrelevance threshold of Hedges’ g − 0.2. No statistically significant or clinically relevant effects were found for the other outcomes or operationalisations. A meta-analysis was not performed because only one study was available for each outcome, operationalisation or analysis point.

For overall improvement in clinical condition, OCD symptoms and overall functioning (Nos. 6, 9 and 10 in Supplementary file 1: Additional file A6.3), the data provided a hint of greater benefit of ST as an add-on to CBT, whereas for the other outcomes there was no evidence (i.e. no proof, indication or hint) of greater or lesser benefit of ST as an add-on to CBT.

Overall, for patients with OCD, the results support the conclusion that there is a hint of greater benefit of ST as an add-on to CBT compared with CBT alone. As the effects in favour of ST were all shown in a study investigating OCD, the conclusion about the benefit of ST is limited to OCD.

Discussion

Summary of results

For 1 of the 14 mental disorder classes investigated (mental and behavioural disorders due to psychoactive substance use), an assessment across all outcomes showed a hint of greater benefit of ST compared with CBT. For eating disorders and hyperkinetic disorders, there was no evidence (i.e. no proof, indication or hint) of greater benefit or harm of ST. For affective disorders, there was a hint of lesser benefit of ST. Despite this null finding, the overall result for ST is considered positive because of the above-mentioned importance of CBT as a control intervention and because positive effects of ST were shown in the mental disorder class with the largest sample size, mental and behavioural disorders due to psychoactive substance use (1378 patients). In this class, there was a hint of greater benefit of ST compared with CBT across all outcomes. The conclusion has a reasonable degree of certainty because it is based on the results of 8 studies. The fact that there was no evidence of greater benefit or harm of ST in 2 classes of mental disorders (eating disorders and hyperkinetic disorders) should not be equated with no benefit at all compared with other treatments, especially as the present review only compared ST with CBT.

For 1 of the classes (OCD) an assessment across all outcomes showed a hint of greater benefit of ST as an add-on to CBT compared with CBT alone. However, with regard to the benefit of ST, this result is not as informative as the results comparing ST with CBT. In addition, the conclusion about the benefit of ST is limited to OCD, as evidence of a benefit of ST as an add-on to CBT was only found in the study on patients with OCD; no such evidence was found in the study on patients with an anxiety disorder.

To some extent the hint of less benefit of ST in affective disorders poses a problem for the otherwise positive body of evidence, as affective disorders are a particularly important class in terms of prevalence. However, this finding was based on the results of only a small study with 72 patients in the relevant population, so the conclusion on affective disorders should be viewed with caution.

Comparison with previous research

Due to methodological differences between our systematic review and previous systematic reviews [2,3,4], the comparability of results is limited.

Notably, none of the reviews included separate comparisons of ST with CBT. In addition, no meta-analyses were performed in von Sydow et al. [3] and Retzlaff et al. [2]. Riedinger et al. [4] performed meta-analyses, but unlike in our review, results were presented only for mental disorder classes, not for specific outcomes or specific comparisons of interventions. Moreover, the study pools of the 3 reviews and our review differed considerably due to the more stringent inclusion criteria in our review (confirmed diagnosis of a mental disorder required) and the more recent (5 to 9 years) literature search. We therefore believe that our review adds robust new evidence to the literature on ST.

Strengths and limitations

As previous research has shown that CBT is superior to placebo [11,12,13], the comparison between ST and CBT is particularly informative with regard to the benefit of ST. Accordingly, comparing ST with CBT helps to reduce the risk of overestimating the effects of ST.

With the exception of mental and behavioural disorders due to psychoactive substance use (1378 patients) the sample sizes for the other mental disorder classes were relatively small (range: 72 to 273 patients) and meta-analyses were not feasible for 3 out of the 5 classes with usable data. We tried to reduce the variation in follow-up periods between the included studies by combining similar follow-up periods into one period in the meta-analyses. Although this made it impossible to draw conclusions about some individual follow-up periods, it increased the overall robustness and validity of the results. In addition, most of the studies had a high risk of bias at the study level (largely due to inadequate or unclear randomisation or allocation concealment), which also led to a high risk of bias at the outcome level. In addition, patients and therapists were not blinded, as blinding is hardly possible with this type of intervention, which also contributes to a high risk of bias. Due to the small number of studies in the analyses, no formal tests for publication bias were performed, which should be taken into account when interpreting the data. Positive conclusions about the benefit of ST could only be drawn for 2 mental disorder classes in this review; it is unclear whether these conclusions apply to other classes due to a lack of appropriate data. In addition, our conclusions apply only to the comparison of ST and CBT, and the comparison of ST as an add-on to CBT and CBT alone, not to ST in general. In fact, the full HTA report [9] compared ST with a range of non-CBT control interventions (or no intervention). For both eating disorders and hyperkinetic disorders, ST showed a benefit compared with non-CBT control interventions. Finally, in all studies the reporting of adverse events was incomplete; therefore, we were not able to comprehensively weigh the benefits and harms of ST. As most studies on psychotherapeutic interventions still lack a standardised approach for adverse event recording [124], this problem should be addressed in future research.

No data on the comparisons included were available for conduct disorders, pervasive developmental disorders and unspecified mental disorders (for which data on the non-CBT control interventions were available in the full HTA report [9]). In addition, no studies were found for the remaining classes of mental disorders specified in the G-BA guideline [7], namely, 1) somatoform disorders and dissociative disorders (conversion disorders), 2) reaction to severe stress, and adjustment disorders, 3) non-organic sleep disorders, 4) sexual dysfunction, 5) personality disorders and conduct disorders, 6) mental illnesses as a result of severe chronic diseases, and 7) schizophrenic and affective psychotic disorders.

Conclusions

Our systematic review of ST for the treatment of children and adolescents with mental disorders shows a hint of greater benefit of ST (or ST as an add-on to CBT) compared with CBT for 2 classes of mental disorders: mental and behavioural disorders due to psychoactive substance use (ST) as well as OCD (ST + CBT), although the finding of greater benefit is less conclusive for OCD. Given the importance of CBT as a control intervention, ST may therefore be considered a beneficial treatment option for children and adolescents with certain mental disorders. Limitations of our review include an overall high risk of bias in the studies and outcomes analysed and a lack of data for several mental disorders. Results from high-quality RCTs are needed to confirm and extend our conclusions.

Availability of data and materials

All data used in this article are available in the full German-language report published on the IQWiG website [9]: https://www.iqwig.de/en/projects/n21-03.html.

Abbreviations

BDI:

Beck Depression Inventory

CBT:

Cognitive behavioural therapy

CGI-I:

Clinical Global Impression – Improvement

COIS-R:

Child Obsessive Compulsive Impact Scale-Revised

CY-BOCS:

Children's Yale-Brown Obsessive-Compulsive Scale

DSM-5:

Diagnostic and Statistical Manual of Mental Disorders 5th Edition

HTA:

Health technology assessment

ICD-10:

International Statistical Classification of Diseases and Related Health Problems 10th Revision

ICTRP:

International Clinical Trials Registry Platform

IQWiG:

Institute for Quality and Efficiency in Health Care

ITT:

Intention to treat

G-BA:

Federal Joint Committee (Gemeinsamer Bundesausschuss)

MD:

Mean difference

No(s). / Nos.:

Number(s)

OCD:

Obsessive-compulsive disorder

OR:

Odds ratio

PRISMA:

Preferred Reporting Items for Systematic reviews and Meta-Analyses

RCT:

Randomised controlled trial

RD:

Risk difference

SHI:

Statutory health insurance

ST:

Systemic therapy

References

  1. Schiepek G. Die Grundlagen der Systemischen Therapie. Göttingen: Vandenhoeck & Ruprecht; 1999.

    Google Scholar 

  2. Retzlaff R, von Sydow K, Beher S, Haun MW, Schweitzer J. The efficacy of systemic therapy for internalizing and other disorders of childhood and adolescence: a systematic review of 38 randomized trials. Fam Process. 2013;52(4):619–52.

    Article  PubMed  Google Scholar 

  3. von Sydow K, Retzlaff R, Beher S, Haun MW, Schweitzer J. The efficacy of systemic therapy for childhood and adolescent externalizing disorders: a systematic review of 47 RCT. Fam Process. 2013;52(4):576–618.

    Article  Google Scholar 

  4. Riedinger V, Pinquart M, Teubert D. Effects of systemic therapy on mental health of children and adolescents: a meta-analysis. J Clin Child Adolesc Psychol. 2017;46(6):880–94.

    Article  PubMed  Google Scholar 

  5. Gemeinsamer Bundesausschuss. The Federal Joint Committee. Available from: https://www.g-ba.de/english/.

  6. Institut für Qualität und Wirtschaftlichkeit im Gesundheitswesen. Responsibilities and objectives of IQWiG. 2023. Available from: https://www.iqwig.de/en/about-us/responsibilities-and-objectives-of-iqwig/.

  7. Gemeinsamer Bundesausschuss. Richtlinie des Gemeinsamen Bundesauschusses über die Durchführung der Psychotherapie: Psychotherapie-Richtlinie 2020 [updated 20.11.2020. Available from: https://www.g-ba.de/downloads/62-492-2400/PT-RL_2020-11-20_iK-2021-02-18.pdf.

  8. Institut für Qualität und Wirtschaftlichkeit im Gesundheitswesen. Systemische Therapie bei Erwachsenen als Psychotherapieverfahren; Abschlussbericht [Abschlussbericht/ -dokument]. 2017 [updated 24.05.2017; cited N14–02. Final report/document; 513. Available from: https://www.iqwig.de/download/N14-02_Abschlussbericht_Systemische-Therapie-bei-Erwachsenen_V1-0.pdf.

  9. Institut für Qualität und Wirtschaftlichkeit im Gesundheitswesen. Systemische Therapie als Psychotherapieverfahren bei Kindern und Jugendlichen; Abschlussbericht [Abschlussbericht/ -dokument]. 2023 [updated 27.01.2023; cited N21–03. Final report/document; 1507. Available from: https://www.iqwig.de/download/n21-03_systemische-therapie-bei-kindern-und-jugendlichen_abschlussbericht_v1-0.pdf.

  10. Jaite C, Seidel A, Hoffmann F, Mattejat F, Bachmann CJ. Guideline-based psychotherapy of children and adolescents in Germany: frequency, treatment modalities, and duration of treatment. Dtsch Arztebl Int. 2022;119(8):132–3.

    PubMed  PubMed Central  Google Scholar 

  11. Gillies D, Taylor F, Gray C, O’Brien L, D’Abrew N. Psychological therapies for the treatment of post-traumatic stress disorder in children and adolescents (Review). Evid Based Child Health. 2013;8(3):1004–116.

    Article  PubMed  Google Scholar 

  12. James AC, Reardon T, Soler A, James G, Creswell C. Cognitive behavioural therapy for anxiety disorders in children and adolescents. Cochrane Database Syst Rev. 2020;11(11):CD013162.

    PubMed  Google Scholar 

  13. Uhre CF, Uhre VF, Lonfeldt NN, Pretzmann L, Vangkilde S, Plessen KJ, et al. Systematic review and meta-analysis: cognitive-behavioral therapy for obsessive-compulsive disorder in children and adolescents. J Am Acad Child Adolesc Psychiatry. 2020;59(1):64–77.

    Article  PubMed  Google Scholar 

  14. Institute for Quality and Efficiency in Health Care. General Methods; Version 6.1 [Abschlussbericht/ -dokument]. 2022 [updated 24.01.2022; cited Methoden. Final report/document. Available from: https://www.iqwig.de/methoden/general-methods_version-6-1.pdf.

  15. Institut für Qualität und Wirtschaftlichkeit im Gesundheitswesen. Systemische Therapie als Psychotherapieverfahren bei Kindern und Jugendlichen; Berichtsplan [Berichtsplan]. 2021 [updated 19.11.2021; cited N21–03. Report plan. Available from: https://www.iqwig.de/download/n21-03_systemische-therapie-bei-kindern-und-jugendlichen_berichtsplan_v1-0.pdf.

  16. Moher D, Liberati A, Tetzlaff J, Altman DG. Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement. PLoS Med. 2009;6(7):e1000097.

    Article  PubMed  PubMed Central  Google Scholar 

  17. Page MJ, McKenzie JE, Bossuyt PM, Boutron I, Hoffmann TC, Mulrow CD, et al. The PRISMA 2020 statement: an updated guideline for reporting systematic reviews. BMJ. 2021;372:n71.

    Article  PubMed  PubMed Central  Google Scholar 

  18. Jakubeit T, Sturtz S, Sow D, Groß W, Mosch C, Patt M, et al. Single-fraction stereotactic radiosurgery versus microsurgical resection for the treatment of vestibular schwannoma: a systematic review and meta-analysis. Syst Rev. 2022;11(1):265.

    Article  PubMed  PubMed Central  Google Scholar 

  19. Zens Y, Barth M, Bucher HC, Dreck K, Felsch M, Groß W, et al. Negative pressure wound therapy in patients with wounds healing by secondary intention: a systematic review and meta-analysis of randomised controlled trials. Syst Rev. 2020;9(1):238.

    Article  PubMed  PubMed Central  Google Scholar 

  20. Biomarkers Definitions Working Group. Biomarkers and surrogate endpoints: preferred definitions and conceptual framework. Clin Pharmacol Ther. 2001;69(3):89–95.

    Article  Google Scholar 

  21. World Health Organization. Internationale Klassifikation psychischer Störungen; ICD-10 Kapitel V (F); Klinisch-diagnostische Leitlinien. 5th ed. Bern: Huber; 2005.

    Google Scholar 

  22. Falkai P, Wittchen HU, Döpfner M, Gaebel W, Maier W, Rief W, et al. Diagnostisches und statistisches manual psychischer Störungen - DSM-5. Göttingen: Hogrefe; 2015.

    Google Scholar 

  23. Levold T, Wirsching M. Systemische Therapie und Beratung: das große Lehrbuch. Heidelberg: Auer; 2014.

    Google Scholar 

  24. Retzlaff R. Spiel-Räume: Lehrbuch der systemischen Therapie mit Kindern und Jugendlichen. 7th ed. Stuttgart: Klett-Cotta; 2019.

    Google Scholar 

  25. Von Schlippe A, Schweitzer J. Lehrbuch der systemischen Therapie und Beratung I; Das Grundlagenwissen. Göttingen: Vandenhoeck & Ruprecht; 2016.

    Book  Google Scholar 

  26. Von Sydow K, Beher S, Retzlaff R, Schweitzer J. Die Wirksamkeit der Systemischen Therapie/Familientherapie. Göttingen: Hogrefe; 2007.

    Google Scholar 

  27. Von Sydow K, Borst U. Systemische Therapie in der Praxis. Weinheim: Beltz; 2018.

    Google Scholar 

  28. ICH Expert Working Group. ICH harmonised tripartite guideline: structure and content of clinical study reports; E3; current step 4 version 1995 [updated 30.11.1995. Available from: https://database.ich.org/sites/default/files/E3_Guideline.pdf.

  29. Moher D, Hopewell S, Schulz KF, Montori V, Gøtzsche PC, Devereaux PJ, et al. CONSORT 2010: explanation and elaboration; updated guidelines for reporting parallel group randomised trials. BMJ. 2010;340:c869.

    Article  PubMed  PubMed Central  Google Scholar 

  30. Institut für Qualität und Wirtschaftlichkeit im Gesundheitswesen. Systemische Therapie als Psychotherapieverfahren bei Kindern und Jugendlichen; Vorbericht (vorläufige Nutzenbewertung) [Vorbericht]. 2022 [updated 12.08.2022; cited N21–03. Preliminary report. Available from: https://www.iqwig.de/download/n21-03_systemische-therapie-bei-kindern-und-jugendlichen_vorbericht_v1-0.pdf.

  31. Higgins JPT, Thomas J, Chandler J, Cumpston M, Li T, Page MJ, et al. Cochrane handbook for systematic reviews of interventions. 2nd ed. Hoboken: Wiley-Blackwell; 2019.

    Book  Google Scholar 

  32. Cohen J. Statistical power analysis for the behavioral sciences. London: Academic Press; 1977.

    Google Scholar 

  33. Veroniki AA, Jackson D, Viechtbauer W, Bender R, Knapp G, Kuss O, et al. Recommendations for quantifying the uncertainty in the summary intervention effect and estimating the between-study heterogeneity variance in random-effects meta-analysis. Cochrane Database Syst Rev. 2015;1(Suppl 1):25–7.

    Google Scholar 

  34. Sutton AJ, Abrams KR, Jones DR, Sheldon TA, Song F. Methods for meta-analysis in medical research. Chichester: Wiley; 2000.

    Google Scholar 

  35. Guddat C, Grouven U, Bender R, Skipka G. A note on the graphical presentation of prediction intervals in random-effects meta-analyses. Syst Rev. 2012;1:34.

    Article  PubMed  PubMed Central  Google Scholar 

  36. Boyer BE, Doove LL, Geurts HM, Prins PJ, Van Mechelen I, Van der Oord S. Qualitative treatment-subgroup interactions in a randomized clinical trial of treatments for adolescents with ADHD: exploring what cognitive-behavioral treatment works for whom. PLoS One. 2016;11(3):e0150698.

    Article  PubMed  PubMed Central  Google Scholar 

  37. Boyer BE, Geurts HM, Prins PJ, Van der Oord S. Two novel CBTs for adolescents with ADHD: the value of planning skills. Eur Child Adolesc Psychiatry. 2015;24(9):1075–90.

    Article  PubMed  Google Scholar 

  38. Boyer BE, Geurts HM, Prins PJ, Van der Oord S. One-year follow-up of two novel CBTs for adolescents with ADHD. Eur Child Adolesc Psychiatry. 2016;25(3):333–7.

    Article  PubMed  Google Scholar 

  39. University of Amsterdam. Training adolescents with ADHD to plan and organize: investigating short- and longterm effects of treatment. 2022. Available from: https://trialsearch.who.int/Trial2.aspx?TrialID=NTR2142.

  40. Barbe RP, Bridge J, Birmaher B, Kolko D, Brent DA. Suicidality and its relationship to treatment outcome in depressed adolescents. Suicide Life Threat Behav. 2004;34(1):44–55.

    Article  PubMed  Google Scholar 

  41. Barbe RP, Bridge JA, Birmaher B, Kolko DJ, Brent DA. Lifetime history of sexual abuse, clinical presentation, and outcome in a clinical trial for adolescent depression. J Clin Psychiatry. 2004;65(1):77–83.

    Article  PubMed  Google Scholar 

  42. Birmaher B, Brent DA, Kolko D, Baugher M, Bridge J, Holder D, et al. Clinical outcome after short-term psychotherapy for adolescents with major depressive disorder. Arch Gen Psychiatry. 2000;57(1):29–36.

    Article  CAS  PubMed  Google Scholar 

  43. Brent DA, Baugher M, Birmaher B, Kolko DJ, Bridge J. Compliance with recommendations to remove firearms in families participating in a clinical trial for adolescent depression. J Am Acad Child Adolesc Psychiatry. 2000;39(10):1220–6.

    Article  CAS  PubMed  Google Scholar 

  44. Brent DA, Birmaher B, Kolko D, Baugher M, Bridge J. Subsyndromal depression in adolescents after a brief psychotherapy trial: course and outcome. J Affect Disord. 2001;63(1–3):51–8.

    Article  CAS  PubMed  Google Scholar 

  45. Brent DA, Holder D, Kolko D, Birmaher B, Baugher M, Roth C, et al. A clinical psychotherapy trial for adolescent depression comparing cognitive, family, and supportive therapy. Arch Gen Psychiatry. 1997;54(9):877–85.

    Article  CAS  PubMed  Google Scholar 

  46. Brent DA, Kolko DJ, Birmaher B, Baugher M, Bridge J. A clinical trial for adolescent depression: predictors of additional treatment in the acute and follow-up phases of the trial. J Am Acad Child Adolesc Psychiatry. 1999;38(3):263–70.

    Article  CAS  PubMed  Google Scholar 

  47. Brent DA, Kolko DJ, Birmaher B, Baugher M, Bridge J, Roth C, et al. Predictors of treatment efficacy in a clinical trial of three psychosocial treatments for adolescent depression. J Am Acad Child Adolesc Psychiatry. 1998;37(9):906–14.

    Article  CAS  PubMed  Google Scholar 

  48. Bridge JA, Barbe RP, Birmaher B, Kolko DJ, Brent DA. Emergent suicidality in a clinical psychotherapy trial for adolescent depression. Am J Psychiatry. 2005;162(11):2173–5.

    Article  PubMed  Google Scholar 

  49. Dietz LJ, Marshal MP, Burton CM, Bridge JA, Birmaher B, Kolko D, et al. Social problem solving among depressed adolescents is enhanced by structured psychotherapies. J Consult Clin Psychol. 2014;82(2):202–11.

    Article  PubMed  PubMed Central  Google Scholar 

  50. Gaynor ST, Weersing VR, Kolko DJ, Birmaher B, Heo J, Brent DA. The prevalence and impact of large sudden improvements during adolescent therapy for depression: a comparison across cognitive-behavioral, family, and supportive therapy. J Consult Clin Psychol. 2003;71(2):386–93.

    Article  PubMed  Google Scholar 

  51. Kolko DJ, Brent DA, Baugher M, Bridge J, Birmaher B. Cognitive and family therapies for adolescent depression: treatment specificity, mediation, and moderation. J Consult Clin Psychol. 2000;68(4):603–14.

    Article  CAS  PubMed  Google Scholar 

  52. Renaud J, Brent DA, Baugher M, Birmaher B, Kolko DJ, Bridge J. Rapid response to psychosocial treatment for adolescent depression: a two-year follow-up. J Am Acad Child Adolesc Psychiatry. 1998;37(11):1184–90.

    Article  CAS  PubMed  Google Scholar 

  53. Stein D, Brent DA, Bridge J, Kolko D, Birmaher B, Baugher M. Predictors of parent-rated credibility in a clinical psychotherapy trial for adolescent depression. J Psychother Pract Res. 2001;10(1):1–7.

    CAS  PubMed  PubMed Central  Google Scholar 

  54. Burleson JA, Kaminer Y, Dennis ML. Absence of iatrogenic or contagion effects in adolescent group therapy: findings from the Cannabis Youth Treatment (CYT) study. Am J Addict. 2006;15(Suppl 1):4–15.

    PubMed  Google Scholar 

  55. Dennis M, Godley SH, Diamond G, Tims FM, Babor T, Donaldson J, et al. The Cannabis Youth Treatment (CYT) Study: main findings from two randomized trials. J Subst Abuse Treat. 2004;27(3):197–213.

    Article  PubMed  Google Scholar 

  56. Dennis M, Titus JC, Diamond G, Donaldson J, Godley SH, Tims FM, et al. The Cannabis Youth Treatment (CYT) experiment: rationale, study design and analysis plans. Addiction. 2002;97(Suppl 1):16–34.

    Article  PubMed  Google Scholar 

  57. Diamond G, Godley SH, Liddle HA, Sampl S, Webb C, Tims FM, et al. Five outpatient treatment models for adolescent marijuana use: a description of the Cannabis Youth Treatment Interventions. Addiction. 2002;97(Suppl 1):70–83.

    Article  PubMed  Google Scholar 

  58. French MT, Roebuck MC, Dennis ML, Diamond G, Godley SH, Tims F, et al. The economic cost of outpatient marijuana treatment for adolescents: findings from a multi-site field experiment. Addiction. 2002;97(Suppl 1):84–97.

    Article  PubMed  Google Scholar 

  59. French MT, Roebuck MC, Dennis ML, Godley SH, Liddle HA, Tims FM. Outpatient marijuana treatment for adolescents. Economic evaluation of a multisite field experiment. Eval Rev. 2003;27(4):421–59.

    Article  PubMed  Google Scholar 

  60. Liddle HA. Multidimensional family therapy for adolescent cannabis users. Rockville (USA): U.S. Department of Health and Human Services; 2002. Available from: http://lib.adai.washington.edu/clearinghouse/downloads/Multidimensional-Family-Therapy-for-Adolescent-Cannabis-Users-207.pdf.

  61. Dakof GA, Henderson CE, Rowe CL, Boustani M, Greenbaum PE, Wang W, et al. A randomized clinical trial of family therapy in juvenile drug court. J Fam Psychol. 2015;29(2):232–41.

    Article  PubMed  PubMed Central  Google Scholar 

  62. University of Miami. Family-based juvenile drug court services. 2012. Available from: https://ClinicalTrials.gov/show/NCT01668303.

  63. Gantner A. Multidimensional family therapy for adolescent clients with cannabis use disorders–results and experience from the INCANT pilot study. Prax Kinderpsychol Kinderpsychiatr. 2006;55(7):520–32.

    PubMed  Google Scholar 

  64. Goorden M, van der Schee E, Hendriks VM, Hakkaart-van RL. Cost-effectiveness of multidimensional family therapy compared to cognitive behavioral therapy for adolescents with a cannabis use disorder: Data from a randomized controlled trial. Drug Alcohol Depend. 2016;162:154–61.

    Article  CAS  PubMed  Google Scholar 

  65. Grichting E, Haug S, Nielsen P, Schaub M. INCANT Hauptstudie; Nationaler Schlussbericht Schweiz. Zürich: Institut für Sucht- und Gesundheitsforschung; 2011.

    Google Scholar 

  66. Har A, Bonnaire C. The end of INCANT: The transition phase from research to clinical work. Thérapie Familiale. 2013;34(4):529–41.

    Article  Google Scholar 

  67. Hendriks V, van der Schee E, Blanken P. Treatment of adolescents with a cannabis use disorder: main findings of a randomized controlled trial comparing multidimensional family therapy and cognitive behavioral therapy in The Netherlands. Drug Alcohol Depend. 2011;119(1–2):64–71.

    Article  PubMed  Google Scholar 

  68. Hendriks V, van der Schee E, Blanken P. Matching adolescents with a cannabis use disorder to multidimensional family therapy or cognitive behavioral therapy: treatment effect moderators in a randomized controlled trial. Drug Alcohol Depend. 2012;125(1–2):119–26.

    Article  PubMed  Google Scholar 

  69. Hendriks VM, van der Schee E, Blanken P. Multidimensional family therapy and cognitive behavioral therapy in adolescents with a cannabis use disorder: a randomised controlled study. Tijdschr Psychiatr. 2013;55(10):747–59.

    CAS  PubMed  Google Scholar 

  70. Lascaux M, Bastard N, Bonnaire C, Couteron J-P, Phan O. INCANT: Comparison of two formalized therapeutic models. Alcoologie et Addictologie. 2010;32(3):209–19.

    Google Scholar 

  71. Lascaux M, Phan O. Comparison of European therapies for cannabis addiction among adolescents. Encephale. 2015;41(Suppl 1):S21–8.

    Article  PubMed  Google Scholar 

  72. Phan O, Bonnaire C, Bastard N, Jouanne C. INCANT project. Psychotropes. 2009;14(3–4):137–56.

    Article  Google Scholar 

  73. Phan O, Henderson CE, Angelidis T, Weil P, van Toorn M, Rigter R, et al. European youth care sites serve different populations of adolescents with cannabis use disorder. Baseline and referral data from the INCANT trial. BMC Psychiatry. 2011;11:110.

    Article  PubMed  PubMed Central  Google Scholar 

  74. Phan O, Jouanne C, Monge S. A random clinical trial concerning the psychotherapy of adolescents addicted to cannabis. [French]. Ann Med Psychol (Paris). 2010;168(2):145–51.

    Article  Google Scholar 

  75. Pol TM, Hendriks V, Rigter H, Cohn MD, Doreleijers TAH, Domburgh L, et al. Multidimensional family therapy in adolescents with a cannabis use disorder: long-term effects on delinquency in a randomized controlled trial. Child Adolesc Psychiatry Ment Health. 2018;12(1):44.

    Article  PubMed  PubMed Central  Google Scholar 

  76. Rigter H, Henderson CE, Pelc I, Tossmann P, Phan O, Hendriks V, et al. Multidimensional family therapy lowers the rate of cannabis dependence in adolescents: a randomised controlled trial in Western European outpatient settings. Drug Alcohol Depend. 2013;130(1–3):85–93.

    Article  PubMed  Google Scholar 

  77. Rigter H, Pelc I, Tossmann P, Phan O, Grichting E, Hendriks V, et al. INCANT: a transnational randomized trial of multidimensional family therapy versus treatment as usual for adolescents with cannabis use disorder. BMC Psychiatry. 2010;10:28.

    Article  PubMed  PubMed Central  Google Scholar 

  78. Rowe C, Rigter H, Henderson C, Gantner A, Mos K, Nielsen P, et al. Implementation fidelity of multidimensional family therapy in an international trial. J Subst Abuse Treat. 2013;44(4):391–9.

    Article  PubMed  Google Scholar 

  79. Schaub MP, Henderson CE, Pelc I, Tossmann P, Phan O, Hendriks V, et al. Multidimensional family therapy decreases the rate of externalising behavioural disorder symptoms in cannabis abusing adolescents: outcomes of the INCANT trial. BMC Psychiatry. 2014;14:26.

    Article  PubMed  PubMed Central  Google Scholar 

  80. Tossmann P, Jonas B. Ergebnisbericht der kontrollierten Behandlungsstudie INCANT. Berlin: Delphi-Gesellschaft für Forschung, Beratung und Prokjektentwicklung; 2010.

    Google Scholar 

  81. Tossmann P, Jonas B, Rigter H, Gantner A. Treating adolescents with cannabis use disorder with Multidimensional Family Therapy (MDFT): main results of a randomized controlled trial (RCT). [German]. Sucht. 2012;58(3):157–66.

    Article  Google Scholar 

  82. van der Pol TM, Henderson CE, Hendriks V, Schaub MP, Rigter H. Multidimensional family therapy reduces self-reported criminality among adolescents with a cannabis use disorder. Int J Offender Ther Comp Criminol. 2018;62(6):1573–88.

    Article  PubMed  Google Scholar 

  83. Verbanck P, Pelc I, Glaesener M, Gladsteen Y, Spapen P. INCANT final report; Period: November 2006-April 2010. Brüssel: CHU Brugmann; 2010.

    Google Scholar 

  84. Erasmus Medical Centre. The effectiveness of outpatient multidimensional family therapy (MDFT) compared with outpatient treatment as usual in adolescents with a cannabis use disorder and other problem behaviour: a multicentre, trans-national randomised controlled trial. 2018 [Yes. Available from: https://www.isrctn.com/ISRCTN51014277.

  85. The Netherlands Ministry of Health, Welfare and Sport. The effectiveness of multidimensional family therapy (MDFT) versus cognitive behavioural therapy (CBT) in Dutch adolescents with a cannabis use disorder: a randomised controlled trial. 2020. Available from: https://www.isrctn.com/ISRCTN00179361.

  86. Lebowitz ER, Marin C, Martino A, Shimshoni Y, Silverman WK. Parent-based treatment as efficacious as cognitive-behavioral therapy for childhood anxiety: a randomized noninferiority study of supportive parenting for anxious childhood emotions. J Am Acad Child Adolesc Psychiatry. 2020;59(3):362–72.

    Article  PubMed  Google Scholar 

  87. Gorrell S, Kinasz K, Hail L, Bruett L, Forsberg S, Lock J, et al. Rituals and preoccupations associated with bulimia nervosa in adolescents: Does motivation to change matter? Eur Eat Disord Rev. 2019;27(3):323–8.

    Article  PubMed  PubMed Central  Google Scholar 

  88. Le Grange D, Lock J, Agras WS, Bryson SW, Jo B. Randomized clinical trial of family-based treatment and cognitive-behavioral therapy for adolescent Bulimia Nervosa. J Am Acad Child Adolesc Psychiatry. 2015;54(11):886.

    Article  PubMed  PubMed Central  Google Scholar 

  89. Matheson BE, Gorrell S, Bohon C, Agras WS, Le Grange D, Lock J. Investigating early response to treatment in a multi-site study for adolescent Bulimia Nervosa. Front Psych. 2020;11:92.

    Article  Google Scholar 

  90. Valenzuela F, Lock J, Le Grange D, Bohon C. Comorbid depressive symptoms and self-esteem improve after either cognitive-behavioural therapy or family-based treatment for adolescent bulimia nervosa. Eur Eat Disord Rev. 2018;26(3):253–8.

    Article  PubMed  PubMed Central  Google Scholar 

  91. Stanford University. Study of treatment for adolescents with Bulimia Nervosa. 2015. Available from: https://ClinicalTrials.gov/show/NCT00879151.

  92. Chinchilla P. Comorbidity as a moderator of process-outcome relations in individual and family therapy for adolescent substance abuse. New York: Fordham University; 2007. p. 6954.

    Google Scholar 

  93. Dauber S. Treatment focus in individual and family therapy for adolescent drug abuse. New York: Fordham University; 2004.

    Google Scholar 

  94. Henderson CE, Dakof GA, Greenbaum PE, Liddle HA. Effectiveness of multidimensional family therapy with higher severity substance-abusing adolescents: report from two randomized controlled trials. J Consult Clin Psychol. 2010;78(6):885–97.

    Article  PubMed  PubMed Central  Google Scholar 

  95. Hogue A, Dauber S, Liddle HA, Samuolis J. Linking session focus to treatment outcome in evidence-based treatments for adolescent substance abuse. Psychotherapy. 2004;41(2):83–96.

    Article  PubMed  Google Scholar 

  96. Hogue A, Dauber S, Stambaugh LF, Cecero JJ, Liddle HA. Early therapeutic alliance and treatment outcome in individual and family therapy for adolescent behavior problems. J Consult Clin Psychol. 2006;74(1):121–9.

    Article  PubMed  PubMed Central  Google Scholar 

  97. Hogue A, Henderson CE, Dauber S, Barajas PC, Fried A, Liddle HA. Treatment adherence, competence, and outcome in individual and family therapy for adolescent behavior problems. J Consult Clin Psychol. 2008;76(4):544–55.

    Article  PubMed  PubMed Central  Google Scholar 

  98. Liddle HA, Dakof GA, Turner RM, Henderson CE, Greenbaum PE. Treating adolescent drug abuse: a randomized trial comparing multidimensional family therapy and cognitive behavior therapy. Addiction. 2008;103(10):1660–70.

    Article  PubMed  Google Scholar 

  99. Rowe CL, Liddle HA, Greenbaum PE, Henderson CE. Impact of psychiatric comorbidity on treatment of adolescent drug abusers. J Subst Abuse Treat. 2004;26(2):129–40.

    Article  PubMed  Google Scholar 

  100. Liddle HA, Dakof GA, Rowe CL, Henderson C, Greenbaum P, Wang W, et al. Multidimensional Family Therapy as a community-based alternative to residential treatment for adolescents with substance use and co-occurring mental health disorders. J Subst Abuse Treat. 2018;90:47–56.

    Article  PubMed  Google Scholar 

  101. University of Miami. Family-based and adolescent residential drug treatment. 2012. Available from: https://ClinicalTrials.gov/show/NCT01737632.

  102. Nyman-Carlsson E, Birgegard A, Engstrom I, Gustafsson SA, Nevonen L. Predictors of outcome among young adult patients with anorexia nervosa in a randomised controlled trial. Eur Eat Disord Rev. 2019;27(1):76–85.

    Article  PubMed  Google Scholar 

  103. Nyman-Carlsson E, Norring C, Engstrom I, Gustafsson SA, Lindberg K, Paulson-Karlsson G, et al. Individual cognitive behavioral therapy and combined family/individual therapy for young adults with Anorexia nervosa: A randomized controlled trial. Psychother Res. 2020;30(8):1011–25.

    Article  PubMed  Google Scholar 

  104. Anorexia Bulimia Unit Child Adolescent Psychiatry Centre. The Gothenburg anorexia nervosa treatment study - a randomized control trial comparing individual cognitive behavioral therapy (I-CBT) and family therapy (FT) for young adults with anorexia nervosa. 2019. Available from: https://www.isrctn.com/ISRCTN25181390.

  105. Kircanski K, Peris TS. Exposure and response prevention process predicts treatment outcome in youth with OCD. J Abnorm Child Psychol. 2015;43(3):543–52.

    Article  PubMed  PubMed Central  Google Scholar 

  106. Peris TS, Piacentini J. Optimizing treatment for complex cases of childhood obsessive compulsive disorder: a preliminary trial. J Clin Child Adolesc Psychol. 2013;42(1):1–8.

    Article  PubMed  Google Scholar 

  107. Peris TS, Rozenman MS, Sugar CA, McCracken JT, Piacentini J. Targeted family intervention for complex cases of pediatric obsessive-compulsive disorder: a randomized controlled trial. J Am Acad Child Adolesc Psychiatry. 2017;56(12):1034.

    Article  PubMed  PubMed Central  Google Scholar 

  108. University of California. Family focused treatment of pediatric obsessive compulsive disorder (PFIT). 2021. Available from: https://clinicaltrials.gov/ct2/show/NCT01409642.

  109. Rohde P, Turner CW, Waldron HB, Brody JL, Jorgensen J. Depression change profiles in adolescents treated for comorbid depression/substance abuse and profile membership predictors. J Clin Child Adolesc Psychol. 2018;47(4):595–607.

    Article  PubMed  Google Scholar 

  110. Rohde P, Waldron HB, Turner CW, Brody J, Jorgensen J. Sequenced versus coordinated treatment for adolescents with comorbid depressive and substance use disorders. J Consult Clin Psychol. 2014;82(2):342–8.

    Article  PubMed  PubMed Central  Google Scholar 

  111. Oregon Research Institute. Sequenced vs. integrated delivery of treatment for adolescent depression and SUD. 2014. Available from: https://ClinicalTrials.gov/show/NCT00680966.

  112. Perkins S, Schmidt U, Eisler I, Treasure J, Yi I, Winn S, et al. Why do adolescents with bulimia nervosa choose not to involve their parents in treatment? Eur Child Adolesc Psychiatry. 2005;14(7):376–85.

    Article  PubMed  Google Scholar 

  113. Schmidt U, Lee S, Beecham J, Perkins S, Treasure J, Yi I, et al. A randomized controlled trial of family therapy and cognitive behavior therapy guided self-care for adolescents with bulimia nervosa and related disorders. Am J Psychiatry. 2007;164(4):591–8.

    Article  PubMed  Google Scholar 

  114. Schmidt U, Lee S, Perkins S, Eisler I, Treasure J, Beecham J, et al. Do adolescents with eating disorder not otherwise specified or full-syndrome bulimia nervosa differ in clinical severity, comorbidity, risk factors, treatment outcome or cost? Int J Eat Disord. 2008;41(6):498–504.

    Article  PubMed  Google Scholar 

  115. The Health Foundation. A randomized controlled evaluation of the cost effectiveness of cognitive-behavioural guided self-care versus family therapy for adolescent bulimia nervosa in a catchment area-based population. 2007. Available from: https://www.isrctn.com/ISRCTN70120585.

  116. Siqueland L, Rynn M, Diamond GS. Cognitive behavioral and attachment based family therapy for anxious adolescents: phase I and II studies. J Anxiety Disord. 2005;19(4):361–81.

    Article  PubMed  Google Scholar 

  117. Guo X, Slesnick N, Feng X. Reductions in depressive symptoms among substance-abusing runaway adolescents and their primary caretakers: a randomized clinical trial. J Fam Psychol. 2014;28(1):98–105.

    Article  PubMed  Google Scholar 

  118. Guo X, Slesnick N, Feng X. Changes in family relationships among substance abusing runaway adolescents: a comparison between family and individual therapies. J Marital Fam Ther. 2016;42(2):299–312.

    Article  PubMed  Google Scholar 

  119. Slesnick N, Erdem G, Bartle-Haring S, Brigham GS. Intervention with substance-abusing runaway adolescents and their families: results of a randomized clinical trial. J Consult Clin Psychol. 2013;81(4):600–14.

    Article  PubMed  Google Scholar 

  120. Slesnick N, Guo X, Brakenhoff B, Feng X. Two-year predictors of runaway and homeless episodes following shelter services among substance abusing adolescents. J Adolesc. 2013;36(5):787–95.

    Article  PubMed  PubMed Central  Google Scholar 

  121. Slesnick N, Guo X, Feng X. Change in parent- and child-reported internalizing and externalizing behaviors among substance abusing runaways: the effects of family and individual treatments. J Youth Adolesc. 2013;42(7):980–93.

    Article  PubMed  Google Scholar 

  122. French MT, Zavala SK, McCollister KE, Waldron HB, Turner CW, Ozechowski TJ. Cost-effectiveness analysis of four interventions for adolescents with a substance use disorder. J Subst Abuse Treat. 2008;34(3):272–81.

    Article  PubMed  Google Scholar 

  123. Waldron HB, Slesnick N, Brody JL, Turner CW, Peterson TR. Treatment outcomes for adolescent substance abuse at 4- and 7-month assessments. J Consult Clin Psychol. 2001;69(5):802–13.

    Article  CAS  PubMed  Google Scholar 

  124. Klatte R, Strauss B, Flückiger C, Färber F, Rosendahl J. Defining and assessing adverse events and harmful effects in psychotherapy study protocols: a systematic review. Psychotherapy (Chic). 2023;60(1):130–48.

    Article  PubMed  Google Scholar 

Download references

Acknowledgements

We would like to express our sincere thanks to all those who supported the HTA report N21-03 [9]. For the present systematic review, we also thank Stefan Sauerland for guidance and Natalie McGauran for editorial support. In addition, we thank the following colleagues for their involvement in various project tasks such as providing methodological advice and data extraction support: Konstanze Angelescu, Thorsten Busan, Wolfram Groß, Michael Hort, Julia Kreis, Anna Margraf, and Sandra Molnar.

Funding

IQWiG has not received any other third-party funding. IQWiG provided non-financial support for the work on this article (input by employees as well as computer hardware and software resources).

Author information

Authors and Affiliations

  1. Institute for Quality and Efficiency in Health Care (Institut für Qualität und Wirtschaftlichkeit im Gesundheitswesen, IQWiG), Im Mediapark 8, Cologne, 50670, Germany

    David Henry Seidel, Martina Markes, Ulrich Grouven, Claudia-Martina Messow, Wiebke Sieben, Marco Knelangen, Sebastian Grümer, Heike Kölsch, Mandy Kromp & Markus von Pluto Prondzinski

  2. Family Therapy Centre (Familientherapeutisches Zentrum gGmbH, FaTZ), Hermann-Walker-Straße 16, 69151, Neckargemünd, Germany

    Rieke Oelkers-Ax

Authors
  1. David Henry Seidel
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Martina Markes
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Ulrich Grouven
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Claudia-Martina Messow
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Wiebke Sieben
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Marco Knelangen
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Rieke Oelkers-Ax
    View author publications

    You can also search for this author in PubMed Google Scholar

  8. Sebastian Grümer
    View author publications

    You can also search for this author in PubMed Google Scholar

  9. Heike Kölsch
    View author publications

    You can also search for this author in PubMed Google Scholar

  10. Mandy Kromp
    View author publications

    You can also search for this author in PubMed Google Scholar

  11. Markus von Pluto Prondzinski
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

DHS, MvPP, MM, MKn and UG were involved in the design of the systematic review. MKn developed and conducted the literature search. MvPP, SG and DHS performed the title and abstract screening, full-text screening, and the inclusion and exclusion of articles (if necessary, final inclusion was checked by MM and ROA). Data were extracted by HK, DHS, MvPP, MKr and SG. UG, CMM and WS performed the statistical analyses and checked data extraction. MvPP and DHS performed assessments of study quality and risk of bias, which were checked by UG, CMM or WS. DHS and MvPP prepared the first draft of the manuscript. All authors interpreted the data and revised the subsequent drafts. All authors reviewed and approved the final version.

Corresponding author

Correspondence to David Henry Seidel.

Ethics declarations

Ethics approval and consent to participate

Not applicable.

Consent for publication

Not applicable.

Competing interests

ROA received financial support from IQWiG for external advice on HTA report N21-03 [9], which is the basis for the present systematic review. All other authors declare that they have no competing interests.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary Information

Supplementary file 1

: Additional file 1. PRISMA 2020 Checklist. Additional file 2. Search strategies. Additional file 3. Study pool of included RCTs. Additional file 4. Characteristics of included RCTs. Additional file 5. Risk-of-bias assessment. Additional file 6. All results.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Seidel, D.H., Markes, M., Grouven, U. et al. Systemic therapy in children and adolescents with mental disorders: a systematic review and meta-analysis. BMC Psychiatry 24, 125 (2024). https://doi.org/10.1186/s12888-024-05556-y

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1186/s12888-024-05556-y

Keywords

BMC Psychiatry

ISSN: 1471-244X

Contact us