- Research article
- Open Access
- Open Peer Review
Evaluation of an interaction-skills training for reducing the burden of family caregivers of patients with severe mental illness: a pre-posttest design
BMC Psychiatryvolume 18, Article number: 84 (2018)
Family members who care for patients with severe mental illness experience emotional distress and report a higher incidence of mental illness than those in the general population. They report feeling inadequately prepared to provide the necessary practical and emotional support for these patients. The MAT training, an Interaction-Skills Training program (IST) for caregivers, was developed to meet those needs. This study used a single-arm pretest-posttest design to examine the impact of the training on caregivers’ sense of competence (self-efficacy) and burden.
One hundred family caregivers recruited from three mental health institutions participated in the training. Burden was assessed using the Involvement Evaluation Questionnaire, and self-efficacy using the Self-Efficacy Questionnaire. Analysis of variance with repeated measures was used to investigate whether participation in the training changed the level of family caregivers’ burden and self-efficacy. Pearson’s correlation was used to examine the relationships between self-efficacy and burden.
Our results indicate that, after the training, self-efficacy increased significantly over time (p < 0.001) and that burden decreased significantly (p < 0.001). However, the results could not demonstrate the expected association between an increase of self-efficacy and decrease of burden. Caregivers expressed high appreciation for the training.
After following the IST program, family caregivers of patients with severe mental illness experienced a greater sense of competence and a significant decrease in burden. The training was greatly appreciated and satisfied caregivers’ need to acquire the skills required in complex caregiving situations.
This study was retrospectively registered (14/01/2018) in the ISRCTN registry with study ID ISRCTN44495131.
A significant proportion of individuals with a severe mental illness (SMI) are unable to manage the demands of daily life. This is mainly caused by their symptoms and the associated impairments of cognitive and social skills. Informal caregivers therefore play an indispensable role in supporting them in their daily activities .
As Ostman & Hansson  indicated, approximately 90% of individuals with a severe mental illness are supported practically and emotionally by family caregivers on an almost daily basis. But as many caregivers are not properly prepared for a role as informal caregiver, they experience a great deal of psychological strain and a sense of burden [3, 4]. Up to 50% of informal caregivers report significantly more psychological distress and a higher incidence of mental illness than people in the general population . They report in particular feelings of depression, distress, fatigue and trouble sleeping. The burden they experience also intrudes upon their social lives in ways that makes them feel trapped in their caregiving role and isolated from society .
Even though previous studies made it clear that patients’ challenging behavior causes caregivers to experience substantial distress and health problems, effective support strategy for reducing the subjective burden are limited . Current interventions to support family caregivers are based mainly on psycho-education . But as caregivers lack the skills and confidence they need to deal effectively with patient’s emotions and behavior , they also need practical training to increase their interaction and communication skills [9, 10].
There is limited scientific evidence regarding the effects of broader psychosocial intervention strategies aimed at the reduction of caregivers’ burden. For example, Martire, Lustig, Schulz, Miller and Helgeson  described the positive effects of behavioral family therapy on caregiver burden, depression and anxiety. The psychosocial intervention in this therapy targeted the patient’s closest family member or relative. Smeerdijk and colleagues found that the combination of family motivational interventions (MI) and the IST was effective in reducing family members’ worrying and burden . A systematic evaluation by Signe & Elmståhl  showed that 90% of family caregivers expressed great satisfaction with respect to psychosocial interventions intended to meet caregiver’s needs. Caregivers seemed to benefit from the network that was created when information and experiences were shared. There is also evidence that caregivers who were exposed to family interventions targeting their caregiver’s needs seemed less disturbed by a patient’s disruptive behavior, because they had learned to attribute it to the illness and not to the patient’s personality .
One explanatory variable with respect to the acquisition of these skills and reduction of caregivers’ burden is self-efficacy , which concerns people’s beliefs that they can produce the desired effects through their actions . Self-efficacy is seen as a crucial determinant of emotional reactions and burden. Research indicates that lower self-efficacy is associated with greater subjective burden .
Late last century, we used available knowledge about burden, self-efficacy and behavioral family interventions to develop the MAT training, being an Interaction Skills Training program (IST) for informal caregivers that would serve the needs of family caregivers. In the present study, we investigated the impact of this training program on the levels of self-efficacy and burden experienced by caregivers caring for a family member with a severe mental illness. We hypothesized that the training was effective in increasing caregivers’ self-efficacy and reducing their burden. Hereinafter, the training will be referred to as “the IST program”.
We used a pretest-posttest design to examine the effect of the IST program on self-efficacy and burden, which were measured on three occasions: at T0 (baseline), T1 (after the training) and T2 (three months after termination of the training). The third measurement also included a brief evaluation of the caregivers’ perspective on the training.
The training program was offered at three mental health institutions in various parts of the Netherlands. Within these three hospitals, there was an open registration for family members to participate in the training. The training was announced through the mental health professionals, local media, or a brochure. The inclusion criteria were as follows: being a family member of, and caring for, a patient with a severe mental illness, defined as not being free of symptoms, having had the mental illness in the long term (> 2 years), and having serious limitations in personal and social functioning . Informed consent was obtained from all participants and confidentiality was assured. One hundred primary caregivers participated in the training, but, due to incomplete data, only 75 were included in the analyses. Twenty-five participants were excluded from the analysis because of missing data at one of the three measurements T0, T1 or T2.
The MAT training
The history of MAT started in 1994 when the development of interaction-skills training programs for use in the mental health sector was initiated by Ypsilon, a Dutch advocacy organization committed to family members and close relatives of people with psychosis. These skill-based programs were intended to serve as a tool to improve the interaction and communication between professionals/informal caregivers and patients with psychosis .
Successful versions of the MAT training have since evolved into the current MAT program, which we used in our study. Its purpose is to provide family caregivers with various interaction skills that can support them in their caregiving role. The MAT is used as a tool to demonstrate through role-play whether or not one is having a problem with another person’s behavior. This is done either by standing on the green section of a mat, which stands for ‘cooperation area’, or on the red section, which stands for ‘problem area’. Either a fellow family member or a MAT trainer takes the position of the patient in the role-plays. In the training, the mat is used to practice several interaction skills that are important in handling difficult confrontations between a family member and a patient, i.e. when one or both of them are standing on the red section of the mat, see Fig. 1.
The complete training comprises seven 3-h sessions over a 10-week period. Our training was taken by a total of eleven groups, with a minimum of eight and a maximum of sixteen participants. It was provided by certified MAT trainers together with trained professionals from a range of disciplines, who also had personal experience of caring for a family member with a severe mental illness. An overview of the training sessions is provided in Table 1. The sessions have a structure that gradually adds new interaction elements to the training with increasing difficulty of the interaction process. The sessions were accompanied by homework assignments with instructions to practice the specific skills that were central in the successive sessions. The participants’ experiences when executing the homework assignments were discussed at the start of the following session.
Self-efficacy was measured with the Self-Efficacy Questionnaire (SEQ). The SEQ is a questionnaire developed by our research group to measure the extent to which caregivers find themselves competent to interact effectively with a family member with a mental illness. The development process of the SEQ involved the following steps. First, a panel of nine experts, both researchers and MAT trainers, compiled an item pool with items referring to essential competencies of caregivers to effectively interact with the patient with severe mental illness. Next, these items were clustered into meaningful units of caregiver competences. The nine experts participated in four commentary rounds to develop the first version of the questionnaire. This first version of the SEQ was then presented to Ypsilon and five MAT trainers for review with respect to content, formulations and construction. In the last stage, nine caregivers completed the latest version to test its comprehensibility and feasibility. The final version of the SEQ consists of 23 items that are rated on a 4 point Likert scale ranging from ‘1 = probably not’ to ‘4 certainly’. Examples of items are: ‘I think I’m able to make the difference in situations when patient is not capable or not willing to do something’. Or: ‘I think I’m able to align my own behavior to the patient’s inabilities’. In our sample, the internal consistency of the SEQ was very high, i.e. 0.94. The SEQ is currently subject to further psychometric evaluation, whose results will be published elsewhere.
The Involvement Evaluation Questionnaire (IEQ) measures the degree of subjective burden of caregivers of people with schizophrenia, depression and mixed psychiatric disorders . It covers all domains of burden relevant to caregivers. It was shown to be valid, reliable, easy to understand, not time consuming, and sensitive to changes . With a high degree of internal consistency, ranging from 0.74 to 0.85 for the subscales and 0.90 for all items of the instrument, the IEQ has proven to have satisfactory reliability .
The original IEQ consists of seven modules. As the second module represents the core module, it was the only module used for this study. The questions concern the burden experienced in the preceding 4 weeks. The items are measured on a 5-point Likert-scale ranging from ‘1 = never’ to ‘5 = (almost) always’. The module contains 31 items which are distributed over the following four subscales: feelings of tension (9 items), which refer to mental or emotional strain; supervision (6 items), which refer to caregiver’s tasks of guarding the patient’s needs; worrying (6 items), which covers feelings of concern and unease; and urging (8 items), which relates to the patient’s activation and motivation to undertake certain activities . The items 28 and 44 are used in more than one scale.
To evaluate the appreciation of the training, we developed a short survey with 11 statements referring to the central aims and components of the training (see Table 4). These items are rated on a 4 point Likert scale, ranging from 1 (‘not important’ to 4 (‘very important’). In addition, the respondents were requested to score the overall quality of the training regarding content and trainers (10 point scale: 1 = low; 10 = high).
The Analysis of Variance with Repeated Measures (RM-ANOVA) was used to examine the main hypothesis, i.e. whether the training had a significant effect over time on family caregivers’ burden and self-efficacy.
Next, on the assumption that an increase in self-efficacy would result in a decrease in burden, we used linear regression analysis to evaluate the relationship between the increased self-efficacy and decreased burden. Any changes in self-efficacy and burden were calculated using delta scores (endpoint score minus baseline score). Additional exploratory analyses were conducted to investigate the relationship between increased self-efficacy and the different subscales of burden. Finally, we evaluated the relationship between caregivers’ appreciation of the training and any changes of burden and self-efficacy.
The SPSS Statistics program version 20.0  was used for all the analyses. A p value of less than 0.05 was considered statistically significant.
A total of 75 participants were included in the analyses, 25 (33%) of them male and 50 (67%) of them female. Their age range was 25 to 78 years (Mean = 55.4, SD = 11.1). Forty-eight percent had a lower level of education; 42% had a medium level (bachelor level); and 10% had attended higher education (master level).
The effect for caregivers’ self-efficacy and burden over time
Mauchly’s Test of Sphericity indicated that the assumption of sphericity (i.e., homogeneity of variance) had not been violated for self-efficacy (χ2(2) = 5.34, p = 0.069), but had been violated for burden (χ2(2) = 30.74, p < 0.001). We therefore corrected degrees of freedom using Lower-Bound estimates of sphericity (ε = 0.74) for burden. The results provide good support for the main hypotheses, showing a significant increase in participants’ self-efficacy after the training (F(2,134) = 33.09, p < 0.001) and a significant decrease in their burden (F(1,70) = 21.37, p < 0.001) (Table 2). Figure 2. illustrates these effects. The effect sizes for these analyses (d = − 0.71 for self-efficacy, d = 0.46 for burden) were classed as large to moderate, in accordance with existing guidelines for interpreting the effect sizes, as offered by Cohen .
Correlations between self-efficacy and burden
Pearson’s correlation was used to assess the relationship between the change in self-efficacy and the change in level of burden, including the changes in the subscales of burden (see Table 3). There was no significant correlation between the change in self-efficacy and change in burden (r = 0.03, n = 70, p = 0.78). Neither did additional analyses of the relationship between change in self-efficacy and change in the subscales of burden show any significant correlation (see Table 3).
Evaluation of the training
The participants expressed great appreciation for the training: on the 10 point scale, the mean score for training content was 8.4 (SD = 0.73); for the trainers it was 8.6 (SD = 0.85). There were no significant associations between their appreciation of the training and their decrease in burden (r(75) = − 0.102, p = 0.384), and their increase in self-efficacy (r(75) = 0.092, p = 0.432). With regard to appreciation of the different components of the training, the highest average score was for ‘Determining whose problem it is: mine or his/hers’ (mean = 3.37, SD = 0.56). Table 4 shows the complete list of components, ranked from the highest appreciation to the lowest.
In this study, we found that the caregiver’s level of self-efficacy after the IST had increased, and that their level of burden had decreased. We may conclude that the training program led to meaningful changes in caregiver’s lives. Our results confirm the findings of an earlier study  where families who completed the IST reduced burden effectively.
This study did not support Solomon and Draine’s hypothesis that higher self-efficacy is associated with decreased subjective burden . Neither was self-efficacy associated with either of the subscales representing burden. One possible explanation for this is that the great reduction in burden may have been due partly to the supporting network created between caregivers who had to cope with the difficulties, see also Signe & Elmståhl . Another possible explanation is that the follow-up period in our study was too short to assess an association between the increase in self-efficacy and decrease in burden. This effect may develop in later stages when the self-efficacy skills learned have been applied for longer. Caregivers in previous successful versions of the MAT training reported not having enough time to practice the skills they learnt during the training; 75% of them asked for follow-up training . The results of a longer follow-up period should either verify or falsify the hypothesis.
As the evaluation of caregiver’s participation in the training shows, the caregivers regarded the interaction skills training program as very successful, expressing great satisfaction with both its content and the trainers. These results are consistent with the findings of Signe & Elmståhl . With regard to the various training components in our study, caregivers reported great appreciation for ‘determining whose problem it is: mine or his/hers’, ‘dealing with the handicaps’ of the person involved and ‘insight into the handicap’. They also reported feeling more confident about the appropriate things to say, and more competent in ensuring that elaborative conversations did not turn into endless circular discussions. These findings point out the importance of interventions that train caregivers how to effectively manage the caregiver situation. This is consistent with previous research that examined ‘a sense of mastery’ as a caregiving concern  and how successful daily interactions between family member and patient were a result of the IST .
This study has two main strengths. First, the results of our study regarding self-efficacy and burden, in combination with the caregivers’ great appreciation of the program, provides evidence of the effectiveness of interactive interventions targeting caregiver self-efficacy and burden. As Rose et al.  claimed that providing information and support alone is not enough to improve caregivers’ understanding of the mental illness; nor does it increase their sense of competence and confidence in their caregiving ability. Given the current trend in mental health care towards more ambulatory treatment  with more caring responsibilities for family members, mental health care should rely less on psycho-educational interventions alone, and more on interaction training programs that include psycho-educational components.
The second strength of our study is that the medium to large effect sizes implicate reliable reduction in burden and improvement of self-efficacy in practical terms for caregivers’ daily life.
Our study also has some limitations. The first was our use of a non-experimental design. Therefore, we could not identify desirable between-group effects by contrasting our results with those of a control group. However, in view of the long-term distress experienced by the family members, spontaneous increase of self-efficacy and decrease of burden over a period of weeks is not very likely . The second limitation is that our findings are based on non-probability sampling. As the caregivers were required to enroll themselves in the training, there may have been selection bias. Similarly, the fact that caregivers were informed about and recruited for the training in various ways may have affected their motivation for signing up for and participating in the training. The third limitation was the relatively small sample size caused by incomplete data; roughly a quarter of our participants were excluded from analyses. Fourth, as there is no follow-up data, we cannot examine the effect over a longer period. Finally, although SEQ showed high face validity, good feasibility and high internal consistancy, further examination of the psychometric properties of the Self-Efficacy Questionnaire (SEQ) is necessary. This psychometric research is currently in progress.
The main purpose of the IST program is to train caregivers in using effective communication and interactive skills when dealing with patients who frequently behave disruptively. After taking the IST program, family caregivers of patients with severe mental illness experienced a greater sense of competence (self-efficacy) and a significant decrease in burden. The training, combining psycho-education and interactive skills, was greatly appreciated, and satisfied the caregivers’ need to acquire the skills they required in complex caregiving situations.
Bentur N, Moualem A. The effect on family members of treating home hospitalized patients. Harefuah. 2001;140(5):386–91.
Ostman M, Hansson L. Family burden and care participation: a test-retest reliability study of an interview instrument concerning families with a severely mentally ill family member. Nordic Journal of Psychiatry. 2000;54:327–32.
Chang KH, Horrocks S. Lived experiences of family caregivers of mentally ill relatives. J Adv Nurs. 2006;53(4):435–43.
Rose L, Mallinson RK, Walton-Moss B. A grounded theory of families responding to mental illness. West J Nurs Res. 2002;24(5):516–36.
Shah AJ, Wadoo O, Latoo J. Psychological distress in carers of people with mental disorders. British Journal of Medical Practitioners. 2010;3(3):18.
Zarit SH, Reever KE, Bach-Peterson J. Relatives of the impaired elderly: correlates of feelings of burden. Gerontologist. 1980;20(6):649–55.
Carter PA. Family caregivers’ sleep loss and depression over time. Cancer Nurs. 2003;26(4):253–9.
Schulze B, Rössler W. Caregiver burden in mental illness: review of measurement, findings and interventions in 2004–2005. Current Opinion in Psychiatry. 2005;18(6):684–91.
Rose LE, Mallinson RK, Gerson LD. Mastery, burden, and areas of concern among family caregivers of mentally ill persons. Arch Psychiatr Nurs. 2006;20(1):41–51.
Given B, Sherwood PR, Given CW. What knowledge and skills do caregivers need? AJN The American Journal of Nursing. 2008;108(9):28–34.
Martire LM, Lustig AP, Schulz R, Miller GE, Helgeson VS. Is it beneficial to involve a family member? A meta-analysis of psychosocial interventions for chronic illness. Health Psychol. 2004;23(6):599.
Smeerdijk M, Keet R, van Raaij B, Koeter M, Linszen D, de Haan L, Schippers G. Motivational interviewing and interaction skills training for parents of young adults with recent-onset schizophrenia and co-occurring cannabis use: 15-month follow-up. Psychol Med. 2015;45(13):2839–48.
Signe A, Elmståhl S. Psychosocial intervention for family caregivers of people with dementia reduces caregiver's burden: development and effect after 6 and 12 months. Scand J Caring Sci. 2008;22(1):98–109.
Zeiss AM, Gallagher-Thompson D, Lovett S, Rose J, McKibbin C. Self- efficacy as a mediator of caregiver coping: development and testing of an assessment model. Journal of Clinical Geropsychology. 1999;5(3):221–30.
Bandura, A. (1994). Self-efficacy. New York City, United States: John Wiley & Sons, Inc.
Solomon P, Draine J. Subjective burden among family members of mentally ill adults: relation to stress, coping, and adaptation. Am J Orthopsychiatry. 1995;65(3):419.
Delespaul PH, De Consensusgroep EPA. Consensus over de definitie van mensen met een ernstige psychische aandoening (epa) en hun aantal in Nederland. Tijdschrift voor Psychiatrie. 2013;55(6):427–38.
Kuipers T, Kuipers-van Bekkum N. Training helpt in dagelijkse omgang. Ypsilon Nieuwsbrief. 2002;100:10–1.
Schene AH, van Wijngaarden B. Family members of people with a psychotic disorder; a study among members of Ypsilon. Amsterdam: Department of Psychiatry, University of Amsterdam; 1993.
Wijngaarden BV, Schene AH, Koeter MWJ, Vazquez-Barquero JL, Knudsen HC, Lasalvia A, McCrone P. Caregiving in schizophrenia: development, internal consistency and reliability of the involvement evaluation Questionaire-European version. EPSILON study 4. Br J Psychiatry. 2000;177(S39):21–7.
IBM Corp. (2011). Released 2011. IBM SPSS statistics for windows, version 20.0. Armonk, NY: IBM Corp.
Cohen, J. (1988). Statistical power analysis for the behavioral sciences. Hillside, NJ: Lawrence Earlbaum Associates.
Bond GR, Drake RE, Mueser KT, Latimer E. Assertive community treatment for people with severe mental illness. Disease Management and Health Outcomes. 2001;9(3):141–59.
Centrale Commissie Mensgebonden Onderzoek (CCMO), http://www.ccmo.nl/en/(2015).
We would like to thank all family members for their participation in this study.
No funding was obtained for this study.
Availability of data and materials
Upon reasonable request, the datasets used and/or analysed during the current study will be provided by the corresponding author.
Ethics approval and consent to participate
Under Dutch law, no medical ethical approval was needed for our study, see CCMO . According to the Medical Research Involving Human Subjects Act (WMO), research falls under the scope of the WMO if:
It concerns medical-scientific research and;
Participants are subject to procedures or are required to follow rules of behavior.
Our study does not concern medical-scientific research, since it is an evaluation study among family members (non-patients) following a training program that is offered as care as usual in several Dutch mental health institutions over the past years.
This was confirmed by the Pro Persona Research Department (February 2016).
After receiving verbal and written information on the study, all participants / family members signed for informed consent.
Consent for publication
The authors declare that they have no competing interests.
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.