Quality of care and its determinants in longer term mental health facilities across Europe; a cross-sectional analysis
BMC Psychiatry volume 16, Article number: 31 (2016)
The Quality Indicator for Rehabilitative Care (QuIRC) is an international, standardised quality tool for the evaluation of mental health facilities that provide longer term care. Completed by the service manager, it comprises 145 items that assess seven domains of care: living environment; treatments and interventions; therapeutic environment; self-management and autonomy; social interface; human rights; and recovery based practice. We used the QuIRC to investigate associations between characteristics of longer term mental health facilities across Europe and the quality of care they delivered to service patients.
QuIRC assessments were completed for 213 longer term mental health units in ten countries that were at various stages of deinstitutionalisation of their mental health services. Associations between QuIRC domain scores and unit descriptive variables were explored using simple and multiple linear regression that took into account clustering at the unit and country level.
We found wide variation in QuIRC domain scores between individual units, but across countries, fewer than a quarter scored below 50 % on any domains. The quality of care was higher in units that were smaller, of mixed sex, that had a defined expected maximum length of stay and in which not all patients were severely disabled.
This is the first time longer term mental health units across a number of European countries have been compared using a standardised measure. Further use of the QuIRC will allow greater understanding of the quality of care in these units across Europe and provide an opportunity to monitor pan-European quality standards of care for this vulnerable patient group.
The European Commission’s Green Paper on Mental Health in 2005  specifically highlighted the need to improve the social inclusion of mentally ill people and to protect their fundamental rights and dignity. It acknowledged the stigma and prejudice they often face and the evidence for better quality of life and improved social inclusion that has arisen from the deinstitutionalisation of mental health services in many countries. However, it also acknowledged that in some Member States, institutions still account for a large share of the mental health services infrastructure and, in many countries are poorly covered by existing health monitoring systems. It highlighted the need to harmonize indicators and information about these facilities across the EU, such that recommendations for best practice in promotion of social inclusion and the protection of human rights of people with longer term mental health problems could be made. The DEMoBinc study was funded by the European Commission  to develop an international toolkit, the Quality Indicator for Rehabilitative Care (QuIRC), to assess the quality of care in psychiatric and social care facilities for adults with longer term mental health problems. It is one facet of the work needed in the EU’s agenda for quality assurance. The development of the QuIRC has been described elsewhere .
In summary, evidence from the results of a systematic literature review , international Delphi exercise  and review of care standards in each of the ten participating countries was triangulated to derive items for the QuIRC. Items were included if there was evidence of their ability to support the recovery of individuals with longer term mental health problems. The QuIRC comprises 145 items of which 86 are used to assess seven domains of care: the Living (built) Environment; the Treatments and Interventions provided, including the use of restraint and seclusion; the Therapeutic Environment (culture of the unit); the promotion of patients’ Self-management and Autonomy; the promotion of their Social Interface with the community and family/carers; the protection of their Human Rights; the implementation of Recovery Based Practice. The remaining items provide descriptive data. The QuIRC is available as a web based application (www.quirc.eu) in the ten languages of the countries that participated in its development. Results are presented in a printable report showing the unit’s performance on each domain as a percentage, on a “spider web” diagram, which also shows the average performance for similar units in the same country. Higher domain scores denote better quality of care in each domain. Although it is known that community based units tend to have a more homely environment [4, 6], whereas inpatient units tend to have a greater proportion of service users with more complex needs who are detained involuntarily, the impact on the quality of care of other characteristics of longer term mental health facilities is not known. Therefore, our aim was to explore associations between the characteristics of units that participated in the DEMoBinc study and the quality of care as assessed by the QuIRC.
Ten European countries participated in the DEMoBinc study (Bulgaria, Czech Republic, Germany, Greece, Italy, Netherlands, Poland, Portugal, Spain and the UK), purposively selected to represent countries at different stages of deinstitutionalisation and a broad geographical spread across the European Union. Since the aim of the DEMoBinc study was to develop a quality assessment tool that could be used across hospital and community based longer term mental health care facilities in different countries, rather than to carry out a representative survey of such facilities, services meeting specific criteria were purposively identified for participation by the principal investigator in each country from personal knowledge and local registration lists. All had to provide longer term care (length of stay at least 6 months), for at least six service users living in a communal setting (i.e. not individual flats/bedsits), with staff on-site, usually 24 h a day. Units that only provided for specialist groups such as those with learning disability, degenerative brain disease or head injuries, substance misuse or dementia were excluded. Both hospital and community based units of different size and geographical location (urban/suburban/rural) were purposively recruited. Most users of these services were male, with a diagnosis of psychotic illnesses such as schizophrenia and a mean length of stay of 9 months (see Killaspy et al. , for more details). Face to face interviews with unit managers were carried out by the researchers in each country in order to complete the QuIRC. Managers were first contacted by the researchers about the study who explained its purpose and process. If they were willing, the researcher arranged to meet with them and they were sent a Participant Information Sheet. They were given at least 2 days to review this and were then able to clarify any queries with the researcher before given informed consent to participate. The study was approved by the relevant ethics committees in each of the ten participating countries involved in developing the QuIRC (Bulgaria - Ethics Committee, Alexandrovska University Hospital, Sofia; Czech Republic - General University Hospital, Ethics Committee, Prague; Germany – Ethik Kommission der Medizinischen Fakultät Carl Gustav Carus an der Technischen Universität, Dresden; Greece - University Medical Mental Health Research Institute, Athens; Italy - Comitato Etico Indipendente, Trieste; the Netherlands - Medical Ethical Committee of the University Medical Centre, Groeningen; Poland - Commission of Bioethics, Wroclaw Medical University; Portugal - Ethical Committee of the New University of Lisbon Medical School; Spain - Comisión Etica de la Universidad de Granada; UK - City and East London Multi Region Ethics Committee).
Using data from a report by Dunlap, Xin & Myers , it was calculated that with a sample size of 213 units there would be 80 % power to estimate the relationship of ten variables of a small to medium effect size of 0.33 with each domain of QuIRC at a 0.7 % significance level. This significance level takes account of the fact that seven regression models were being fitted.
The following unit characteristics were investigated for their association with QuIRC domain scores: unit type (hospital or community based); location (urban, suburban, rural); size (total number of beds); whether there was a maximum length of stay; whether the unit was single or mixed gender; the proportion of patients generally able to do very little without assistance; the proportion of patients detained involuntarily; staffing intensity (ratio of the number of full-time staff to beds); and staff turnover (the proportion of staff who had left, retired, died or been dismissed in the previous two years). Descriptive data on the input from different staff disciplines was also recorded in terms of whether the unit had access to the discipline from outside the unit or whether staff were employed within the unit, but these variables could not be included in the model due to very small numbers of staff from some disciplines in some countries.
The QuIRC domain scores were normally distributed and were each analysed as dependent variables for associations with the variables listed above. Multiple linear regression was used, with categorical independent variables entered as dummy variables. It was important to adjust for clustering of units within countries so parameters of the linear regression were computed using robust clustered standard errors, the country identifier being the cluster variable. Un-standardised regression coefficients and their 95 % confidence intervals (estimated using bootstrapping) are presented for all parameter estimates.
Table 1 shows the units’ characteristics. The QuIRC was completed in 213 units across 10 European countries. Around half of these units were in the inner city, and two-thirds were community based facilities. The size of units varied greatly (IQR from 12 to 35 patients). The majority (n = 172, 81 %) did not have a maximum length of stay, but where present this was usually 2 years. One quarter of units were single sex. The majority of units (59 %) had no detained patients, while in a small number (n = 14, 7 %) more than 50 % of patients were detained. The functional impairment of patients varied, with about one quarter of units having no patients who were able to do very little without assistance, but in 23 (8 %) units the majority of patients required assistance with most things.
All units employed nurses and support workers. Two-thirds of units employed a psychiatrist, but only half employed a clinical psychologist, only half employed a social worker, and one-third employed a lead for vocational rehabilitation. The mean number of full time equivalent (FTE) staff to beds was 0.4 (4 staff to 10 beds) but this ranged from 0.1 to 9.1 across units. Staff turnover in the last 2 years (per unit bed) ranged from 0.0 to 4.0.
Associations between unit quality (QuIRC domain scores) and unit characteristics
Table 2 shows mean and standard deviation QuIRC domain scores by country.
Table 3 shows the association of QuIRC domain scores with unit characteristics after adjustment.
The Living Environment domain had an overall mean score across countries of 59.8 %, (mean range 73.8 % [Germany] to 46.5 % [Spain]). This was associated with the type of unit, size of unit and the percentage of patients involuntarily detained: community units scored 11.2 % more than hospital units; larger units scored lower on Living Environment, every extra bed reducing the score in this domain by 0.2 %; for every 1 % increase in detained patients the Living Environment score reduced by 0.1 %.
The Therapeutic Environment domain had an overall mean score across countries of 52.1 % (mean range 64.5 % [UK] to 45.6 % [Bulgaria]). This was associated with the type of unit, location of unit, size of unit, having a maximum length of stay and the percentage of patients able to do very little without assistance: community based units scored 3.4 % lower than hospital units; rural units scored 3.3 % lower than those based in urban areas; larger units scored lower, every extra bed reducing the Therapeutic Environment domain score by 0.1 %; units with a maximum length of stay scored 9.0 % higher than those without; every 1 % increase in patients able to do very little without assistance was associated with a reduction in the Therapeutic Environment domain score of 0.1 %.
Treatments and interventions
The Treatments and Interventions domain had an overall mean score across countries of 50.6 % (mean range 59.5 % [UK] to 46.2 % [Poland]). This was associated with whether the unit had a maximum length of stay and the percentage of patients able to do very little without assistance: those with a maximum length of stay scored 6.0 % more; for each 1 % increase in patients able to do very little without assistance, this domain score reduced by 0.1 %.
Self-management and autonomy
The Self-Management and Autonomy domain had an overall mean score across countries of 55.5 % (mean range 71.9 % [Germany] to 44.1 % [Poland]). This was associated with the size of unit, whether it was single or mixed gender, the percentage of patients able to do very little without assistance and the percentage of patients detained: for every additional bed this domain score reduced by 0.2 %; single sex units scored 8.6 % lower than mixed gender units; for each 1 % increase in patients able to do very little without assistance, this domain score reduced by 0.2 %; for every 1 % increase in detained patients the score reduced by 0.1 %.
The Social Interface domain had an overall mean score across countries of 48.5 % (mean range 59.5 % [Spain] to 40.1 % [Poland]). This was associated with the type, size and location of the unit, having a maximum length of stay and the percentage of patients able to do very little without assistance. Community based units scored 7.8 % lower than hospital units; for every additional bed this domain score reduced by 0.2 %; units with a maximum length of stay scored 6.7 % higher than those without; and for every 1 % increase in patients able to do very little without assistance, this domain score reduced by 0.2 %.
The Human Rights domain score had an overall mean across countries of 56.7 % (range 70.8 % [Netherlands] to 48.1 % [Italy]). Scores were associated with whether the unit was mixed or single gender and the level of disability of the patients: single sex units scored 7.9 % lower than mixed gender units; and for every 1 % increase in patients able to do very little without assistance, this domain score reduced by 0.1 %.
Recovery based practice
The Recovery Based Practice domain had an overall mean score across countries of 52.7 %, (mean range 65.9 % [UK] to 44.2 % [Portugal]). This was associated with the location of the unit, having a maximum length of stay, whether single or mixed gender, and the level of disability of the patients: units in rural areas scored 3.7 % lower than those based in urban areas; units with a maximum length of stay scored 5.8 % higher than those without; single gender units scored 4.8 % less than mixed gender units; and for every 1 % increase in patients able to do very little without assistance, this domain score reduced by 0.2 %.
Although there was wide variation in QuIRC domain scores between individual units, across countries, only 25 % of units scored less than 50 % on individual QuIRC domain scores. We found that a number of characteristics of units were associated with the quality of care delivered. The characteristic that had an influence on the largest number of domains was the level of disability of the patients; units with a higher proportion of poorer functioning patients scored lower on six of the seven QuIRC domains (all except Living Environment). This suggests that units that are managing a majority of people with high levels of disability may struggle to deliver better quality services and therefore having patients with a range of functioning may be important. However, since the number of units surveyed in most countries was relatively small (see below), we do not know whether this is an option as we do not know whether there is a range of disability of the patients in longer term units in different countries.
Whether the unit had a maximum length of stay was positively associated with four of the seven domains (Therapeutic Environment, Treatments and Interventions, Social Interface, and Recovery Based Practice). Having an expected length of stay is likely to help clarify the aims of treatment and instil a sense of hope in patients and staff. Units that focus on patients’ recovery and move on within a defined timeframe may make more effort to build links and informal support networks for patients in the local community and with their family and carers, and thus develop a more therapeutically optimistic environment. However, this variable is dependent on there being somewhere suitable for patients to move on to. It may be that a lack of appropriately supported community accommodation in many countries was one of the reasons why relatively few units had a maximum length of stay.
Larger units had poorer quality scores for four of the seven QuIRC domains (Living Environment, Therapeutic Environment, Self-Management and Autonomy, and Social Interface). This association might be explained in cases where units had previously been part of a large psychiatric hospital (prior to “deinstitutionalisation”) and perhaps continued the culture of the older institutions, promoting less autonomy of their patients. This finding also suggests that the risk of poorer quality, more institutional practice is greater in larger units and therefore the size of units is an important factor for policy makers and service planners to take into consideration when developing contemporary services that aim to promote individuals’ independence and community discharge.
Single sex units scored lower on three domains (Self-Management and Autonomy, Human Rights and Recovery Based Practice) which suggests that mixed gender units are preferable (wherever clinically appropriate).
The location of the unit did not seem to have a major influence on quality; however, those based in rural areas scored lower on the Therapeutic Environment and Recovery Based Practice domains, perhaps due to isolation and lack of access to other services.
The type of unit had the largest positive effect on any single domain, with community units scoring up to 11 % higher for “Living Environment” compared to hospital units. However, this characteristic was negatively associated with the Therapeutic Environment and Social Interface domain scores. These findings therefore only partially concur with previous studies that have reported that community based units provide a more homely and therapeutic environment that facilitates greater community access than hospital based facilities [4, 6].
We did not find that staffing intensity or staffing turnover were associated with any domain scores. However, we were unable to investigate the association of specific professions with the quality of care provided.
Strengths and limitations
The sampling of units was not random and thus we cannot say that those surveyed are representative of the countries that took part. The units were invited to participate in a study to develop a quality indicator for assessing longer term residential mental health care, and though there were no eligibility criteria based on performance, it is possible that selection bias occurred. Units were known to the principal investigators, and may have represented above (or below) average quality for that country. Furthermore, given only 20 units participated in each country, we had insufficient power to make comparisons between countries. Although our results give a snapshot of provision for those with severe mental health problems requiring longer term care across Europe, it is important to stress that in some countries, such as Portugal, almost all relevant units were included in the sample, whereas in other countries, such as Germany, Italy and the UK, the sample comprised only a small proportion of units in one geographic area. A major strength of our study was the use of the QuIRC to assess quality. Not only has it been shown to be a reliable instrument , but it has excellent content validity since the individual items included were identified from triangulation of three different evidence sources. Thus it provides a standardised measurement of quality of care provided in the units studied, rather than a more subjective or value judgement. Our approach to the analysis was strengthened by taking account of clustering of units within a country and ensuring that the number of variables included in our regression analysis was appropriate for our sample size. Furthermore, we have previously found little positive association between service user characteristics and quality of care in these same longer term mental health facilities and in a national survey of longer term inpatient mental health units in England, so we are confident that the range of quality we found was not determined by service user variables [7, 9].
The results of the study have important implications for policy and organisation of services. They show that the average quality of units for longer term patients in the ten European countries involved in the study, although not excellent, was quite reasonable: they scored more than 50 % in six of the seven domains and 48.5 % in the seventh domain. The results also show that only a quarter scored less than 50 % on individual QuIRC domain quality scores, including the Living Environment, Self-management and Autonomy, and Recovery Based Practice. This seems to suggest that better quality care is being provided than that which prevailed in European psychiatric institutions in the past, though of course we have no QuIRC data for the older institutions. However, across all domains there are still units with very low quality scores, indicating that a lot of work still remains to be done, both at the country and at the EU level.
Bearing these caveats in mind, our results are able to provide tentative guidance for those planning longer term mental health facilities. While this study cannot suggest an ideal size of unit, smaller units appear to have an advantage in terms of quality of care provision. Units situated in the community had considerably better quality scores for their living environment, and should, whenever possible, be provided rather than hospital based units. Having a larger proportion of patients with greater need was negatively associated with a number of quality domains and units should therefore avoid having only patients with severe disability or those who are detained, and instead try to work with people with a range of functioning. Finally, our results on gender would suggest that mixed sex units are to be preferred.
There are many possible socioeconomic reasons for the differences we found in domain scores across European countries. These include variability in the spend on mental health services; the history of service development; the level of service utilisation; the dispersal of settings for mental health care, including the criminal justice system; cultural norms about individualisation, self-management and family support; and the different expectations of the general public in different countries . Ongoing collection of QuIRC data can provide an opportunity to collect and monitor national quality benchmarking for these services.
Two areas cannot be addressed from this study. Firstly, we do not know which of the QuIRC domains is most important in terms of patient experience of care and clinical outcomes. The identification of critical success factors for this group of patients would obviously help guide the organisation and management of services. Secondly, although the QuIRC provides an accessible and free tool that allows units to review their performance against national averages (www.quirc.eu), it is not yet known what interventions are feasible and effective in different contexts in helping staff to improve the quality of care they offer. Further study is required in both these areas.
This is the first time longer term mental health units across a number of European countries have been compared using a standardised measure. The collection of QuIRC data constitutes the beginning of a database across Europe that will allow continuing assessment of quality of care in these units. It will also contribute to the development of realistic targets and timescales for the development of minimum quality of care standards for this stigmatised and vulnerable group of patients.
European Commission. Green Paper: Improving the Mental Health of the Population: Towards a Strategy on Mental Health for the European Union. Brussels: European Commission, Health and Consumer Protection Directorate-General; 2005. Ec.europa.ey/health/phdeterminants/lifestyle/mental/greenpaper/mentalgpen.pdf.
Killaspy H, King M, Wright C, White S, McCrone P, Kallert T, et al. Study protocol for the development of a European measure of best practice for people with long term mental health problems in institutional care (DEMoBinc). BMC Psychiatry. 2009;9:36.
Killaspy H, White S, Wright C, Taylor T, Turton P, Schützwohl M, et al. The development of the Quality Indicator for Rehabilitative Care (QuIRC): a measure of best practice for facilities for people with longer term mental health problems. BMC Psychiatry. 2011;11:35.
Taylor T, Killaspy H, Wright C, Turton P, White S, Kallert T, et al. A systematic review of the international published literature relating to quality of institutional care for people with longer term mental health problems. BMC Psychiatry. 2009;9:55.
Turton P, Wright C, White S, Killaspy H, Taylor T, Onchev G, et al. Promoting recovery in long term mental health institutional care: an international Delphi study of stakeholder views. Psychiatr Serv. 2010;61(3):293–9.
Brunt D, Hansson L. A comparison of the psychosocial environment of two types of residences for persons with severe mental illness: small congregate community residences and psychiatric inpatient settings. Int J Soc Psychiatry. 2002;48(4):243–52.
Killaspy H, White S, Wright C, Taylor T, Turton P, Kallert T, et al. Quality of longer term mental health facilities in Europe: validation of the quality indicator for rehabilitative care against service users’ views. PLoS One. 2012. doi:10.1371/journal.pone.0038070.
Dunlap W, Xin X, Myers L. Computing aspects of power for multiple regression. Behav Res Methods Instrum Comput. 2004;36:701.
Killaspy H, Marston L, Omar R, Green N, Harrison I, Lean M, Holloway F, Craig T, Leavey G, M. Service quality and clinical outcomes: an example from mental health rehabilitation services in England. Br J Psychiatry. 202:28-34. doi:10.1192/bjp.bp.112.114421
Caldas de Almeida JM, Killaspy H. Long-term mental health care for people with severe mental disorders. Report for the European Commission. European Union. 2011.
The study was funded by the Sixth Framework of the European Commission and the authors gratefully acknowledge their support. The authors would like to thank all the unit managers and service users who participated in the research. They would also like to acknowledge the contributions of the members of the International Expert Panel throughout the study and thank them for their valuable input: Mr Jerry Tew (social scientist, UK); social care – Mr Tony Ryan (independent consultant on out of area placements, UK), Mr Michael Clark (Care Services Improvement Partnership, UK); rehabilitation psychiatry and psychology – Professor Tom Craig (UK), Dr Frank Holloway (UK), Professor Jaap van Weeghel (Netherlands), Dr Joanna Meder (Poland), Professor Geoff Shepherd (UK); service user perspective – Mr Maurice Arbuthnott (UK), Ms Vanessa Pinfold (Rethink, UK); human right law – Associate Professor Luis Fernando Barrios-Flores (University of Granada, Spain); mental health law – Professor Peter Bartlett (Nottingham University, UK); disability rights – Ms Liz Sayce (Royal Association for Disability and Rehabilitation, UK); care standards – Dr Geraldine Strathdee (Healthcare Commission, UK). We also acknowledge the work and support of members of the Maristán network http://redmaristan.org/
The authors declare that they have no competing interests.
HK, MK, CW and SW conceived and designed the study in collaboration with GC, JMCdeA, PT, TT, MS, MSchus, JC, PB, JR, LK, GO, SA, RM, PR, DW, EV, AK, TA, DP, FG . The data were collected by GC, PT, TT, MSchus, PB, LK, SA, EV, TA, FG. The data were analysed by SW. CW made the first draft of the article which was reviewed and revised by HK, GC, JMCdeA and MK. All authors reviewed and amended the draft and agreed the final version for publication (HK, GC, JMCdeA, PT, TT, MS, MSchus, JC, PB, JR, LK, GO, SA, RM, PR, DW, EV, AK, TA, DP, FG, MK).
About this article
Cite this article
Killaspy, H., Cardoso, G., White, S. et al. Quality of care and its determinants in longer term mental health facilities across Europe; a cross-sectional analysis. BMC Psychiatry 16, 31 (2016). https://doi.org/10.1186/s12888-016-0737-5