This is the first study to evaluate the POD approach in a large NHS mental health trust, delivering a standalone POD team, as complete care to NHS mental health service users presenting in crisis within a community setting. This model was set up from scratch to test it in the NHS and this has proved possible. There were a number of barriers to this POD team model as, contrary to most NHS service models, it worked trans-diagnostically and across crisis and recovery discrete community pathways of traditional services. The POD team also in-reached to inpatient services.
We collected data from service users, network members and clinicians regarding the implementation and performance of the POD approach. Aspects investigated included; self-reported and clinician-reported clinical outcomes, satisfaction of service users and network members; and recorded broad data for the same time period between community and crisis teams. The group of service users treated were predominantly young people, mean age 35 years, slightly more males than females in an area where generally the population is white British, with a student population attracted by the local universities. The approach is organised for a shared decision making process. In health services there is an emerging focus on co-production and active citizenship in recovery [32] such that there is a strong case for enhancing the implementation of shared decision making to promote recovery. This is fundamental to POD which is democratic, collaborative and sees each person in the network meeting as an equal or potential partner in recovery. In a service user led report it was identified that most service users who took part in the project which gathered their views felt that social approaches to mental health, which take account of the whole person and wider societal issues affecting them, are the most helpful [33].
Clinical Outcomes
POD clinical outcomes are encouraging with all clinical measures of service user outcomes showing benefits in terms of recovery and function from three self-report measures and one clinician-rated measure. All four measures show improvements from the baselines score at the 3 and 6 month time point with statistically significant improvements recorded in all the outcomes. There are continual improvements for all measures from baseline to 3 months and also from 3 to 6 months. The SWEMWBS scores show there was a significant improvement in mental wellbeing between baseline and the 3 month timepoint. The significance of the 4.48 reduction in the SWEMBS score between baseline and 6 months can be viewed in that a change of between 1 and 3 points is reported to meet the thresholds for statistically important change [34]. The WASAS scores record ongoing improvement in self-reported functioning in work and social activities at each time point with the scores between baseline and 3 months being significantly different. The WASAS score at baseline was 27.28 with a score above 20 suggesting moderately severe or worse psychopathology. This contrasts with WASAS scores of between 10 and 20 (as recorded by this cohort at 3 months and 6 months) which are associated with significant functional impairment but less severe clinical symptomatology [35]. This suggests that improvements in clinical outcomes happen reasonably quickly but are maintained with some additional improvements after the 3 month period. These results show benefits of reduced symptom severity, improved mental wellbeing and increased engagement in work and social activities. These improvements go hand in hand with an improved quality of life for the service users and indicates reduced distress and care needs with a better prognosis for their future. This is of great significance to service users.
There are statistically significant improvements for CMHS between baseline and 6 months. For satisfaction, it appears the satisfaction score is already high at baseline but improves, and is maintained whilst they are receiving POD. This approach appears to be well accepted by service users and a positive experience.
National Health Service user experiences are measured each year using the CQC Community Mental Health Survey. Our observations of the scores show that service users rated the POD service more highly than other local and national data and suggests this approach may address some outstanding issues with mental health services around service user satisfaction.
Evaluation is the basis for improving care and it has been suggested that mental health care services can be evaluated on two dimensions; whether they are beneficial or harmful, and whether they offer value for money [36]. This service was demonstrated to be beneficial to service users in terms of clinical outcomes and overall satisfaction. It incorporates shared decision making and the option to mobilise social support networks with a high level of choice offered to the service user and this may address national quality issues of service user satisfaction identified by the Care Quality Commission [37].
Clinical outcomes were consistent across mental wellbeing and work/social adjustment (self-reported), CMHS and HoNOS scores (clinician-reported) and showed that this approach is clinically effective with significant improvements across all measures. The Healthy London Partnership guide to analysing HoNOS data suggest that using effect size statistics such a Cohen’s d can aid interpretation of HoNOS data because it calculates the significance of the change. The change between the mean score recorded at 6 months as opposed to baseline equates to a Cohen’s d effect size of 1.24. It is proposed that any effect size of over 0.8 is indicative of an improvement of critical clinical importance [38]. Reduced symptoms, better functioning with service satisfaction are highly desirable goals of any mental health service and even small improvements can have a clinically valuable benefit for those in receipt of and providing services in terms day to day life and work fulfilment.
Hospitalisation rates
The mean number of bed days for the service users receiving the POD service was 0.44 in hospital during their complete period of continuous care starting from their first referral to date of their final discharge. This seems a low figure and, as a reference point, traditional services reported a mean number of 2.24 bed days for service users accessing the locality CRHT/CMHT services over the same period; over five times higher. The general observations of POD relative to traditional community care are of interest and point to the need for further work looking at comparisons. In general it is notable that any reduction in bedday usage is highly impactful on cost of care within the health system as a whole and needs further exploration and research. A long term data analysis will be needed before conclusions can be drawn about cost-effectiveness and value for money.
Contact rates
The mean duration of each contact between POD services and users was 88.14. This level of contact can be seen in context when looking at the reported average duration of 62 min for meetings between CRHT/CMHT traditional services and service users. This is congruent with the POD model which is responsive to need as perceived and expressed by the service user rather than on the basis of appointment slots or capacity of clinicians in clinics unless a service user declares their issue urgent in traditional services when they then receive duty worker support. The decision about frequency of contact is made by the service user with the team and this may have been empowering for the service user, a collaborative approach which may have improved the outcomes for service users. This could be explored in greater detail in future studies. The current evidence tentatively indicates improved outcomes when family and social network are involved in terms of reduced hospitalisation and recovery [39, 40].
Wellbeing of family and social network
The CWS support scale scores seem favourable given the findings of Quirk et al. [31] during the development of the measure. However the population under investigation in the Quirk study were different so the findings are not directly comparable. The 6 month score of 48.0 in our study shows a marked increase in the how carers perceived they were being supported by the POD team. It is also striking to note that at baseline the POD participants record a score of 41.67 suggesting that carers acknowledged the initial support provided by the POD service even at this early stage of the intervention.
There seems to be a wish for the involvement of family and friends as over half the meetings were attended by carers. When carers are involved in care, they are more involved in the service users recovery and with reduced risk of mortality by suicide [39]. One of the strengths of the POD model lies in the mobilisation of the social network. In this study it was found that almost half of the meetings did not have any network members present. Possible reasons for this could be due to stigma or a low level of social support in this population overall. Our data is for network attendance at the first ten meetings and it would be useful in future studies to monitor this data for all network meetings and observe any variations. However having carers involved in the half the total meeting with service users is much higher than our own clinical experiences of community services general involvement of family and support networks. Typically service users are seen singly and then family and social network consulted at the conclusion of the appointment if at all [40,41,42]. The assessment of service users alone often leads to concern from families about not being consulted. We acknowledge though that an evaluation of how often family and social network are involved in ordinary consultations needs to be considered objectively. Despite this finding, the high levels of satisfaction suggest that the POD team provided suitable support for individuals when other support was absent or unwanted. This could be explained by the POD team’s practice of engaging with important voices through the service user as a way of understanding relational aspects of the service user. The meetings contained the same dialogical conversations whether the service user was alone or not with the same decision making process. All the voices were valued in the meetings to emphasise a relational understanding of the service user’s distress. Barriers to shared decision making in practice have been described and can be addressed by shared care planning [40]. POD provides specific training for collaborative working which is likely to enhance the wellbeing in the social network.
Implementing the Open Dialogue Approach
A review of the evidence for Open Dialogue found that there has been considerable variation in how the Finnish OD approach has been implemented in different locations making comparisons unreliable [8]. The clinicians reflections on the Dialogic Practice adherence tool guided the POD service in treatment practice principles outlined by the Finnish researchers and developers of the Dialogic Practice adherence tool. [1, 5] (Ziedonis D, Olson M, Seikkula J: 10 Organizational criteria of open dialogue, unpublished). High practice adherence is important as there are a number of attempts to set up OD treatments around the world which struggle to be faithful to the practice and fully informed by the organisational criteria. The lack of a validated practice adherence instrument is an impediment to this. This study has looked at the implementation of a true OD practice approach modified only by the addition of peer workers to the clinical team. Ensuring onsistency of the approach being maintained ensures psychological continuity for service users. The Dialogic Practice Adherence tool supported reflection on practice to the approach and helped the team develop in their relationships and dialogue with each other.
Open Dialogue requires a reflective space for clinicians to have time to consider their practice and relational way of being with each other and the service users and social network [15,16,17]. This enabled a deeper understanding of each other as practitioners but also helped to take account of power positions in meetings including between practitioners. This usually took place without an external facilitator [16]. Essentially this was a model of reflective peer supervision for all members of the team. It is important to note that no clinical decisions were made in this supervision space as those decisions remained in the domain of the social network meetings.
A qualitative study examining the introduction of a POD service in England reported that clinicians positively viewed this way of working [43]. There were, however, mixed views from service users, including being unsure as to the purpose of the network meetings and finding the reflective conversations strange, though, the majority felt listened to and understood. The study was carried out during the training of the clinicians so may not accurately reflect the opinions of an established POD service from users and professionals.
The evidence base for the Open Dialogue approach, in a UK setting, remains sparse and currently there has not been any published quantitative data examining the POD model [43, 44].
Challenges
This study begins to grapple with a number of challenges in the delivery of Open Dialogue in mainstream NHS services. We were required to negotiate these challenges within the NHS Trust and recruitment to the research was successful having an acceptable level of attrition, with 74% of recruited participants completing the six month measures, allowing quantitative data to be collected.
The team functioned alongside the existing mental health services which included traditional community teams’ entry points for; inpatients, crisis resolution and home treatment, community mental health team, early intervention in psychosis and specialist teams. There were major challenges to develop a service that did not follow the established pathways for entry into care under traditional team boundaries. There were further challenges in continuing to work with people until they felt they had achieved recovery and were ready for discharge rather than the clinician and service design making those decisions as is traditional in the medical model.
We had to remain within the governance structures of a large NHS trust. The standard operating procedure written for the team included agreements with all parts of the system as to how the POD team would interface and work safely within and seamlessly with the usual system of mental health care delivery. For example, we had to consider the Care Programme Approach (care and treatment plan) [45, 46] and risk assessment and management [47] whilst continuing to work dialogically. It was essential not to increase risk to any service user or family member or indeed staff member during the course of this innovation. We also monitored quality and performance in line with all other Trust services.
The Kent POD Team was a small scale clinical approach in the very big system of a mental health trust within the NHS, with approximately 3300 staff serving a population of 1.7 million people. It was also an enormous challenge to continue working with people until they felt they had achieved recovery and were ready for discharge rather than the clinician making those decisions as is traditional in the medical model. Organisational investment is also needed to allow the implementation of service user choices including training clinicians in shared decision making as within the POD model.
Strengths
This trial was run in a real life NHS clinical setting taking a pragmatic approach.
Limitations
There was also a possibility of bias, in particular social desirability bias, impacting on the findings through the use of self-report measures. The authors aimed to minimise the likelihood of bias through adopting methods to reduce the prospect of social desirability bias detailed by Nederhof [48]. This entailed the team collecting the data from a range of different sources (service users, carers and clinicians), using well established validated measurement tools ensuring the participants and assuring the participants their answers would remain confidential. In addition, any missing data was not included in the analysis. This is also a potential source of bias as some data from the non-respondents could differ systematically from those that responded. This is a small scale study, not randomised and looked at one team. A full randomised control trial is therefore indicated.