Refining a model of collaborative care for people with a diagnosis of bipolar, schizophrenia or other psychoses in England: a qualitative formative evaluation
BMC Psychiatry volume 19, Article number: 7 (2019)
Many people diagnosed with schizophrenia, bipolar or other psychoses in England receive the majority of their healthcare from primary care. Primary care practitioners may not be well equipped to meet their needs and there is often poor communication with secondary care. Collaborative care is a promising alternative model but has not been trialled specifically with this service user group in England. Collaborative care for other mental health conditions has not been widely implemented despite evidence of its effectiveness. We carried out a formative evaluation of the PARTNERS model of collaborative care, with the aim of establishing barriers and facilitators to delivery, identifying implementation support requirements and testing the initial programme theory.
The PARTNERS intervention was delivered on a small scale in three sites. Qualitative data was collected from primary and secondary care practitioners, service users and family carers, using semi-structured interviews, session recordings and tape-assisted recall. Deductive and inductive thematic analysis was carried out; themes were compared to the programme theory and used to inform an implementation support strategy.
Key components of the intervention that were not consistently delivered as intended were: interaction with primary care teams, the use of coaching, and supervision. Barriers and facilitators identified were related to service commitment, care partner skills, supervisor understanding and service user motivation. An implementation support strategy was developed, with researcher facilitation of communication and supervision and additional training for practitioners. Some components of the intervention were not experienced as intended; this appeared to reflect difficulties with operationalising the intervention. Analysis of data relating to the intended outcomes of the intervention indicated that the mechanisms proposed in the programme theory had operated as expected.
Additional implementation support is likely to be required for the PARTNERS model to be delivered; the effectiveness of such support may be affected by practitioner and service user readiness to change. There is also a need to test the programme theory more fully. These issues will be addressed in the process evaluation of our full trial.
ISRCTN95702682, 26 October 2017.
It has been well established that people with a diagnosis of schizophrenia or bipolar have poorer physical health and social outcomes. The factors contributing to this are multiple, reciprocally interacting and include difficulties with engagement between healthcare providers and service users, along with health related behaviours. [1, 2]. There is evidence that primary care is centrally involved with the care of people with a diagnosis of schizophrenia, bipolar or other psychoses in the UK: Reilly et al.  found that nearly a third were seen only in primary care and those seen in secondary care received only minimal support. These authors identified a number of obstacles to primary care practitioners supporting this group to achieve improved outcomes, including a lack of continuity of care within primary care teams and poor continuity and information exchange at the interface with secondary care . Primary care practitioners also often lack the necessary time and training to effectively support people with these diagnoses in addressing their mental health needs [4,5,6]. Collaborative care has been identified as a potential strategy for overcoming these obstacles  and has been found to be effective in improving mental, physical and social functioning across a range of mental health conditions [7, 8]. However, few trials have included people with a diagnosis of schizophrenia or similar forms of psychosis and so it is not possible to draw conclusions about its effectiveness for this group [5, 8]. Further, trials that have included people with a diagnosis of bipolar, schizophrenia or other psychoses have principally taken place in the USA, with none in the UK, where the organisation and funding of healthcare provision is substantially different.
The PARTNERS2 (develoPing integrAted primaRy care for paTieNts with sERiouS mental illness) programme intends to address this gap through the development of a model of collaborative care for people with a diagnosis of bipolar, schizophrenia or other psychoses, which will be tested through a randomised controlled trial in three sites in England. The process of the development of the model and the intervention theory are described in more detail in Gwernan-Jones et al. . A Cochrane review  of collaborative care approaches for people with severe mental illness, defined as schizophrenia or other types of schizophrenia like psychosis (e.g. schizophreniform and schizoaffective disorders), bipolar affective disorder or other psychosis, has identified 14 components of collaborative care interventions. These are listed in Table 1, alongside a description of how they are operationalised in the PARTNERS intervention.
As with other collaborative care approaches , the PARTNERS model is based on Wagner’s Chronic Care Model . Additionally, and as a subtle departure from other collaborative care interventions, which tend to be focussed on specific disorders, it incorporates personal recovery processes  as a specific orientation to supporting service users to become more informed and active in managing their mental health, as well as ensuring care is focussed to support their priorities. Coaching was selected as the psycho-social intervention to support this recovery orientation and an approach supporting goals identified by individuals.
The PARTNERS programme theory was developed using realist principles  aiming to identify underlying causal mechanisms by which the intervention had its effects and the influence of context on the operation of these mechanisms. Within the realist approach, mechanisms are defined as the reasoning and reactions of human agents in response to the resources provided by an intervention. Mechanisms thus reflect the internal mental processes which take place within the individuals involved in an intervention that lead to them choosing to make the desired changes in practices or behaviours necessary to implement the intervention or that form the outcomes of the intervention . Mechanisms include a diverse range of processes, such as increasing acceptance, motivation, knowledge and skills, trust, and self-confidence [14,15,16]. The initial programme theory development resulted in a large number of ‘explanatory statements’ , which are summarised in Fig. 1 and Table 2.
MRC guidelines for the evaluation of complex interventions [17, 18] recommend a feasibility and piloting phase, during which formative evaluation can take place, in order to test and refine the underlying programme theory and to establish what support would be needed to ensure delivery of the intervention as intended, and thus allow effective outcome evaluation during the main trial. As implementation of collaborative care for mental health in general has been limited, despite evidence of its efficacy , factors affecting delivery were a particularly important focus for the formative evaluation of the PARTNERS model.
Two recent systematic reviews, which included 9 UK studies, have identified barriers and facilitators to the implementation of collaborative care for depression and anxiety [20, 21]. These included practitioners’ understanding of collaborative care and their attitudes towards change, the presence or absence of regular communication between practitioners and structures which supported this and the qualities, knowledge and skills of the case manager, in engaging both with other practitioners and service users. A potential barrier was that some service users preferred not to have mental and physical health integrated but generally the experience of service users was not investigated. These findings are consistent with other systematic reviews of factors affecting implementation of collaborative care across all conditions [22, 23] suggesting they were also likely to operate during our feasibility study. There is further some evidence that facilitation can lead to improved implementation of collaborative care models for mental health conditions [24, 25].
In the context of the above evidence, the aims of our formative evaluation were therefore to:
Assess how well the intervention as delivered matched the model
Identify the barriers and facilitators to delivering the model as intended
Identify additional support for implementation likely to be required in the main trial
Test and refine our initial theory, through exploring any perceived effects of the intervention and comparing these to the programme theory
The use of qualitative data is recommended for addressing such aims  and so this methodology was adopted.
Ethical approval was obtained from the West Midlands – Edgbaston Ethics Committee (14/WM/0052).
The formative evaluation consisted of the small scale delivery of the PARTNERS intervention in three geographically diverse regions and qualitative data collection about the experiences of the intervention from all stakeholders. In order to maximise rigour, we used multiple methods of data collection: semi-structured interviews, recordings of consultations between care partners and service users and tape-assisted recall interviews about these recordings with service users and care partners. This enabled us to triangulate multiple perspectives, including our own observations of the intervention in action, and participants’ reflections both on the intervention overall and specific interactions, allowing us to build a more detailed and accurate understanding of how the intervention had been delivered and experienced .
Through negotiations with the NHS Trusts providing secondary care mental health services in each site, agreements were reached whereby a qualified mental health worker and a senior clinician would have a portion of their time re-allocated to delivering the PARTNERS service as a care partner and supervisor respectively. The portion of time allocated varied across the sites with one Trust agreeing to a full time equivalent and another agreeing to half a day a week. The individual practitioners were nominated by the Trusts as being available for and interested in the roles. The care partners were two mental health nurses and a social worker and the supervisors were a mental health nurse, a social worker and an occupational therapist currently working as team leaders.
A number of GP (general practitioner) practices in each site were approached to establish interest in receiving the intervention. The number recruited in each site varied according to the time allocated to the care partner, with a total of six practices receiving the PARTNERS service.
To deliver the components of the intervention involving direct patient contact, care partners met individually with service users, usually at their GP practice although this was negotiated individually and some meetings took place at service users’ homes or community venues. Consultations, known as intervention sessions, varied in length according to need, preference and stage of the intervention, lasting between 20 and 90 min. Initial sessions were used to build a therapeutic relationship and develop a ‘shared understanding’ of the service user’s life experiences and priorities for change. This fed into the identification of the service user’s personal goals, the exploration of resources which could help them meet those goals and the agreement of action plans, which would be reviewed in subsequent sessions, along with routine monitoring of service users’ mental health and wellbeing using standardised measures: the CORE-10 (Clinical Outcomes in Routine Evaluation 10-item)  and the WEMWBS (Warwick Edinburgh Mental Wellbeing Scale) . Care partners would liaise with other people and services who could provide support for the service user to work towards their goals on an individually agreed basis.
Care partners were based at GP surgeries for the allocated time with the expectation that they would become active members of that primary care team, engaging in two way consultation with other team members in whatever way was appropriate and recording information about service users in computerised record systems.
Care partners were expected to meet with supervisors at least once a fortnight for guidance on goal setting, level of intensity of care and decision-making in relation to working collaboratively with other services and agencies. A protocol was provided to ensure that care partners and supervisors maintained an overview of all service users on the caseload.
The inclusion criteria for receiving the intervention were: adults with a diagnosis of schizophrenia or bipolar, including those only receiving mental health care from general practice and those with stable but ongoing mental health needs under specialist services; individuals receiving acute crisis care were excluded. Service users were identified by screening primary and secondary care records and their suitability was confirmed by a practitioner who was known to them. They were then approached by the practitioner to discuss receiving the new service. The number of service users varied across sites, according to the time allocated to the care partner, with a total of thirty-eight service users receiving the intervention.
The intervention was delivered over a period of eight to 10 months; expectations of the minimum number of contacts a service user would receive were adjusted so that they would have at least two contacts during this period. The intervention was delivered concurrently in sites A and C but the start was delayed by 6 months in site B. No changes were made to the intervention in Site B of the basis of the evaluation of the other sites but this did influence the data collection (see below).
Participants in the formative evaluation were service users receiving the intervention, their family carers, and practitioners involved in delivering and supporting the service. A required sample size for each category of participant in each site was agreed prior to data collection, with an aim to achieve maximum diversity in the sample rather than data saturation. This was a pragmatic decision made in relation to funding constraints and we judged that these sample sizes would provide sufficient data about the range of experiences. Data was collected from sixteen service users, five family carers, three care partners, three supervisors, four GPs and six other primary or secondary care practitioners.
Topic guides for semi-structured interviews were developed collaboratively by the research team, with input from members of the PARTNERS2 Lived Experience Advisory Panels (LEAPs). The LEAPs consisted of people with personal experience of living with schizophrenia or bipolar, as a service user or a family carer, and an interest in research. The study has one LEAP per site, three in total.
Topic guides focussed on the key elements of the PARTNERS service, as specified in the manual and associated service user guide, whether and how the participant had experienced these in practice and what difference this made in terms of the quality of care delivered. Time pressure on sampling prevented pilot testing of topic guides. Copies of the topic guides are available on request from the authors.
We adopted the tape-assisted recall method outlined by Cape et al.,  whereby consultations were audio-recorded and reviewed by researchers, to identify five or six excerpts from the session where there seemed to be divergence from the model or where the care partner and service user may have perceived the care partner’s action differently. Service users and care partners were then invited to participate in separate tape-assisted recall interviews, in which these excerpts were played back and the participant was asked for their views of what had been happening at that point and whether this was helpful or unhelpful.
After service users had been receiving the intervention for a few weeks, researchers contacted them by phone to attempt to recruit them to the evaluation. Service users were asked to consent separately to the different forms of data collection. In site A, service users were sampled purposively for interviews to achieve a range of gender, diagnosis and experiences of mental health services but were identified for session recordings by the care partner. In site B, service users were identified for interviews and session recordings by the care partner on the basis of their availability and willingness to participate. In Site C, all service users were approached by researchers for interviews, although not all were available. One service user declined to have a session recorded and one was not approached for a session recording as the care partner judged it would be too distressing. Participating service users were asked to identify family carers who could be approached. Family carers were then approached by phone; all family carers who were approached agreed to be interviewed. Practitioners who had some form of contact with the PARTNERS service were contacted by phone and asked to consent to data collection relevant to their role; all those approached agreed to participate.
Data was collected over a 6 month period from sites A and C, where the intervention was delivered concurrently. Preliminary analysis of this data, which identified which components of the intervention had not been delivered as intended in these sites and possible barriers and facilitators informed the approach to data collection in site B, such that the care partner and supervisors were re-interviewed towards the end of the intervention period to see what learning through experience might have taken place.
Data was collected by a number of researchers (MC, LGa, LGi, RGJ, CMc, SR, AR), including two service user researchers. All received training in qualitative methods as appropriate, and the use of the specific interview techniques, from members of the research team experienced in qualitative research (RGJ, NB, VP, LGa).
Semi-structured interviews lasted one to 2 h and tape-assisted recall interviews lasted about 1 h. All service user and family carer interviews were conducted face to face, at the participant’s GP practice or home, according to their preference. Two service users chose to have family members present during the interview but otherwise participants were alone. Practitioner participants were interviewed face to face at their regular place of work or by telephone, according to their preference. Service user participants, care partners and supervisors had met the researcher previously as part of the process of setting up the PARTNERS service and so the interviewer would have been familiar to them. Other categories of participant had not met the interviewer previously. Service user researchers were encouraged to disclose their experience of using mental health services to service user participants. Otherwise participants were informed on Participant Information Sheets that the interviewers belonged to a group of ‘very experienced mental health researchers’. All interviews were audio-recorded and interviewers also made field notes after the interview, using an agreed template. Service users and carers received a £10 shopping voucher for each interview and recorded intervention session.
Several participants contributed to more than one source of data and the total number of data sources by type is shown in Table 3.
All data sources were transcribed and imported into NVivo 11. Transcripts were checked against the recordings for accuracy by researchers but not returned to participants due to time pressures created by preparing for the full trial. Preliminary deductive thematic analysis, was carried out by a number of the interviewers (MC, LGi, RGJ, CMc, AR). Data from sites A and C was coded against the detailed sub-components of the model and intended outcomes as articulated in the manual and wider programme theory. A descriptive analysis was then conducted (RGJ) that was reported and discussed at a researcher meeting, where areas for further exploration were prioritised as needing to be understood to support planning for the main trial. Written summaries of key issues relating to each component were then produced for discussion at a stakeholder meeting.
The stakeholder meeting was attended by the whole research team, care partners and supervisors and members of the LEAPs, enabling data interpretation to be grounded in lived experience. This meeting focussed on understanding why some components had not been delivered or experienced as intended and identifying strategies to address these issues; the outcome of these discussions formed the basis of the implementation support strategy and informed revisions to programme materials in advance of the full trial.
Subsequent to this meeting, further data was collected; this consisted of two tape-assisted recall interviews with the care partner and one interview with a service user in site C and all the data collection in site B. There was insufficient time to revise topic guides to take account of the emerging themes but the preliminary analysis informed the development of new topic guides for the follow up interviews with care partners and supervisors in site B. Once all the data had been collected, the entire data set was analysed by one researcher (EB) with more detailed themes, and the relationships between them, being identified and defined. Coding of data from the diverse range of sources required a degree of interpretation, especially where observational data was inconsistent with interview data or two informants gave conflicting accounts of the same event. These interpretations were supported by reference to field notes and discussion with a researcher who had been involved in collecting data and the initial coding (RGJ) and an experienced qualitative researcher (NB), taking into account the main coder’s professional background as a clinical psychologist. A framework analysis  was then used to systematically explore differences between data sources and reach conclusions about how best to understand the data. Interpretations were shared and discussed with the wider researcher team.
In order to address the aims of the study, the analysis initially focussed on barriers and facilitators to delivery of the intervention components. The themes identified were compared against the ongoing actions agreed at the stakeholder meeting to ensure that the emerging implementation support strategy still reflected the data. Subsequent analysis focussed on the intended outcomes of the intervention. The themes identified for each outcome domain were compared to the mechanisms hypothesised in the programme theory, as articulated in the intervention logic model and ‘explanatory statements’  (see Fig. 1 and Table 2).
Sixteen service users, five family carers and sixteen practitioners participated in the formative evaluation; distribution of participants across the three sites is shown in Table 4, along with the demographic data.
Aim 1. Evidence of whether the intervention as delivered matched the model
The preliminary analysis identified that a number of components of the model had not been delivered as intended across the sites and one component required clearer specification as it was not clear whether what had been delivered was consistent with the intervention. The extent to which each component of the model was found to have been delivered is summarised in Table 5.
Aim 2. Barriers and facilitators to delivering the model as intended
Further analysis initially focussed on identifying barriers and facilitators in relation to the components that had been identified as not being delivered. The most informative data was available in relation to three components: systematic communication, psychological intervention (in the form of coaching and goal setting) and multi-disciplinary working (in the form of supervision by senior mental health practitioners); we will therefore present detailed findings in relation to these.
Limited information was available about the barriers to provider integration as our topic guides did not focus on systemic issues and few of our participants worked at an organisational level where they would have been able to comment on this. Interpretation of the data in relation to the other components was complicated by inconsistencies between data sources. Data from session recordings, service user interviews and primary care practitioner interviews indicated that care partners were not fully delivering the model; however this was not necessarily recognised either by the care partner or their supervisor. Consequently care partners and supervisors did not offer any account of barriers and descriptions of facilitators were considered only partial accounts. In relation to systematic monitoring and shared decision making this prevented any firm conclusions being reached about barriers and facilitators. The themes that were identified as barriers and facilitators to systematic communication, coaching and goal setting, and supervision are shown in Table 6.
The extent to which care partners communicated with primary care teams appeared to be influenced both by the atmosphere of the primary care service and the communication style of the care partner. Communication was most likely to take place when the primary care service was hospitable and the care partner had an interpersonal route in to the care team, who could raise awareness of the care partner’s role and provide practical support:
the manager…was very welcoming and introduced me to as many people as possible and… assigned a – I think she’s an admin worker or something, to me, so if I have any problems I just go to M. and M. does everything, and it’s great. [care partner 3]
Conversely if the care partner’s presence went ‘under the radar’ [GP4] this acted as a barrier.
Care partners were still able to successfully communicate with primary care teams, if they were pro-active in creating and capitalising on opportunities to do so, either routinely, through using records or attending meetings, or for a specific clinical purpose. However, care partners’ lack of knowledge about primary care systems and processes could reduce their confidence to approach primary care team members, such that they focussed on more familiar clinical work rather than liaison:
in [name of surgery] I feel that it’s kind of in process now, it’s working, people are turning up and it’s almost like, well, do I need to do something? People are aware that I’m there, but I don’t feel that they have the quality understanding of why I’m there [care partner 3]
The support that care partners sought, and were offered, to overcome barriers to communication was a further influence, with some supervisors not being aware that this was an area of difficulty and so the issue was not addressed:
I can’t remember him saying that there was ever a particular issue? Not one, maybe, that he brought to supervision. [supervisor 2]
In contrast, another care partner sought support from the research team, which seemed to lead to greater engagement with primary care.
If service users wanted the care partner’s support in accessing physical health care this also created opportunities for the care partner to liaise. However some service users did not see the value in this, which acted as a further barrier to liaison:
If I need to see me GP about health problems, I just go to me GP, I don’t involve [name of care partner]. [service user 1]
Coaching and goal setting
A goal-setting approach was most likely to be delivered when both care partners and service users saw the value of it, and some service users identified it as a key active ingredient of the intervention:
the goal-oriented approach for me is the crucial factor [service user 11]
However, there were times when both service users and care partners questioned the value of goal setting, suggesting that the emotional support provided by the sessions was as important:
I was just thinking about sometimes allowing the sessions to just be, because whilst there is an agenda, … of the coaching… sometimes I’ve found people don’t want to necessarily be coached, but they want to come along to the sessions [care partner 3]
This appeared to be linked to an interpretation of ‘goals’ as relating to practical achievements rather than psychological needs, which service user 11 described as ‘airy fairy, nebulous goals’.
The care partner’s pre-existing personal style, and whether this was congruent with a coaching approach was also influential. Where coaching was too great a contrast to the care partner’s usual practice, this tended to result in the prioritisation of their perception of the service user’s needs, especially in relation to managing risk:
To me, me getting [name of service user] and taking her physically and saying ‘Come on, [name of service user]’ and if I have to ‘There, there’ and wrap her up in cotton wool for a little bit, I’ll know the job was done and dusted then… I’ll know if she’s safe. [care partner 1]
The availability, or lack, of supervision in the use of a coaching approach was also linked to delivery. One supervisor was comfortable and familiar with coaching, which supported the care partner, but another was not able to provide such guidance, leaving the care partner struggling:
I was getting meself bogged down …and I’d go to me manager and I’d say …‘ I need help here’, and it was like ‘Read your manual’ and I felt like saying [shouts] ‘You read the manual!’ [care partner 1]
Service user motivation also impacted on the delivery of coaching and goal setting, with some not engaging with the approach as they did not wish to make changes and others taking it up readily:
some people have already set their own goals, they’ve been very good… very insightful into their own difficulties … just thinking of somebody that I’m working with at [name of surgery], she’s been very good at planning what she needs to do to improve her quality of life [care partner 3]
The delivery of systematic communication and coaching was therefore underpinned by the delivery of supervision. While some supervisors were able to prioritise providing PARTNERS supervision, in one site there were too many competing demands:
we took on the single point of access team, recovery team as well as the crisis team, so my role sort of really expanded. So I feel really bad because I didn’t apply myself the way I probably should have done [supervisor 1]
Consequently, the care partner felt inhibited about seeking supervision:
I know he’s busy [care partner 1]
The supervisor’s availability was associated with the extent to which the supervisor understood the model. When the supervisor found it hard to understand, it was more likely to be experienced as an additional demand. Conversely, regularly engaging with the model increased the supervisor’s familiarity with it and commitment to the intervention:
the manuals are on the table but they’re closed because – I know how we’ve been working over time and I understand the model [supervisor 3]
Whether the content of supervision was consistent with the model appeared to be influenced by the extent to which both supervisors and care partners recognised the need to challenge the care partner to engage in detailed discussions about their work. Where supervisors assumed the care partner was competent and the care partner did not recognise a need to change their practice, supervision did not focus on the intervention elements, as intended. This focus was provided when the supervisor engaged the care partner in ‘extended case discussions’ [supervisor 3] and the care partner valued the opportunity to reflect on their practice:
what I like is supervisions where people are challenging me and… giving me ideas of how I can improve upon things… rather than saying ‘Oh yeah, it’s really great’ [care partner 3]
Aim 3. Additional support for implementation likely to be required in the main trial
Drawing on these findings and approaches to facilitation that have been found to be effective in supporting the implementation of collaborative care for mental health [24, 25], we developed an implementation support strategy for the main trial. This included actions in relation to each of the three components where barriers and facilitators were identified.
Actions to facilitate systematic communication between care partners and primary care practitioners
A link person to be identified within the primary care team, who will contribute to a local needs assessment, clarifying the most effective strategies for the care partner to communicate with the team about their work.
Regular contact between researchers and care partners to ensure these strategies are being used and identify any concerns, and regular contact with the practice link person for problem-solving.
The supervision protocol to be amended to include reviewing communication with primary care practitioners and providing support for developing communication strategies.
Actions to facilitate delivery of coaching and goal setting
Revisions to care partner and supervisor manual and training, clarifying that goals can be psychological as well as practical and may include addressing the need for ongoing emotional support.
Revisions to care partner and supervisor manual and training to provide clearer examples of coaching approaches and more opportunities for care partners and supervisors to practice through role play.
Additional follow up training, in which care partners reflect on audio-recordings of their own work, together with supervisors to support learning from practice.
Actions to facilitate delivery of supervision
Clear, negotiated agreements with secondary care service leadership about supervisors’ time commitments and regular contact between researchers and supervisors and researchers and service leadership for progress checking and trouble shooting.
Follow up training focussed on audio recordings of care partners’ practice, to encourage critical self-refection and support supervisors engaging care partners in detailed discussion of their work.
Aim 4. Comparison of perceived effects to the programme theory
Our preliminary analysis also identified a number of components that had been delivered but had not been experienced as intended. This mostly related to the experience of a few service users and were not universally reported. These were: not relating to the language of one of the measures used for ‘systematic monitoring’ and experiencing its use as burdensome rather than supportive; concerns about the use of coaching as the ‘psychological intervention’ because of its focus on identifying goals, which was felt to be unrealistic in the context of complex difficulties, and finding the service user guide, intended as a support for ‘shared decision making’ to be too lengthy, complex or paternalistic. Additionally, despite the provision of ‘education for mental health providers’ we found that care partners and supervisors sometimes did not have the knowledge or skills required to deliver their role in key areas following training, including physical health care and use of IT systems.
In relation to the programme theory, these findings indicate that practitioners did not react as expected to the resources provided the intervention training, in that they did not develop the knowledge required to deliver the intervention. Additionally, service users did not react as expected to some of the resources provided by the care partner, in that rather than becoming activated in managing their mental health they felt directed to attend to it in particular ways. Discussion of these findings within the team concluded that the failure of these mechanisms to operate as intended was a consequence of inadequate operationalisation of the programme theory, through the care partner manual and training and the materials used with service users, such that these did not adequately convey what was intended. It was agreed that, rather than revisions being required to the programme theory itself, revisions should be made to these practical expressions of it, in consultation with members of the LEAPs, in order to find forms of expression that would be experienced as more empowering for service users.
Further analysis focussed on the intended outcome domains shown in Fig. 1: physical health, stability of mental health and quality of life, including the personal recovery processes of connectedness, hope, identity, meaning and empowerment. The themes identified in each of these domains were compared to the programme theory, to see whether it reflected the operation of the hypothesised mechanisms.
We found evidence for three mechanisms operating as predicted by the programme theory. Firstly, care partner liaison with GPs could increase GP’s understanding of service users’ needs, resulting in GPs providing appropriate support, including feeling able to attend to the service user’s physical health concerns rather than mental health:
I can see that one patient that did come to see… my colleague, he was able to focus on his shoulder and his other thing that he’d come about and not try to do everything all at once. ‘Cos the danger is, you see a patient who’s on those sorts of drugs, you think ‘Oh god, I really need to think about their mental health’ and ‘Oh my god I need to check they’re not seeing things or they’re not in danger’, and then… there’s no time to look at stuff like stopping smoking or what their blood pressure might be [GP 4]
Secondly, closer working with primary care could increase the care partner’s awareness of physical health issues and the support available, resulting in them introducing this as a possible goal with service users:
it’s also having that wider perspective of kind of interest… looking at… physical history, which I wouldn’t normally have access to those kind of records [care partner 3]
Thirdly, when the care partner shared knowledge about physical health, this could increase service user knowledge, empowering them to seek appropriate support from primary care:
the other week she [care partner] was explaining that… I should have been having more… heart checks and stuff with the medication I’m on…and that’s the only thing that is a little bit weak, ‘cos I only had my first ever ECG [electrocardiogram]… a couple of weeks ago, and that’s ‘cos I asked for it. [service user 12]
Quality of life and CHIME framework outcomes
Also consistent with the programme theory, we found evidence that service users responded to the resources provided by the care partner, particularly the coaching approach and consistency of contact, by experiencing increased hope and confidence in their ability to change as well as trust in the support from the care partner. This resulted in them taking small steps towards goals such as starting voluntary work or increasing their social contact:
it’s giving me the confidence to get back out on the tightrope knowing that there’s safety net underneath me [service user 11]
There was also evidence that this set up a virtuous cycle, whereby increased activity reinforced service users’ confidence, as predicted by the programme theory:
we take the dog over there and we walk around the park a couple of times and I come home and I feel, yeah, I've got out and that's an achievement. Where beforehand I would have thought, ooh, no, no, I just can't do that. [service user 7]
Stability of mental health
We also found evidence that service user responses to coaching and consistency of contact with the care partner resulted in service users changing their behaviours in ways which would help maintain the stability of their mental health, in the ways predicted. This included trust in the care partner leading to a service user becoming more open and gaining reassurance:
there was a sense that [name of service user] felt that we were in a comfortable space, a comfortable environment for him to be able to talk about… he said that he could have bizarre, dark thoughts at times in the past. And I guess it’s a place that we can talk through those and then kind of – we can explore whether it’s something that’s out of the ordinary or whether it’s just kind of eccentric feelings, views, thoughts or whatever, to talk those things through [care partner 3]
Additionally there was some evidence that monitoring of service users’ mental health could increase service user hope and confidence about their ability to maintain stability and access to support if necessary:
even though I find putting a number on it is quite hard, but at the same time it gives you that general score of where you are… you might be depressed, but you're not having bad thoughts, you're not kind of not confident but you don’t need to worry [service user 12]
There was also evidence that care partner liaison with secondary care practitioners led to increases in their understanding of individual service user needs and preferences, resulting in appropriate support being provided, in the way expected:
links to the psychiatrist, that’s been pretty good, ‘cos there was somebody …whose medication had been changed around and as a result she’d become a lot less motivated … so I spoke with a psychiatrist here, ‘cos I knew that she was going to be meeting as an outpatient, um, to see what their thoughts were about medication, and they’ve changed it back and she’s very pleased about that [care partner 3]
This formative evaluation of a novel collaborative care intervention for people with a diagnosis of bipolar, schizophrenia or other psychoses in England had four aims: we will discuss our findings in relation to each in turn.
How well the intervention as delivered matched the model
Our data indicated that while many of the intervention components were delivered as intended, a number were only partially or inconsistently delivered, with one component requiring clearer specification as it was not clear whether it had been delivered as intended. Key components that were not delivered in at least two, if not all three, sites were: increased interaction between care partners and primary care practitioners, either through becoming integrated into primary care teams or through routinely sharing information about service users, supervision of the intervention, use of a coaching and goal-setting approach, and a collaborative style of interaction with service users.
These findings are consistent with those of other qualitative evaluations of the delivery of collaborative care for mental health in England, with limited integration of specialist mental health workers into primary care teams being most commonly identified [32,33,34]. Other evaluations have also identified inconsistent delivery of supervision  and practitioners having difficulty adapting to a new therapeutic style . This indicates that these components may need additional implementation support in order to enable evaluation of the intervention in a full trial and that such support may also need to be considered if more widespread implementation is indicated.
Barriers and facilitators to delivering the model
We identified a number of factors that influenced the delivery of increased communication, the use of coaching, and supervision: service leadership that supported organisational change through raising awareness of new roles and making resources available, the use of strategies to support regular communication between practitioners, the communication skills of care partners, both with other practitioners and service users, practitioners’ understanding of the intervention, effective supervision and service user motivation. Our results indicate that recovery-focussed collaborative care for people with a diagnosis of bipolar, schizophrenia or other psychoses is most likely to be implemented when the primary care team are receptive and the secondary care practitioner is prepared to adopt new working practices, including a pro-active approach to liaison and a more positive approach to risk. Further, that such changes to practice may be most effectively supported by supervisors with sufficient time and knowledge and when practitioners are sufficiently self-aware to engage in focussed reflection on practice. The intervention is most likely to be accepted by service users who are ready and willing to make changes, especially in addressing their physical health.
The barriers and facilitators that we identified are largely similar to those described in two recent systematic reviews of the implementation of collaborative care for depression [20, 21]. However, there has been limited previous investigation of service user views and experiences of collaborative care for mental health ; our findings indicate that service user motivation may also be a contributing factor. These reviews conclude that evidence-based approaches to implementation should be used to overcome such barriers, especially adequate training and supervision and setting up robust systems for communication; we have attempted to achieve this through our implementation strategy.
Implementation support required for the main trial
Our implementation strategy aims to support increased interaction between care partners and primary care practitioners and the delivery of supervision through researcher facilitation, which has previously been found to be effective in supporting the implementation of collaborative care for mental health [24, 25]. This will identify communication strategies tailored to the local context and monitor their use, and monitor the delivery of supervision, engaging with service leadership to support the delivery of the new roles and systems where necessary.
We aim to improve care partner understanding and skills in relation to coaching, goal setting and collaborative interaction styles through additional training and clearer written information in the manual. The extent to which care partners adopted a coaching approach appeared to reflect their preferred style of practice. Clinicians’ personal style is a well-established barrier to implementation in primary care settings and was also identified as influential during a randomised controlled trial of a team level pro-recovery intervention, which had a coaching component [38, 39]. The process evaluation of this trial identified that training was most effective when it involved opportunities to use the skills in practice, through role play with colleagues or initial experiences with service users, and receive direct personalised feedback. We have revised our training process to provide increased opportunities for skills practice and feedback, through role play and the use of audio recordings of sessions. Whether these strategies will be successful in influencing practitioners’ personal style is a remaining ‘key uncertainty’  and will therefore be included as a specific research question in the process evaluation of the full trial.
Our data also indicated that when service users were ready and motivated for a collaborative care style intervention that they responded well. As, through its use of a coaching approach, the PARTNERS intervention focusses on building readiness for change, increased support for care partner skills development should also address barriers arising from service user motivation. Whether the reliable application of such an intervention can enable service users who are less receptive to become more engaged is an additional question that can be explored in the process evaluation of the full trial.
Comparison to the programme theory
The partial delivery of key intervention components limited the extent to which we were able to test the programme theory. Our preliminary analysis identified instances where intervention components were delivered but not experienced as intended; these appeared to reflect difficulties with operationalising the intervention and learning from these contributed to the process of revising the written materials and training that supported intervention delivery.
Where there was evidence of the intervention being delivered as intended, the themes identified in relation to each of the intended outcome domains reflected the mechanisms proposed within the programme theory. This indicates that the programme theory itself does not require revision in advance of the full trial and provides some validation of the comprehensive theory development process outlined in Gwernan-Jones et al. . However, the model requires more robust testing during the full trial, when we anticipate that increased implementation support will lead to greater fidelity and there will be thus be more opportunity to explore in depth how the intervention may have its effects.
Strengths and limitations
In contrast to existing studies of the implementation of collaborative care for mental health, which did not collect data from all professional groups  or service users  we collected data from a wide range of sources,. We also made use of session recordings and linked tape assisted recall interviews, which deepened our understanding of interactions between care partners and service users . However, there were inconsistencies between some data sources, and we did not make direct observations of how care partners related to primary care teams or supervisors. Consequently, interpretation of this data required a greater degree of inference about potential influences on implementation and uncertainty remains. Care partners and supervisors were possibly motivated to present a positive impression of the intervention rather than engage in critical reflection. This indicates a need for a different approach to understanding care partner and supervisor experience in the process evaluation of the full trial. Increased researcher facilitation may result in better rapport between researchers and practitioners and create opportunities for honest feedback. We have also revised our interview topic guides, to attempt to convey a spirit of mutual enquiry rather than evaluation.
Sampling of service users was to some extent reliant on care partners and is therefore likely to have been influenced by their perceptions of service users. This may have resulted in a positively skewed response from service users. A consistently purposive approach to sampling will be important in the main trial to enable exploration of the full range of service user experiences.
This study provides an example of the contribution of formative evaluation in the development of a complex intervention, of which few are available . However as delivery of the intervention was variable, interpretations about how the intervention worked in practice in comparison to the realist programme theory and facilitators to delivery are based on limited data. Further, the initial data analysis was driven by a pragmatic need to address key issues in planning for the main trial which was imminently due to start and therefore did not involve detailed comparison between the mechanisms specified in the programme theory and the data. However, as the themes derived from our subsequent analysis were consistent with the logic model and detailed theory articulation, this provides some validation of our programme theory. Data analysis in the full trial will adopt a systematic approach to whether the mechanisms specified by the programme theory are operating.
This study found that enhanced implementation support is likely to be required for the PARTNERS model to be delivered as intended; this has implications both for the full trial and for healthcare services attempting to establish collaborative care for mental health as routine practice. In particular, additional support may be required to improve understanding of, and increase practitioner confidence to implement, new working practices. Practitioner training may need to focus on practical skills development for supporting service user activation. It may be that the impact of introducing a recovery-focussed collaborative care intervention is limited by both practitioner and service user readiness to change and there is a need for a greater understanding of the contextual influences which contribute to this. This question will be addressed through the process evaluation in our main trial and has shaped our approach to data collection. Although we found promising indications that the PARTNERS model may operate as predicted by the programme theory, there is a need to test this more fully, an opportunity which the full trial will also provide.
Connectedness, hope, identity, meaning, empowerment
Clinical outcomes in routine evaluation 10-item
Lived experience advisory panel
Developing integrAted primaRy care for paTieNts with sERiouS mental illness
Warwick Edinburgh mental wellbeing scale
Reilly S, Olier I, Planner C, et al. Inequalities in physical comorbidity: a longitudinal comparative cohort study of people with severe mental illness in the UK. BMJ Open. 2015. https://doi.org/10.1136/bmjopen-2015-009010.
Sweeney S, Air T, Zannettino L, Galletly C. Psychosis, socioeconomic disadvantage, and health service use in South Australia: findings from the second Australian national survey of psychosis. Front Public Health. 2015. https://doi.org/10.3389/fpubh.2015.00259.
Reilly S, Planner C, Hann M, Reeves D, Nazareth I, Lester H. The role of primary care in service provision for people with severe mental illness in the United Kingdom. PLoS One. 2015. https://doi.org/10.1371/journal.pone.0036468.
Bindman J, Johnson S, Wright S, et al. Integration between primary and secondary services in the care of the severely mentally ill: patients’ and general practitioners’ views. Br J Psychiatr. 1997. https://doi.org/10.1192/bjp.171.2.169.
Goodrich DE, Kilbourne AM, Nord KM, et al. Mental health collaborative care and its role in primary care settings. Curr Psychiatry Rep. 2013. https://doi.org/10.1007/s11920-013-0383-2.
Lester H, Tritter JQ, Sorohan H. Patients’ and health professionals’ views on primary care for people with serious mental illness: focus group study. BMJ. 2005. https://doi.org/10.1136/bmj.38440.418426.8F.
Woltmann E, Grogan-Kaylor A, Perron B, Georges H, Kilbourne AM, Bauer MS. Comparative effectiveness of collaborative chronic care models for mental health conditions across primary, specialty, and behavioral health care settings: systematic review and meta-analysis. Am J Psychiatr. 2012. https://doi.org/10.1176/appi.ajp.2012.11111616.
Reilly S, Planner C, Gask L, Hann M, Knowles S, Druss B, Lester H. Collaborative care approaches for people with severe mental illness. Cochrane Database Syst Rev. 2013. https://doi.org/10.1002/14651858.CD009531.pub2.
Gwernan-Jones R, Baker E, Allard J, Gill L, Lloyd H, Rawcliffe T et al. A worked example of the programme theory development phase in theory-driven evaluation: PARTNERS2 collaborative care for people who have experienced psychosis in England. 2018. Manuscript submitted for publication.
Wagner E, Austin B, Von Korff M. Organizing care for patients with chronic illness. Milbank Q. 1996;74(4):511–44.
Leamy M, Bird V, Le Boutillier C, Williams J, Slade M. A conceptual framework for personal recovery in mental health: systematic review and narrative synthesis. Br J Psychiatry. 2011;199:445–52.
Pawson R, Tilley N. Realistic evaluation. London: Sage; 1997.
Lacouture A, Breton E, Guichard A, Ridde V. The concept of mechanism from a realist approach: a scoping review to facilitate its operationalization in public health program evaluation. Implement Sci. 2015;10:153. https://doi.org/10.1186/s13012-015-0345-7.
Bertotti M, Frostick C, Hutt P, Sohanpal R, Carnes D. A realist evaluation of social prescribing: an exploration into the context and mechanisms underpinning a pathway linking primary care with the voluntary sector. Prim Health Care Res Dev. 2017;19(3):232–45.
Dieleman M, Gerretsen B, van der Wilt GJ. Human resource management interventions to improve health workers’ performance in low and middle income countries: a realist review. Health Res Policy Syst. 2009;7:1–13.
Wand T, White K, Patching J. Applying a realist (ic) framework to the evaluation of a new model of emergency department based mental health nursing practice. Nurs Inq. 2010;17(3):231–9.
Craig P, Dieppe P, Macintyre S, Michie S, Nazareth I, Petticrew M. Developing and evaluating complex interventions: the new Medical Research Council guidance. Int J Nurs Stud. 2013;50:587–92.
Moore GF, Audrey S, Barker M, Bond L, Bonell C, Hardeman W, et al. Process evaluation of complex interventions: Medical Research Council guidance. BMJ. 2015;350:h1258.
Kroenke K, Unutzer J. Closing the false divide: sustainable approaches to integrating mental health services into primary care. J Gen Intern Med. 2017. https://doi.org/10.1007/s11606-016-3967-9.
Overbeck G, Davidsen AS, Kousgaard MB. Enablers and barriers to implementing collaborative care for anxiety and depression: a systematic qualitative review. Implement Sci. 2016;11:165. https://doi.org/10.1186/s13012-016-0519-y.
Wood E, Ohlsen S, Ricketts T. What are the barriers and facilitators to implementing collaborative care for depression? A systematic review. J Affect Disord. 2017. https://doi.org/10.1016/j.jad.2017.02.028.
Davy C, Bleasel J, Liu H, Tchan M, Ponniah S, Brown A. Factors influencing the implementation of chronic care models: a systematic literature review. BMC Fam Pract. 2015;16(1):102.
Kadu MK, Stolee P. Facilitators and barriers of implementing the chronic care model in primary care: a systematic review. BMC Fam Pract. 2015;16:12.
Ritchie MJ, Parker LE, Edlund CN, Kirchner JE. Using implementation facilitation to foster clinical practice quality and adherence to evidence in challenged settings: a qualitative study. BMC Health Serv Res. 2017. https://doi.org/10.1186/s12913-017-2217-0.
Waxmonsky J, Kilbourne AM, Goodrich DE, Nord KM, Lai Z, Laird C, et al. Enhanced fidelity to treatment for bipolar disorder: results from a randomized controlled implementation trial. Psychiatr Serv. 2014;65:1,81–90.
O’Cathain A, Hoddinott P, Lewin S, Thomas KJ, Young B, Adamson J, et al. Maximising the impact of qualitative research in feasibility studies for randomised controlled trials: guidance for researchers. Pilot Feasibility Stud. 2015;1(1):32.
Baker E, Gwernan-Jones R, Britten N, McCabe C, Gill L, Byng R et al. Using interpersonal process recall to understand empowerment processes in a collaborative care intervention for people with a diagnosis of psychosis. 2018. Manuscript submitted for publication.
Barkham M, Bewick B, Mullin T, Gilbody S, Connell J, Cahill J, et al. The CORE-10: a short measure of psychological distress for routine use in the psychological therapies. Couns Psychother Res. 2013;13(1):3–13.
Tennant R, Hiller L, Fishwick R, Platt S, Joseph S, Weich S, et al. The Warwick-Edinburgh mental well-being scale (WEMWBS): development and UK validation. Health Qual Life Outcomes. 2007;5(1):63.
Cape J, Geyer C, Barker C, Pistrang N, Buszewicz M, Dowrick C, et al. Facilitating understanding of mental health problems in GP consultations: a qualitative study using taped-assisted recall. Br J General Pract. 2010;60(580):837–45.
Ritchie J, Spencer L. Qualitative data analysis for applied policy research. In: Bryman A, Burgess RG, editors. Analysing qualitative data. Abingdon: Taylor and Francis; 1994. p. 173–94.
Byng R, Norman I, Redfern S, Jones R. Exposing the key functions of a complex intervention for shared care in mental health: case study of a process evaluation. BMC Health Serv Res. 2008. https://doi.org/10.1186/1472-6963-8-274.
Coupe N, Anderson E, Gask L, Sykes P, Richards DA, Chew-Graham C. Facilitating professional liaison in collaborative care for depression in UK primary care; a qualitative study utilising normalisation process theory. BMC Fam Pract. 2014;15:78.
Knowles S, Chew-Graham CA, Coupe N, Adeyemi I, Keyworth C, Thampy HK, et al. Better together? A naturalistic qualitative study of inter-professional working in collaborative care for co-morbid depression and physical health problems. Implement Sci. 2013;8:110.
Murphy R, Ekers D, Webster L. An update to depression case management by practice nurses in primary care: a service evaluation. J Psychiatr Ment Health Nurs. 2014;21(9):827–33.
Bennett M, Walters K, Drennan V, Buszewicz M. Structured pro-active care for chronic depression by practice nurses in primary care: a qualitative evaluation. PLoS One. 2013. https://doi.org/10.1371/journal.pone.0075810.
Lau R, Stevenson F, Ong BN, Dziedzic K, Treweek S, Eldridge S, et al. Achieving change in primary care—causes of the evidence to practice gap: systematic reviews of reviews. Implement Sci. 2016;11:40.
Leamy M, Clarke E, Le Boutillier C, Bird V, Janosik M, Sabas K, et al. Implementing a complex intervention to support personal recovery: a qualitative study nested within a cluster randomised controlled trial. PLoS One. 2014;9:e97091.
Slade M, Bird V, Clarke E, Le Boutillier C, McCrone P, Macpherson R, et al. Supporting recovery in patients with psychosis through care by community-based adult mental health teams (REFOCUS): a multisite, cluster, randomised, controlled trial. Lancet Psychiatry. 2015;2(6):503–14.
Brand SL, Quinn C, Pearson M, Lennox C, Owens C, Kirkpatrick T, et al. Building programme theory to develop more adaptable and scalable complex interventions: realist formative process evaluation prior to full trial. Evaluation. 2018. https://doi.org/10.1177/1356389018802134.
This report is independent research funded by the National Institute for Health Research (Programme Grants for Applied Research, PARTNERS2: Development and pilot trial of primary care based collaborative care for people with serious mental illness, RP-PG-0611-20004). Nicky Britten and Richard Byng were partially supported by the National Institute for Health Research (NIHR) Collaboration for Leadership in Applied Health Research and Care South West Peninsula. Max Birchwood is partially supported by the National Institute for Health Research (NIHR) Collaboration for Leadership in Applied Health Research, West Midlands. The views expressed are those of the author(s) and not necessarily those of the NHS, the NIHR or the Department of Health. The funders had no role in the study design, data collection and analysis, the writing of the report, or the decision to submit the paper for publication.
Availability of data and materials
Transcripts will not be shared in their entirety to protect the anonymity of service users and care partners delivering the intervention. However, requests for excerpts of the data will be considered on an individual basis. Please contact the corresponding author.
EB is trained as a clinical psychologist and received her Clin.Psy.D. from the University of Birmingham. She is currently working as a research fellow on the process evaluation of the PARTNERS2 trial. Her clinical and research interests include recovery-based practice, especially as applied to psychosis.
NB is Professor of Applied Health Care Research at the University of Exeter College of Medicine and Health. She has a PhD in medical sociology and is an experienced qualitative researcher. Her research interests include patient-practitioner communication about medicines.
MC received a Bsc. (Hons) in Psychology from Lancaster University. She has received post graduate qualifications in Mental Health Research and Primary Care Mental Health Practice. She worked as a research associate on the PARTNERS2 project and has also worked within the NHS in both primary care and secondary care settings.
LGa trained in psychiatry at the University of Manchester, where she received her PhD, and is currently Emerita Professor of Primary Care Psychiatry. Her research interests are improving the quality of care for people with mental health problems in the setting of primary care, the management of depression and mental health communication skills. She also has personal experience of mental health problems.
LGi received an MSc in Social Research from the University of Plymouth. She worked on the Partners2 project as a Service User Research Assistant, bringing lived experience to the role. She also runs a voluntary help group for those with mild to moderate mental health problems.
RGJ received her PhD in Education from the University of Exeter and worked on the PARTNERS2 project as a research fellow. She is a qualitative researcher interested by participatory research, reviews and realist approaches. Her interests include disability, stigma and marginalisation, how these interact with identity and the social model of disability.
CMc received her Masters in Sociology from Simon Fraser University and worked on the PARTNERS2 project as a research assistant. Her research interests include mental health, sexuality and virtual spaces.
SR is Senior Lecturer in Health Research at the Lancaster University. Her research interests are improving the quality of care and evidence base for the delivery of health and social care services for people living with severe mental illness and people living with dementia.
AR received her DPhil in Public Health from the University of Oxford and worked as a research associate on the PARTNERS2 project. She is interested in using mixed methods to evaluate and develop health and social policy in relation to individuals from vulnerable communities.
Ethics approval and consent to participate
Ethical approval was obtained from the West Midlands – Edgbaston Ethics Committee (REC number 14/WM/0052). All participants provided written informed consent before taking part in this study.
Consent for publication
The authors declare that they have no competing interests.
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
About this article
Cite this article
Baker, E., Gwernan-Jones, R., Britten, N. et al. Refining a model of collaborative care for people with a diagnosis of bipolar, schizophrenia or other psychoses in England: a qualitative formative evaluation. BMC Psychiatry 19, 7 (2019). https://doi.org/10.1186/s12888-018-1997-z