This article has Open Peer Review reports available.
Longitudinal comparative evaluation of the equivalence of an integrated peer-support and clinical staffing model for residential mental health rehabilitation: a mixed methods protocol incorporating multiple stakeholder perspectives
© The Author(s). 2016
Received: 10 March 2016
Accepted: 20 May 2016
Published: 2 June 2016
A novel staffing model integrating peer support workers and clinical staff within a unified team is being trialled at community based residential rehabilitation units in Australia. A mixed-methods protocol for the longitudinal evaluation of the outcomes, expectations and experiences of care by consumers and staff under this staffing model in two units will be compared to one unit operating a traditional clinical staffing. The study is unique with regards to the context, the longitudinal approach and consideration of multiple stakeholder perspectives.
The longitudinal mixed methods design integrates a quantitative evaluation of the outcomes of care for consumers at three residential rehabilitation units with an applied qualitative research methodology. The quantitative component utilizes a prospective cohort design to explore whether equivalent outcomes are achieved through engagement at residential rehabilitation units operating integrated and clinical staffing models. Comparative data will be available from the time of admission, discharge and 12-month period post-discharge from the units. Additionally, retrospective data for the 12-month period prior to admission will be utilized to consider changes in functioning pre and post engagement with residential rehabilitation care. The primary outcome will be change in psychosocial functioning, assessed using the total score on the Health of the Nation Outcome Scales (HoNOS). Planned secondary outcomes will include changes in symptomatology, disability, recovery orientation, carer quality of life, emergency department presentations, psychiatric inpatient bed days, and psychological distress and wellbeing. Planned analyses will include: cohort description; hierarchical linear regression modelling of the predictors of change in HoNOS following CCU care; and descriptive comparisons of the costs associated with the two staffing models. The qualitative component utilizes a pragmatic approach to grounded theory, with collection of data from consumers and staff at multiple time points exploring their expectations, experiences and reflections on the care provided by these services.
It is expected that the new knowledge gained through this study will guide the adaptation of these and similar services. For example, if differential outcomes are achieved for consumers under the integrated and clinical staffing models this may inform staffing guidelines.
Community based residential rehabilitation for mental health consumers in Australia has become increasingly available through non-government organisations (NGOs) and public health services . These are bed-based services that focus on improving the independence and community functioning of persons affected by severe and persisting mental illness, predominantly those with a diagnosis of schizophrenia. The growth in availability of residential rehabilitation over the past 20 years has been linked in part to the recovery movement and research evidence promoting a more optimistic view of the potential for improvement among people with severe mental illness than has been previously assumed [2–4]. This paradigm shift has impacted the landscape of mental health policy and practice , facilitating a more holistic approach to treatment planning, and increasing the focus on addressing consumers’ functioning and attainment of personal goals . However, at this time, there is limited evidence to guide service users, service providers and funding bodies about the effectiveness of residential rehabilitation service models  and how they should function.
There is limited research examining the outcomes of clinically focused community residential mental health rehabilitation services in Australia; much of what is available focuses on the consumers following their deinstitutionalisation . With regards to non-clinical services, a 2012 consultation paper commissioned by the state of Victoria recommended discontinuation of bed-based adult rehabilitation services due to a lack of evidence of consumer outcomes and recovery oriented care . Despite limitations in the evidence base, there has been substantial recent investment in additional capacity by the Queensland Government, with six new clinically operated community residential mental health rehabilitation services (126 beds) expected to open over the 2015–2016 period . Novel approaches to the inclusion of peer workers have been considered for these units. There is a paucity of research to guide policy-makers, service providers and users as to the differences between models of staffing of residential mental health rehabilitation with regards to consumer preferences and outcomes. Better evidence about the effectiveness of these services, as well as the implications of integrating peer support workers into staffing models, is clearly needed.
Another important aspect of mental health policy and practice linked to the recovery movement is the increasing emphasis on the availability of peer support [10–12]. The concept of peer support has been formalised in roles such as ‘peer workers’ or ‘peer support workers’ where an individual with a lived experience of mental illness is employed with the expectation these experiences will be explicitly utilised in supporting consumers of the service . It is argued that this lived experience facilitates the sharing of experiential knowledge about coping strategies and pathways to recovery. Peer support roles are being increasingly encouraged and trialled as a component of mental health service provision. However, despite this strong policy backing, available evidence regarding the effectiveness of peer support has been criticised with regards to methodological rigor and potential bias [10, 13]. Much of the evidence relating to the value of peer support is qualitative in nature, and quantitative studies have been completed in outpatient and inpatient settings rather than community residential services. Notwithstanding these limitations, the literature suggests that peer workers employed as case managers achieve equivalent outcomes for service users in comparison with professional case managers, with potential advantages in terms of consumer engagement and benefits for peer workers and the services they work with [12–15]. The possibility of improved engagement through the inclusion of peer-support is promising, particularly given increasing interest in the impact of consumer engagement on rehabilitation outcomes [16, 17].
Different ways of incorporating peer workers into rehabilitation have been considered by non-clinical NGO services and clinically focussed public mental health services. Some of these services have utilised ‘buy in’ approaches whereby clinical time or peer support is made available from an external provider. Additionally, clinical services have increasingly employed peer workers in positions that provide advocacy and support to facilitate improved understanding of consumer perspectives by the clinical team . Each of these approaches have potential limitations. The employment of sole peer workers may be tokenistic, and there is a risk of peer worker values and attitudes assimilating with those of clinical staff over time. There may also be resistance to peer roles from clinical staff [13, 18]. Buy-in staffing models may not favour the potential for cross-fertilization of recovery paradigms that can occur from representing the peer work force as another discipline within a more integrated staffing model .
Continuing Care Units (CCUs) are a model of community-based residential rehabilitation care established in Australia in the 1990s as an alternative to long term psychiatric hospitalisation . The model involves cluster housing in a community setting, with onsite mental health professionals providing individualised care with the purpose of maximising independence in the community. CCUs were initially intended to provide permanent accommodation and support, however, the philosophy of care has evolved. While the focus on delivering individualised care remains, the units no longer focus on providing lifelong care for formerly institutionalised consumers [7, 21]. Rather, they provide time-limited rehabilitation support, currently between 6 and 24 months, preparing individuals to live more independently in an alternative residential setting . There is evidence to suggest that CCUs were able to meet their original purpose of providing an alternative and preferable care environment to long-term hospitalisation [7, 23]. However, many consumers continued to have significant disability and unmet needs post-discharge from the CCU . Despite CCU availability since the 1990s there has been limited literature exploring their outcomes since their inception, or the experiences of care from the perspectives of staff and consumers [7, 24].
Characteristics of the Coorparoo, Bayside and Logan CCUs, and their referring districts
Distance from state capital (km)
LGA Index of Relative Socio-economic Disadvantage, 2011a 
Availability of acute inpatient services
Availability of community mental services
Availability of inpatient rehabilitation mental health beds
Availability of transitional housing team
Availability of community based non-residential rehabilitation team
Availability of dedicated mental health homelessness team
Philosophy of care
Designated rehabilitation focus
Voluntary engagement in rehabilitationb
Individualised care planning
Role of peer support in care planning and delivery
Maximum occupancy (consumers)
Number of self-contained independent living units
Number of disabled access units
Shared recreation and leisure facilities
Available treatment & support
Individual psychotherapy support (CBT)
Living skills support and development
Structured leisure and physical activities
Evidence based therapeutic group programmes
Total FTE staff
Total FTE peer-support staff
Total FTE clinical staff
Peer support : Clinical staff ratio
Staff : Consumer ratio
“Recognises and embraces the possibilities for recovery and wellbeing created by the inherent strength and capacity of all people experiencing mental health issues
Maximises self-determination and self-management of mental health and wellbeing
Assists families to understand the challenges and opportunities arising from their family member’s experiences (p3).”
The concept of personal recovery is defined under the framework as: “being able to create and live a meaningful and contributing life in a community of choice with or without the presence of mental health issues” (p2).
“The CCU service is for adult consumers aged 18 to 65 years who have a severe and persistent mental illness. The symptoms substantially impair their psychosocial function and the capacity for independent living. These consumers have, or are likely to have, difficulty functioning within their community and have had problems accessing other services for support. It is expected that CCU consumers will benefit from 24-hour mental health care.
The diagnostic profile of eligible consumers includes individuals with a range of diagnoses associated with a severe and persistent mental illness, a cohort of whom may also have a comorbid drug or alcohol disorder. Consumers whose treatment is complicated by an intellectual disability and/or personality disorder are also eligible for this recovery-based consumer rehabilitation program. Consumers who benefit from participating are often not acutely unwell but in a recovery phase and readjusting to living in a social setting and require more support than is available to them in other settings such as home or social housing. This includes consumers who have symptoms that may be slow to respond to treatment or experience behavioural disturbances that make living in alternative community settings difficult” (p7).
The aforementioned gaps in the literature regarding the outcomes of engagement with CCU care, and experiences of care, are concerning given the investment in new service capacity that has occurred over recent years. The opening of the two new CCUs has presented a unique opportunity to evaluate the outcomes and experiences of care associated with CCUs offering an integrated staffing model in comparison to a traditional clinical staffing model. Understanding consumer perspectives is critical to ensuring service delivery contributes to recovery outcomes. Additionally, understanding staff expectations and experiences of working in these services presents an opportunity to explore the function of these units and the extent to which this aligns with the designated model of service. This study protocol outlines the proposed collection of evaluative data from both consumers and staff, across multiple stages of the implementation of the new staffing model. Data from the clinical staffing model will be collected to facilitate comparative analysis between traditional clinical and integrated staffing models.
Aims and benefits
The study examines the outcomes of engagement with CCUs, and considers the impact of the integrated staffing model on both the outcomes and experiences of care in comparison to a clinically staffed CCU.
The operation of an identical model of service across both the clinical and integrated sites, hence the expectation that equivalent care will be delivered. If equivalent care was realised was a staffing model where the majority of staff are employed on the basis of their ‘lived experience’ rather than clinical expertise this would support future decision-making about staffing on the basis of secondary considerations  such as cost-effectiveness and consumer preference.
Limitations in the quantitative evidence base relating to the impact of the integration of peer support to justify a hypothesis of superiority. Where quantitative findings are available these have tended to discuss equivalent outcomes to clinical staff. Furthermore, a failure to demonstrate superiority of the integrated model would not answer the question of whether such a staffing approach may be appropriate to be considered by other services .
The limitations of testing a non-inferiority hypothesis with regards to establishing the equivalence of the two models where qualitative differences in the experience of care are theorised, and the impact of these on consumer outcomes is unknown.
Who are the consumers who currently utilise community based residential rehabilitation services (cohort description)?
Does provision of CCU rehabilitation care result in change in domains of relevance to serious and persistent mental illness between admission and discharge (secondary outcomes)?
What factors predict a consumer psychosocial outcome through the provision of CCU based rehabilitation support (hierarchical linear regression modeling)?
How do the operational costs associated with running a CCU under the clinical and integrated staffing models compare (description)?
Defining and comparing the expectations of care at the CCU under the integrated staffing model for consumers and staff
Comparing the experience of care in a traditional clinical staffing model with the novel integrated model for both consumers and staff
Examining reflections on the experience of rehabilitation care post-discharge for consumers under both the clinical and integrated CCU staffing models.
Principles of participatory action research  have informed the approach to the qualitative component of the study. Both consumer and staff participants will be actively involved as co-creators of knowledge during the analysis and manuscript preparation. Additionally, this knowledge is expected to drive future service development.
The understanding of the residential rehabilitation service and the implications of the integrated staffing model achieved through this project will have implications for future service planning. The outcomes will be relevant to the staffing and processes of similar units in Australia, as well as guiding service providers internationally about novel ways of incorporating peer support into routine clinical care.
The mixed methods approach will facilitate understanding of not only ‘what’ works but also consider ‘why’ this works and for whom, as well as ‘how’ similar services may be modified to better meet the needs of consumers. The approach has relevance given the socially complex nature of these services , the limited literature examining similar services internationally, and the acknowledged uncertainty as to exactly what mental health rehabilitation services are and whom they should be for . Whilst the quantitative and qualitative aspects will be dissociable, the qualitative analysis will provide depth to understanding emergent outcomes as well as the opportunity to complete secondary analyses focussing on subgroups of participants achieving favourable and unfavourable outcomes through the provision of residential rehabilitation support.
Ethical approval and informed consent
Both the University of Queensland and the Metro South Human Research Ethics Committee have provided ethical clearance for this study (HREC/14/QPAH/62). Participation of both staff and consumers is on the basis of voluntary informed consent. The CCUs accept consumers aged 18–65 who have a severe and persistent mental illness, predominantly schizophrenia, that substantially impairs psychosocial functioning and capacity for independent living . For those consumers lacking capacity to provide this consent, the consent of their relevant guardian for the inclusion of routine assessment and outcomes data in the evaluation will be sought. The nature of the care delivered to a consumer at the CCUs will not be altered by study participation. A small gratuity for consumers involved in the semi-structured individual interviews is provided. This gratuity recognises the investment of participant time, and the limited direct benefits through participation.
Routine clinical assessment battery and administrative data collection over the baseline, admission, discharge and follow-up collection points
(0–6 weeks post)
(0–6 weeks prior)
Health of the Nation Outcome Scale (HoNOS) A
Life Skills Profile (LSP-16) B
Mental Health Inventory (MHI-38) B
Mental health related Emergency Department presentations
Psychiatric inpatient bed days
Clinical assessment battery
Alcohol Use Disorders Identification Test (AUDIT) 
Brief Psychiatric Rating Scale (BPRS) 
Scale for the Assessment of Negative Symptoms (SANS) 
Allen’s Cognitive Levels (ACL) 
Social Functioning Scale (SFS) 
Perceive Recall Plan & Perform System of Task Analysis (PRPP) 
Stages of Recovery Instrument (STORI-30) 
Burden Assessment Scale (BAS) 
Adult Carer Quality of Life (AC-QoL) 
The project is currently in progress. The cohort will comprise all consumers admitted to the three MSAMHS CCUs (Bayside, Coorparoo and Logan) between December 2014 and December 2017 who provide informed consent to participate. Baseline and follow-up data will be sourced from administrative datasets for the 12-month period prior to and following CCU admission and discharge respectively.
As the study takes a naturalistic observational design, the only defined inclusion criteria is acceptance for a trial of CCU based rehabilitation care. Each CCU provides a six-week assessment period to establish suitability for engagement under the model of service. Given this, consumers who are discharged prior to the conclusion of this assessment period will be excluded from the study. This exclusion criterion assists in ensuring that included participants are aligned with the model of service. This reflects a modified intention-to-treat approach , as participants will be included regardless of their mode of exit from the CCU (planned/unplanned (CCU initiated)/unplanned (consumer initiated).
Based on anticipated service throughput, with a median length of stay at the Coorparoo CCU of 0.9 years (inter-quartile range 0.50-1.8 years) at a point estimate in 2013, and assuming a 70 % response rate, it was expected that at least 60 consumers from the integrated staffing model and 40 from the clinical staffing model would agree to participate over the study period. This minimum sample likely reflects an under-estimate given 37 unique admissions occurred to the Coorparoo CCU in the 12-month period from July 2012. Additionally, since the opening of the Bayside and Logan CCUs 50 of the 63 consumers admitted across all three sites met the inclusion criteria (79.4 %) and 42 (84.0 %) of eligible consumers provided consent to participate.
The adequacy of the estimated minimum sample size was considered using the Epi Info™ StatCalc program  for unmatched or cross-sectional cohort studies. This minimum sample was estimated to provide greater than 80 % power to detect a 15 % difference in outcome on the primary outcome measure at the 95 % confidence interval using the Fleiss calculation. The acceptability of this power is supported by an emergent difference of 20 % in outcome HoNOS scores in a recent Australian study examining outcomes of a community residential service for similar consumers .
Primary outcome measure
Has high validity, reliability and sensitivity to change, as well as routinely informing service policy and planning decisions in Australia ;
Is considered appropriate for tracking changes in social functioning over time, and more sensitive in doing so than the CANSAS (a needs based measure) ;
Is supported by an established national assessment protocol and training processes to facilitate inter-rater reliability , and with evidence to support higher levels of acceptability  and completion than the other core measures of routine outcome suite 
It is used and available across multiple international settings , increasing the replicability of the methodology.
Utilising the criterion of the highest (i.e. greatest dysfunction) score in the 12-month period pre and post CCU entry limits the risk of conflating the benefits of active rehabilitation support with the benefits associated with support and accommodation in the CCU setting.
Planned secondary outcome measures and descriptive data
Completion of relevant measures and explanation of routine clinical assessment battery
Responsible person for ensuring completion
Principle service provider
Notes on assessment battery
Life Skills Profile (LSP-16)a
•These measures form part of the routine assessment battery to be completed with all consumers residing at the Community Care Units.
•The battery was developed on the basis of the clinical experience and knowledge at the Coorparoo CCU, with extensive staff consultation.
•Standardisation of the assessment process provides an opportunity to better evaluate and compare consumer outcomes, both at the individual and group level.
•Whilst there many other relevant measures, the battery chosen aims to broadly cover relevant domains in a manner that does not place excessive demands on consumers, carers and staff.
•There will not be an opportunity to change the components of the battery for the duration of the CCU evaluation project. For this reason, it will be critical to find ways in which the chosen measures can be utilised to enhance the quality and experience of care at the units.
•It is expected that the measures completed by carers and consumers will form the basis of a subsequent discussion with members of the clinical team.
•It is likely that multiple team members may be present at the time a given measure is collected given the multi-disciplinary nature of the work undertaken at the CCUs.
•The Principle Service Provider role under both the integrated and clinical staffing models is performed only by clinical staff members.
•For the purposes of this study a Carer is considered to be a person (e.g. family member or friend) who the Consumer nominates as having a significant unpaid caring role for them.
Mental Health Inventory (MHI)b
Health of the Nation Outcome Scale (HoNOS)a
Alcohol Use Disorders Identification Test (AUDIT)c
Brief Psychiatric Rating Scale (BPRS)d
Scale for the Assessment of Negative Symptoms (SANS)d
Allen’s Cognitive Levels (ACL)d
Social Functioning Scale (SFS)c
Perceive Recall Plan & Perform System of Task Analysis (PRPP)d
Stages of Recovery Instrument (STORI-30)b
Burden Assessment Scale (BAS)b
Adult Carer Quality of Life (AC-QoL)b
Planned secondary outcome measures will be derived from this assessment battery and routine administrative data sets. Secondary outcome assessment will specifically focus on mean changes in symptomatology (Brief Psychiatric Rating Scale (BPRS)  & Scale for the Assessment of Negative Symptoms (SANS) ), disability (Life Skills Profile-16 ), recovery orientation (Stages of Recovery Instrument (STORI-30) ), carer quality of life (Adult Carer Quality of Life (AC-QoL) ), emergency department presentations, psychiatric inpatient bed days, and psychological distress and wellbeing (MHI-38) ). For each of these secondary measures it is again hypothesised that no differences in mean change will emerge between the clinical and integrated staffing models.
Descriptive details of consumer characteristics and care experience will also be recorded. Relevant information will include: age at CCU commencement, gender, ethnicity, marital status, years of education, diagnosis, involuntary treatment order status, medication, length of stay, and the mode of exit from the CCU (planned/unplanned (consumer initiated)/unplanned CCU initiated).
Information about the operational costs under the integrated and clinical staffing models over the December 2014 - December 2017 period will be reported descriptively. Additional information about the operational experience under the integrated and clinical staffing models including staff retention and absences will also be reported. Estimates of the cost of ongoing service utilisation following CCU engagement based on ED Presentations and acute inpatient bed days in the 12-months post-CCU discharge will be calculated using the most recently available state-wide cost estimates.
Planned predictors for hierarchical linear regression modelling
A limited number of predictors of positive psychiatric rehabilitation outcome have been supported by the international literature over the past decade. These include: medication adherence [52, 53]; lower baseline disability scores ; family engagement with the program and provision of peer support/self-help ; increasing age [53, 54]; female sex ; being diagnosed with schizoaffective disorder rather than schizophrenia  or any disorder other than schizophrenia ; and having been engaged with formal education . The contribution of two potentially relevant predictors, namely cognitive function and rehabilitation engagement had not been explored. Cognitive functioning has emerged as a strong predictor in the vocational rehabilitation literature  as well as of functional outcomes in schizophrenia generally . There has also been discussion about the importance of considering consumer engagement as a factor in rehabilitation outcomes [16, 17], and there is clear face validity of this concept considering the proportion of consumers receiving support from rehabilitation services who are not engaged with the available support [17, 58] Additionally, alcohol use has historically been the most common non-tobacco related substance use issue arising in Queensland CCUs and has been anecdotally perceived by staff as detrimental to the rehabilitation process.
Planned variables for inclusion in hierarchical linear regression modelling
3 - Organisational
Features with potential impact on consumer experience
2 - Site
Length of Stay
1 - Consumer
Based on previous prediction studies
Categorical (Schizophrenia/Schizoaffective Disorder/Other)
LSP-16 (Item 5)
Psychiatric inpatient bed days
Mode of discharge
Categorical (Planned/Unplanned - Consumer initiated/Unplanned - CCU initiated)
Identical processes for data collection will operate at the three sites. The clinical assessment incorporates a combination of clinician, consumer, clinician-and-consumer and carer/family completed measures (see Table 2). For the purposes of this study a Carer is considered to be a person (e.g. family member or friend) who the Consumer nominates as having a significant unpaid caring role for them. Staff for which each measure reflects their core-skills set and routine practice will be responsible for measure completion (see Table 3).
The research protocol was explained to staff prior to the commencement of data collection. Each discipline completing relevant assessments will work to maximise fidelity through regular peer based supervision. Standardised training in the use of routine outcome measures is provided to all staff, this includes opportunities to calibrate scoring of HoNOS and Life Skills Profile-16 (LSP)  across the study sites. Fidelity of data relating to symptom measures will be enhanced through the use of the anchored scoring version of the Brief Psychiatric Rating Scale (BPRS-A)  and a semi-structured interview guide developed in an Australian context for raters with low-clinical experience , as well as a structured interview guide for the Scale for the Assessment of Negative Symptoms (SANS) measure . Inter-rater reliability data will also be calculated from calibration sessions for the BPRS and SANS involving relevant staff.
A research assistant will collate the data from the pre-admission and post-discharge phases from established administrative data sets, where the relevant information is routinely collected.
As is commonly expected in health research [65, 66], it is anticipated that missing data will be present in both the administrative datasets and assessment battery. Categorical codes will be available for non-completed consumer and carer measures, and for clinician rated measures dependent on structured assessment. These codes will designate the reason for non-completion as being due to refusal by the consumer/carer or due to the measure not being offered to the consumer/carer. Missing Values Analysis will be completed in SPSS Version 22 and it is planned for missing data to be handled using the multiple imputation method  assuming this is missing at random. Characteristics and patterns of missing data, and the assumed mechanisms will be reported.
The cohort of consumers utilising the services across the three sites will be defined using descriptive statistics. Differences between the sites and staffing models will be examined using t-tests and chi-squared analyses. Assuming comparability of the cohorts is established it is planned that the data from the two integrated staffing model sites will be able to be combined.
In order to answer the question of equivalence of the integrated and clinical staffing model for CCUs, differences in the total HoNOS score pre admission and post discharge will be assessed using a t-test or Mann–Whitney U test depending on the normality of the distribution.
The question of whether CCU rehabilitation care results in change in specific clinically relevant outcome domains, will be answered through analysis of the secondary outcome measures. It is anticipated that data from the secondary outcome measures will be dichotomised and subjected to Mann–Whitney U analysis with appropriate correction for multiple comparisons.
Predictors of change in psychosocial functioning following CCU care will be explored using hierarchical linear regression modelling. The dependent variable is the same as the primary outcome variable for the overall study, mean change in total HoNOS score pre admission and post discharge from CCU care. The independent variables will be entered cumulatively with the organisational (Level 3) variable considered first, followed by the site related (Level 2) variables and then the consumer (Level 1) variables. A hierarchical approach is appropriate given the expectation of inter-correlation between the predictors .
Planned data collection phases, timeframes, focuses and estimated sample size for each cell of data collection
Clinical staffing model
Integrated staffing model
Site 1 (Coorparoo)
Site 2 (Bayside)
Site 3 (Logan)
Expectations of carea
n = 4 (6–10)
n = 4 (6–10)
n = 4 (6–10)
n = 4 (6–10)
n = 4 (6–10)
Experience of care
12-18 months post-commencement
n = 4 (6–10)
n = 4 (6–10)
n = 4 (6–10)
n = 4 (6–10)
n = 4 (6–10)
n = 4 (6–10)
Reflections on care and transition
12-18 months post-discharge
n = 4 (6–10)
n = 4 (6–10)
n = 4 (6–10)
Number of estimated total interviews
n = (6–10)
n = (18–30)
n = (12–20)
n = (18–30)
n = (12–20)
n = (18–30)
Estimated total sample pool for each site/group (N)
A sub-sample of consumers who consent to have their data included in the quantitative analysis from each site will participate in the sequential semi-structured interviews. Within each cell for qualitative data collection, convenience sampling is employed due to the time constraints associated with the collection of expectation data as well as the limited size of the total sample pool in each case. The aim is to be as inclusive as practicable as well as consistent with a grounded theory approach by sampling to achieve thematic saturation. However, the requirements of exhaustive sampling must be balanced against the relatively small sizes of these groups and the need to avoid the possibility of creating real or perceived pressure on staff and consumers to participate if they do not want to. For the integrated staffing model sites, a balance of peer and clinical worker interviews is planned.
A minimum of four interviews will be completed for each cell of data collection prior to determination of the final sample size. We expect the total size for each cell to be between six and ten, but, consistent with a grounded theory approach, sample size will depend on considerations of when, and whether, thematic saturation is reached, as well as the level of interest from each consumer or staff group at each site and time point.
During Phase One consumers commencing at each of the three CCU sites were approached to be involved in the evaluation project during the initial six weeks of their care. Interview participation was prioritized on the basis of earliest provision of consent, and the availability and willingness to complete the interviews at the proposed time. Effort was made to involve the same consumer participants at each of the two subsequent interviewing phases (Phase 2 and Phase 3), with the option to include new participants in the case of attrition or iterative need. Consumer inclusion in the Phase 2 and 3 interviews will be prioritized on the basis of:  participation in Phase 1 interviews;  balancing consumers who remain resident at the CCU with those who have left;  for consumers no longer resident at the CCU balancing those who had planned and unplanned (consumer and CCU initiated) discharges; and  availability at the designated interview time. Consumers will be given the opportunity to participate in feedback and discussion sessions regardless of whether or not they were involved in the interviews.
During Phase One staff commencing at the integrated model sites were approached to be involved in the evaluation project in the initial six weeks of their employment. Phase Two will include both the clinical and integrated staffing model sites. Staff interview participation during Phase 1 was prioritized on the basis of achieving a balance of clinical and peer workers from the integrated staffing model sites. Additional considerations in prioritization of interview allocation were the earliest provision of consent and the participant’s availability and willingness to complete the interviews at the proposed time. For the integrated staffing model participants, effort will be made to involve the same staff in the Phase One and Two interviews, with the option to include new participants in the case of attrition or iterative need. Additional considerations in the prioritization of interview allocation for Phase Two staff interviews will be:  balancing clinical and peer staff roles at the integrated staffing model sites and  proportionate balancing of staff who have stayed on and those who have left at the integrated staffing model sites.
During each interview a trained independent interviewer will encourage participants to elaborate on relevant themes that emerge, and to give their reasons for their perspective and suggestions for improvements to service delivery. Initially an inductive approach will be taken in the analysis of each data cell. Confirmation or refutation of emergent themes and theory will be guided by an inductive-deductive interplay including involvement of staff and consumer participants in feedback and discussion. Data collection, analysis and theorising will occur in tandem from the outset.
Principles of participatory action research  have informed the approach, with efforts made to collaboratively engage staff and consumers in the co-creation of knowledge. Attention will be provided to fostering democratic principles, including individual and group discussion with consumers and staff to encourage and maintain buy in and ownership of the qualitative components of the evaluation. The articulation of dissenting opinions will be valued in building the understanding required for incremental improvement to services. Efforts to offset the inherent power differentials affecting consumers in an institutional setting will be made through the use of an independent interviewer and provision of payment for participation. Explicit statements will be made about the role of this gratuity as a token of acknowledgement of the value of consumer involvement. Additionally, the independence of the interviewer, the availability of choice in interview setting, staff review and approval of transcripts, and the de-identification of transcripts will work towards creating a ‘safe space’ for consumers and staff to disclose their personal opinions. Multiple opportunities for participant involvement and feedback will be made available during the analyses, around emergent concepts and potential action.
All interviews will be completed by EN, a research assistant and doctoral level psychology candidate. EN is not involved in the delivery of clinical assessment or care at the CCU sites. CM, an experienced qualitative researcher, has provided training in qualitative interviewing to EN prior to project commencement; and the research team will provide regular supervision and guidance including early review of initial transcripts. The initial interview schedules (Appendix 1) will evolve iteratively in consultation amongst the researchers following each interview. Both consumer and staff interviews are expected to take between 30-min and 1-h to complete.
Consumer interviews for all Phase 1 interviews were conducted in a private non-clinical area on the grounds of the CCU. The option of completing Phase 2 interviews at a neutral site or by telephone will also be available for consumer participants. The AUD$25 gratuity will be provided to consumer participants following each interview in acknowledgement of their participation. All staff interviews will be completed during their work hours in a private room, and for Phase Two interviews staff will be given the option of completing the interview offsite. All interviews will be audio-recorded, independently transcribed by an external transcription company and then de-identified by EN. For the staff interviews the de-identified transcript will be returned via email to participants for approval prior to them being made available to the research team for analysis. During this approval stage staff will be able to edit their transcript. The de-identification of transcripts, conducting interviews in non-clinical areas and providing interview transcripts to staff to review, are measures intended to contribute to the creation of a safe space for participation.
The decision to provide the option of transcript review to staff participants was made based on concerns about the re-identifiability of staff participants, given the relatively small target population and that the investigation was initiated by senior staff within the mental health service. Whilst the interview procedures were designed to establish a safe space for participation, it is foreseeable that staff may experience anxiety about how their information may be utilized and its implication on their personal employment situation. This poses a possible barrier to participation and/or disclosure. Providing additional review overcomes this barrier by allowing staff participants an opportunity to remove any information if they have such concerns. The number of participants who elect to edit their transcripts and the nature of this editing (e.g. addition, redaction, variation or some combination of these) will be detailed in the results. These risks were considered lower for consumer participants, however, the decision not to provide transcripts to consumer participants for review was made for logistic reasons, and reflects a limitation of the methodology.
Interviews at each time point are open-ended and allow the researchers to assess how priorities and preferences, as well as views, change over time. The semi-structured interviews guide the interviewee through topics in historical order, so that they might recall earlier experiences first and more recent experiences last. Open-ended questions are presented initially, with the interview then tapering to more direct prompts. This process allows participants to discuss matters most important and relevant to them, without prejudging what these may be, while also ensuring maximum consistency and comparability in topics discussed across interviews. Both interviewer and interviewees are encouraged to think of the interview as a conversation.
The interviewer will explore emergent themes as the interviews progress, particularly where these appear to diverge from those previously identified. Where an interviewee provides a lengthy answer to a question, the interviewer will paraphrase what they understood to be the key point being made and ask for clarification and confirmation on this. This is anticipated to be important to being able to represent the views of consumers where formal thought disorder or delusional thinking is present and it will also provide additional means of verifying the interviewer’s interpretations of participants’ views. The interviewer will rephrase and simplify questions that are misinterpreted or not understood. In the case of emergent distress, the interviewer will seek ongoing consent for the interview to continue.
The two principal investigators involved in the qualitative aspects of the study, SP and FD, work within the services in clinical leadership roles. From this ‘insider’ position they are theoretically sensitised and familiar with the relevant literature . Potential conflicts relating to the dual role of clinical-researcher are acknowledged. Other members of the research team are engaged from the ‘outsider’ position, with preconceived ideas based on broader knowledge and experience, but not on the specifics of the organisations being evaluated. The involvement of CM and EN in the planning, implementation and review will help facilitate regular discussion and consideration of investigator positionality, and acknowledgement of preconceived beliefs and values of all investigators, to ensure that such prior knowledge does not unduly distort the iterative analysis of empirical material collected.
De-identified transcripts will be uploaded into an electronic database (NVivo10)  for analysis. Multiple researchers (SP, FD, EN, CM) will review the initial transcripts for each data-collection phase, working collaboratively towards consensus on an initial coding scheme. This discussion will inform revision of the interview schedule and final decisions around sample sizes to achieve thematic saturation. The coding process will pay attention to, and distinguish between, first order themes relating to concrete and easily articulated issues and experiences (e.g. specific concerns about infrastructure, work conditions and/or interventions) as well as second order themes that may be more nebulous (e.g. pertaining to identity, aspirations and relationships). The research team will work collaboratively to consider limitations in the coding, and identify instances where theorising ceases to be adequately grounded in the available data. Multiple tactics will be consciously applied to generating meaning through analysis at each cell of the data collection . These tactics will include generating meaning, testing and confirming hypotheses and considering the quality of the conclusions.
In the case of apparent convergence between the two integrated staffing model sites it is anticipated that the data will be analysed in composite for both staff and consumers at these sites. The resultant themes will be used to examine the similarities and differences between staff and consumer expectations and experiences of care under the two staffing models. Additionally, comparison and efforts to link consumer expectations, experience and reflections on care under the two staffing models will be used to develop our understanding of the consumer journey through these units. Feedback on both the content and concepts raised in interviews will be presented at least 6-monthly to consumer and staff groups at each site for feedback and consideration of relevant adaptations to service delivery that can be made. A record will be kept of changes to service delivery based on feedback at each time point in order to provide relevant context and depth to the analysis. Draft manuscripts will be made available to interested participants for comment prior to submission for publication.
Comparison of emergent themes relating to the integrated and clinical staffing models will inform the understanding and interpretation of the quantitative findings. Matching of participant identification codes for quantitative and qualitative data will allow focused exploration of experiential aspects of groups of consumers identified as benefiting and non-benefiting from CCU support. Additionally, the Phase 2 and 3 interview schedules may be adapted in response to initial data and analyses from the quantitative data-set.
The protocol is novel in both the focus of investigation as well as the longitudinal, multi-perspective approach with intentional combination of quantitative and qualitative consumer data over multiple time points. This approach allows a rich understanding of the dynamic nature of the phenomena under consideration. The naturalistic design increases the relevance of the study outcomes to the delivery of complex interventions to consumers in real world clinical settings. The resultant data set will facilitate understanding of whether equivalent outcomes can be realized under a staffing model where the majority of the members of the rehabilitation team are employed on the basis of lived experience of mental illness, as well as how this staffing model impacts the experience of care.
A conceptual limitation of the study is the emphasis on the CCU and it staff as a key consideration in the outcomes of people with a diagnosis of serious and persistent mental illness. The factors impacting on entry to, exit from and the outcomes of rehabilitation care are likely multifactorial and dependent on resources and supports external to the unit . Considerations in the socio-economic environment include the availability of housing, employment and welfare opportunities; funding of external clinical and non-clinical services, and broader issues relating to social and structural stigma impacting functional outcomes. Exploration of these external factors is beyond the scope of the present study, but may emerge in the qualitative analysis.
Several limitations to the quantitative aspects of the methodology need to be considered. The primary outcome (HoNOS) measures constructs of relevance to clinically defined recovery outcomes rather than personal recovery outcomes or the recovery process. Reliance on the HoNOS in Australia has been criticized with reference to its focus on consumers deficits and problems . Additionally, the lack of correlation between this measure and consumer defined recovery measures has been noted . However, it has been suggested this finding is best explained in relation to the difference between clinical and consumer rated measures . Both the qualitative and secondary quantitative analyses will be important in interpreting the meaningfulness of observed change or failure to observe change in the primary outcome measure.
There are inherent limitations with relying on routinely collected administrative and clinical data, including missing data and recording bias . Furthermore, the absence of blinding of raters provides a potential source of bias, however the primary outcome assessments in during the baseline and discharge periods will be completed by trained assessors independent of the CCU context. The absence of randomized allocation between the staffing models hinders our ability to make causal inferences regarding relationships between service provision and changes in outcome over time. However, the inappropriateness of running randomized placebo-controlled trials in socially complex services with reference to ethical and practical barriers has been acknowledged in the literature . Regardless, the utility of equivalence testing depends in part on the underlying assumption that the comparator has been convincingly proven to have superior efficacy to placebo . This assumption cannot be definitively asserted on the basis of the available literature.
The naturalistic approach to sample selection confers both benefits and costs. Whilst the defined exclusion criterion aims to facilitate inclusion of only participants initially deemed to align with the model of service, this inclusive approach risks masking the effectiveness and differences between the two treatment arms through the inclusion of participants who do not complete their planned rehabilitation. Whilst the option of a per-protocol analysis approach , including only those participants whose CCU discharge was planned, will be available such an approach would not reflect the reality of rehabilitation practice. It has been argued that limited benefits are to be expected through provision of rehabilitation care without consumer engagement in the process . There is an emerging interest in engagement scales for use in residential rehabilitation settings, including adaptation of an established measure used in assertive community treatment . However, the broad suitability of this measure in application has not yet been established. Engagement with rehabilitation support over the CCU stay is not being specifically measured in this study. It is anticipated that the qualitative interviews with consumers will provide rich detail about relative levels of engagement under the two staffing models. Length of stay (whether a consumer remains for the minimum 6-month length of stay detailed in the model of service) will provide an additional proxy measure of engagement.
The longitudinal approach to sampling adds additional complexity to the evaluation. There is an expectation that service delivery will change over time. Generalizability of findings may also be affected by the expectation of incomplete sampling and site related differences in referral patterns. Given this, careful consideration of the characteristics of participants and non-participants at each is site is warranted.
The qualitative aspects of the methodology deviate from a purist reading of grounded theory insofar as it acknowledges the applied importance of reporting (largely descriptive) content as part of a formative evaluation. However, the methodology does work towards fulfilling the key criteria by which qualitative research is judged i.e. credibility, originality, resonance and usefulness . The approach provides opportunity to derive both theory and describe content that is relevant and responsive to the needs of mental health service planners, staff and consumers. Practical and theoretical complexities associated with longitudinal qualitative analyses revolve around the task of constructing a consistent narrative that weaves through expectations to reality and reflections on care at the CCUs while preserving the ability to represent changes in participants’ priorities and preferences over time. It cannot be assumed that what is important and relevant to staff and consumers will remain stable, and that the evaluation can simply measure changes in their views on matters raised at the beginning of their journey. The analyses need to be sensitive to the evolving nature of the services under investigation. These changes need to be documented over the course of the study.
The qualitative component of the study does not include the voice of carers. Carers are identified as a key stakeholder in mental health service provision . The omission of carers from the qualitative analysis reflects pragmatic limitations in the ability to realise a comprehensive longitudinal evaluation. Future exploration of the expectations and experiences of CCU based care from the perspectives of carers is encouraged, and it is possible that divergence from the views of both staff and consumers may emerge.
Multiple sources of potential bias need to be considered in the qualitative analysis. Retrospective biases are likely to impact comparison recall of alternative care and work settings for both consumers and staff . There are specific challenges associated with the consumer sample. The risk of deficits and bias in the self-report of persons with schizophrenia, especially around functional capacity, are well documented . There is also a risk that the provision of a gratuity to consumers creates an undue incentive to participate , and that a self-selected and financially incentivized consumer group do not adequately reflect the general population of service users . The inherent power differential associated with the consumer role within the service under investigation creates a risk of pseudo-participation . Additionally, for staff participants the dual roles of several members of the research team may inhibit more negative disclosures about their work experiences. Given these issues, careful and ongoing attention to the collaborative involvement of participants and the creation of a safe space for participation is needed.
There is very limited research exploring the outcomes and experience of community based residential mental health rehabilitation care. The generalisability of results from the present study will be limited by the focus on a single model of service and two sites for the integrated staffing and a single site for the traditional clinical staffing model. It is possible that idiosyncratic aspects of practice at each site or under the two staffing models will impact on the outcomes of the study. However, the richness of detail facilitated by taking a longitudinal mixed methods approach incorporating multiple stakeholder perspectives is anticipated to facilitate relevant inferences to guide practice in alternative contexts.
ACL, allen’s cognitive levels; AC-QoL, adult carer quality of life; AUDIT, alcohol use disorders identification test; BPRS, brief psychiatric rating scale; BPRS-A, brief psychiatric rating scale – annotated; CCU, community care unit; LSP, life skills profile - 16; MHI-38, mental health inventory - 38; MSAMHS, metro south addiction and mental health service; NFROMHS, national framework for recovery oriented mental health service; NGO, non-government organisation; PSW, peer support worker; RANZCP, royal Australian and New Zealand college of psychiatrists; SANS, scale for the assessment of negative symptoms; STORI-30, stages of recovery instrument.
The consumer component of the qualitative evaluation has been funded in part by a RANZCP New Investigator Grant. MSAMHS have provided additional funding to support this project.
Availability of data and material
Informed consent for publication of the data set has not been sought from participants or approved by the MSAMHS institutional review board. External storage and/or publication of the de-identified clinical data set is inconsistent with the data-security policies of the MSAMHS.
SP, Initial draft of protocol, finalisation of ethics application, funding applications, data analysis, manuscript drafting and review. FD, Concept development, authorisation of in kind support from MSAMHS, content expertise (rehabilitation), data analysis, review of protocol and manuscripts. EN, Initial drafting of ethics application and associated forms, review of protocol, completion of semi-structured interviews, data analysis, manuscript drafting and review. NK, Contribution to the development and piloting of the assessment battery, review of protocol and the qualitative interview schedule. CM, Content expert (qualitative methods), review of protocol, funding applications, data analysis, manuscript drafting and review. DS, Review of protocol and provision of advice regarding quantitative methodology and mixed methods synthesis. MH, Review of protocol and provision of advice regarding quantitative methodology and mixed methods synthesis. All authors read and approved the final manuscript.
SP is a male early career Psychiatrist working at the Rehabilitation Academic Clinical Unit of MSAMHS. He has a clinical leadership role at the Logan and Bayside CCUs and is involved in the delivery of direct care. He is also a PhD candidate with the School of Population Health at the University of Queensland, Australia, working on the topic of ‘What works for whom in community based mental health rehabilitation.’ He holds the qualifications of Cert Adult Psych, Cert Psychotherapy, FRANZCP, BSc (Psychology, Hon 1) and BBus (Man).
FD is a female senior Psychiatrist who is the Clinical Director of the Rehabilitation Academic Clinical Unit of MSAMHS. She has a clinical leadership role covering the full range rehabilitation services provided by MSAMHS. Within this role she provides direct clinical support at all three CCU sites on an infrequent basis. She is a PhD candidate with the School of Population Health at the University of Queensland, Australia, working on the topic ‘Comparing the implementation and dissemination of cognitive behavioural therapy for psychosis (CBTp) and cognitive remediation (CRT) in two mental health service districts.’ She holds the qualifications of FRANZCP and MBBS.
EN is a female research assistant within the Rehabilitation Academic Clinical Unit of MSAMHS. She is a provisional psychological and doctor of clinical psychology candidate with the School of Psychology and Counselling at Queensland University of Technology, Australia. Her doctoral dissertation addresses the determinants of parental and psychological functioning in families of children with congenital heart disease. She holds the following qualifications BPsySc (Psychology), BPsych. Hons (Psychology).
NK is a female Psychiatrist working at the Rehabilitation Academic Clinical Unit of MSAMHS. She has a clinical leadership role at the Coorparoo CCUs and is involved in the delivery of direct care. She holds the qualifications of FRANZCP and MBBS.
CM is a female Post-doctoral Research Fellow with the School of Public Health, at the University of Queensland, Australia. Her interests are in integrating cross-disciplinary perspectives to improve the ways in which new ideas and technological innovations are used to address societal needs and challenges, ethically and effectively. To this end, her research is based on the principles of user-centred design and she carries out mixed methods research and evaluation, including stakeholder engagement and analysis of media and communication. She holds the following qualifications BSc (Philosophy), Grad Dip Sc. (Mathematics), M.Sc. (Hons, Mathematics and Statistics) and PhD (Anthropology).
DS is a male clinical academic psychiatrist working at the Rehabilitation Academic Clinical Unit of MSAMHS in the Mobile Intensive Rehabilitation Team. Although he does not provide direct clinical services to the consumers in the CCUs, consumers may be under his clinical governance before and/or after their time in the CCU. His research interests include treatment refractory schizophrenia, physical health co-morbidity in people living with mental illness and mental health service evaluation. He holds the following qualifications MBBS, MPH, PhD, FRANZCP.
MH is a female Senior Research Fellow with the School of Public Health, at the University of Queensland, Australia. Her research focuses on how the health system can be designed to maximise reduction in the burden of mental and substance use disorders. Themes include: clinical and psychosocial outcomes of mental health treatment services and models; population patterns of health service utilisation for mental health problems, and; excess mortality and morbidity among people with mental disorders. She holds the following qualifications BA (Hons, Psychology), MPASR, and MPH.
The authors declare that they have no competing interests.
Consent for publication
Ethics approval and consent to participate
Ethical approval has been obtained from the University of Queensland and the Metro South Human Research Ethics Committee (HREC/14/QPAH/62).
Open AccessThis article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated.
- Commonwealth of Australia. The Senate Select Committee on Mental Health: A national approach to mental health – from crisis to community First Report. Australia: Commonwealth of Australia; 2006.Google Scholar
- Harding CM. Course types in schizophrenia: an analysis of European and American studies. Schizophr Bull. 1988;14(4):633–43.View ArticlePubMedGoogle Scholar
- Harrison G, Hopper K, Craig T, Laska E, Siegel C, Wanderling J, et al. Recovery from psychotic illness: a 15- and 25-year international follow-up study. Br J Psychiatry J Ment Sci. 2001;178:506–17.View ArticleGoogle Scholar
- Taylor TL, Killaspy H, Wright C, Turton P, White S, Kallert TW, et al. A systematic review of the international published literature relating to quality of institutional care for people with longer term mental health problems. BMC Psychiatry. 2009;9:55.View ArticlePubMedPubMed CentralGoogle Scholar
- Ramon S, Healy B, Renouf N. Recovery from Mental Illness as an Emergent Concept and Practice in Australia and the UK. Int J Soc Psychiatry. 2007;53(2):108–22.View ArticlePubMedGoogle Scholar
- Rosen A, O’Halloran P. Recovery entails bridging the multiple realms of best practice: towards a more integrated approach to evidence-based clinical treatment and psychosocial disability support for mental health recovery. East Asian Arch Psychiatry. 2014;24(3):104–9.PubMedGoogle Scholar
- Chopra P, Herrman HE. The long-term outcomes and unmet needs of a cohort of former long-stay patients in Melbourne, Australia. Community Ment Health J. 2011;47(5):531–41.View ArticlePubMedGoogle Scholar
- State of Victoria. Psychiatric Disability Rehabilitation and Support Services Reform Framework. In: Health Do. Melbourne: State of Victoria; 2007.Google Scholar
- Meehan T, Stedman T, Parker S, Curtis B, Jones D. Comparing clinical and demographic characteristics of people with mental illness in hospital- and community-based residential rehabilitation units in Queensland. Aust Health Rev. 2016. Apr 28. doi: 10.1071/AH15207. [Epub ahead of print]Google Scholar
- Lloyd-Evans B, Mayo-Wilson E, Harrison B, Istead H, Brown E, Pilling S, et al. A systematic review and meta-analysis of randomised controlled trials of peer support for people with severe mental illness. BMC Psychiatry. 2014;14(1):39.View ArticlePubMedPubMed CentralGoogle Scholar
- Tait L. Encouraging user involvement in mental health services. Adv Psychiatr Treat. 2005;11(3):168–75.View ArticleGoogle Scholar
- Doughty C, Tse S. The effectiveness of service user-run or service user-led mental health services for people with a mental illness: A systematic literature review. Wellington, New Zealand: Mental Health Commission of New Zealand; 2005.Google Scholar
- Gillard S, Holley J. Peer workers in mental health services: literature overview. Adv Psychiatr Treat. 2014;20(4):286–92.View ArticleGoogle Scholar
- Davidson L, Chinman M, Sells D, Rowe M. Peer support among adults with serious mental illness: a report from the field. Schizophr Bull. 2006;32(3):443–50.View ArticlePubMedPubMed CentralGoogle Scholar
- Rogers ES, Kash-MacDonald M, Brucker D. Systematic Review of Peer Delivered Services Literature 1989 – 2009. Boston: Boston University, Sargent College, Center for Psychiatric Rehabilitation; [updated 2009. Available from: http://www.bu.edu/drrk/research-syntheses/psychiatric-disabilities/peer-delivered-services/.
- Meaden A, Hacker D, De Villiers A, Carbourne J, Paget A. Developing a measurement of engagement: the Residential Rehabilitation Engagement Scale for psychosis. J Ment Health. 2012;21(2):182–91.View ArticlePubMedGoogle Scholar
- Meaden A, Commander M, Cowan C, Edwards T. Patient engagement and problematic behaviours in nurse-staffed residential rehabilitation units. Psychiatr Bull. 2014;38(6):260–4.View ArticleGoogle Scholar
- Health Workforce Australia. Mental Health Peer Workforce Literature Scan: Health Workforce Australia; 2014 [Available from: http://pandora.nla.gov.au/pan/133228/20150419-0017/www.hwa.gov.au/sites/default/files/HWA_Mental%20health%20Peer%20Workforce%20Literature%20scan_LR.pdf.
- Parker S, Dark F, Vilic G, Doyle C, Lendich B, O’Sullivan R, et al. Integrated staffing model for mental health rehabilitation (in press). Ment Health Soc Incl. 2016;20:2.Google Scholar
- Trauer T, Farhall J, Newton R, Cheung P. From long-stay psychiatric hospital to Community Care Unit: evaluation at 1 year. Soc Psychiatry Psychiatr Epidemiol. 2001;36(8):416–9.View ArticlePubMedGoogle Scholar
- Gerrand V, Bloch S, Smith J, Goding M, Castle D. Reforming mental health care in Victoria: a decade later. Australas Psychiatry. 2007;15(3):181–4.View ArticlePubMedGoogle Scholar
- Queensland Health. Community Care Unit Model of Service: Queensland Public Health Services. Queensland: State of Queensland (Queensland Health); 2015.Google Scholar
- Hamden A, Newton R, McCauley-Elsom K, Cross W. Is deinstitutionalization working in our community? Int J Ment Health Nurs. 2011;20(4):274–83.View ArticlePubMedGoogle Scholar
- Benson J. Living at a Community Care Unit with a serious mental illness: An ethnographic study of residents’ experiences and satsifaction. [Honours]. Bundoora, Victoria: La Trobe University; 2001.Google Scholar
- Australian Health Ministers Advisory Council. A national framework for recovery-oriented mental health services: Guide for practitioners and providers. Australia: Commonwealth of Australia; 2013.Google Scholar
- Wing JK, Beevor AS, Curtis RH, Park SB, Hadden S, Burns A. Health of the Nation Outcome Scales (HoNOS). Research and development. Br J Psychiatry J Ment Sci. 1998;172:11–8.View ArticleGoogle Scholar
- Piaggio G, Elbourne DR, Pocock SJ, Evans SJ, Altman DG, Group C. Reporting of noninferiority and equivalence randomized trials: extension of the CONSORT 2010 statement. JAMA. 2012;308(24):2594–604.View ArticlePubMedGoogle Scholar
- Kummervold PE, Johnsen J-AK, Skrøvseth SO, Wynn R. Using Noninferiority Tests to Evaluate Telemedicine and E-Health Services: Systematic Review. J Med Internet Res. 2012;14(5), e132.View ArticlePubMedPubMed CentralGoogle Scholar
- Bergold J, Thomas S. Participatory Research Methods: A Methodological Approach in Motion. Historical Social Research/Historische Sozialforschung. 2012;37(4 (142)):191–222.Google Scholar
- Wolff N. Using randomized controlled trials to evaluate socially complex services: problems, challenges and recommendations. J Ment Health Policy Econ. 2000;3(2):97–109.View ArticlePubMedGoogle Scholar
- Killaspy H, Harden C, Holloway F, King M. What do mental health rehabilitation services do and what are they for? A national survey in England. J Ment Health. 2005;14(2):157–65.View ArticleGoogle Scholar
- Community Care Unit Model of Service. Queensland Public Health Services, February 2015. Queensland: State of Queensland (Queensland Health); 2015.Google Scholar
- Von Elm E, Altman DG, Egger M, Pocock SJ, Gotzsche PC, Vandenbroucke JP, et al. The Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) statement: guidelines for reporting observational studies. Lancet. 2007;370(9596):1453–7.View ArticlePubMedGoogle Scholar
- Greene CJ, Morland LA, Durkalski VL, Frueh BC. Noninferiority and equivalence designs: issues and implications for mental health research. J Trauma Stress. 2008;21(5):433–9.View ArticlePubMedPubMed CentralGoogle Scholar
- CDC. Introducing EpiInfo TM 7: Centers for Disease Control and Prevention; 2013 [15/11/15]. Available from: http://wwwn.cdc.gov/epiinfo/.
- Siskind D, Harris M, Kisely S, Brogan J, Pirkis J, Crompton D, et al. A retrospective quasi-experimental study of a community crisis house for patients with severe and persistent mental illness. Aust N Z J Psychiatry. 2013;47(7):667–75.View ArticlePubMedGoogle Scholar
- Burgess P, Pirkis J, Coombs T. Do adults in contact with Australia’s public sector mental health services get better? Aust New Zealand Health Policy. 2006;3:9.View ArticlePubMedPubMed CentralGoogle Scholar
- Burgess P, Pirkis J, Coombs T. Routine outcome measurement in Australia. Int Rev Psychiatry. 2015;27(4):264–75.View ArticlePubMedGoogle Scholar
- Slade M, Beck A, Bindman J, Thornicroft G, Wright S. Routine clinical outcome measures for patients with severe mental illness: CANSAS and HoNOS. Br J Psychiatry J Ment Sci. 1999;174:404–8.View ArticleGoogle Scholar
- Trauer T. Outcome measurement in chronic mental illness. Int Rev Psychiatry. 2010;22(2):99–113.View ArticlePubMedGoogle Scholar
- Eager K, Trauer T, Mellsop G. Performance of routine outcome measures in adult mental health care. ANZJP. 2005;39:713–8.View ArticleGoogle Scholar
- Williams B, Speak B, Hay P, Muncer SJ. An evaluation of the independence of the Health of the Nation Outcome Scales. Australas Psychiatry. 2014;22(5):473–5.View ArticlePubMedGoogle Scholar
- Coombs T, Stapley K, Pirkis J. The multiple uses of routine mental health outcome measures in Australia and New Zealand: experiences from the field. Australas Psychiatry. 2011;19(3):247–53.View ArticlePubMedGoogle Scholar
- Trauer T, Eager K, Gaines P, Bower A. New Zealand Mental Health Consumers and their Outcomes Auckland. New Zealand: Health Research Council of New Zealand; 2004.Google Scholar
- Trauer T. Outcome Measurement in Mental Health : Theory and Practice. Cambridge: Cambridge University Press; 2010. Available from: http://uql.eblib.com.au/patron/FullRecord.aspx?p = 542904.Google Scholar
- Flemenbaum A, Zimmermann RL. Inter- and intra-rater reliability of the Brief Psychiatric Rating Scale. Psychol Rep. 1973;32(3):783–92.View ArticlePubMedGoogle Scholar
- Andreasen NC. Scale for the Assessment of Negative Symptoms Scale. 1982.Google Scholar
- Rosen A, Trauer T, Hadzi-Pavlovic D, Parker G. Development of a brief form of the Life Skills Profile: the LSP-20. Aust N Z J Psychiatry. 2001;35(5):677–83.View ArticlePubMedGoogle Scholar
- Andresen R, Caputi P, Oades L. Development of a short measure of psychological recovery in serious mental illness: the STORI-30. Australas Psychiatry. 2013;21(3):267–70.View ArticlePubMedGoogle Scholar
- Joseph S, Becker S, Elwick H, Silburn R. Adult carers quality of life questionnaire (AC-QoL): development of an evidence-based tool. Ment Health Rev J. 2012;17(2):57–69.View ArticleGoogle Scholar
- Veit CT, Ware JE. The structure of psychological distress and well-being in general populations. J Consult Clin Psychol. 1983;51(5):730–42.View ArticlePubMedGoogle Scholar
- Chatterjee S, Pillai A, Jain S, Cohen A, Patel V. Outcomes of people with psychotic disorders in a community-based rehabilitation programme in rural India. Br J Psychiatry J Ment Sci. 2009;195(5):433–9.View ArticleGoogle Scholar
- Killaspy H, Zis P. Predictors of outcomes for users of mental health rehabilitation services: a 5-year retrospective cohort study in inner London, UK. Soc Psychiatry Psychiatr Epidemiol. 2013;48(6):1005–12.View ArticlePubMedGoogle Scholar
- Grinshpoon A, Abramowitz MZ, Lerner Y, Zilber N. Re-hospitalization of first-in-life admitted schizophrenic patients before and after rehabilitation legislation: a comparison of two national cohorts. Soc Psychiatry Psychiatr Epidemiol. 2007;42(5):355–9.View ArticlePubMedGoogle Scholar
- Gonda T, Deane FP, Murugesan G. Predicting clinically significant change in an inpatient program for people with severe mental illness. Aust N Z J Psychiatry. 2012;46(7):651–8.View ArticlePubMedGoogle Scholar
- Evans JD, Bond GR, Meyer PS, Kim HW, Lysaker PH, Gibson PJ, et al. Cognitive and clinical predictors of success in vocational rehabilitation in schizophrenia. Schizophr Res. 2004;70(2–3):331–42.View ArticlePubMedGoogle Scholar
- Brune M, Schaub D, Juckel G, Langdon R. Social skills and behavioral problems in schizophrenia: the role of mental state attribution, neurocognition and clinical symptomatology. Psychiatry Res. 2011;190(1):9–17.View ArticlePubMedGoogle Scholar
- Ferdinandi A, Yoottanasumpun V, Pollack S, Bermanzohn P. Rehab Rounds: Predicting rehabilitation outcome among patients with schizophrenia. Psychiatr Serv. 1997;49(7):907–9.View ArticleGoogle Scholar
- Velligan DI, Bow-Thomas CC, Mahurin R, Miller A, Dassori A, Erdely F. Concurrent and predictive validity of the Allen Cognitive Levels Assessment. Psychiatry Res. 1998;80(3):287–98.View ArticlePubMedGoogle Scholar
- Babor T, Higgins-Biddle J, Saunders J, Monteiro M. The Alcohol Use Disorders Identification Test: Guidelines for Use in Primary Care. 2nd ed: WHO. 2001.Google Scholar
- Steyerberg EW, Eijkemans MJ, Harrell Jr FE, Habbema JD. Prognostic modelling with logistic regression analysis: a comparison of selection and estimation methods in small data sets. Stat Med. 2000;19(8):1059–79.View ArticlePubMedGoogle Scholar
- Woerner MG, Mannuzza S, Kane JM. Anchoring the BPRS: an aid to improved reliability. Psychopharmacol Bull. 1988;24(1):112–7.PubMedGoogle Scholar
- Crippa JA, Sanches RF, Hallak JE, Loureiro SR, Zuardi AW. A structured interview guide increases Brief Psychiatric Rating Scale reliability in raters with low clinical experience. Acta Psychiatr Scand. 2001;103(6):465–70.View ArticlePubMedGoogle Scholar
- Schoemaker J, Gaur R, Sajatovic M, Ramirez L, De Santi S. A structured interview guide for the Scale for Assessment of Negative Symptoms (IG-SANS)..2nd European Conference on Schizophrenia Research. 2009.Google Scholar
- Masconi KL, Matsha TE, Echouffo-Tcheugui JB, Erasmus RT, Kengne AP. Reporting and handling of missing data in predictive research for prevalent undiagnosed type 2 diabetes mellitus: a systematic review. EPMA J. 2015;6(1):7.View ArticlePubMedPubMed CentralGoogle Scholar
- Rombach I, Rivero-Arias O, Gray AM, Jenkinson C, Burke O. The current practice of handling and reporting missing outcome data in eight widely used PROMs in RCT publications: a review of the current literature. Qual Life Res. 2016. Jan 28. [Epub ahead of print]Google Scholar
- He Y. Missing data analysis using multiple imputation: getting to the heart of the matter. Circ Cardiovasc Qual Outcomes. 2010;3(1):98–105.View ArticlePubMedPubMed CentralGoogle Scholar
- Scialfa CT, Games PA. Problems with step-wise regression in research on aging and recommended alternatives. J Gerontol. 1987;42(6):579–83.View ArticlePubMedGoogle Scholar
- Tong A, Sainsbury P, Craig J. Consolidated criteria for reporting qualitative research (COREQ): a 32-item checklist for interviews and focus groups. International J Qual Health Care. 2007;19(6):349–57.View ArticleGoogle Scholar
- Dicicco-Bloom B, Crabtree BF. The qualitative research interview. Med Educ. 2006;40(4):314–21.View ArticlePubMedGoogle Scholar
- McGhee G, Marland GR, Atkinson J. Grounded theory research: literature reviewing and reflexivity. J Adv Nurs. 2007;60(3):334–42.View ArticlePubMedGoogle Scholar
- QSR. NVivo qualitative data analysis Software - Version 10. Version 10 ed: QSR International Pty Ltd. 2014.Google Scholar
- Miles MB, Huberman AM, Saldaña J. Qualitative data analysis: A methods sourcebook. Thousand Oaks, Califorinia: SAGE Publications, Inc.; 2014.Google Scholar
- Chopra P, Harvey C, Herrman H. Continuing Accommodation and Support Needs of Long-Term Patients with Severe Mental Illness in the Era of Community Care. Curr Psychiatr Rev. 2011;7(1):67–83.View ArticleGoogle Scholar
- Andresen R, Caputi P, Oades LG. Do clinical outcome measures assess consumer-defined recovery? Psychiatry Res. 2010;177(3):309–17.View ArticlePubMedGoogle Scholar
- Trauer T. Comment on “do clinical outcome measures assess consumer-defined recovery?”. Psychiatry Res. 2011;186(2–3):469. author reply 70.View ArticlePubMedGoogle Scholar
- Scott IA. Non-inferiority trials: determining whether alternative treatments are good enough. MJA. 2009;190(6):326–30.PubMedGoogle Scholar
- Bright FA, Kayes NM, Worrall L, McPherson KM. A conceptual review of engagement in healthcare and rehabilitation. Disabil Rehabil. 2015;37(8):643–54.View ArticlePubMedGoogle Scholar
- Charmaz K. Constructing Grounded Theory : A Practical Guide through Qualitative Analysis. London: SAGE Publications; 2006. Available from: http://uql.eblib.com.au/patron/FullRecord.aspx?p = 585415.Google Scholar
- Callander R, Ning L, Crowley A, Childs B, Brisbane P, Salter T. Consumers and carers as partners in mental health research: reflections on the experience of two project teams in Victoria, Australia. Int J Ment Health Nurs. 2011;20(4):263–73.View ArticlePubMedGoogle Scholar
- Gilburt H, Slade M, Rose D, Lloyd-Evans B, Johnson S, Osborn DP. Service users’ experiences of residential alternatives to standard acute wards: qualitative study of similarities and differences. Br J Psychiatry Suppl. 2010;53:s26–31.View ArticlePubMedGoogle Scholar
- Gould F, McGuire LS, Durand D, Sabbag S, Larrauri C, Patterson TL, et al. Self-Assessment in Schizophrenia: Accuracy of Evaluation of Cognition and Everyday Functioning. Neuropsychology. 2015.Google Scholar
- Dunn LB, Kim DS, Fellows IE, Palmer BW. Worth the risk? Relationship of incentives to risk and benefit perceptions and willingness to participate in schizophrenia research. Schizophr Bull. 2009;35(4):730–7.View ArticlePubMedGoogle Scholar
- Fudge N, Wolfe CD, McKevitt C. Assessing the promise of user involvement in health service development: ethnographic study. BMJ. 2008;336(7639):313–7.View ArticlePubMedPubMed CentralGoogle Scholar
- Commonwealth of Australia. 2033.0.55.001 - Census of Population and Housing: Socio-Economic Indexes for Areas (SEIFA), Australia, 2011 Australia2013 [updated 18/07/2013. Available from: http://www.abs.gov.au/ausstats/abs@.nsf/DetailsPage/2033.0.55.0012011?OpenDocument.
- Queensland Health. Metro South Health Demographic and Health Status Profile 2011–2012. Queensland Government (Metro South Health). 2012.Google Scholar
- Andreasen NC. Negative symptoms in schizophrenia. Definition and reliability. Arch Gen Psychiatry. 1982;39(7):784–8.View ArticlePubMedGoogle Scholar
- Birchwood M, Smith J Fau - Cochrane R, Cochrane R Fau - Wetton S, Wetton S Fau - Copestake S, Copestake S. The Social Functioning Scale. The development and validation of a new scale of social adjustment for use in family intervention programmes with schizophrenic patients. (0007–1250 (Print)).Google Scholar
- Aubin G, Chapparo C, Gelinas I, Stip E, Rainville C. Use of the Perceive, Recall, Plan and Perform System of Task Analysis for persons with schizophrenia: a preliminary study. Aust Occup Ther J. 2009;56(3):189–99.View ArticlePubMedGoogle Scholar
- Reinhard SC, Gubman GD, Horwitz AV, Minsky S. Burden assessment scale for families of the seriously mentally ill. Eval Program Plann. 1994;17(3):261–9.View ArticleGoogle Scholar