Skip to main content

Step-up, step-down mental health care service: evidence from Western Australia’s first – a mixed-method cohort study

Abstract

Background

Mental health Step-up, Step-down services (SUSD), also known as subacute services or Prevention and Recovery Services, have emerged to fill an identified gap between hospital-based inpatient care and clinical community-based mental health support. Evidence for the effectiveness of the SUSD service model is limited but growing. Accordingly, this study looked to add to the extant body of knowledge, by (i) assessing change outcomes in mental health and wellbeing, and predictors of these changes, for patients who accessed Western Australia’s first SUSD service; and (ii) evaluating patients’ satisfaction with service, and what patients value from their stay.

Methods

This was a mixed-method retrospective cohort study. Participants comprised 382 patients who accessed a 22-bed Mental Health SUSD facility and incurred 551 episodes of care during the 01/07/2014–30/06/2016 period. Patients’ change outcomes in psychological distress, general self-efficacy, and work and social adjustment from service entry to service exit were analyzed using generalized linear modeling. Simple Pearson’s correlation coefficients were calculated for preliminary assessment of the associations between patients’ service satisfaction and their change outcomes. Qualitative outcomes that patients valued from their stay were analyzed thematically according to a semi-grounded theoretical approach.

Results

Significant improvements were observed in patients’ self-reported psychological distress, self-efficacy, and work and social adjustment (all p < 0.0001). A strong and persistent baseline effect existed across the three measures. Older age, female gender, and having a dependent child in the same household were protective/enhancing factors for the patients’ recovery. Satisfaction with service was high. Patients valued having the time and space to recuperate, gain insight, focus, and create changes in their lives.

Conclusion

The encouraging findings, regarding both patients’ change outcomes and satisfaction with service, support the value of the SUSD service model for patients with mental illnesses. Strengths and limitations were discussed; ensued recommendations were offered to both service providers and researchers to enhance the robustness of future research findings, to help inform more effective policy and funding decisions related to mental health care.

Peer Review reports

Background

Mental health policy in Australia is undergoing reform to deliver a stepped care approach, to ultimately see people receiving supports matched to their level of need; that is, the right level of the right type of clinical care is received at the right time [1]. This requires access to a full continuum of services from low intensity, early intervention services through to acute inpatient services. As part of this reform process mental health Step-up, Step-down services (SUSD), also known as subacute services or Prevention and Recovery Services (PARCS), have emerged to fill an identified gap between hospital based inpatient care and clinical community based mental health support. Adult SUSD mental health services provide short- to medium-term residential care, either as a means of early intervention (Step-up) or to support the transition from hospital to home (Step-down). Since 2003, SUSDs have been embedded in the Victorian mental health service system; other Australian states have also begun to adopt them [2].

The Australian literature on the SUSD model is currently limited but growing. Existing studies have described aspects of the model as applied in different service settings such as location, capacity, goals and parameters, service usage demographics and service approach and program activities [3,4,5,6,7,8,9]. In addition, most of these studies include an evaluative component such as stakeholder perspectives in relation to how practice aligns with the service approach and patients’ experiences of service.

In the historic and international context, the SUSD model can be said to have evolved from the movement of ‘deinstutionalization’ beginning several decades ago in countries such as the United Kingdom (UK) and the United States (US) [10]. In the recent years, despite its uptake in several other countries worldwide, Australia included, the understanding of the SUSD model, or of stepped care in general, in terms of utility, scope, effectiveness and cost, remains limited [11]. This is because of the widely varied nature of the specific models implemented in each country, or each setting. Such parameters may include: admission/inclusion criteria, diagnosis, typical length of stay offered, types of service (e.g., clinical, educational, psychosocial etc) and service provider (e.g., government, private, not-for-profit etc) [12, 13]. For example, publications [14, 15] from a study of 17 SUSD services in one Australian state highlighted that while care was generally recovery-oriented there was wide variation in the “structure, resourcing, and content and quality of care” ([15],p.10). This study reinforced how local service systems, beyond the SUSD, influence the operation of SUSD services. With these varied characteristics, it is expected that different service models look to achieve different ‘outcomes’ [13]. In addition, SUSD-type services have been developed to address the immediate needs of certain target patient populations and local regions, with less focus on the additional tasks of evaluation and research [16].

At the more fundamental level, outcomes for the service users, referred to as patients henceforth, which purportedly are a key driver for the birth of this type of models, have been variably assessed using different measures, with mixed rationales. Examples of the measures used in relevant studies thus far include: the Mental Health Recovery Star (MHRS [4];), Health of the Nation Outcome Scales (HoNOS [4, 8] [9];);, Kessler Psychological Distress Scale (K10 [4];), Behaviour and Symptom Identification Scale (BASIS-32 [8] [9];);, Life Skills Profile – 16 (LSP-16 [9];), Scottish Recovery Indicator (SRI 2 [3];), Individual Recovery Plans (IRP’s [3];), and Assessment of Quality of Life (AQoL [17];). Findings from these studies provide an early indication that the SUSD has value for people beyond reductions in hospital usage, but exactly what the additional values are remain to be defined.

This lack of clarity in choice and rationale of outcome measures used in evaluating the SUSD service model is understandable, given the relatively new and unique position of this type of model, compared with the more traditional, or more established care models in the acute hospital setting, or the day-visit community setting [18, 19]. That is, the needs of the patients accessing SUSD services may differ from those in the other care settings. As also seen in the above summary of previous research, this diversity, and complexity, in choice and rationale of outcome measures, becomes even more compounded as one also considers whether the recovery/change outcomes for the patients themselves are also congruent with the outcomes expected of, and/or by, the service providers and funders. More specifically, in Australia, government funding goals for SUSDs are predominantly framed in relation to system and financial efficiencies, improvements in service delivery and better health outcomes [20].

There is also a case for pitching service outcome goals and measurement instrument design to ‘what is wanted’ in terms of quality of life and wellbeing outcomes that go beyond the absence of illness [21,22,23,24,25]. Exploring outcomes from the patients’ perspective in terms of the value and contribution of a stay in the SUSD to their overall lives may add to the knowledge in this area [25,26,27,28,29].

Taken together, while there is a small and growing volume of early research suggesting the value of the SUSD model, it is clear that substantial further research is needed to shed more light on the diverse facets of its effectiveness. Meaningful assessment of the model’s effectiveness can be realized via the adoption and/or development of measures of change or recovery that are relevant or salient to the patients, as well as informative to both the service providers and the health system collectively. In this light, the current study is set out to primarily assess change outcomes in mental health and wellbeing for a cohort of patients who accessed Western Australia’s Joondalup SUSD service, which is the first of its kind in this Australian state. At set-up, the service administrators conceptualized the Joondalup SUSD as being a setting where people make recovery gains that are assisted by the development of skills and confidence to self-manage upon return to home.

Recovery in this SUSD context is defined by “a deeply personal, unique process of changing one’s attitudes, values, feelings, goals, skills and/or roles” [30], and “the establishment of a fulfilling, meaningful life and a positive sense of identity founded on hopefulness and self-determination” [31]. Accordingly, to assess change outcomes, two measures recommended use in Australian settings – the Kessler 10 (K10 [32];), to measure level of psychological distress, and the Work and Social Adjustment Scale (WSAS [33];) – along with an additional measure, the General Self-Efficacy Scale (GSES [34];), were chosen as aligning with the service goals. The study also evaluates the patients’ experience with the service, namely their service satisfaction, and what they value from their service stay. The purpose of this secondary component is to seek to validate the relevance of the three mentioned scales as measures of the patients’ change outcomes in a SUSD setting.

Methods

Design

This was a retrospective cohort study, with mixed methods for data collection and analysis. The main part of the study, on patients’ change outcomes, utilized a simple pre-post design (with data collected at both service entry and exit) and quantitative data analysis. The secondary component of the study has a cross-sectional design, where data on patients’ satisfaction with, and perception of, the service, were collected only at service exit. This secondary component involved both quantitative and qualitative data analyses.

Settings

The 22-bed facility, Neami Joondalup Mental Health Step-Up Step-Down service (JMHSS), is run by Neami National, a not-for-profit community mental health service supporting people with mental illness “to improve their wellbeing, live independently and pursue a fulfilling life” [35]. Neami as a psychosocial provider is directly contracted by WA Mental Health Commission to deliver the service. Neami employs mixture of psychosocial and clinical staff, for example registered nurses, enrolled nurses and allied health clinicians. A memorandum of understanding underpins the relationship with the local clinical mental services – inpatient and community based [36]. In addition to meeting the needs of the local region it is a statewide service accepting patients from across much of Western Australia. In 2013, JMHSS received its first patients. The JMHSS’s service practice is underpinned by a recovery-oriented strengths-based coaching framework – The Collaborative Recovery Model [37]. The framework is designed to support personal empowerment and self-agency through the development of individualized wellbeing plans based on personal aspirations and context [37,38,39]. Program activities such as The Optimal Health Program [40] are delivered to enhance self-efficacy and skills in wellbeing planning and management.

Participants and data source

Participants were patients with mental health issues who accessed JMHSS service any time during the period from 1 July 2014 to 30 June 2016 inclusive. Patients’ data were routinely collected by the service provider JMHSS (part of Neami National) at service entry and exit. Patients may choose, or refuse, to provide the data in question. Patients are aware that Neami National may use the data for research and/or service evaluation and improvement purposes, provided that the data are anonymized in reporting and dissemination. In the specific context of this study, relevant patients’ data were made available in de-identified format to the first author, an independent researcher external to Neami National, for analysis.

Materials

Patients accessing the JMHSS service were requested, but not obligated, to complete self-report measures of their state of health and functioning (namely: K-10, GSES, and WSAS, as mentioned above), in addition to providing their contact and demographic details. The K-10 consists of 10 questions, with each rated from 1 to 5 in order of increasing frequency of a particular (negative) feeling; thus a person may score as low as 10 (i.e., least distress) or as high as 50 (most distress). The GSES also has 10 items, with each rated from 1 to 4; hence total scores may range from 10 to 40, with higher scores indicating high self-efficacy. The WSAS has 5 items, with each rated from 0 (no impairment at all) to 8 (severe impairment), yielding a maximal possible range of 0 to 40 for total scores. All three scales K-10, GSES, and WSAS have been validated widely in other mental health service settings ([32,33,34]). In this study, these instruments were completed by the patients at treatment entry and exit. Patients were also invited to complete an Exit Questionnaire on their exit, to provide feedback on their experience of their stay at JMHSS (see Additional file 1).

Outcome measures and definitions

Outcome measures were score changes in K-10, GSES, and WSAS – from service entry to service exit, and patients’ satisfaction with service upon exit. Score changes were defined as “score at exit minus score at entry”. It is noted that scales K-10 and WSAS measure ‘undesired’ constructs, namely psychological distress and work and social impairments. As such, a negative score change on K-10 or WSAS (i.e., a reduction in distress or impairment) is desired. The GSES, on the other hand, measures self-efficacy, a ‘desired’ construct. Therefore, a positive score change on GSES (i.e., an increase in self-efficacy) is desired.

Statistical analysis

First, the study cohort was described using descriptive statistics: mean and standard deviation for continuous measures, and frequency and percentage for categorical measures. Descriptions involved demographic, clinical, and systemic characteristics. Demographic characteristics included age, gender, socio-economic indexes for areas (SEIFA [41];), country of birth, living arrangement, and employment. Clinical factors included primary psychiatric diagnosis, and service history with JMHSS (i.e., both total number of service episodes and length of stay). Referral pathway (i.e., step-up from home/community, or step-down from hospital) was the sole systemic factor available for analysis.

Second, for the primary aim, among those who had ‘paired’ data (i.e., both at entry and exit), change outcome measures were tested for statistical significance using paired t-test. Magnitude of change was also expressed as percentage of change relative to baseline for added context. Statistical predictivity of patients’ characteristics in change outcomes was assessed using generalized linear modelling, given the response variable being continuous. For the secondary aim, among patients who completed an Exit Questionnaire, their satisfaction with service was profiled using descriptive statistics and column charts. Simple Pearson’s correlation coefficients were also calculated for preliminary assessment of the associations between patients’ service satisfaction and their change outcomes. Where relevant, these associations were investigated further using generalized linear modelling.

As the outcome measures were available and relevant at the episode level, all the main analyses were conducted at the episode level accordingly. The only exception was the profiling/ characterization of the patient cohort (the descriptive analysis) which was carried out at the (unique) patient level; this was consistent with Neami National’s data management practice where the patient’s characteristics at the most recent service episode were allowed to override those recorded for their earlier episode(s), should they have incurred multiple episodes. Quantitative data analyses were conducted in SAS (Statistical Analysis Systems), version 9.3. Statistical significance was set at α = 0.05; hence 95% Confidence Intervals (CIs) were presented in conjunction with estimates where appropriate.

Qualitative data on what patients valued about their JMHSS stays – used interchangeably ‘valued outcomes’ henceforth – were analyzed thematically according to a semi-grounded theoretical process as illustrated by Braun and Clarke (2006). The qualitative analysis process involved firstly a collective familiarity with all the response statements to the question ‘what was valuable to your stay?’ at service exit. This informed the nature of the enquiry which was to understand the type of personally valued outcomes that patients experienced as a result of their stay at the JMHSS, and to understand how these outcomes were facilitated. Secondly, the response statements were broken into smaller response units. For example the response “I learnt a lot about myself and seem to have got my medications right, as well the staff have been so caring and helpful through this tough time” was broken into 4 smaller units as follows: 1) learnt about self, 2) medications working, 3) caring staff, and 4) helpful staff. Each response unit was deemed equivalent to a unique idea within a statement. Response units were then coded and grouped into themes according to either A) valued personal process outcomes (for example, ‘learnt about self’), or B) facilitators of valued outcomes (for example, ‘caring staff’).

Ethics approval

This study was approved by The University of Western Australia’s Human Research Ethics Committee (RA/4/1/8805).

Results

Description of patient cohort

A total of 551 service episodes, corresponding to 382 unique patients, between 1 July 2014 and 30 June 2016 were included for analysis. Patients’ characteristics are described in Table 1. Mean age was 37.5 (range 18–65) years, with the majority of patients being female (61%). The majority of patients (82%) did not identify themselves as Aboriginal, and were living in a less disadvantaged area (78%). More than half of the patients were born in Australia, and nearly half of all patients were living with family at service entry and had no dependent children. Only 7% reported to be employed at service entry. More patients were referred from the ‘Step Up’ pathway (59%) than from ‘Step Down’ (37%). Patients with depression made up the largest subgroup (34%) in terms of primary diagnosis. Nearly three-quarters have had only 1 admission episode with Neami. Mean length of stay was approximately 3.5 weeks.

Table 1 Profile description of 382 patients accessing JMHSS service (total 551 episodes) between 1 July 2014 and 30 June 2016

Patients’ self-reported recovery outcomes, via changes in K-10, GSES, and WSAS scores

Significant improvements were seen in all three self-reported outcome measures at service exit, compared to at service entry (Table 2). The most pronounced improvement was in the patients’ psychological distress level, which was a 20% reduction compared to the baseline measure, with an associated strong Cohen’s d index of 0.7 in absolute value. Improvement in general self-efficacy was of comparable magnitude. Improvement in work and social adjustment was also strongly significant statistically, although of more moderate magnitude, as indicated by the Cohen’s d index.

Table 2 Summary of scores on: Kessler Psychological Distress Scale (K-10), General Self-Efficacy Scale (GSE), and Social and Work Adjustment Scale (WSAS). Data are based on episodes (see n below), not unique patients

Predictors of patients’ self-reported recovery outcomes

Table 3 shows patients with higher K-10 score (i.e., higher psychological distress) at baseline improved more than those with lower distress level at baseline, by 0.6 (95% CI 0.5, 0.7) point in score change for every 1-point increment in K-10 at baseline. Older age, female gender, non-Indigenous status, and having a dependent child living with the patient are also statistical predictors of greater improvement (or reduction) in K-10 scores (Table 3). Compared to patients with psychosis, those with personality disorder or with depression also had their psychological distress level improved to a greater extent. When all the significant predictive factors mentioned above were assessed concurrently in the same statistical model, only baseline K-10 scores remained significant predictors of K-10 score changes.

Table 3 Factors associated with improvement in K-10 scores (reduction desired) and GSES scores (increase desired), from service entry to exit

For GSES scores, baseline measure was also a strong statistical predictor of change at service exit, in that patients with higher GSE score (i.e., higher efficacy) at baseline improved less than those with lower efficacy at baseline, by 0.6 (CI 0.4–0.7) point in score change for every 1-point increment in GSES at baseline (Table 3). Older age, female gender, and having a dependent child living in the same household also enhanced the recovery or improvement in GSES measure. Patients with personality disorder or depression improved more than those with psychosis (Table 3). When all of the significant predictive factors were assessed concurrently in the same statistical model, only baseline GSE scores and primary diagnosis grouping remained significant predictors of GSES score changes.

Of the 13 factors assessed, only the WSAS baseline scores were predictive of the WSAS score change at service exit. Specifically, patients with higher WSAS score (ie higher impairment) at baseline improved more than those with lower impairment at baseline, by 0.3 (CI 0.2–0.5) point in score change for every 1-point increment in WSAS at baseline. No other factor was significant in predicting WSAS score changes.

Patients’ satisfaction with JMHSS service

Up to 251 episodes (or 45.6% of all 551 episodes) had data on patients’ satisfaction reported via the Exit Questionnaire, hence analyzed. Potential biases, or differences, were checked for Exit Questionnaire completers versus non-completers. Additional file 2 shows comparable profiles between the two groups, except that those who completed the Exit Questionnaire on average stayed 3 days longer than those who did not.

As seen in Fig. 1, patients’ satisfaction with JMHSS service overall (Question 9) was rated highly, with 90% of the respondents reporting “Satisfied” or “Very Satisfied” with their stays. Similarly favourable ratings were also recorded for items on Staff support (94% positively rated) and Safety (87% positively rated). For the other four items (on Peer Engagement, Group Work, Daily Routine, and Health Plan), at least two-thirds of the respondents provided a positive rating.

Fig. 1
figure 1

Distribution of patients’ ratings of satisfaction with service upon exit

All seven indexes of patients’ satisfaction with service positively and significantly correlated with the patients’ reduction in psychological distress (Table 4). Improvement in self-efficacy positively and significantly correlated with five of the seven indexes of service satisfaction (i.e., all but Questions 2 and 5, for peer engagement and feeling safe, respectively). Improved work and social adjustment (i.e., reduced impairment in work and social domains) only correlated significantly and positively with the patients’ rating of confidence with their health plan (Question 6).

Table 4 Correlations between patients’ Exit Questionnaire ratings (of satisfaction with service) and their score changes on K-10, GSES, and WSAS

Given that Question 6 taps on the core focus of Neami’s service model, and that it is most consistently associated with the three outcome measures (score changes), we assessed its effect further with generalized linear modelling. Patients who were most confident (rating 5) with their health plan on average observed a further reduction in psychological distress level (estimate 6.3 points; 95% CI 2.3, 10.3; p = 0.0022), relative to those who were less confident (ratings 1–4). Similarly, patients with highest confidence with their health plan also reported a greater reduction in work and social impairment by a further 5.3 points (95% CI 1.9, 8.6; p = 0.0021). For general self-efficacy, however, there was only a marginally significant effect of confidence with health plan (p = 0.0538).

Patient exit interview qualitative feedback

There were 235 episodes (or 42.6% of all 551 episodes) contributing qualitative responses to the question ‘what was valuable to your stay?’ via the Exit Questionnaire. The 235 responses were broken into 531 smaller response units, which were subsequently categorized into two broad themes of (A): Valued outcomes, and (B): Outcome facilitators. As seen in Table 5, each broad theme encompasses four subthemes (Table 5). That is, four top valued outcomes from the patients’ stay at JMHSS were: (i) time and space to recuperate, (ii) perspective and insight; (iii) focus and direction; and (iv) changes, particularly in relation to establishing connections and supports, establishing a routine, gaining knowledge, self- validation and confidence. Facilitators of these valued outcomes included: (i) the environment dedicated to improving mental health, (ii) staff attributes, (iii) program activities, and (iv) interactions with others.

Table 5 Themes and ideas derived from participants’ responses for (A) what outcomes they valued from their stays at the Joondalup Mental Health Step-Up Step-Down (JMHSS) and (B) facilitators of those valued outcomes

Discussion

Summary and interpretation of key findings

This study provides evidence that receipt of service at the JMHSS was strongly associated with significant improvement in patients’ psychological wellbeing (i.e., reduced distress), general sense of self-efficacy, and work and social adjustment. There was a strong and persistent baseline effect present across the three measures (K10, GSE, and WSAS), which showed that the less well patients benefited more from the service. This makes sense, as the further one is from the ‘ideal’ position, the more ‘room’, or potential, one has to improve, or to get there. Older patients and female patients benefited more from the services, compared to their younger or male counterparts, respectively. Further analyses (data not shown) confirmed the underlying baseline effect, which stipulated that older or female patients had reported poorer health status (i.e., higher psychological distress and lower self-efficacy) at baseline than their respective younger or male peers. Parenting responsibility and role model expectation appeared to serve as a protective factor for the patient’s psychological wellbeing and general self-efficacy during their recovery journey with JMHSS. There was some mild indication that repeated patients (i.e., those with more than one service episode) might have benefited more, via slightly greater reduction in psychological distress scores and increase in self-efficacy scores, than their single-episode peers. If true, this would not be a surprising finding, as anecdotally in focus groups run by Neami, patients did share that they appreciated using the JMHSS as a way of ‘top-up’ for improving their mental health.

Patients’ satisfaction with the service, as reported via the Exit Questionnaire, in general was high, with higher satisfaction generally correlating with greater improvements in the three outcome measures (reduced psychological distress, enhanced self-efficacy, and reduced impairment in work and social activities). Features that patients valued most from their stays with JMHSS include being afforded with the time and space to recuperate, gain perspectives, focus, and create changes in their lives. Facilitating these valued features were: aspects of the environment (physical space, atmosphere and approach), staff attributes, program activities, and interactions with other residents and staff. Thematically and conceptually, the patients’ qualitative feedback concurs with the three quantitative outcome measures. For example, reduction in psychological distress (measured via K-10) would be expected to be ‘linked’ to the patients having the time and space to recuperate. Similarly, gaining insight, focus, direction, and the ability to create change (such as establishing new routines, new coping strategies, support network) should foster one’s sense of self-efficacy (measured by GSES) as well as enhance one’s work and social functioning (measured by WSAS). Notably, patients who completed the Exit Questionnaire on average stayed three days longer than those who did not complete the questionnaire. This may be because longer stays afforded more opportunities for patients to be contacted to complete the Exit Questionnaire, although longer stays were not associated with more beneficial change outcomes, as already seen in the Results.

Contribution to the literature - comparison with other studies

Findings from this study build on recent Australian literature for the value of the SUSD setting to patients, or service users, in terms of a) change outcomes identified by routinely administered assessment tools and b) qualitative feedback obtained from consumers who have accessed this type of service [4, 5, 8, 9]. They also add to recent efforts to form evaluation practices from program logic modelling [3]. Specifically, the three validated measures K-10, GSES, and WSAS were used as theoretically they were thought to reflect the desired changes in the key domains of the mental health patient’s wellbeing, namely reduced psychological distress, improved self-efficacy and work and social functioning. Whilst these measures may allow for comparisons between mental health services, the use of qualitative feedback adds a nuanced understanding of the outcomes that are valued by the patients – how these are experienced, and which aspects of service that facilitate these valued outcomes. Having time and space to recuperate in an environment dedicated to mental health and wellbeing whilst also being enabled to develop skills and confidence to face and manage future challenges, appears to be the principal process underpinning the positive change outcomes identified in this study. From a program logic modelling perspective, the assessment of change outcomes undertaken in this study suggest that there is a reasonable alignment between the program activities and service approach and the measures utilised in this setting to assess outcomes.

The added qualitative component to assess the patient’s satisfaction with the service, particularly the valued aspects of their stay, has taken one step further in the direction towards ‘recovery-orientation’ [42], and away from a negative, deficit- [28] and disability- orientation [43]. Literature from the motivation and behavioral sciences demonstrate that a condition for behavior is what attention is drawn to [21]. Translated to the mental health services field, there is a case for shifting the focus of measurement tools towards that of desired gains [21, 26], an idea also supported by studies that have actively involved service users [29].

Strengths

On many counts, this study has several strengths compared to previous research in the field. Specifically, our study has a much larger sample, in contrast to several previous studies employing an effective sample size of up to 40–50 participants (e.g., [3, 5, 9]). The more sizeable sample allows for multiple outcomes to be assessed concurrently and informatively. To our knowledge, this is the first study in the field that has systematically assessed the statistical predictivity of several patient variables in change outcomes.

Assessment of relationships between patients’ satisfaction with service and their outcome measures has added another level of insight not only into the patients’ reception and perception of the service, but also the potential areas that need focusing to help enhance their recovery. Further, nuanced qualitative feedback from patients have afforded the researchers, service providers, and stakeholders alike with an expanded understanding of what matters most to them along their ‘recovery’ or change process, which in turn will help develop outcome measures that are more meaningful to the patients, for whom the services are designed.

Limitations

There are limitations to our study. First, this is a retrospective cohort study with only ‘cases’ and no ‘controls’ to act as an appropriate comparator. One therefore is not able to ascertain if the observed positive changes would have also occurred (e.g., by default) in non-SUSD patients. Second, a belated inclusion of an independent research collaborator precluded applications of stringent research principles, such as the absence of express, informed, and written consent from patients regarding the use of their data for research purposes, and more robust data collection methods, as noted next,

Third, Neami’s current system of data recording and storage allows for the patient’s last record and associated details to override all previous records, if the patient has had more than one service episode. This inadvertently removes an amount of potentially valuable longitudinal data, especially those on more ‘dynamic’ characteristics of the patient (such as employment status). This data removal also prohibits the potential examination of associations between these dynamic characteristics and the patient’s recovery outcomes at each episode.

Fourth, the outcome measure WSAS may have not been the most optimal in measuring the patient’s change in work and social adjustment. Although there was strong evidence for improvement in levels of functioning (or reduced impairment) in the patient’s work and social activities post-JMHSS, there was no significant predictor of this, other than its baseline level. Coupled with this is the observation (from Table 1) that the vast majority of the patient cohort was “Not in the labour force”, which together implies that an alternative instrument may more meaningfully measure a patient’s recovery. Given that employment and/or vocational activities are central to the mental wellbeing for most people, including those with severe mental illness (e.g., [44]), it is important that changes related to such outcomes are meaningfully tracked, to help enable a comprehensive evaluation of the SUSD model.

Finally, the qualitative analysis was limited by the material gathered by the routine assessment performed retrospectively, which might have introduced potential for bias (for example, if the person administering the Exit Questionnaire was a staff member that the patient had had contact with, or received support from, during their stay). Nevertheless, as alluded to earlier, the qualitative component has provided a supplementary first-person narrative about the value of the SUSD model to those who access it. Furthermore, the qualitative data come from the same population as the rest of the study, thus allowing the opportunity to compare these two sets of data, quantitative and qualitative.

Conclusion

Overall, this study has showcased compelling, albeit preliminary, evidence for the need for more transitional/ community services like the JMHSS program, with a strong focus on both prevention and recovery. The program has shown very promising outcomes for patients, both in terms of their change measures and their satisfaction with several aspects of the service. Findings in this study build on the emerging evidence base for the value of the SUSD setting in terms of positive outcomes for consumers, and contribute knowledge that may advance program logic modelling and evaluation practices for mental health SUSD settings.

From a service viewpoint, the following aspects, among others, may require further attention. First, procedures and systems need to be in place such that the individual patient’s needs can be continually monitored and effectively met. For example, this may require a delicate balance in the provision of clinical and psychosocial care, depending on the patient’s journey or progress. Second, patient intake needs to be reasonably balanced such that SUSD services do not become overflow wards for acute cases, as anecdotally noted. In the current study of the JMHSS, this was not the case, with almost 60% of admission episodes coming from the ‘intervention/preventative’ Step-up stream (instead of Step-down). Third, services need to ensure equity in access and accessibility to patients of Indigenous and/or other disadvantaged ethnic backgrounds. In our study, 2.6% of the patients self-identified as Indigenous, which is approximately in line with the 2.1% estimate from the 2016 census for Greater Perth [45]. Nevertheless, nearly 28% of patients did not have information on country of birth recorded, with an additional 2.6% of patients who were born outside of Australia, New Zealand, the UK, and other European countries.

In terms of research enhancement, further conceptualizations and/or assessments of patient outcomes in alignment with the expectations of both the patients, the service providers and funders, particularly with the aid of more rigorous qualitative research methods, would be beneficial and recommended. Lastly, extensions of similar studies, with the inclusion of an appropriate control group, would help elucidate the robustness of the findings.

Availability of data and materials

In upholding the principles of research integrity and replicability, the authors are willing to provide detailed instructions of how the data analyses were conducted.

The source dataset for this manuscript is not available publicly because it contains sensitive information on patients. Requests to access the dataset should be directed to the Data Custodian, Neami National, via their representatives: Keren Wolstencroft keren.wolstencroft@neaminational.org.au or Priscilla Ennals priscilla.ennals@neaminational.org.au.

Abbreviations

SUSD:

Step-Up Step-Down (model of care or service)

JMHSS:

Neami Joondalup Mental Health Step-Up Step-Down Service

PARCS:

Prevention and Recovery Services

K-10:

Kessler Psychological Distress Scale (10-item version)

GSES:

General Self-Efficacy Scale

WSAS:

Work and Social Adjustment Scale

References

  1. Department of Health. Australian Government response to contributing lives, thriving communities. Review of mental health programmes and services. Canberra: Australian Government; 2015.

    Google Scholar 

  2. Galloway J, Scollo G, Thomson N: Mental health prevention and recovery care: a clinical and community partnership model of sub-acute mental health care. In. Edited by services DoHaH. Melbourne: Victorian Government; 2016.

  3. Heyeres M, Kinchin I, Whatley E, Brophy L, Jago J, Wintzloff T, Morton S, Mosby V, Gopalkrishnan N, Tsey K. Evaluation of a residential mental health recovery Service in North Queensland. Front Public Health. 2018;6:123.

    Article  PubMed  PubMed Central  Google Scholar 

  4. Frost BG, Turrell M, Sly KA, Lewin TJ, Conrad AM, Johnston S, Tirupati S, Petrovic K, Rajkumar S. Implementation of a recovery-oriented model in a sub-acute intermediate stay mental health unit (ISMHU). BMC Health Serv Res. 2017;17(1):2.

    Article  PubMed  PubMed Central  Google Scholar 

  5. Thomas KA, Rickwood DJ. Facilitators of recovery for step-up and step-down clients of a sub-acute residential mental health service. J Ment Health. 2016;25(4):343–9.

    Article  PubMed  Google Scholar 

  6. Thomas KA, Rickwood DJ. Recovery environment of a sub-acute mental health service. Ment Health Rev J. 2016;21(2):95–106.

    Article  Google Scholar 

  7. Thomas KA, Rickwood DJ, Bussenschutt G. Adult step-up step-down: a sub-acute short-term residential mental health service. Int J Psychosoc Rehabil. 2015;19(1):1–1.

    Google Scholar 

  8. Lee SJ, Collister L, Stafrace S, Crowther E, Kroschel J, Kulkarni J. Promoting recovery via an integrated model of care to deliver a bed-based, mental health prevention and recovery Centre. Australas Psychiatry. 2014;22(5):481–8.

    Article  CAS  PubMed  Google Scholar 

  9. Thomas KA, Rickwood DJ, Brown PM. Symptoms, functioning and quality of life after treatment in a residential sub-acute mental health service in Australia. Health Soc Care Community. 2015;25(1):243–54.

    Article  PubMed  Google Scholar 

  10. Lyoyd-Evans B, Slade M, Jagielska D, Johnson S. Residential alternatives to acute psychiatric hospital admission: systematic review. Br J Psychiatry. 2009;195:109–17.

    Article  Google Scholar 

  11. Hewlett E, Moran V. Executive Summary. In: Making Mental Health Count: The Social and Economic Costs of Neglecting Mental Health Care. Paris: Organisation for Economic Co-operation and Development (OECD) 2014.

  12. NICE. Transition between inpatient mental health settings and community or care home settings: NICE guideline full version. In: National Institute for Health & Care Excellence; 2016.

    Google Scholar 

  13. Dawda P, Russell L. Sub-acute care: an international literature review. Canberra: Australian Primary Health Care Research Institute; 2014.

    Google Scholar 

  14. Harvey C, Brophy L, Tibble H, Killaspy H, Spittal MJ, Hamilton B, Ennals P, Newton R, Cruickshank P, Hall T, et al. Prevention and recovery Care Services in Australia: developing a state-wide typology of a subacute residential mental health service model. Frontiers Psychiatry. 2019;10:383. https://doi.org/10.3389/fpsyt.2019.00383.

  15. Fletcher J, Brophy LM, Killaspy H, Ennals P, Hamilton B, Collister L, Harvey CA. Prevention and recovery Care Services in Australia: describing the role and function of sub-acute recovery-based residential mental health services in Victoria. Frontiers Psychiatry. 2019;10:735.

    Article  Google Scholar 

  16. Thomas KA, Rickwood D. Clinical and cost-effectiveness of acute and subacute residential mental health services: a systematic review. Psychiatr Serv. 2013;64:1140–9.

    Article  PubMed  Google Scholar 

  17. Richardson J, Iezzi A, Khan MA, Maxwell A. Validity and reliability of the assessment of quality of life (AQoL)-8D multi-attribute utility instrument. Patient Patient Centered Outcomes Res. 2014;7(1):85–96.

    Article  Google Scholar 

  18. Gilburt H, Slade M, Rose D, Lloyd-Evans B, Johnson S, Osborn DP. Service users' experiences of residential alternatives to standard acute wards: qualitative study of similarities and differences. Br J Psychiatry Suppl. 2010;53:s26–31.

    Article  PubMed  Google Scholar 

  19. Osborn DP, Lloyd-Evans B, Johnson S, Gilburt H, Byford S, Leese M, Slade M. Residential alternatives to acute in-patient care in England: satisfaction, ward atmosphere and service user experiences. Br J Psychiatry Suppl. 2010;53:s41–5.

    Article  PubMed  Google Scholar 

  20. Council of Australian Government. Council of Australian Government 19 and 20 April 2010 Communique. Canberra: Governments CoA. p. 2010.

  21. Stallman HM, Wilson CJ. Can the mental health of Australians be improved by dual strategy for promotion and prevention? Aust N Z J Psychiatry. 2018;52(6):602.

    Article  PubMed  Google Scholar 

  22. Huppert FA: The State of Wellbeing Science: Concepts, Measures, Interventions, and Policies. In: Interventions and Policies to Enhance Wellbeing: A Complete Reference Guide. Volume Volume VI, edn. Edited by Huppert FA, Cooper CL. West Sussex: Wiley.; 2014: 1–50.

  23. Slade M. Mental illness and well-being: the central importance of positive psychology and recovery approaches. BMC Health Serv Res. 2010;10:26.

    Article  PubMed  PubMed Central  Google Scholar 

  24. World Health Organisation: Constitution of the World Health Organisation. 2006.

    Google Scholar 

  25. Kilbourne AM, Beck K, Spaeth-Rublee B, Ramanuj P, O'Brien RW, Tomoyasu N, Pincus HA. Measuring and improving the quality of mental health care: a global perspective. World Psychiatry. 2018;17(1):30–8.

    Article  PubMed  PubMed Central  Google Scholar 

  26. Thornicroft G, Slade M. New trends in assessing the outcomes of mental health interventions. World Psychiatry. 2014;13(2):118–24.

    Article  PubMed  PubMed Central  Google Scholar 

  27. Happell B. Determining the effectiveness of mental health services from a consumer perspective: part 2: barriers to recovery and principles for evaluation. Int J Ment Health Nurs. 2008;17(2):123–30.

    Article  PubMed  Google Scholar 

  28. Crawford MJ, Robotham D, Thana L, Patterson S, Weaver T, Barber R, Wykes T, Rose D. Selecting outcome measures in mental health: the views of service users. J Ment Health. 2011;20(4):336–46.

    Article  PubMed  Google Scholar 

  29. Trivedi P, Wykes T. From passive subjects to equal partners: qualitative review of user involvement in research. Br J Psychiatry. 2002;181(6):468–72.

    Article  PubMed  Google Scholar 

  30. Anthony WA, CMR FMD, Bachrach LL. Psychiatric rehabilitation. Boston: Center for Psychiatric Rehabilitation, Boston University, Sargent College of Allied Health Professions; 1990.

    Google Scholar 

  31. Andresen R, OL CP. The experience of recovery from schizophrenia: towards an empirically validated stage model. Aust N Z J Psychiatry. 2003;37:586–94.

    Article  PubMed  Google Scholar 

  32. Kessler RCAG, Colpe LJ, Hiripi E, Mroczek DK, Normand SL, Walters EE, Zaslavsky AM. Short screening scales to monitor prevalences and trends in non-specific psychological distress. Psychol Med. 2002;32(6):959–76.

    Article  CAS  PubMed  Google Scholar 

  33. Mundt JCMI, Shear MK, Greist JH. The work and social adjustment scale: a simple measure of impairment in functioning. Br J Psychiatry. 2002;180:461–4.

    Article  PubMed  Google Scholar 

  34. JR SR, editor. Generalized self-efficacy scale. Windsor: NFER-NELSON; 1995.

    Google Scholar 

  35. [https://www.neaminational.org.au/find-services/neami-jmhss/].

  36. Western Australian Mental Health Commission. Western Australian Mental Health Commission. (2017). Western Australian community mental health step up/ step down servcies: model of servcie. Perth: Government of Western Australia. p. 2017.

  37. Oades L, Crowe T, Nguyen M. Leadership coaching transforming mental health systems from the inside out: the collaborative recovery model as person-centred strengths based coaching psychology. Int Coach Psychol Rev. 2009;4(1):25–36.

    Google Scholar 

  38. Oades LG, Deane FP, Crowe TP, Lambert WG, Kavanagh D, Lloyd C. Collaborative recovery: an integrative model for working with individuals who experience chronic and recurring mental illness. Aust Psychiatry. 2005;13(3):279–84.

    Google Scholar 

  39. Oades LG, Deane FP, Crowe TP. Collaborative Recovery Model: From Mental Health Recovery to Wellbeing. In: Slade M, Oades L, Jarden A, editors. Wellbeing, Recovery and Mental Health. Cambridge: UK: Cambridge University Press; 2017. p. 99.

    Chapter  Google Scholar 

  40. Gilbert MM, Chamberlain JA, White CR, Mayers PW, Pawsey B, Liew D, Musgrave M, Crawford K, Castle DJ. Controlled clinical trial of a self-management program for people with mental illness in an adult mental health service - the optimal health program (OHP). Aust Health Rev. 2012;36(1):1–7.

    Article  PubMed  Google Scholar 

  41. Australian Bureau of Statistics. Census of Population and Housing: Socio-Economic Indexes for Areas (SEIFA), Australia, 2011. Canberra: Australian Bureau of Statistics 2014.

  42. Oades L, Anderson J. Recovery in Australia: Marshalling strengths and living values. Int Rev Psychiatry. 2012;24(1):5–10.

    Article  PubMed  Google Scholar 

  43. Andresen R, Caputi P, Oades LG. Do clinical outcome measures assess consumer-defined recovery? Psychiatry Res. 2010;177(3):309–17.

    Article  PubMed  Google Scholar 

  44. Petrakis M, Stirling Y, Higgins K. Vocational support in mental health service delivery in Australia. Scand J Occup Ther. 2019;26(7):535–45.

    Article  PubMed  Google Scholar 

  45. Australian Bureau of Statistics. Estimates of Aboriginal and Torres Strait Islander Australians, June 2016. Canberra; 2018.

Download references

Acknowledgements

The authors wish to acknowledge Mr. Arthur PapaKotsias, CEO of Neami National at the time of the study, for his part in initiating this study in collaboration with The University of Western Australia.

Funding

Apart from the potential competing interests as declared above, Neami National, the ‘funder’, has agreed to fund the cost of the publication of this study.

Author information

Authors and Affiliations

Authors

Contributions

HN contributed to the study design; analyzed, interpreted and presented the quantitative data; and wrote most of the manuscript. PE contributed to the study design; and critically reviewed and edited the manuscript drafts. ST managed and supplied the source data for analysis; and critically reviewed and edited the manuscript drafts. EG and AC provided input in the initial stage of the study conception; and critically reviewed and edited the manuscript drafts. KW contributed to the study design; analyzed, interpreted and presented the qualitative data; and contributed to the write-up and editing of the manuscript. The funder Neami National, and its employees, did not have input into, or veto over, HN’s analysis of the quantitative data or the interpretation of these results. All authors approved the submitted version.

Corresponding author

Correspondence to Hanh Ngo.

Ethics declarations

Ethics approval and consent to participate

JMHSS patients provided consent for their data to be used for the purpose of personalised service, service evaluation and research. The patients did not provide their consent explicitly for this study. As such, the authors sought for a Waiver of Consent consideration from the University of Western Australia’s Human Research Ethics Committee (HREC). This request was subsequently granted by the mentioned HREC (Reference Number RA/4/1/8805).

Consent for publication

Not applicable.

Competing interests

Within her employment with the University of Western Australia, Hanh Ngo was contracted to provide an independent evaluation of the Joondalup Mental Health Step-Up Step-Down service, run by Neami National. Elizabeth Geelhoed received a fee for service from Neami National, in another evaluation component subsequent to that reported in the current manuscript. Priscilla Ennals, Serhat Turut, and Keren Wolstencroft were employees of Neami National.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary information

Additional file 1.

Sub-acute consumer exit questionnaire

Additional file 2.

Comparing baseline profiles of Exit Questionnaire completers (n=251) versus non-completers (n=300)

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Ngo, H., Ennals, P., Turut, S. et al. Step-up, step-down mental health care service: evidence from Western Australia’s first – a mixed-method cohort study. BMC Psychiatry 20, 214 (2020). https://doi.org/10.1186/s12888-020-02609-w

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1186/s12888-020-02609-w

Keywords