Setting
Data were collected in the context of a routine outcome monitoring procedure. Assessments were performed by trained independent raters (usually psychologists) and were planned every six to twelve months. These routine outcome monitoring assessments were available for clinicians to use in clinical practice when discussing treatment progress with the patient. Routine outcome monitoring data-collection was approved by the Dutch Committee for the Protection of Personal Data. Data for this study apply to the period from February 2002 to April 2012, and were used anonymously. By Dutch law, studies only using questionnaires, do not need formal evaluation by a Medical Ethical Committee, when data are used anonymously [13]. In addition, in a study such as this, not needing formal evaluation by a Medical Ethical Committee, also no informed consent was required considering the observational nature of the study and because all assessments were collected in the context of a Routine Outcome Monitoring procedure, without any additional burden on the patient.
The study involved patients from seven Assertive Community Treatment (ACT) teams in the city of Rotterdam, the Netherlands. Criteria for treatment by an ACT team were a) age 18 or older, b) having a severe mental illness, usually a psychotic or bipolar disorder (with or without a co-morbid substance use disorder); and c) lack of motivation to be treated at the start of ACT, such that assertive outreach was necessary.
The model fidelity of the ACT teams was assessed using the Dartmouth Assertive Community Treatment Scale (DACTS) [14]. The mean of the total DACTS scores of the ACT teams was 3.5 (range: 2.9 – 3.8), meaning that, on average, ACT had been implemented with moderate success. On the human resources subscale, model fidelity was high (i.e., items that were awarded with scores 4–5). Low scores (i.e., items that were awarded with scores 1–2) were awarded to items pertaining to the nature of services subscale, such as intensity of services, frequency of contact, provision of dual disorder treatment groups, and role of consumers on team (i.e. consumers not involved in providing service) [15].
Measures
Camberwell Assessment of Need Short Appraisal Schedule (CANSAS)
The CANSAS – a modified version of the Camberwell Assessment of Need (CAN) [16] – consists of 22 items [17]. To assess the need for care, it assesses health and social needs across the following domains: accommodation, food, looking after the home, self-care, daytime activities, physical health, psychotic symptoms, information, psychological distress, safety to self, safety to others, alcohol, drugs, company, intimate relationships, sexual expression, childcare, basic education, telephone, transport, money, and benefits. Each item is scored 0 (no problem), 1 (met need) or 2 (unmet need). The reliability of the English version of the CANSAS is acceptable [18],[19]. The needs for care were assessed using a Dutch translation of the CANSAS [20],[21].
Quality of life scale
The Cumulative Needs for Care Monitor (CNCM) quality of life scale was used to measure subjective quality of life (assessed in Dutch) [20],[21]. This instrument was based on the Lancashire Quality of Life Profile [22] and was very similar to the Manchester Short Assessment of Quality of Life scale (MANSA) [23], which consists of six items [24]: financial situation, accommodation/living situation, relationship with others, physical health, psychological health, and life as a whole. These are rated on a 7-point scale (1 = “Couldn’t be worse” to 7 = “Couldn’t be better”). This scale has strong correlations with the Lancashire Quality of Life Profile [21].
Motivation item
The scale for assessing motivation for treatment was adapted from the Severity of Psychiatric Illness scale [25]-[27], an observer-rated scale covering the last two weeks. It was scored in five categories (0 = “Highly motivated” to 4 = “Lack of motivation”) in the same way as the Health of the Nation Outcome scales (HoNOS; [28],[29]). The psychometric properties of the English and Dutch HoNOS total scores have been found to be acceptable [28],[29].
Analyses
SPSS version 18.0 was used for all analyses. In the routine outcome monitoring data we identified 827 eligible patients, i.e., patients who had had at least two routine outcome monitoring assessments. For analytical purposes we used only complete records of CANSAS (from which no more than 5 items were missing; missing values were then regarded as no need for care) and QoL assessment, and found that 251 patients had completed both the CANSAS and QoL in two consecutive assessments. Descriptive statistics (i.e. means, standard deviations, median, inter-quartile range and percentages) were calculated for outcome variables and patient characteristics. Pearson’s chi-square tests were used for categorical data, and independent samples t-tests for normally distributed data. Paired samples T-tests were used to compare pairwise baseline and follow-up scores for normally distributed data; related-samples Wilcoxon Signed Rank Tests were used for non-normally distributed data. For analytical purposes, the scale for assessing motivation for treatment was dichotomized into two groups (score 0 to 2 and score 3 to 4).
To analyze the associations between changes in QoL over time and changes in the number of unmet needs, we used a regression analysis in which the dependent variable was QoL total score (T1; second available routine outcome monitoring assessment). The determinants were 1) number of unmet needs (T0; first available routine outcome monitoring assessment); 2) QoL total score (T0; first available routine outcome monitoring assessment); 3) changes in number of unmet needs over time, and 4) an interaction (T0 QoL total score * change in number of unmet needs over time).
Then, to study the association between treatment duration and outcome in QoL over time (i.e. to determine whether or not a patient responded to treatment [30]), we determined 1) criteria for clinically meaningful change, and 2) cut-off points between a low and high level of QoL [31]. This combination of both approaches allowed us to look beyond the traditional method of change scores, as it created a hierarchy of outcomes for the QoL scores over time on the basis 1) of the degree of change and 2) of the classification of the total score.
These criteria were calculated using a distribution-based method and an anchor-based method. Distribution-based methods, such as the standard error of measurement (SEM; for formula see Appendix), use statistical characteristics of the data (e.g., standard deviation and measurement precision of the instrument), and provide an estimation of test error which can be used to interpret a patients’ score in a test. The SEM corresponds to a clinically meaningful amount of change in total score (difference between T0 and T1) [30]-[33].
Anchor-based methods are used to identify cut-off points to differentiate between a low and high level of QoL [33]. Total score cut-off points can be calculated by determining the likelihood that patients who report satisfaction in all QoL domains will exceed that of patients who report dissatisfaction in one or more QoL domain. Patients were considered to have a low QoL if they did not report being satisfied in all the QoL items (i.e., a score below 5 in one or more of the QoL items). The clinical significance (CS) cut-off point between a low and high level of QoL was computed (for formula see Appendix). After these calculations, we combined these criteria (SEM and CS cut-off) to create a model in which meaningful change and a classification of the severity were integrated (according to the Jacobson and Truax approach [34]). By combining the classification (CS-cut-off of 33) and the meaningful change criteria (4-point change), this created 10 possible groups, which were further combined into 4 QoL-change groups for analytic purposes.
The QoL meaningful change and outcome classification was:
-
[1]
Very poor: WORSENED from high to low quality of life; WORSENED within low quality of life
-
[2]
Fair to poor: STABLE within low to high quality of life; STABLE within low quality of life; STABLE within high to low quality of life
-
[3]
Good: IMPROVED within low quality of life
-
[4]
Very good: STABLE within high quality of life; IMPROVED within high quality of life; IMPROVED from low to high quality of life
To investigate the change and outcome in individual unmet needs for care (score 2 on individual CANSAS items) in relation to 1) level of QoL and 2) meaningful change in QoL, we created 4 groups for each CANSAS item.
Classification of change and outcome in individual CANSAS items:
-
[1]
Very poor: T0 unmet need & T1 unmet need
-
[2]
Poor: T0 no unmet need & T1 unmet need
-
[3]
Good: T0 unmet need & T1 no unmet need
-
[4]
Very Good: T0 no unmet need & T1 no unmet need
To examine the relation between change and outcome in individual CANSAS items (group 1–4) and meaningful change and outcome in QoL (groups 1–4), we used bivariate Spearman correlation coefficients (22 correlations).
After these preliminary analyses, we performed an ordinal regression analysis that included CANSAS items as determinants (22 items; each categorized into 4 groups) and QoL as dependent variable (the 4 groups above). The ordinal regression started with stepwise forward selection in which determinants required a probability value of P < 0.25 for entry into the model. Then, using stepwise backward elimination and a log likelihood test, the determinants were removed at a probability value of P > 0.05 [35]. As two determinants (CANSAS items 10 `safety to self’ and 21 `money’) violated the proportional odds assumption, they were excluded from the model-fitting procedure.