Prospective study of factors influencing conditional discharge from a forensic hospital: the DUNDRUM-3 programme completion and DUNDRUM-4 recovery structured professional judgement instruments and risk

Abstract

Background

We set out to examine whether structured professional judgement instruments DUNDRUM-3 programme completion (D-3) and DUNDRUM-4 recovery (D-4) scales along with measures of risk, mental state and global function could distinguish between those forensic patients detained in a secure forensic hospital (not guilty by reason of insanity or unfit to stand trial) who were subsequently discharged by a mental health review board. We also examined the interaction between these measures and risk, need for therapeutic security and eventual conditional discharge.

Methods

A naturalistic observational cohort study was carried out for 56 patients newly eligible for conditional discharge. Patients were rated using the D-3, D-4 and other scales including HCR-20, S-RAMM, START, SAPROF, PANSS and GAF and then observed over a period of twenty three months during which they were considered for conditional discharge by an independent Mental Health Review Board.

Results

The D-3 distinguished which patients were subsequently discharged by the Mental Health Review board (AUC = 0.902, p < 0.001) as did the D-4 (AUC = 0.848, p < 0.001). Item to outcome analysis showed each item of the D-3 and D-4 scales performed significantly better than random. The HCR-20 also distinguished those later discharged (AUC = 0.838, p < 0.001) as did the S-RAMM, START, SAPROF, PANSS and GAF. The D-3 and D-4 scores remained significantly lower (better) for those discharged even when corrected for the HCR-20 total score. Item to outcome analyses and logistic regression analysis showed that the strongest antecedents of discharge were the GAF and the DUNDRUM-3 programme completion scores.

Conclusions

Structured professional judgement instruments should improve the quality, consistency and transparency of clinical recommendations and decision making at mental health review boards. Further research is required to determine whether the DUNDRUM-3 programme completion and DUNDRUM-4 recovery instruments predict those who are or are not recalled or re-offend after conditional discharge.

Background

The Butler Report [1] described conditional discharge as the “most valuable feature of the system of restriction orders” and it has been shown that patients who have been conditionally discharged from forensic hospitals have lower rates of recidivism than those released without conditions [2]. Coid et al. showed that longer stay and restriction on discharge are protective factors against reoffending in patients discharged from medium secure forensic psychiatry services [3]. However despite the benefits of conditional discharge, the decision to recommend a patient for conditional discharge is one of the most difficult taken by a forensic psychiatrist. Mental health tribunals and review boards make decisions that require the balancing of rights and risks according to law. Patients should not be detained at a level of therapeutic security higher than that which is deemed necessary, for their own safety and the safety of others [4] however this must be balanced with public safety and victim rights issues.

Risk assessment is a key part of the process when making decisions regarding a patient’s readiness for discharge to the community. Dolan and Khawja [5] showed that the HCR-20 was a predictor of readmission and self reported violence in groups of discharged male medium secure patients. It has also been shown that the HCR-20 is a good predictor of both violent and non-violent offending in patients released from forensic psychiatric hospitals [6]. Doyle at al have also shown that dynamic (“current” and “risk”) items on the HCR-20 significantly improved the accuracy of prediction of violence after discharge from forensic units [7]. It has been shown that factors such as a higher score on PCL-R and a younger age at the time of first criminal offence were significantly related to release recommendations in a group of NGRI patients in the USA [8]. However evidence presented to Criminal Law (Mental Health) Review boards, when recommending a patient for a move to less secure places, consists of more than risk assessment. In practice evidence given by clinicians also includes or takes account of factors such as rapport, insight, therapeutic alliance and use of leave from the hospital. Risk assessment in forensic psychiatry has evolved in recent years from the use of unstructured clinical judgement to the use of structured professional judgement instruments. These instruments add to the transparency and accountability of the process of risk assessment, uniting the tasks of prediction, assessment, clinical management and communication [9]. The aim of the DUNDRUM toolkit is to provide the same transparency and accountability to the decisions regarding admission to and discharge from secure hospital settings [10].

The DUNDRUM toolkit is a set of five structured professional judgement instruments developed at the Central Mental Hospital Dundrum, the Republic of Ireland’s only secure forensic hospital [10]. The first two instruments of the DUNDRUM Toolkit are concerned with the admission of patients to secure hospitals, acting as structured professional judgement instruments to decide the level of need for therapeutic security and the urgency of that need [10]. We have previously shown that these two instruments distinguished between patients who required admission to different levels of therapeutic security and distinguished the urgency of need amongst patients [11, 12]. The DUNDRUM-3 and DUNDRUM-4 scales are structured professional judgement instruments to rate a patient’s programme completion and recovery and therefore the patient’s continuing need for therapeutic security [10]. We have also shown that the DUNDRUM-3 programme completion items distinguished significantly between levels of therapeutic security and the DUNDRUM-4 recovery items consistently distinguished those given unaccompanied leave outside the secure hospital setting [13]. The fifth scale is a self-rated instrument which patients use to rate their own recovery and their own ongoing need for therapeutic security and the validation of this instrument is currently underway. We have recently shown that the DUNDRUM-1 triage security scale and HCR-20 instrument for the assessment of risk of violence distinguished between those who were moved from higher to less secure wards within the hospital and also from less secure back to more secure wards, and the DUNDRUM-3 and DUNDRUM-4 also distinguished those who were moved [14]. Our hypothesis is that it should be possible to use structured professional judgement to guide decision making about conditional or absolute discharge from hospital.

The Criminal Law (Insanity) Act 2010 [15] was introduced in the Republic of Ireland in March 2011. It introduced, for the first time in Ireland, the power for mental health review boards (MHRBs) to grant conditional discharge to those detained following a verdict of not guilty by reason of insanity (NGRI) or unfit to stand trial (UST). This ’experiment of nature’ offered the opportunity to carry out a prospective observational study on the clinical facts that influence how clinicians make such recommendations and MHRBs make such decisions.

In this prospective cohort study we examined whether the DUNDRUM-3 programme completion and DUNDRUM-4 recovery instruments [10] measured at baseline just before the commencement of the new legislation could distinguish between those who were or were not subsequently discharged either conditionally or unconditionally from this high and medium secure forensic hospital setting, and whether measures of risk, need for therapeutic security, mental state and global function also played a part in the decision. Because we are interested in the use of structured professional judgement itself, we also wished to study the ‘item to outcome’ results for these instruments, since a study of the scale scores would be insufficient, amounting to an actuarial study only.

Methods

Study design

This was a naturalistic prospective observational cohort study. Data were gathered as part of the routine clinical audit of service delivery and outcomes. The study was approved by the research ethics, audit and effectiveness committee of the Central Mental Hospital. All participants were given information about the nature and purpose of the assessments and gave informed consent.

Structured professional judgement instruments and measures of mental state and global function were assessed at baseline in February 2011 just before the commencement of the Criminal Law (Insanity) Act 2010 which permitted conditional discharge for the first time. This allowed the MHRB to grant conditional discharge where appropriate and safe to do so, and subject to conditions regarding both treatment adherence and supports according to the individual’s risks and needs.

The Mental Health Review Board was supplied with a report from the treating consultant psychiatrist, had access to all clinical notes, assessments and reports and heard oral evidence from the patient, the treating consultant psychiatrist and the patient’s lawyer, and any other evidence the MHRB wished to hear. The clinicians and MHRB were blind to the assessments reported here, carried out independently of the treating clinicians prior to the commencement of the new legislation. However neither the clinicians nor the members of the MHRB could be blinded to the unstructured factual clinical information used to make the ratings for the various instruments, nor could they have been blinded to such information. Clinicians decided to recommend or not to recommend absolute or conditional discharge in accordance with their normal practice, a synthesis of the results of structured professional judgement instruments such as the HCR-20 (rated by themselves) and broad unstructured bio-psycho-social assessment including reports of progress in hospital. The MHRB received reports from the treating psychiatrists and other clinicians and was independent in the exercise of its powers [15].

Setting

The Central Mental Hospital is the only secure forensic psychiatric hospital in the state, providing high, medium and low secure therapeutic wards for a population of 4.6 million. It is the only hospital designated under the Criminal Law (Insanity) Acts 2006 & 2010 for the reception and treatment of patients found unfit to stand trial or not guilty by reason of insanity, or transferred from prison to hospital for psychiatric treatment. Male patients in the Central Mental Hospital are admitted to a high secure admission ward. The hospital is organised into a series of progressively less secure wards and patients are moved from ward to ward along this recovery pathway, progressing from high secure admission units, through medium secure units and on to low secure pre-discharge rehabilitation units [16]. These placements correspond to a coherent patient-centred pathway through care as patients are placed at the level of therapeutic security deemed most appropriate for their needs. Placements also correspond to a stratified risk management system [17].

Legal structures

Decisions to offer conditional or unconditional discharges to patients detained under the Criminal Law (Insanity) Acts 2006 & 2010 on the grounds of unfitness to stand trial (UST) or not guilty by reason of insanity (NGRI) were made by a Mental Health (Criminal Law) Review Board (MHRB). Those detained under the Mental Health Act 2001 were reviewed by a different body, the Mental Health Tribunal [18] that could grant absolute discharges but not conditional discharges or community treatment orders.

Participants

All patients in the hospital were assessed between February and March 2011 of whom 56 were newly eligible for conditional discharge to the community by the mental health review board. Data on the eligible patients and their subsequent discharges from the hospital were then gathered up to 31st December 2012. Those detained because UST (8 patients) were eligible for conditional discharge by the MHRB in the same way as those detained because found NGRI (48 patients), though those detained because UST could also be returned to court by the treating consultant if they regained fitness.

Variables

The primary outcome measure was absolute or conditional discharge by the MHRB from this high and medium secure forensic hospital setting following the commencement of the Criminal Law (Insanity) Act 2010 on 8th February 2011. Patient discharges were documented up to 31st December 2012, an observation period of twenty three months. During this period each of the 56 patients eligible for discharge had at least three Criminal Law Review Board hearings thus ensuring that each patient had the opportunity to be granted a conditional or absolute discharge.

Data sources and measurement

The DUNDRUM-1 triage security instrument, DUNDRUM-3 programme completion (D-3) and DUNDRUM-4 recovery (D-4) scales are part of a set of four structured professional judgement instruments for the assessment of need for therapeutic security [10]. The DUNDRUM-1 triage security scale is a static measure while the DUNDRUM-3 and DUNDRUM-4 are dynamic measures sensitive to change and response to treatment. The content of these instruments is different from but complementary to risk assessment. When assessing readiness for discharge to the community, clinicians are likely to take more than risk assessment alone into account. Factors such as mental health, physical health, self care and activities of daily living, family and social networks, use of leave from the hospital and other such factors are all given strong consideration. These items are often included in clinician’s unstructured reports to mental health tribunals and review boards to assist these bodies in their decision making with regard to a patient’s readiness for conditional or absolute discharge, or neither. The items scored in the DUNDRUM-3 and DUNDRUM-4 include the above items and the item definitions and ratings are based on motivation theory, cycle of change and engagement. The DUNDRUM-3 programme completion instrument consists of seven items - physical health, mental health, drugs and alcohol, problem behaviours, self care and activities of daily living, occupation, education and creativity and family and social networks. The six items of the DUNDRUM-4 recovery scale are stability, insight, rapport and working alliance, leave, HCR-20 dynamic risk items and victim sensitivity. Each item is accompanied by a series of definitions and rated from ‘0’ to ‘4’. A patient scoring mostly ‘4’s is unlikely to be ready for a move to a less secure place, a patient scoring mainly ‘3’s is likely to be ready to move from high to medium security, mainly ‘2’s is likely to be ready to move from medium security to PICU, mainly ‘1′s is ready for placement in the community setting and mainly ‘0’s is likely to be ready for an absolute discharge.

We have previously shown that the DUNDRUM-3 programme completion and the DUNDRUM-4 recovery scales have excellent psychometric properties [13] and distinguished between patients who moved between levels of therapeutic security within the forensic hospital setting [14] while the DUNDRUM-1 triage security scale and the HCR-20 risk assessment score [19] accounted for most of the variation in this decision [14].

As part of the hospital’s routine data gathering for audit and outcome measures, all in-patients were assessed at baseline prior to the commencement of the Criminal Law (Insanity) Act 2010 by MD using the DUNDRUM-1 triage security instrument, DUNDRUM-3 programme completion instrument and DUNDRUM-4 recovery instrument between February and March 2011. All patients were also assessed for the Positive and Negative Symptom Score (PANSS) [20] and Global Assessment of Function (GAF) [21] by post membership psychiatrists (OG and LN) who were blind to the ratings of DUNDRUM-3 and DUNDRUM-4. The HCR-20 [19] and S-RAMM [22] were rated by the treating clinicians and collated by AN. The HCR-20 historical item H7 ‘psychopathy’ was omitted. ZA rated all patients using the START [23] and the Structured Assessment of Protective Factors for Violence Risk (SAPROF) [24]. The treating clinicians were blind to these audit and outcome ratings when making their reports to the MHRB, as were the MHRB.

Confounding and bias

Possible sources of confounding and bias were considered, including the need for therapeutic security at baseline measured by the DUNDRUM-1 triage security instrument [10] since historical considerations might influence clinicians to be more conservative when recommending whether a patient be offered a conditional or unconditional discharge, or neither. The DUNDRUM-1 was also used to enable benchmarking against other services and other studies. Other confounders considered included a measure of risk of violence the HCR-20 [19], risk of suicide and self harm measured by the S-RAMM [22] and protective factors the START-S [23] and SAPROF [24], a measure of mental state (PANSS) [20] and global function (GAF) [21].

Statistical methods

No data were missing. All data were entered in SPSS-18 [25].

Association with outcome was tested using the receiver operating characteristic (ROC) area under the curve (AUC), where a significant result for the AUC is one that differs significantly from the ‘random’ AUC of 0.5 - as a minimum the 95% confidence interval for the AUC does not overlap 0.5.

Where analysis of variance was carried out, differences between groups can be assessed from the over-lap of confidence intervals. Univariate analysis of a general linear model was carried out for discharge, co-varying for the HCR-20 total score at the beginning of the period of observation because the HCR-20 measure of dynamic risk is already established as a predictor of success or failure in such discharges [5–8]. Similarly, we tested whether the HCR-20 scores differentiated between those discharged and those not when co-varying for DUNDRUM-3 and DUNDRUM-4 scores.

Regression analysis was used to examine the antecedent covariates of discharge. Discharge was treated as a binary variable, and binary logistic regression was used with either forward or backward stepwise likelihood ratios as appropriate.

Covariates were chosen according to the results of the receiver operating characteristic.

Unadjusted odds ratios were used as an indicator of the magnitude of effect of individual items, where these could be calculated.

Results

Participants

There were 56 eligible patients. The mean follow up period was 1.748 years (SD 0.248). Twelve were conditionally discharged and none were absolutely discharged as a first outcome during the follow up period (Table 1). One who had been conditionally discharged was subsequently absolutely discharged – this was not double counted.

Table 1 Receiver operating characteristic area under the curve (AUC), with 95% confidence intervals and asymptotic ‘p’ value for null hypothesis that AUC does not differ from 0.5, for 56 patients of whom 12 were granted a conditional discharge

Descriptive data

At baseline, mean age was 43.7 years (SD 12.8) and mean length of stay was 11.1 years (SD 11.3) for the 56 eligible patients. All were Irish born. There were no significant differences between those discharged by the MHRB (n = 12) and those who were not discharged (n = 44) though there was a tendency for those conditionally discharged to be older (44.1(10.0) years vs 42.3(13.6)) and to have had longer lengths of stay (11.6(8.7) years vs 10.9(11.9)). Diagnosis according to ICD-10 criteria [26] was schizophrenia (F20) 41 (73%), schizoaffective disorder (F25) 4 (7%), bi-polar affective disorder (F31) 5 (9%), recurrent depressive disorder, severe with psychotic symptoms (F33.3) 4 (7%), intellectual disability (mental retardation with significant impairment of behaviour F71.1 and 72.1) 2 (4%). The legal status was unfit to stand trial 8 (14%), not guilty by reason of insanity 48 (86%).

The need for therapeutic security was assessed at baseline using the DUNDRUM-1 triage security scale [10]. The mean DUNDRUM-1 score for the 56 eligible participants was 28.6 (SD 4.2). As there are 11 items in the D-1 scale all scored from ‘0’ to ‘4’ this equates to a mean item score of 2.6, where ‘2’ would be a score consistent with a low secure profile and ‘3’ would be a score consistent with a medium secure profile [15], the mean for all was therefore higher than the level of a ‘typical’ low secure patient.

Outcome data

Of the 8 patients admitted because they were deemed unfit to stand trial 5 remained as in-patients during the period of observation while three were found fit to stand trial in court and returned to prison or the community. All had been considered by the Mental Health Review Board but had not been discharged. Of the 48 patients who had the legal status of Not Guilty by Reason of Insanity (NGRI) 12 were discharged subject to conditions by the MHRB while the remaining 36 continued to be detained as in-patients in hospital at the end of the period of observation.

Main results

Receiver operating characteristics

For the 56 eligible for conditional or absolute discharge by the MHRB, Table 1 shows the area under the curve (AUC) of the receiver operating characteristic with discharge (all conditional) by the MHRB as the outcome measure.

Neither length of stay at baseline (AUC = 0.598, 96% CI 0.434 – 0.763, p = 0.299) nor age at baseline (AUC = 0.558, 95% CI 0.393 – 0.722, p = 0.539) was associated with subsequent discharge.

The DUNDRUM-1 triage security scale was not associated with subsequent conditional discharge (AUC = 0.624, 95% CI 0.446 – 0.802, p = 0.192). The DUNDRUM-3 Programme completion scale distinguished which patients went on to receive a discharge from the Criminal Law (Mental Health) Review board (AUC = 0.902, p < 0.001) as did the DUNDRUM-4 Recovery Scale (AUC = 0.848, p < 0.001).

The HCR-20 total score also distinguished which patients went on to receive a discharge (AUC = 0.838, p < 0.001) as did the HCR-20 historical items (AUC = 0.745, p = 0.010) and HCR-20 dynamic (clinical and risk) items (AUC = 0.788, p = 0.002).

The S-RAMM background items did not distinguish which patients went on to receive a discharge from the Criminal Law (Mental Health) Review board (AUC 0.546, p = 0.633), however the S-RAMM dynamic (current and future) items (AUC = 0.738, p = 0.013) and S-RAMM total score (AUC = 0.869, p = 0.045) were significant.

The START strengths distinguished which patients went on to receive a conditional discharge (AUC = 0.904, p < 0.001) as did the START vulnerabilities (AUC = 0.899, p < 0.001) and the SAPROF (AUC = 0.806, p = 0.001).

The Global Assessment of Functioning (GAF) Scale distinguished which patients received a discharge from the Criminal Law (Mental Health) Review board (AUC = 0.930, p < 0.001).

The PANSS positive scale distinguished which patients went on to receive a discharge (AUC = 0.758, p = 0.007) as did the PANSS negative scale (AUC = 0.809, p = 0.001), PANSS general symptoms score (AUC = 0.809. p = 0.001) and the total PANSS score (AUC = 0.846, p < 0.001).

Analysis of variance

Table 2 shows that the DUNDRUM-3 and DUNDRUM-4 scores, including the DUNDRUM-4 score omitting the item for ‘risk’ were all significantly lower (better) at baseline for those who went on to be conditionally discharged. The GAF score was significantly higher (better) and PANSS positive, negative and general scores were significantly lower (better) at baseline for those who were later discharged. The HCR-20 historical, clinical, risk and total scores, S-RAMM current and future scores were lower (better) as was the START-V score, while the SAPROF and START-S were higher (better) in those who were later discharged. When co-varied for HCR-20 total score, only the DUNDRUM-3 and DUNDRUM-4 scores remained significantly better at baseline in those who went on to be discharged, though the GAF and START scores neared statistical significance.

Table 2 Determinants of discharge: crude data and marginal means adjusted for HCR-20 total score

Table 3 shows that when the HCR-20 sub-scales and total scores were adjusted for DUNDRUM-3 or DUNDRUM-4 (omitting the risk item from the DUNDRUM-4) they were no longer significantly better for those conditionally discharged.

Table 3 HCR-20 scores and discharge by tribunals - crude data and marginal means adjusted for the DUNDRUM-3 programme completion score and DUNDRUM-4 recovery score (omitting item D-4-R item 5 ′risk’ to avoid circularity)

Logistic regression

Binary logistic regression was performed using forward stepwise likelihood ratios for the variables with significant areas under the curve. These were the DUNDRUM-3, DUMDRUM-4, HCR-20 total score, S-RAMM current score, START-S, SAPROF, GAF and PANSS total scores. The START-V was omitted and HCR-20 sub-scales combined to reduce multiple co-linearity while the START-S was also omitted to avoid co-linearity with SAPROF. The iterative process resolved in one step for the GAF as the only significant variable remaining in the regression equation. The model was acceptable with Hosmer and Lemeshow X² = 8.45, df = 8, p = 0.391, Nagelkerke R² = 0.610, and GAF odds ratio = 1.258 (95% confidence interval 1.089 – 1.454) p = 0.002. This model correctly distinguished 95.1% of those not discharged, 83.1% of those who were discharged and 92.5% overall. This model was not robust with backwards logistic regression yielding a different result, resolving in five steps (DUNDRUM-3 odds ratio = 0.682, 95% CI 0.470 – 0.991, p = 0.045; SAPROF OR = 0.633, 95% CI 0.413 – 0.967, p = 0.034; GAF OR = 1.387, 95% CI 1.054 – 1.825, p = 0.020, Hosmer and Lemeshow X² = 3.629, df = 8, p = 0.889, Nagelkerke R² = 0.739, correct predictions as before).

Because the model was not robust, the logistic regression was repeated, this time omitting the GAF. This was done because of the apparent interactive effects between the GAF and SAPROF, evident also when the START-S was included instead of the SAPROF. Backward logistic regression resolved in five iterations, Nagelkerke R² = 0.603, Hosmer and Lemeshow X² = 6.05, df = 8, p = 0.642, with 97.6% correct for those not discharged, 66.7% correct for those discharged and 90.6% correct overall. The model included two variables DUNDRUM-3 OR = 0.717, 95% CI 0.561 – 0.916, p = 0.008 and PANSS total score which did not reach significance OR = 0.913, 95% CI 0.821 – 1.016, p = 0.097. This model was reasonably robust and when repeated by forward logistic regression yielded a model in one iteration with Nagelkerke R² = 0.525, Hosmer and Lemeshow X² = 2.37, df = 7, p = 0.927, 92% correct for those not discharged, 66.7% correct for those discharged and 86.8% correct overall and included a single variable, DUNDRUM-3 odds ratio = 0.698, 95% CI 0.563 – 0.866, p < 0.001.

Secondary analysis

On an item to outcome analysis, all items of the DUNDRUM-3 programme completion scale and all items of the DUNDRUM-4 recovery scale were significantly related to eventual discharge (Table 4). Although the DUNDRUM-1 triage security scale was not a significant indicator of subsequent conditional discharge, a lower score on item TS7 ‘preventing access’ was associated with conditional discharge (AUC = 0.726, 95% CI 0.539 – 0.912, p = 0.018) and a lower score on a further item, TS5 ′specialist forensic need’ was strongly associated with subsequent conditional discharge (AUC = 0.845, 95% CI 0.725 - 0.966, p = 0.003).

Table 4 Item to outcome measures for DUNDRUM-3 programme completion and DUNDRUM-4 recovery

For the HCR-20 historical (fixed) items only low scores on H3 ‘relationship instability’ and H4 ‘employment problems’ distinguished subsequent discharge, though H7 psychopathy was omitted. Amongst the dynamic items the associations of subsequent discharge were lower scores for C1 ‘lack of insight’, C3 ‘active symptoms of major mental illness’, C5 ‘unresponsiveness to treatment’ and R1 ‘plans lack feasibility’ (Table 5).

Table 5 Item to outcome measures for HCR-20

For the SAPROF, the items that distinguished those later discharged were item 4 ‘coping’, item 5 ′self control’, item 6 ‘work’, item 8 ‘financial management’, item 11 ‘life goals’ and item 13 ‘social network’ (Table 6).

Table 6 SAPROF and conditional discharge

For START strengths, the significant items for subsequent discharge were higher scores on item 2 ‘relationships’, item 3 ‘occupational’, item 6 ‘mental state’, item 10 ‘external triggers‘, item 17 ‘insight’, item 18 ‘plans’, item 19 ‘coping’ and item 20 ’treatability‘ (Tables 7 and 8).

Table 7 START and conditional discharge

Table 8 START and conditional discharge

For START vulnerabilities the items that distinguished those not subsequently discharged were item 1 ‘social skills’, item 2 ‘relationships’, item 3 ‘occupational’, item 6 ‘mental state’, item 11 ‘social support’, item 17 ‘insight’, item 18 ‘plans’, item 19 ‘coping’ and item 20 ‘treatability’ (Tables 7 and 8).

For the S-RAMM, an assessment of risk of suicide and self harm, the items that distinguished those later discharged were lower scores for item C4 ‘treatment adherence’, C7 ‘psychosocial stress’, F2 ‘future service contact’, F3 ‘future response to drug treatment’ and F4 ‘future response to psychological interventions’ (Table 9).

Table 9 S-RAMM items related to outcomes

The PANSS symptoms that were associated with subsequently not being discharged were P1 ‘delusions’, P6 ‘suspiciousness / persecution’, N1 ‘blunted affect’, N4 ‘passive/apathetic/social withdrawal’, N5 ‘difficulty in abstract thinking’ and G12 ‘lack of judgement and insight’ (Table 10).

Table 10 PANSS items and AUC for discharge

Table 11 shows the 61 items from all of the above scales and instruments with confidence intervals for the AUC that did not overlap the random value of 0.5, arranged in order of magnitude of the AUC. A few themes appear to emerge including global function and coping skills, insight and mental state, problem behaviours and specialist forensic need, negative symptoms, therapeutic rapport, substance misuse problems, victim issues and physical health.

Table 11 Variables in order of magnitude of area under the curve for prediction of discharge

Discussion

Key results

This was not an actuarial study. This study was not primarily about predicting discharge though this forms one part of validation. The main purpose was to validate the content of structured professional judgement instruments to aid decision making regarding discharge from a forensic hospital. The MHRBs were blind to the instrumental ratings made prior to the commencement of the new legislation. They were not blind to the clinical facts of the individual patient’s progress. The object of the study was to examine whether the content of the instruments captured the clinical information that influenced the decision makers. A successful validation would be evidence from receiver operating characteristics and in regression models that the instruments accounted for most of the outcomes. This proved to be the case. We believe the study demonstrated that simple risk assessments account for only some of the outcome, with treatment completion and recovery measures also contributing. Further, we wanted to examine whether all items in these structured professional judgement instruments were relevant. We found that some instruments that had scale scores associated with conditional discharge had only a few items that were relevant to this outcome, though other items might be relevant to other outcomes such as later reoffending or recall to hospital. Odds ratios also demonstrated that not all items were equally influential, some having larger effects (stronger associations with outcomes) than others.

This study occurred as a result of an experiment of nature. In February 2011, legislation was introduced for the first time in the Republic of Ireland, granting to the Mental Health Review Board the power to grant conditional discharge to patients who were detained in a forensic hospital having been found unfit to stand trial or not guilty by reason of insanity. We were therefore able to test the relevance of the content of structured professional judgement instruments assessing need for therapeutic security, risk assessment, assessment of progress in treatment, recovery, symptoms and global function by assessing these prior to the first hearings under the new legislation. During the period of observation, the DUNDRUM-3 programme completion and the DUNDRUM-4 recovery scales distinguished which patients later received a conditional discharge. The DUNDRUM-1 assessment of need for therapeutic security - a static measure - did not distinguish subsequent conditional discharge apart from a low score on ‘specialist forensic need’, an item describing arson, sadism, paraphilias and lack of cooperation. Low scores on the HCR-20 scales for risk of violence also distinguished which patients received a conditional discharge, as did low scores on the S-RAMM for assessing risk of self-harm and suicide. High (better) scores for the Global Assessment of Function (GAF) had a better AUC than any of the risk assessment scales. Low scores on the PANSS scales for positive, negative and general symptoms also distinguished subsequent conditional discharge.

Co-varying for the HCR-20 score left only the DUNDRUM-3 and DUNDRUM-4 as significantly associated with conditional discharge, and the DUNDRUM-4 was eliminated when its risk item was omitted. However when the HCR-20 scales were co-varied for either the DUNDRUM-3 or the DUNDRUM-4 (omitting its risk item) the HCR-20 was no longer statistically significant for conditionally discharged patients.

It is particularly interesting that the Global Assessment of Function (GAF) emerged as one of the strongest precursors of conditional discharge, and dominated the logistic regression models. The DUNDRUM-3 and DUNDRUM-4 correlated strongly with the GAF (Spearman r for DUNDRUM-3 and GAF, r = −0.804, p < 0.001, DUNDRUM-4 and GAF r = −0.729, p < 0.001) and the DUNDRUM-3 programme completion scale emerged as the next strongest statistical precursor. It is possible that the items of these scales represent elements of what is taken into account when rating the GAF, or it may be that an element in the rating of all of these items, like the GAF, is a rating of some ‘structured intuition’ as recently suggested [27]. The ratings of the seven items of the DUNDRUM-3 programme completion scale and the six items of the DUNDRUM-4 recovery scale have in common that they are based on motivation theory, cycle of change and engagement.

On an item to outcome analysis all items of the DUNDRUM-3 programme completion scale and of the DUNDRUM-4 recovery scale significantly predicted conditional discharge while five of the HCR-20 items, four of the SAPROF items, seven of the START-S items, nine of the START-V items, five of the S-RAMM items and six PANSS items contributed to the statistical relationship with the outcome.

Limitations

The ratings of the instruments and their component items were carried out by researchers who were blind to each others’ ratings and the research ratings were not provided to the clinicians or MHRBs. The clinicians and MHRB were therefore blind to the baseline ratings. The success or failure of the instruments as statistical predictors depends on the extent to which they emulate the ways in which the clinicians and MHRBs make their recommendations and decisions. The clinical information contained in the structured professional judgement instruments was inevitably available in unstructured form to the clinicians and MHRBs who could not be deprived of it. What was studied is the extent to which the structured instruments and their contents at baseline, were associated with the later behaviour of the decision makers. The statistical inference is that the decision makers relied to a greater or lesser extent on this information (in unstructured form). This was an observational study and therefore no special effort could be made to blind clinicians who gave evidence or the members of the MHRB to such unstructured information. Since the purpose of the study was to identify in a defined structured form those clinical facts that influence conditional discharge, it would be self-defeating and probably impossible to blind clinicians and MHRB members to such unstructured information.

This study did not attempt to distinguish between the recommendations of the treating psychiatrists and the decisions of the mental health review board (MHRB) since that would have required greater numbers and access to the deliberations of a judicial body.

A further limitation of this study is that many of the measures are dynamic. Ratings made up to 18 months before a hearing may therefore be attenuated by subsequent change. Significant associations (receiver operating characteristics, odds ratios) are therefore conservative estimates.

This was a study of factors influencing the decision by a mental health review board (MHRB) to conditionally discharge, it was not a study of adverse outcomes following discharge. One of the weaknesses of this study is that a number of patients had been considered ready for conditional discharge for some time prior to this being introduced into Irish legislation. Because these patients had been delayed so long, many of the patients achieving conditional discharge had near perfect scores on the DUNDRUM-3 and DUNDRUM-4 scales. We believe this problem has to an extent been overcome by the period of follow-up, examining conditional discharges after three MHRB hearings rather than just the first hearing under new legislation. In the future patients may be offered a conditional discharge with less perfect scores.

Interpretation

This is a study of the overall process by which clinicians make recommendations and MHRB members decide who to discharge. We have shown that clinicians and decision makers appear to take into account more than risk alone when deciding to release a patient from a secure hospital setting.

A thematic analysis of the items listed in Table 11 suggests that amongst the strongest statistical discriminants of subsequent discharge are items specifically related to societal considerations such as victim sensitivity, psychosocial stress and family and social networks. A second prominent theme concerns rehabilitation and recovery items such as stability, global function, insight, self care, occupation, negative symptoms and mental health. A third theme could be summarised as features of offending such as problem behaviours, dynamic risk, specialist forensic need and early maladjustment . Indicators of good working relationships with clinicians such as responsiveness to treatment, therapeutic rapport and use of leave outside the hospital (trust) also formed a significant theme.

The handbook of the DUNDRUM toolkit [10] emphasises the importance of motivation for change and fulfilment of personal needs, as well as working alliance and interpersonal trust in clinicians [28]. It may be that these ‘higher order’ themes account for the apparent success of the DUNDRUM-3 treatment completion and DUNDRUM-4 recovery scales in predicting which patients clinicians will recommend for conditional discharge and who the members of the MHRB will decide should be discharged.

Generalisability

The DUNDRUM-3 programme completion and the DUNDRUM-4 recovery scales appear to predict moves to the community from the forensic hospital setting. In another naturalistic prospective cohort study in the same population but a year earlier we reported that the DUNDRUM-1 triage security scale and the HCR-20 measure of risk of violence predicted which patients would be moved from more secure to less secure wards within the hospital, and who would be moved back from less secure to more secure wards [14]. The HCR-20 has been shown to predict those who have adverse outcomes on discharge from forensic hospitals [5–7]. Future research will be required to assess whether or not the DUNDRUM-3 programme completion and the DUNDRUM-4 recovery scales can predict those patients who will succeed in the community setting and those patients who will require to be recalled to the forensic hospital setting. It may be that different sets of items within the various structured professional judgement instruments and symptom inventories will predict violent recidivism, relapse or readmission. Those items that appear to be ‘inactive’ in this study may prove more important in the community. A follow up study over a prolonged period will be necessary to examine that question. We believe that a multi-centre, international study may be needed to achieve statistical power for a sufficiently detailed analysis.

The development of structured professional judgement instruments to assist decision making in areas other than risk assessment should be of broad interest to clinicians other than forensic psychiatrists. The decision to discharge any in-patient is among the most professionally challenging in any medical or psychiatric setting, accompanied as it is by risks of subsequent self neglect, relapse, social and environmental hazards, suicide and in rare cases harm to others. The efforts here to describe this decision making process, leading to future validation of those factors other than risk that might be associated with positive or adverse outcomes, is of benefit for a range of applications. Such structured professional judgement may be a way of containing naturally occurring bias in decision making [29]. Medical educators may find this useful as well as those concerned with decision making in mental health tribunals and review boards, the commissioners of services and service managers who plan treatment programmes and aftercare.

Conclusions

The DUNDRUM-3 and DUNDRUM-4 structured professional judgement instruments were designed to compliment risk assessment instruments as a means of guiding the clinicians and decision makers concerning those who are ready for the move from secure forensic hospitals to the community. In this naturalistic observational study of a cohort of patients with measured needs for therapeutic security at medium secure level (using the DUNDRUM-1 triage security measure), the DUNDRUM-3 and DUNDRUM-4 appear to capture information relevant to the decision to discharge, adding significantly to the information gained from risk assessment alone. Whether they also predict those who are not recalled or do not reoffend after conditional discharge remains to be studied.