Validation of the Chinese SAD PERSONS Scale to predict repeated self-harm in emergency attendees in Taiwan

Background Past and repeated self-harm are long-term risks to completed suicide. A brief rating scale to assess repetition risk of self-harm is important for high-risk identification and early interventions in suicide prevention. The study aimed to examine the validity of the Chinese SAD PERSONS Scale (CSPS) and to evaluate its feasibility in clinical settings. Methods One hundred and forty-seven patients with self-harm were recruited from the Emergency Department and assessed at baseline and the sixth month. The controls, 284 people without self-harm from the Family Medicine Department in the same hospital were recruited and assessed concurrently. The psychometric properties of the CSPS were examined using baseline and follow-up measurements that assessed a variety of suicide risk factors. Clinical feasibility and applicability of the CSPS were further evaluated by a group of general nurses who used case vignette approach in CSPS risk assessment in clinical settings. An open-ended question inquiring their opinions of scale adaptation to hospital inpatient assessment for suicide risks were also analyzed using content analysis. Results The CSPS was significantly correlated with other scales measuring depression, hopelessness and suicide ideation. A cut-off point of the scale was at 4/5 in predicting 6-month self-harm repetition with the sensitivity and specificity being 65.4% and 58.1%, respectively. Based on the areas under the Receiver Operating Characteristic curves, the predictive validity of the scale showed a better performance than the other scales. Fifty-four nurses, evaluating the scale using case vignette found it a useful tool to raise the awareness of suicide risk and a considerable tool to be adopted into nursing care. Conclusions The Chinese SAD PERSONS Scale is a brief instrument with acceptable psychometric properties for self-harm prediction. However, cautions should be paid to level of therapeutic relationships during assessment, staff workload and adequate training for wider clinical applications.


Background
Suicide can cause significant years of life lost and bring impacts to service providers and the society as a whole [1]. The challenge of high suicide numbers but relatively few research in Asian countries have called for more attention in prevention strategies in Asia [2]. In this context, predicting suicidal behaviour for appropriate interventions has become an emerging issue for the clinicians [3,4].
Given the limited evidence on suicide risk assessment in predicting risk levels [5][6][7], more research are needed to investigate the validity of risk screening tools.
Past and repeated self-harm or suicide attempts were long-term risks to completed suicide [8][9][10]. Recognizing repetition risks of suicide by non-mental health professionals was an important variable that could improve patient outcome [11]. Such risk identification task can be performed through the nurses and motivated general health workers [12,13]. A simple scale and adequate risk assessment skills for first-line physicians and nurses would provide basic objective indicators of risk level [14] and raise awareness of the clinicians to offer professional support and attention to facilitate help-seeking behaviour among people with self-harm [15]. Though there was limited evidence on the accuracy of screening tools [7], risk screening under guidance of an effective and efficient tool appeared to be a critical and promising method for high-risk identification, follow-ups and further interventions [16,17].
Few established scales of suicide risk assessment were developed and studied [18], with some assessing indirect risks such as suicide ideation [19], depression or other general health indicaters [20]. Though depression scales such as the Patient Health Questionnaire provided acceptable diagnostic properties [21], its predictive capability may not be equal to that of direct risk assessment given reliable responses acquired. Supporting evidence for suicide scale validation was still limited [22]; even more limited in the Chinese populations. Moreover, few studies compared the differences among scales that directly assess suicide-related risks such as suicide intention (e.g. the Pierce Suicide Intention Scale or Beck Suicide Intention Scale), hopelessness (e.g. the Beck Hopelessness Scale) or a relatively comprehensive suicide risk assessment (e.g. the SAD PERSONS Scale) that was originally developed to assess the need to be hospitalized among emergency attendees with self-harm. The aim of the study was to demonstrate the psychometric properties and an adequate cut-off point of the Chinese version of the SAD PERSONS Scale (CSPS) to predict 6-month self-harm repetition and its applicability in general medical settings.

Study settings
The study was performed in northern Taiwan where the latest suicide rate was reported at 14.7 per 100,000 in 2011 [23]. The healthcare and medical context of the study has been previously reported [24]. A general assessment approach of suicide risk factors for consecutive patients with self-harm in the emergency department of Mackay Memorial Hospital (MMH) has been adopted since 2005. The researchers at the MMH suicide prevention center used the Modified SAD PERSONS scale [25] for suicide risk screening and provided service referral and case management interventions. In order to test the applicability and reliability of the scale, the first author further collected a group of general nurses' opinions towards the use of the CSPS in another general hospital, the National Taiwan University Hospital (NTUH), which situated in the same area as another study hospital. In order to assure our study quality, the authors have established standardized procedure for assistant training and made sure of the consistency in data collection to avoid information bias. Moreover, the researchers who performed CSPS assessment were blinded of the baseline CSPS scores in order to derive reliable results in 6-month predictive performance of the CSPS assessment.

Study subjects and procedures
The ethical approval was acquired from the two study hospitals respectively (reference numbers: MMH-I-S-202 and 201107036RB) before participants recruitment. The patients and nurses were interviewed by two trained research assistants and the first author respectively at the two study hospitals after they provided written consent forms. All participants with self-harm (the cases) admitted to the emergency department and patients without selfharm (the controls) from the family medicine department of the MMH provided informed consent. Self-harm was defined as infliction of a harmful act upon oneself in any form, self-poisoning or self-injury, with a non-fatal outcome [26]. Patients with self-harm who admitted to the emergency department were assessed by research assistants irrespective of the presence of suicide idea and based on self-inflicted harming behavior rather than diagnosis; those who denied self-harm were judged if the acts exceeded the tolerance of a person and caused harm. The control group was defined as those with general medical diagnoses without prior self-harm acts and was adopted based on the rationale of establishing the differentiation validity of the CSPS. The overall data quality was ensured by the corresponding author who supervised the entire study procedures in the suicide prevention center. Patients who were unable to complete their first interviews during hospital contact were further arranged an interview during outpatient visit within one week of their admission or at their homes. On the other hand, we randomly sampled 54 general nurses to join interactive group discussions using clustered randomization drawn from Medical, Surgical, Emergency/Intensive Care sectors in the NTUH. The main methodology description of the group intervention was specified elsewhere (under submission) and briefly introduced below. Each group was consisted of 5-10 nurses who joined two similar group sessions discussing case scenario using the CSPS. The first session included instruction of suicide epidemiology and discussion of how and what to assess the case's suicide risk factors, and the second session strengthened the concepts of risk assessment using the same approach. After each group session, a satisfaction survey was carried out to evaluate the nurses' opinions of the group intervention and the applicability and feasibility of using the CSPS. The CSPS applicability and reliability were then analyzed.

The scale applicability and reliability
Using case vignette approach invented by the research team that describes medical inpatients with potential suicide risk factors (see an example in the Appendix), the first author discussed these factors and how they could be assessed by the general nurses in three interactive discussion groups. The pilot results showed that the scale of CSPS was generally accepted by the nurses who acknowledged its application in daily nursing care [15]. In this study we attempted to present the nurses' evaluation towards the use of the underlying CSPS risk assessment in the postintervention satisfaction survey and to analyze the testretest reliability of their ratings at baseline and 3-week reassessment. The reliability was tested given that the correct answers of each CSPS item rating and the nurses' rating performances were blinded to them.

Measurements
The patients were interviewed at baseline by the CSPS and other psychometric scales measuring depression, suicide ideation and hopelessness as shown below. We followed up all the patients using all the scales at the sixth month of baseline interviews and evaluated self-harm repetition and the change in scores. Besides, we collected the nursing participants' evaluations towards the CSPS using a response sheet in the satisfaction survey.

The Chinese version of SAD PERSONS scale (CSPS)
This 10-item mnemonic scale was designed for nonmental health professional use in assessing the overall suicide risk level among emergency attendees. The sensitivity and specificity were 94% and 71% respectively in the original study [25]. The cut-off of over 6 points of score was found to identify the need for hospitalization among emergency patients who self-harm [27,28]. Upon permission acquired from the scale inventor, we performed scale translation into Chinese version and the back translation, which reflected the same meanings of the contents in the original scale based on the researchers' consensus; we then created inquiries for each item for the research assistants to draw responses from the patients. These inquiries were based on the mnemonic contents and were developed into semi-structural format. Each item was scored 2, 1 or 0 with the sum of 14 (see Additional file 1); higher scores indicate higher suicide risk level. Next, we performed a pilot testing of the applicability of CSPS in clinical setting by general nurses and found it a feasible and brief tool for quick assessment towards the inpatients' suicide risk [15]. The rationale of selecting this scale for validation was that it could inform suicide risk level in self-harm patients who seek emergency care [25] or predict repetition of selfpoisoning [29]. It has acceptable face validity and provides an opportunity for initial psychosocial assessment in clinical settings for people with suicide risks.

Pierce suicide intent scale (PSIS)
The PSIS encompasses 12 items that assess four aspects of suicide intent: self-harm context (item 1-4), suicide warning signs (item 5-6), subjective perceptions (item 7-10), and medical seriousness (item [11][12]. It is notable that only those with actual self-harm acts can be rated using this scale rather than those with suicide ideation. Each item was scored 2 or 3 with a sum of 25 points, with higher scores indicating higher level of intent [30]. The internal consistency tested in this study was 0.81.

Hamilton rating scale for depression (HAMD)
This clinician-rated questionnaire was used to measure severity of depressive symptoms. We used the 24-item Chinese version for validation with other psychometric scales in this study [31]. Each question has 3-5 possible answers, with higher scores indicating higher levels of depression. The trained clinician must choose the possible responses to each question through interviews and by observation. It has been demonstrated to have a good interrater and internal reliability as well as satisfying concurrent validity [31]. The internal consistency tested in this study was 0.86.

Beck scale for suicide ideation (BSSI)
This 19-item instrument was developed to measure a person's severity of suicide ideation and his/her plans and wishes to commit suicide [32]. All items were administered via semi-structural interviews by trained clinicians. Each item was rated 0-2, deriving a total score of 0-38 with higher score indicating severer conditions. The BSSI was shown to have good internal consistency with the Cronbach's alpha values over 0.90s in the original study [32] as well as in ours (Cronbach's α = 0.93).

Beck hopelessness scale (BHS)
This 20-item self-report inventory was designed to measure three major aspects of hopelessness: feelings about the future, loss of motivation, and expectations [33]. Each item requires a true (score 1) or false (score 0) response and is scored for indications of pessimism or denial of optimism. The total score ranged between 0-20, with lower scores indicating higher levels of hopelessness. It had good internal reliability and fair concurrent validity with the CSPS in this study. The internal consistency of this scale was found to be 0.82 (positive items) and 0.84 (negative items) in this study.

The satisfaction survey sheet
In a response sheet, we used a 5-point rating scale to measure questions evaluating the nurses' opinions towards the applicability of the CSPS risk assessment in clinical settings. The items in the sheet included, for example, the depth and width of the CSPS and how they perceived the feasibility and applicability of adopting such scale in their units in the hospital. Moreover, we further designed an open-ended question asking about their suggestions towards scale applicability and opinions of each CSPS item (Additional file 1).

Data analysis
The sociodemographic and background information regarding self-harm risk were analyzed using the SPSS 16.0. Among the cases with self-harm, we checked the concurrent validity of the CSPS with PSIS, HAMD, BSSI, BHS and also confirmed the predictive validity with the number of self-harm acts during 6-month follow-up period. We evaluated the associations of continuous variables by Pearson's correlation and compared the means or percentages between groups of self-harm and non-self-harm using Independent T-test or Chi-squared test. In order to determine the optimal cut-off points for the CSPS in predicting the 6-month self-harm repetition, we compared predictive performance of the CSPS with other psychometric scales using areas under the Receiver Operating Characteristic (ROC) curves via the R Project for Statistical Computing. Furthermore, we presented sensitivity, specificity, positive predictive values and negative predictive values to decide a suitable cut-off score for future self-harm risk detection. Besides, two main parts of the nursing data were analyzed, i.e. the test-retest reliability and the nurses' opinions from the satisfaction survey. The first author calculated mean difference of the ratings at two time points, three weeks apart, to evaluate the differences in rating performance and the stability of simulated nursing assessment by applying the CSPS in the case vignette approach. We also calculated the mean scores of the 5-point ratings (0-5 points) of the nurses' opinions towards scale feasibility and applicability in clinical settings. Higher scores indicates that the nurses perceived the scale as more useful and suitable to be adopted in nursing assessment. Moreover, the openended question about their responses of scale adaptation to hospital inpatient assessment for suicide risks and suggestions were analyzed using content analysis.

Results
The patient participants were consisted of 147 people with self-harm from the Emergency Room at baseline and the 6 th month and collected 284 patients without self-harm from the Family Medicine Department in the same hospital. The response rate at 6-month follow-up was 74.8%. Table 1 showed sample characteristics. In this female predominating (72.1%) sample mean-aged 35.6 (standard deviation 13.9) years, the 6-month repetition rate was 21.8% (regardless of self-harm types). Table 2 shows the results of internal consistency of main scales used for assessing the participants' mental health status or suicide risks. The findings indicated that all the scales had good reliability performance with the Cronbach's alpha values ranged between 0.82-0.93. The CSPS was not examined using internal consistency because each item was independent and derived from a variety of suicide risk factors.

Validity and reliability of the CSPS
Among people with self-harm, the total score of the CSPS were significantly correlated with all other psychosomatic scale scores at baseline, with a few scales moderately associated with the CSPS at 6-month follow-up assessment (i.e. all the r values were at 0.40s) ( Table 3). The CSPS had low association with the scores of the PSIS (r = 0.19). In terms of the difference between the correlation coefficients of baseline and 6-month values, we found that both the scores of 6-month HAMD and BHS had stronger associations with the CSPS, while the predictive validity of BSSI remained stable (i.e. the r value was 0.42 at baseline and 0.41 at 6-month follow-up). On the other hand, the 3-week test-retest reliability revealed that the two ratings by 54 general nurses were significantly different (r = 0.12, p = 0.38), indicating its stability and reliability in repeated measurements.

The receiver operating characteristic (ROC) curves
Using the ROC curves to estimate the optimal cut-off point of the CSPS, Table 4 indicated relatively acceptable psychometric values at cut-off 4/5 to predict repetition of selfharm within a 6-month follow-up (i.e. score over 5 and above refers to the high-risk group). With this cut-off value, clinicians could have 65.4% sensitivity and 58.1% specificity to predict 6-month repetition by any self-harm method. From clinical perspectives and based on the research team's consensus, the positive and negative predictive values were also relatively acceptable at this cut-off point.
Moreover, Table 5 showed the performance of the CSPS scores in predicting 6-month self-harm act when comparing to the other four psychometric scales. Judged by the areas under the curves (AUCs) derived from the ROC method, we found that the CSPS had relatively better performance in prediction (AUC = 0.66, p = 0.013, 95% Confidence intervals = 0.54-0.78), followed by the BSSI, HAMD, BHS, and PSIS. We presented the visualized ROC curves in Figure 1. It appeared that the CSPS was a relatively considerable scale to be used for selfharm repetition prediction in this sample.

The nurses' evaluations of clinical applicability of the CSPS
The main results of the nursing intervention were reported elsewhere. In this study we simply presented the findings according to the current study aims. The mean scores of their perceptions of feasibility and applicability were both 4.5 points. They perceived the CSPS as a brief framework and useful guide in nursing assessment for hospital inpatients. They revealed that it is a reminder of suicide risk factors based on which they could provide timely and effective referral after assessment. Many participants recognized its feasibility across different wards and applicability in daily nursing assessment given heavy nursing workloads. Under the challenge of patient characteristics and illness features in various wards, they acknowledged the need to enhance suicide risk awareness of the patients via using this short scale and also disclosed the need for adequate training (see Additional file 1).

Discussion
The study provided evidence in identifying an optimal cut-off value of the CSPS to predict self-harm repetition with an acceptable concurrent and predictive validity. The qualitative findings also highlighted its applicability in general medical settings by non-mental health professionals. The scale was relatively brief and valid as well as adequate for suicide risk assessment in the clinical sample. Given its low internal consistency derived from different *Chi-squared test for categorical variables or independent t-test for continuous variables were performed to derive the statistics of X 2 or t respectively. The cut-off value used to define higher risk group was over 5 points; see Additional file 1 for the contents of each item. Abbreviations: S.D.: standard deviation; CSPS: The Chinese version of SAD PERSONS Scale.
components of risk factors, the ten-item scale appeared to predict a higher risk for self-harm repetition. At cut-off of 5 point and above, the CSPS could reasonably classify a relatively high-risk group for future self-harm.
The findings of the CSPS validation were limited to comparability due to rarity of such studies [7,34]. In a Canadian study, the CSPS was not found to accurately predict future suicide attempts owing to a low sensitivity (40%) and low positive predictive value (PPV) (7.4%) [22]. In applying the 2 point cutoff approach to predict repetition as used in the above study, our results similarly indicated that the CSPS had good classification performance (sensitivity 56.5%, PPV 87.4%) and supported the predictive value of the scale. But we failed to proof its predictive performance by logistic regression analysis suggested by the above study in using separate CSPS items or total score alone to predict 6-month repetition, as we found only the item of Depression could predict future self-harm risk rather than many other items (data shown on request). Although using the CSPS alone might result in considerable false-positives, personnel workload, and extra resources allocated in staff training, it was evident that the scale was comparatively an appropriate option for suicide risk prediction than other scales tested in our study. Besides, the relatively low predictive ability of the PSIS shown in Figure 1 worth attention; it was possibly related to item contents which rated the severity of suicidality from one single self-harm episode that mostly reflects cross-sectional severity rather than a person's consequent suicide risk level. Whereas in comparing to other scales with the CSPS, it also performed significantly better than depression, hopelessness, or suicide ideation measurements in self-harm prediction. Therefore it would be beneficial to build up staff awareness and train the first-line clinicians to use the CSPS for suicide risk assessment. This implication was similar to that of the others which emphasizing the importance of nesting risk screening in routine care by general nurses in general medical settings [35,36]. But according to nurses' opinions, specific training may be needed on how to establish therapeutic relationship and get reliable responses for Item 4,8,10 in the CSPS.
The study was to the best of our knowledge the first that applied a relatively comprehensive method to validate the CSPS in a Chinese population and provided evidencebased data for its validation. We believe that the concerns for low PPV of the CSPS might be significantly improved if risk assessment is combined with another short scale for suicide risk evaluation such as depression. The strategy of augmenting risk prediction ability by using two brief and valid tools concurrently to screen for suicide risk level was also suggested by other study [37]. For example, one may include the CSPS with another brief depression scale such as the BSRS [19,38], the PHQ-2 [39] or other tool specifically for a certain age group such as the elderly (e.g. the Geriatric Depression Scale) to increase precision in predicting the risk level. The CSPS might provide a wider clinical usage in the general medical context for suicide risk identification rather than focusing on people with psychiatric illness, given that the latter group of people had different set of risk factors compared to the general population [34].
The study was among the few that discussed validation of the SAD PERSONS scale and presented both concurrent and predictive validity for relatively objective conclusions.    The strengths included that it explored feasibility and applicability of clinical use in general hospitals and involved service providers' (i.e. the nurses') opinions which complement the study results. Our findings were salient in its mixed data sources with rigorous quality control performed across two major general hospitals. However, the results should be interpreted under several limitations. Firstly, it could be a concern that we tested the scale validity in the emergency department at one hospital but evaluated its applicability based on the nurses' opinions from another hospital. Nevertheless, in the medical context of Taiwan, people are likely to visit doctors between general hospitals due to physician's reputation and affordable medical costs. The risk for subsequent self-harm in the patients from the two hospitals and the source of patients that nurses care for in these two hospitals are regarded as similar due to the approximate characteristics (e.g. mean age and gender ratio) of the patient source to that of a case register study in Taipei [40]. Thus recruitment from the two medical settings would not raise serious problem in interpretation of results. Secondly, the outcome of scale prediction was repeated self-harm. The results derived from such measurement may not be comparable to that of completed suicide. But repetition of self-harm was recognized as an important risk factor more prevalent than suicide ideation and has been adopted in other studies [22,37]. Therefore we regarded this to be an acceptable outcome measurement. Thirdly, we failed to recruit more patients and/or follow up the participants for longer than 6 months due to resource shortage, thus generalization was limited to emergency attendees with self-harm and the exclusion of those with severe physical complications after the index self-harm acts, cognitive dysfunctions, or those who refused to participate at baseline assessment. Given the fact that we failed to collect 6-month data in the control group which may affect the richness of the results and limit the analysis of this group, we focused on establishing the psychometric properties using the case group in this study and had the follow-up period longer than that of other study, e.g. three months [37]. To have a better understanding of patient source of the study groups and the limitation of the hospital controls in this study, refer to our previous publication [41]. Finally, there were factors not considered to be tested in the study but might regarded as repetition risk predictors, such as impulsivity, drug abuse, and life event. However, some of those factors were unforeseen (e.g. life event) and not detrimental to predict 6-month repetition of self-harm [37], so the influence of not including those in the study would be small. Future studies could validate using the CSPS in predicting completed suicide in longitudinal studies or generalizing the results to longer term risk prediction for higher risk of repeated self-harm in a wider clinical populations.

Conclusion
We concluded that the CSPS is a valid and brief instrument suitable for self-harm repetition prediction. It was a considerable instrument for suicide risk screening based on its brevity and feasibility. However, cautions should be paid to level of therapeutic relationships during assessment, staff workload and adequate training for wider clinical applications.