Skip to main content

Developing a clinical decision support system software prototype that assists in the management of patients with self-harm in the emergency department: protocol of the PERMANENS project



Self-harm presents a significant public health challenge. Emergency departments (EDs) are crucial healthcare settings in managing self-harm, but clinician uncertainty in risk assessment may contribute to ineffective care. Clinical Decision Support Systems (CDSSs) show promise in enhancing care processes, but their effective implementation in self-harm management remains unexplored.


PERMANENS comprises a combination of methodologies and study designs aimed at developing a CDSS prototype that assists clinicians in the personalized assessment and management of ED patients presenting with self-harm. Ensemble prediction models will be constructed by applying machine learning techniques on electronic registry data from four sites, i.e., Catalonia (Spain), Ireland, Norway, and Sweden. These models will predict key adverse outcomes including self-harm repetition, suicide, premature death, and lack of post-discharge care. Available registry data include routinely collected electronic health record data, mortality data, and administrative data, and will be harmonized using the OMOP Common Data Model, ensuring consistency in terminologies, vocabularies and coding schemes. A clinical knowledge base of effective suicide prevention interventions will be developed rooted in a systematic review of clinical practice guidelines, including quality assessment of guidelines using the AGREE II tool. The CDSS software prototype will include a backend that integrates the prediction models and the clinical knowledge base to enable accurate patient risk stratification and subsequent intervention allocation. The CDSS frontend will enable personalized risk assessment and will provide tailored treatment plans, following a tiered evidence-based approach. Implementation research will ensure the CDSS’ practical functionality and feasibility, and will include periodic meetings with user-advisory groups, mixed-methods research to identify currently unmet needs in self-harm risk assessment, and small-scale usability testing of the CDSS prototype software.


Through the development of the proposed CDSS software prototype, PERMANENS aims to standardize care, enhance clinician confidence, improve patient satisfaction, and increase treatment compliance. The routine integration of CDSS for self-harm risk assessment within healthcare systems holds significant potential in effectively reducing suicide mortality rates by facilitating personalized and timely delivery of effective interventions on a large scale for individuals at risk of suicide.


Self-harm presents a significant, yet preventable, public health issue, affecting a minimum of 14.6 million individuals globally each year [1]. Individuals with self-harm have elevated risk for repetition of self-harm [2,3,4] and premature death by suicide [2, 3] and other causes [5, 6]. Over 700,000 people die by suicide each year [7], representing an estimated loss of 34.6 million years of life [8]. Furthermore, up to 135 individuals may be affected or bereaved by each suicide [9]. In Europe, suicide rates are notably high (12.8/100,000 in 2019), yet understanding the prevalence and determinants of self-harm and suicidal behaviour in Europe remains a challenge. This is due to the absence of comprehensive surveillance efforts and the considerable differences in definitions and methodologies used across studies and countries [10].

Between 49% and 60% of individuals who die by suicide had visited an emergency department (ED) in the year before their death [11,12,13]. EDs play a pivotal role in the care for people with self-harm [14, 15] as they often represent the first medical contact after self-harm and can offer specialized risk assessment and treatment referral. Several effective interventions for preventing self-harm exist [16]. These include brief interventions such as care coordination, development of safety plans, offering brief follow-up contacts, and targeted therapeutic interventions. When administered during a single encounter with patients at risk of suicide, such as those who have self-harmed and visited the ED, these interventions may not only reduce subsequent suicide attempts but also improve the likelihood of connecting individuals to follow-up care [17]. It is however unclear which interventions are most effective for particular suicide risk profiles, underlining the need for studies to support precision medicine approaches [18,19,20].

A longstanding concern in mental healthcare is the low uptake of proposed treatment by patients that present with self-harm at the ED. According to a recent meta-analysis encompassing 131 distinct studies, roughly two-thirds of these patients receive referrals for either in- or outpatient care, with considerable variability in the factors influencing the allocation of treatment. Importantly, only about one in five effectively receive the recommended treatment [21] and non-attendance among those referred is a strong predictor for subsequent death by suicide [22].

One potential explanation for this low treatment uptake may lie in the varying quality of care provided at the ED to individuals who self-harm. Quality of care as it is perceived by patients ranges from genuine and empathetic healthcare interactions that empower patients to explore reasons for their distress, leading to more effective care, to superficial, disconnected contacts. The latter can include rushed, formulaic risk assessments, a tendency to minimize patients’ distress, and perceived lack of trust, all of which impair willingness to engage in and comply with treatment [23,24,25,26,27,28]. Moreover, there is often a lack of involving and educating informal caregivers when treating people with self-harm [29, 30]. Crucially, unsupportive care has shown to be significantly associated with repeat self-harm [31]. Related to this, so-called frequent self-harm repeaters have been identified, i.e., small proportions of patients with self-injury (< 1%) that are frequent users of (emergency) services but with low adherence and response to mental health treatment [32] which contribute to the high societal costs related to self-harm.

Sub-optimal quality of care for individuals who self-harm could stem, in part, from healthcare providers experiencing feelings of insecurity or uncertainty, often related to stigma and lack of specialist training. Risk detection and intervention efforts at the ED are complicated because self-harm represent difficult-to-predict complex behaviour [20, 33, 34]. Unassisted clinician assessments are insufficient to accurately identify patients at highest risk for repeated self-harm and suicide [35, 36]. Clinicians are, therefore, prone to heuristic-based decision making (i.e., using simplified mental shortcuts or rules of thumb) by linking a limited (and often arbitrary) set of risk factors directly to suicide potential [37]. Addressing the self-perceived lack of training among ED clinicians [38,39,40] may mitigate this problem. However, even among trained mental healthcare professionals, there is high variability in clinical judgement regarding suicide risk assessment [41, 42]. Psychiatrists report that suicide risk assessments are often based on semi-intuition or gut feeling, leading to concerns of being unprofessional and to feelings of loneliness and insecurity [43]. This, in turn, may lead to ineffective clinical decision-making, poor patient experience, and adverse outcomes such as involuntary admissions based on uncertainty and preventable deaths by suicide. Relying on the use of standardized risk assessment scales does not solve this issue as these scales have proven to be inaccurate [44] and may lead to the false impression that risk management is based on objective evidence [30, 43].

Clinical Decision Support Systems (CDSS) have shown the potential to enhance care processes in diverse clinical settings by boosting the precision of risk assessments and refining evidence-based treatment allocation based on risk stratification [45, 46]. Increasingly, CDSSs are being developed with the capability to conduct assessments based on information otherwise unobtainable by humans, using machine learning-based prediction models that allow for considering the complex interactions over time between a high number of patient characteristics, derived from large sets of electronic healthcare registry data. In suicide research, it has been shown that machine learning-based suicide risk predictions outperform all widely researched theories of suicide [47], opening the possibility to stratify patients with self-harm according to risk for adverse outcomes, such as death by suicide, and provide tailored intervention preventions. Despite their clear usefulness in other clinical domains [48,49,50,51], evidence of effective implementation of these techniques into clinically useful prediction tools for self-harm and related adverse outcomes is lacking. This may be due to the absence of a user-oriented personalised approach, i.e., the failure to actively involve both patients and clinicians in the development of this kind of software tools [52]. Qualitative studies reveal healthcare providers’ interest in machine learning–based risk prediction systems, highlighting concerns like liability, alert fatigue, and increased healthcare system demand [53]. In addition, more research is needed on how to integrate CDSS tools to improve suicide prevention training frameworks that take into account both risk assessment and recovery-oriented approaches [54].

Summary of relevant previous studies

While statistical risk prediction models have shown some promise in improving suicide risk prediction accuracy [55,56,57,58] and cost-effectiveness [59], their application in clinical settings remains limited. The U.S. Veterans’ Health Administration utilized a statistical model in the REACH VET program to identify at-risk veteran patients and link them with care [60, 61]. Jaspr Health introduced a tablet-based digital assistant, featuring an AI-driven chatbot aiding ED patients with suicide risk and establishing evidence-based discharge plans, proving both feasible and acceptable [62]. e-Connect is a digital CDSS that improves suicide risk identification and referral among youth in the justice system by probation officers [63]. Finally, the OxRisk project [64] developed tools that provide suicide risk probability scores for individuals presenting with self-harm (OxSATS, [65]) and severe mental illness (OxMIS, [66]). To the best of our knowledge, no prior study has developed a CDSS specifically designed to facilitate personalized assessment and management for unselected populations of patients presenting with self-harm at the emergency department, including risk estimation for various relevant adverse outcomes, such as self-harm repetition and death by suicide. Additionally, prior research has not integrated implementation research to ensure the CDSS’s patient focus, practical value, and usability in real-world healthcare settings.


Here, we outline the research protocol of the PERMANENS project ( PERMANENS is a European research project that brings together expertise in clinical mental health research, public health, biostatistics, and biomedical informatics to develop a CDSS software prototype that can assist clinicians in the personalized detection, assessment and management of risk for key adverse clinical outcomes among ED patients presenting with self-harm. The key adverse clinical outcomes that we aim to tackle in this project include repeat self-harm, self-harm method lethality escalation, death by suicide, premature death, and not receiving mental health treatment following discharge from ED. Trained on evidence accumulated in clinical settings and based on the patient’s particular clinical history, the CDSS will provide the clinician with personalized risk profiles for these adverse outcomes, and propose an evidence-based treatment plan, tailored to the patient’s specific risk profile.

The proposed CDSS will introduce a structured professional judgement approach [67] for self-harm management at the ED– a systematic approach for clinicians to evaluate risk factors, consider historical data, and weigh various elements in the decision making process in a comprehensive way. Importantly, this implies that the proposed CDSS will not be a mere actuarial or statistical instrument, but instead focuses on guiding the clinician-patient interaction to ensure that all relevant areas of clinical risk are evaluated and that adequate treatment is being offered in a highly personalised way. This approach is expected to improve standardization of care as well as clinicians’ confidence in their judgement by guiding the creation of a clinical formulation and a risk management plan. This approach will also enable clinicians to dedicate mental resources to conducting more empathetic and collaborative assessments, acknowledging specific patients’ needs and values, hereby further improving patient satisfaction and treatment adherence.


Study design

The PERMANENS project consists of a combination of methodologies and study designs oriented to the development of a CDSS software prototype, as briefly summarized in Fig. 1. In summary, the CDSS software backend (i.e., the part of the software that allows it to operate but cannot be accessed by the user) will consist of two main components: (1) a series of prediction models for key adverse clinical outcomes, obtained by machine learning-based techniques from transnationally harmonized health record data; and (2) a computerized clinical knowledge base on effective suicide prevention interventions, obtained through a systematic review of the literature, including clinical guidelines. The proposed CDSS software frontend (i.e., the part that interacts with the end-user) will exploit the backend resources to facilitate personalized risk assessment and the delivery of a personalized treatment plan, tailored to the patient’s specific risk profile and needs. User-oriented implementation research will consist of periodic user-advisory group meetings throughout the project, mixed-methods research on currently unmet needs in self-harm management, and small-scale usability testing of the CDSS prototype. Below we describe each of the research activities in more detail.

Fig. 1
figure 1

Overview of PERMANENS CDSS software prototype development and related research activities

Collection of transnationally harmonized electronic registry data

The data used for the development of prediction models for key clinical adverse outcomes consists of individual-level quantitative structured data from existing electronic registries from four separate sites, i.e., three countries (Ireland, Norway, Sweden) and one region (Catalonia, Spain; see Table 1). Types of registry data available at all sites include routinely collected electronic health record (EHR) data, mortality data, and administrative data. Routinely collected EHR data is coded using the International Classification of Diseases-9th revision-Clinical Modification (ICD-9-CM) and ICD-10-CM disease classification systems. This data will be used to identify ED visits of patients presenting with self-harm, including information on repeat self-harm, method escalation (i.e., transitioning to the use of more lethal self-harm methods), and the patients’ clinical history, encompassing mental and substance use disorders, and pertinent somatic conditions. Mortality data is coded using the International Classification of Disease 10 (ICD-10) coding system and will be used to extract information on death by suicide and premature mortality. Administrative data will be used to extract information on sex, age, and socio-economic status. On some sites, data will be complemented with pharmaceutical registry data (Catalonia, Spain, and in Sweden) and/or with specific self-harm case registry data (i.e., the National Self-Harm Registry in Ireland [68]; the Suicide Risk Code Registry in Catalonia, Spain [69]).

Table 1 Overview of electronic registry data used in the PERMANENS project

The Observational Medical Outcomes Partnership (OMOP) Common Data Model (CDM) [70] developed by the Observational Health Data Sciences and Informatics (OHDSI) [71] will be used in order to transform and harmonise all registry data from the four sites into a common data format using standard terminologies, vocabularies and coding schemes.

Machine learning-based prediction models for key adverse clinical outcomes

Next, to facilitate analysis of registry data across sites, we will use a federated data analysis methodology [72, 73]. This methodology involves within-country analyses of individual-level data, producing local aggregated results, saved as query objects (i.e., analysis result that has been transformed or aggregated without any individual-level data remaining, e.g., predictive models or descriptive tables). These results are then exported and centralized for further aggregated analysis at a transnational level, eliminating the need for centralized data storage or cross-border access to local individual-level data. This approach ensures compliance with national and European personal data privacy regulations.

In order to prepare the data for subsequent predictive modelling, we will delineate a patient cohort, operationalize the key adverse clinical outcomes, define the predictor variables for these outcomes, and identify clinically relevant subgroups of self-harm patients. The cohort will consist of patients presenting with self-harm at the ED, and index visits will consist of the first ED visit for self-harm during a predefined period of time. Key adverse clinical outcomes following index ED visit will include repeat self-harm (i.e., having subsequent ED visits for self-harm after the index visit), self-harm method escalation, death by suicide, premature death (e.g., death by any cause before age 70), and not receiving any post-discharge treatment (i.e., not having any inpatient or outpatient psychiatric care registered in the period following ED discharge). Building on previous evidence [74, 75], data-driven diagnostic algorithms will be developed to facilitate accurate identification of self-harm incidents in the registry data.

Predictor variables (features) for the key adverse clinical outcomes will reflect patients’ available clinical information prior to index ED visit, including self-harm, mental and substance use disorders, somatic conditions, and receipt of inpatient or outpatient mental healthcare. Composite predictor variables reflecting mental and/or somatic comorbidity will also be created, as well as separate predictor variables reflecting different retrospective time periods with respect to the index visit (e.g., past 30, 90, 180, 365 days). Clinically relevant subgroups of self-harm patients will be identified (e.g., based on gender, age, socio-economic status, mental disorder category) and the convenience of developing separate prediction models for them will be evaluated. These subgroups will be identified based on literature review, expert opinion as well as data analysis, ranging from simple descriptive analysis to unsupervised machine learning approaches.

Next, we will develop and validate a series of clinically interpretable prediction models which will enable the accurate stratification of patients according to risk for the key adverse clinical outcomes. We will develop machine learning-based models, including ensembles of decision trees (random forests and gradient boosting methods) and deep learning models if the size and depth of the data justify its use. To address class imbalance, pseudo-sampling of cases and under-sampling of controls will be considered. To enable clinical interpretability of prediction models [76], a set of most important predictors will be identified for each model (e.g., using SHapley Additive exPlanations [SHAP] metrics) in terms of their contribution to the overall prediction accuracy. Also, clinical interpretability will be one of the criteria (alongside accuracy metrics such as sensitivity and specificity) to choose the final predictive model to be deployed.

Prediction models will first be developed separately at the local (i.e., within-country) level. These local models will be stored as query objects to allow them to be shared across sites without the need of remotely accessing the original individual-level data, in compliance with national and European data security and privacy regulations. These models will be used to build an ensemble model, which can be exploited for: (1) externally cross-validating each of the models in each of the other countries’ data, substantially increasing the models’ robustness; (2) investigating key differences across countries with respect to the importance of the clinical variables; (3) identifying predictors that are in need for assessment in specific countries; and (4) pooling together the predictive power of each predictor variable across countries using meta-analytic techniques to obtain estimates of prediction accuracy of the global model on the conjoint set of transnational data.

Potential prediction bias in the prediction models with regard to relevant variables (e.g., sex, age group, socio-economic status) will be investigated and eliminated by systematically including these variables as fixed covariates throughout model development as well as by developing separate models stratified by different values of these variables, whenever it will be considered necessary.

Clinical knowledge base on effective suicide prevention interventions

A systematic review will be conducted to develop a knowledge base for evidence-based assessment and treatment options for individuals with self-harm or suicidal behaviour. This will consist of a systematic literature search of available clinical practice guidelines (CPGs) for patients aged 18 or older presenting in any healthcare setting. Inclusion of CPGs will not restricted by language and will be translated to English prior to full-text review. CPGs will be identified from Ovid MEDLINE, Ovid MEDLINE In-Process & Other Non-Indexed Citations, EMBASE, Web of Science, PsycINFO, and the Cochrane Library. Gray literature will be searched using Guidelines International Network (GIN), Trip Medical Database, and the National Institute for Health and Care Excellence in the United Kingdom (NICE). The search will be complemented by sourcing guidelines documents from researchers in the International Academy of Suicide Research. The AGREE II tool [77,78,79] will be used to assess the quality of included CPGs, considering the following domains: Scope and purpose, Stakeholder involvement, Rigour of development, Clarity of presentation, Applicability, and Editorial independence. Findings from the systematic review will include quality of the reviewed CPGs, recommendation matrices for outcomes included in high-quality CPGs, the levels of evidence underpinning recommendations, comparative treatment preferences, and emerging themes in the material. The systematic review protocol is registered with PROSPERO (CRD42023488333).

CDSS prototype software development

Development of the software backend (see Fig. 1) will consist of implementing the machine learning-based prediction models and the computerized clinical knowledge base on effective suicide prevention interventions. Using this system, and based on the data of an individual patient, the system will assign a predicted risk score for the key adverse clinical outcomes and provide suggestions for treatment interventions. The software will work as a web-service, exposing a RESTful application programming interface (API) through which it will interchange input and output data with the front-end.

Development of the software frontend will consist of designing a graphical user interface (GUI) that will ensure intuitive and user-friendly human-system interactions. The CDSS will enable for clinicians to apply a tiered evidence-based approach in the clinical management of self-harm at the ED. The personalised CDSS main outputs will consist of: (1) risk scores (0-100%) and visualization, including the risk of key adverse clinical outcomes, and specifying the most important predictors; (2) recommendations of risk factors in need for assessment to further optimise the ongoing risk evaluation; and (3) a personalised treatment plan aimed at delivery of effective prevention interventions and at ensuring continuity of care. It is important to stress that final assignment of treatment will not be entirely data-driven (i.e., rely on predicted risk scores) but will also be based on expert opinion on which treatment is most indicated considering relevant clinical variables such as age, gender, and mental disorder diagnosis. Risk for intrusive interventions provoked by false-positive model predictions will be prevented by the fact that the final decision regarding treatment and individual rights restriction always lies with the end-user (clinician and/or patient) and never the CDSS. Nevertheless, intensity of the proposed interventions will always depend on the level of certainty of the prediction, and proportional to the risk detected.

We will also develop an format for a transferable personal healthcare record, co-created with patients, and fully acknowledging them as end-users and managers of their data. Such healthcare record format could consist of (but is not limited to) a comprehensive summary of a patient’s risk factors, past healthcare use trajectories, and individualised treatment plans. The final format of the personal healthcare record will consist of a digital portable file (outputted by the CDSS software) that the patient could share with other involved healthcare providers after the risk assessment. The final goal of the transferable personal healthcare record is to improve continuity of care, by enabling more effective communication between patients’ different healthcare providers.

Evaluation of software development will consist of both technical validation (no bugs, adherence to specifications) and functional validation (verifying whether the software meets all users’ real needs). Importantly, we aim for the personalised CDSS to be used as a stand-alone application, not depending on the connection with electronic healthcare systems for its functioning. This means that the input data can consist entirely of user-entered data, i.e., clinicians providing all necessary input data manually through the software GUI. Although this will add user burden at this stage of CDSS development, this is necessary to provide a feasible and flexible software that can also be tested and fine-tuned in healthcare settings with poor system integration [46]. To lower user burden in terms of data that needs to be entered manually, we will test, in a pilot study, the use of prediction models based on sets of predictors with highest predictive accuracy (and minimal loss of overall model prediction accuracy), and explore the implementation of multistage assessments, first prioritizing models with high sensitivity, and subsequently using models with optimum sensitivity and specificity trade-off. Evidently, full integration of the personalised CDSS into electronic healthcare record systems will be desirable as a future step to optimise the integration of the CDSS into the clinical workflow.

Implementation research with end-users

We will organize periodic meetings with user-advisory groups (UAG) to obtain the necessary knowledge and insights throughout the project, and to ensure commitment from local stakeholders for adoption and future implementation of the CDSS. UAG members will consist of people with personal lived experience of ED visits for self-harm and their caregivers, members of advocacy or patient groups representing persons with lived experience, and ED mental healthcare professionals. At least two meetings will be held yearly at each site, including 4–8 participants in each UAG. To standardize procedures across the four sites, meetings will be held in parallel around the same time of year and using a common meeting agenda. The meeting format will be online group meeting, but the option of in-person personal interviews will be actively offered to people with lived experience, in case they prefer this format due to perceived privacy issues. Moderators for the UAG will consist of research team members with sufficient experience and/or training to handle the sensitive nature of the discussed topics. Recruitment for the UAG will be by personal invitation and/or snowball sampling. Participant information leaflets will be developed for that purpose. All participants will be aged 18 or more. The end-deliverable format of the UAG meetings will be a detailed summary report, including a synthesis of findings and results across the four sites.

Using a mixed-methods research design, we will investigate currently unmet needs in self-harm risk assessment at the ED to ensure that the proposed CDSS’ objectives and methodology maximally address users’ needs. Mixed-methods research will consist of focus groups and web-based surveys. Focus groups will be held at the Ireland and Spain sites only. Recruitment methods for the focus groups, type of participants, moderators, and meeting format will be identical as in the UAG meetings. A predefined list of relevant topics (topic guide) will serve as a guide for the focus groups. A gender- and age-tailored recruitment approach will be used to guarantee representative participation. All participants will be aged 18 or more. Audio-recorded group sessions will be transcribed and subjected to thematic and content analysis. Short web-based surveys implemented using Qualtrics will be used to assess currently unmet needs in suicide risk assessment among ED staff members, Clinical Nurse Specialists, Psychiatric Nurses, Crisis Nurses, Psychiatry residents and registrars, and Clinical Psychologists working in the hospital setting. Snowball sampling methods will be applied for the dissemination of the survey which will be conducted via relevant national networks and professional organizations.

We will conduct a small-scale pilot study for usability testing and validation of the CDSS prototype software with people with personal lived experience of ED visits for self-harm and with ED mental healthcare professionals. Clinician-patient dyads will be established for this purpose, in safe clinical settings (e.g., the emergency or psychiatry department) but outside of routine clinical healthcare. Sample size will be determined through the principle of saturation. This will facilitate an iterative evaluation process that will be conducted using an established methodological framework, providing feedback loops to ongoing tasks in software development. Usability, feasibility, and acceptability of the CDSS will be assessed using well-validated quantitative instruments (e.g., the System Usability Scale) and through qualitative interviews, which will be based on a thorough revision of the literature. Audio-recorded interviews will be transcribed and subjected to thematic and content analysis. By consulting regional and national regulatory authorities, we will start exploring the future feasibility of implementing the CDSS on a larger scale for further clinical validation and implementation.


In this paper, we have presented the protocol of the PERMANENS project, a comprehensive European research initiative focused on the development of a Clinical Decision Support System (CDSS) software prototype. The objective of this prototype is to aid clinicians in the personalized detection, assessment, and management of critical adverse clinical outcomes in emergency department patients presenting with self-harm.

The decision to design a project aimed at developing a CDSS that improves self-harm management at the ED was made without direct patient consultation. However, patient involvement will be maximized throughout all PERMANENS project phases, starting from the outset and continuing across all outlined research activities. Patient involvement will be guaranteed through the implementation research with CDSS end-users to ensure that: (1) the research and in particular the CDSS is influenced by principals of citizenship, accountability, and transparency; (2) to ensure that research is acceptable, accessible, sensitive, and representative of the perspectives of people with lived experience and those involved in providing (mental) health services; (3) to facilitate open and collaborative dialogue between people with lived experience and the research team throughout the development of the CDSS; and (4) to complement the team’s knowledge regarding topics relating to suicide and self-harm.

Dissemination of the PERMANENS project will be conducted through the project’s website (; through open-access scientific publications, conferences, workshops and webinars. We will also develop communication strategies (co-created) towards the general public and patients with the focus on creating awareness of the need for a personalised medicine approach in suicide risk assessment. Moreover, we will provide public health policy makers with suggestions of revision in suicide prevention frameworks; and produce an updated clinical guideline for suicide risk assessment, including an online training manual for the clinical use of the CDSS.

Noteworthy aspects of the PERMANENS project include the creation of machine learning-based ensemble prediction models utilizing registry data from diverse European countries, the inclusion of a range of clinically relevant adverse outcomes within the prediction models, and the inclusion of implementation research, actively involving individuals with lived experience and healthcare professionals. This engagement ensures the tailoring of the CDSS software prototype to meet clinical needs, emphasizing its patient focus, practical value, and usability in real-world healthcare settings. Routine implementation of CDSS for self-harm risk assessment in healthcare systems has high potential in effectively reducing suicide mortality in the population, by enabling personalised and timely delivery of effective treatment at large scale among individuals with suicide risk.

Data availability

No datasets were generated or analysed during the current study.



Emergency Department


Clinical Practice Guidelines


International Classification of Diseases-9th revision-Clinical Modification


International Classification of Diseases-10th revision-Clinical Modification


Observational Health Data Sciences and Informatics


Observational Medical Outcomes Partnership


Clinical Decision Support Systems


Application Programming Interface


Graphical User Interface


User-Advisory Groups


Centre for Reviews and Dissemination


International Prospective Register of Systematic Reviews


Representational State Transfer


SHapley Additive exPlanations


Guidelines International Network


National Institute for Health and Care Excellence


  1. Global Burden of Disease Collaborative Network. Global burden of disease study 2019 (GBD 2019) results. Seattle U S. Published Online First: 2019.

  2. Ribeiro JD, Franklin JC, Fox KR, et al. Self-injurious thoughts and behaviors as risk factors for future suicide ideation, attempts, and death: a meta-analysis of longitudinal studies. Psychol Med. 2016;46:225–36.

    Article  CAS  PubMed  Google Scholar 

  3. Liu B-P, Lunde KB, Jia C-X, et al. The short-term rate of non-fatal and fatal repetition of deliberate self-harm: a systematic review and meta-analysis of longitudinal studies. J Affect Disord. 2020;273:597–603.

    Article  PubMed  Google Scholar 

  4. Wang M, Swaraj S, Chung D, et al. Meta-analysis of suicide rates among people discharged from non‐psychiatric settings after presentation with suicidal thoughts or behaviours. Acta Psychiatr Scand. 2019;139:472–83.

    Article  CAS  PubMed  Google Scholar 

  5. Bergen H, Hawton K, Waters K, et al. Premature death after self-harm: a multicentre cohort study. Lancet. 2012;380:1568–74.

    Article  PubMed  Google Scholar 

  6. Carr MJ, Ashcroft DM, Kontopantelis E, et al. Premature death among primary care patients with a history of self-harm. Ann Fam Med. 2017;15:246–54.

    Article  PubMed  PubMed Central  Google Scholar 

  7. World Health Organisation (WHO). Suicide: Key Facts. Suicide. 2019. (accessed 16 November 2023).

  8. Naghavi M. Global burden of Disease Self-Harm collaborators. Global, regional, and national burden of suicide mortality 1990 to 2016: systematic analysis for the global burden of Disease Study 2016. BMJ. 2019;364:l94.

    Article  PubMed  PubMed Central  Google Scholar 

  9. Cerel J, Brown MM, Maple M, et al. How many people are exposed to suicide? Not six. Suicide Life Threat Behav. 2019;49:529–34.

    Article  PubMed  Google Scholar 

  10. Jakobsen SG, Nielsen T, Larsen CP, et al. Definitions and incidence rates of self-harm and suicide attempts in Europe: a scoping review. J Psychiatr Res. 2023;164:28–36.

    Article  PubMed  Google Scholar 

  11. Vasiliadis H-M, Ngamini-Ngui A, Lesage A. Factors Associated with suicide in the Month following contact with Different Types of Health Services in Quebec. Psychiatr Serv. 2015;66:121–6.

    Article  PubMed  Google Scholar 

  12. Morrison KB, Laing L. Adults’ use of health services in the year before death by suicide in Alberta. Health Rep. 2011;22:15–22.

    PubMed  Google Scholar 

  13. Ahmedani BK, Simon GE, Stewart C, et al. Health care contacts in the year before suicide death. J Gen Intern Med. 2014;29:870–7.

    Article  PubMed  PubMed Central  Google Scholar 

  14. Asarnow JR, Babeva K, Horstmann E. The Emergency Department: challenges and opportunities for suicide Prevention. Child Adolesc Psychiatr Clin N Am. 2017;26:771–83.

    Article  PubMed  PubMed Central  Google Scholar 

  15. Larkin GL, Beautrais AL. Emergency departments are underutilized sites for suicide prevention. Crisis. 2010;31:1–6.

    Article  PubMed  Google Scholar 

  16. Mann JJ, Michel CA, Auerbach RP. Improving suicide Prevention through evidence-based strategies: a systematic review. Am J Psychiatry. 2021;178:611–24.

    Article  PubMed  PubMed Central  Google Scholar 

  17. Doupnik SK, Rudd B, Schmutte T, et al. Association of Suicide Prevention Interventions with subsequent suicide attempts, linkage to follow-up care, and Depression symptoms for Acute Care settings: a systematic review and Meta-analysis. JAMA Psychiatry. 2020;77:1021–30.

    Article  PubMed  Google Scholar 

  18. O’Connor RC, Worthman CM, Abanga M, et al. Gone too soon: priorities for action to prevent premature mortality associated with mental illness and mental distress. Lancet Psychiatry. 2023;10:452–64.

    Article  PubMed  Google Scholar 

  19. Barrigon ML, Courtet P, Oquendo M, et al. Precision Medicine and suicide: an opportunity for Digital Health. Curr Psychiatry Rep. 2019;21:131.

    Article  PubMed  Google Scholar 

  20. Kessler RC, Bernecker SL, Bossarte RM et al. The Role of Big Data Analytics in Predicting Suicide. In: Passos IC, Mwangi B, Kapczinski F, eds. Personalized Psychiatry: Big Data Analytics in Mental Health. Cham: Springer International Publishing 2019:77–98.

  21. Witt K, McGill K, Leckning B, et al. Global prevalence of psychiatric in- and out-patient treatment following hospital-presenting self-harm: a systematic review and meta-analysis. EClinicalMedicine. 2023;65:102295.

    Article  PubMed  PubMed Central  Google Scholar 

  22. Qin P, Stanley B, Melle I, et al. Association of Psychiatric Services Referral and Attendance Following Treatment for Deliberate self-harm with prospective mortality in Norwegian patients. JAMA Psychiatry. 2022;79:651.

    Article  PubMed  PubMed Central  Google Scholar 

  23. Uddin T, Pitman A, Benson G et al. Attitudes toward and experiences of clinical and non-clinical services among individuals who self-harm or attempt suicide: a systematic review. Psychol Med. 2023;1–19.

  24. MacDonald S, Sampson C, Turley R, et al. Patients’ experiences of Emergency Hospital Care following Self-Harm: systematic review and thematic synthesis of qualitative research. Qual Health Res. 2020;30:471–85.

    Article  PubMed  Google Scholar 

  25. Lindgren B-M, Svedin CG, Werkö S. A systematic literature review of experiences of Professional Care and Support among people who Self-Harm. Arch Suicide Res off J Int Acad Suicide Res. 2018;22:173–92.

    Article  Google Scholar 

  26. Swedish Council on Health Technology Assessment. Self-harm: Patients’ Experiences and Perceptions of Professional Care and Support. Stockholm: Swedish Council on Health Technology Assessment (SBU) 2015. (accessed 17 November 2023).

  27. Rees N, Rapport F, Thomas G, et al. Perceptions of paramedic and emergency care workers of those who self harm: a systematic review of the quantitative literature. J Psychosom Res. 2014;77:449–56.

    Article  PubMed  Google Scholar 

  28. Taylor TL, Hawton K, Fortune S, et al. Attitudes towards clinical services among people who self-harm: systematic review. Br J Psychiatry J Ment Sci. 2009;194:104–10.

    Article  Google Scholar 

  29. Lavers G, Andriessen K, Krysinska K. A systematic review of the experiences and Support needs of Informal caregivers for people who have attempted suicide or experienced suicidal ideation. Int J Environ Res Public Health. 2022;19:5181.

    Article  PubMed  PubMed Central  Google Scholar 

  30. Graney J, Hunt IM, Quinlivan L, et al. Suicide risk assessment in UK mental health services: a national mixed-methods study. Lancet Psychiatry. 2020;7:1046–53.

    Article  PubMed  Google Scholar 

  31. Cully G, Leahy D, Shiely F, et al. Patients’ experiences of Engagement with Healthcare Services following a high-risk self-harm presentation to a Hospital Emergency Department: a mixed methods study. Arch Suicide Res off J Int Acad Suicide Res. 2022;26:91–111.

    Article  Google Scholar 

  32. Ness J, Hawton K, Bergen H, et al. High-volume repeaters of self-harm. Crisis. 2016;37:427–37.

    Article  PubMed  Google Scholar 

  33. Fazel S, Runeson B, Suicide. N Engl J Med. 2020;382:266–74.

    Article  PubMed  PubMed Central  Google Scholar 

  34. Franklin JC, Ribeiro JD, Fox KR, et al. Risk factors for suicidal thoughts and behaviors: a meta-analysis of 50 years of research. Psychol Bull. 2017;143:187–232.

    Article  PubMed  Google Scholar 

  35. Woodford R, Spittal MJ, Milner A, et al. Accuracy of clinician predictions of future Self-Harm: a systematic review and Meta-analysis of Predictive studies. Suicide Life Threat Behav. 2019;49:23–40.

    Article  PubMed  Google Scholar 

  36. Corke M, Mullin K, Angel-Scott H, et al. Meta-analysis of the strength of exploratory suicide prediction models; from clinicians to computers. BJPsych Open. 2021;7:e26.

    Article  PubMed  PubMed Central  Google Scholar 

  37. Gabbay J, le May A. Evidence based guidelines or collectively constructed ‘mindlines?’ Ethnographic study of knowledge management in primary care. BMJ. 2004;329:1013.

    Article  PubMed  PubMed Central  Google Scholar 

  38. Roy W, Roaten K, Downs D, et al. Suicide risk Assessment and Management: real-world experience and perceptions of Emergency Medicine Physicians. Arch Suicide Res. 2017;21:365–78.

    Article  PubMed  Google Scholar 

  39. Derblom K, Lindgren B-M, Johansson A, et al. Engagement, responsibility, collaboration, and abandonment: nurses’ experiences of assessing suicide risk in Psychiatric Inpatient Care. Issues Ment Health Nurs. 2021;42:776–83.

    Article  PubMed  Google Scholar 

  40. McClatchey K, Murray J, Chouliara Z, et al. Suicide risk assessment in the emergency department: an investigation of current practice in Scotland. Int J Clin Pract. 2019;73:e13342.

    Article  PubMed  Google Scholar 

  41. Sequeira L, Strudwick G, De Luca V, et al. Exploring uniformity of Clinical Judgment: a Vignette Approach to understanding Healthcare professionals’ suicide risk Assessment practices. J Patient Saf. 2022;18:e962–70.

    Article  PubMed  Google Scholar 

  42. Chacko M, Job A, Kim DJ, et al. Examination of physician factors influencing psychiatric assessment of acutely suicidal patients. Psychiatry Res. 2021;297:113736.

    Article  PubMed  Google Scholar 

  43. Waern M, Kaiser N, Renberg ES. Psychiatrists’ experiences of suicide assessment. BMC Psychiatry. 2016;16:440.

    Article  PubMed  PubMed Central  Google Scholar 

  44. Carter G, Milner A, McGill K, et al. Predicting suicidal behaviours using clinical instruments: systematic review and meta-analysis of positive predictive values for risk scales. Br J Psychiatry J Ment Sci. 2017;210:387–95.

    Article  Google Scholar 

  45. Kwan JL, Lo L, Ferguson J, et al. Computerised clinical decision support systems and absolute improvements in care: meta-analysis of controlled clinical trials. BMJ. 2020;370:m3216.

    Article  PubMed  PubMed Central  Google Scholar 

  46. Sutton RT, Pincock D, Baumgart DC, et al. An overview of clinical decision support systems: benefits, risks, and strategies for success. Npj Digit Med. 2020;3:1–10.

    Article  Google Scholar 

  47. Schafer KM, Kennedy G, Gallyer A, et al. A direct comparison of theory-driven and machine learning prediction of suicide: a meta-analysis. PLoS ONE. 2021;16:e0249833.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  48. Demuth S, Müller J, Quenardelle V, et al. Strokecopilot: a literature-based clinical decision support system for acute ischemic stroke treatment. J Neurol. 2023;270:6113–23.

    Article  PubMed  Google Scholar 

  49. Shi X, He J, Lin M, et al. Comparative effectiveness of Team-Based Care with and without a clinical decision support system for Diabetes Management: a Cluster Randomized Trial. Ann Intern Med. 2023;176:49–58.

    Article  PubMed  Google Scholar 

  50. Chekin N, Ayatollahi H, Karimi Zarchi M. A clinical decision support system for assessing the risk of Cervical Cancer: development and evaluation study. JMIR Med Inf. 2022;10:e34753.

    Article  Google Scholar 

  51. Anchala R, Kaptoge S, Pant H, et al. Evaluation of effectiveness and cost-effectiveness of a clinical decision support system in managing hypertension in resource constrained primary health care settings: results from a cluster randomized trial. J Am Heart Assoc. 2015;4:e001213.

    Article  PubMed  PubMed Central  Google Scholar 

  52. Abell B, Naicker S, Rodwell D, et al. Identifying barriers and facilitators to successful implementation of computerized clinical decision support systems in hospitals: a NASSS framework-informed scoping review. Implement Sci. 2023;18:32.

    Article  PubMed  PubMed Central  Google Scholar 

  53. Bentley KH, Zuromski KL, Fortgang RG, et al. Implementing machine learning models for suicide risk prediction in clinical practice: Focus Group Study with Hospital Providers. JMIR Form Res. 2022;6:e30946.

    Article  PubMed  PubMed Central  Google Scholar 

  54. Kar Ray M, Wyder M, Crompton D, et al. Relational safety based suicide prevention training frameworks. Int J Ment Health Nurs. 2020;PROTECT:29:533–43.

    Article  Google Scholar 

  55. Barak-Corren Y, Castro VM, Nock MK, et al. Validation of an Electronic Health Record-based suicide risk prediction modeling Approach across multiple Health Care systems. JAMA Netw Open. 2020;3:e201262.

    Article  PubMed  Google Scholar 

  56. Ripperger MA, Kolli J, Wilimitis D, et al. External validation and updating of a statistical civilian-based suicide risk model in US Naval Primary Care. JAMA Netw Open. 2023;6:e2342750.

    Article  PubMed  PubMed Central  Google Scholar 

  57. Wilimitis D, Turer RW, Ripperger M, et al. Integration of Face-to-face Screening with Real-time machine learning to Predict risk of suicide among adults. JAMA Netw Open. 2022;5:e2212095.

    Article  PubMed  PubMed Central  Google Scholar 

  58. Simon GE, Johnson E, Lawrence JM, et al. Predicting suicide attempts and Suicide Deaths Following Outpatient Visits Using Electronic Health Records. Am J Psychiatry. 2018;175:951–60.

    Article  PubMed  PubMed Central  Google Scholar 

  59. Ross EL, Zuromski KL, Reis BY, et al. Accuracy requirements for cost-effective suicide risk prediction among primary care patients in the US. JAMA Psychiatry. 2021;78:642–50.

    Article  PubMed  Google Scholar 

  60. Matarazzo BB, Eagan A, Landes SJ, et al. The Veterans Health Administration REACH VET program: suicide predictive modeling in practice. Psychiatr Serv. 2023;74:206–9.

    Article  PubMed  Google Scholar 

  61. Kessler RC, Hwang I, Hoffmire CA, et al. Developing a practical suicide risk prediction model for targeting high-risk patients in the veterans health administration. Int J Methods Psychiatr Res. 2017;26:e1575.

    Article  PubMed  PubMed Central  Google Scholar 

  62. Dimeff LA, Jobes DA, Koerner K, et al. Using a tablet-based app to deliver evidence-based practices for suicidal patients in the Emergency Department: pilot randomized controlled trial. JMIR Ment Health. 2021;8:e23022.

    Article  PubMed  PubMed Central  Google Scholar 

  63. Elkington KS, Wasserman GA, Ryan ME, et al. E-Connect: linking probation youth at risk for suicide to behavioral health services. J Consult Clin Psychol. 2023;91:547–57.

    Article  PubMed  Google Scholar 

  64. OxRisk| Oxford Forensic Psychiatry Risk Tools. OxRisk. (accessed 27 November 2023).

  65. Fazel S, Vazquez-Montes MDLA, Molero Y, et al. Risk of death by suicide following self-harm presentations to healthcare: development and validation of a multivariable clinical prediction rule (OxSATS). BMJ Ment Health. 2023;26:e300673.

    Article  PubMed  PubMed Central  Google Scholar 

  66. Botchway S, Tsiachristas A, Pollard J, et al. Cost-effectiveness of implementing a suicide prediction tool (OxMIS) in severe mental illness: economic modeling study. Eur Psychiatry J Assoc Eur Psychiatr. 2022;66:e6.

    Article  Google Scholar 

  67. Bouch J, Marshall JJ. Suicide risk: structured professional judgement. Adv Psychiatr Treat. 2005;11:84–91.

    Article  Google Scholar 

  68. Self-Harm Registry– National Suicide Research Foundation. (accessed 13 December 2023).

  69. Pérez V, Elices M, Prat B, et al. The Catalonia suicide risk code: a secondary prevention program for individuals at risk of suicide. J Affect Disord. 2020;268:201–5.

    Article  PubMed  Google Scholar 

  70. Stang PE, Ryan PB, Racoosin JA, et al. Advancing the science for active surveillance: Rationale and Design for the Observational Medical Outcomes Partnership. Ann Intern Med. 2010;153:600–6.

    Article  PubMed  Google Scholar 

  71. Informatics OHDS. and. The Book of OHDSI. (accessed 13 December 2023).

  72. McMahan HB, Moore E, Ramage D et al. Communication-Efficient Learning of Deep Networks from Decentralized Data. 2023.

  73. Ludwig H, Baracaldo N, Thomas G et al. IBM Federated Learning: an Enterprise Framework White Paper V0.1. 2020.

  74. Qin P, Mehlum L. Deliberate self-harm: case identification and incidence estimate upon data from national patient registry. PLoS ONE. 2020;15:e0231885.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  75. Simon GE, Shortreed SM, Boggs JM, et al. Accuracy of ICD-10-CM encounter diagnoses from health records for identifying self-harm events. J Am Med Inf Assoc JAMIA. 2022;29:2023–31.

    Article  Google Scholar 

  76. Brown LA, Benhamou K, May AM, et al. Machine learning algorithms in suicide Prevention: clinician interpretations as barriers to implementation. J Clin Psychiatry. 2020;81:19m12970.

    Article  PubMed  Google Scholar 

  77. Brouwers MC, Kho ME, Browman GP, et al. Development of the AGREE II, part 2: assessment of validity of items and tools to support application. CMAJ Can Med Assoc J J Assoc Medicale Can. 2010;182:E472–478.

    Article  Google Scholar 

  78. Brouwers MC, Kho ME, Browman GP, et al. Development of the AGREE II, part 1: performance, usefulness and areas for improvement. CMAJ Can Med Assoc J J Assoc Medicale Can. 2010;182:1045–52.

    Article  Google Scholar 

  79. Brouwers MC, Kho ME, Browman GP, et al. AGREE II: advancing guideline development, reporting and evaluation in health care. CMAJ Can Med Assoc J J Assoc Medicale Can. 2010;182:E839–842.

    Article  Google Scholar 

Download references


The PERMANENS project in Ireland is being conducted under the NSRF-UCC Memorandum of Collaboration. In Catalonia, this study will be conducted using anonymized data provided by the Agency for Health Quality and Assessment of Catalonia (AQuAS) within the framework of the Data Analytics Program for Health Research and Innovation (PADRIS) Program. We sincerely thank Professor Ricard Gavaldà Mestre from the Universitat Politècnica de Catalunya and founder from AmalfiAnalytics ( for his advisory role in the PERMANENS project. We would like to acknowledge the valuable input from Dr Paul Corcoran and Sofia Bettella into the PERMANENS project.


This work was supported by Instituto de Salud Carlos III (ISCIII) and by the European Union NextGenerationEU, Mecanismo para la Recuperación y la Resiliencia (AC22/00006; AC22/00045), the Swedish Innovation Agency (no. 2022 − 00549), the Research Council of Norway (project no. 342386) and the Health Research Board Ireland (ERAPERMED2022) under the frame of ERA PerMed. In addition, this work was supported by ISCIII-FSE Miguel Servet co-funded by the European Social Fund Plus (ESF+; CP21/00078); ISCIII and co-funded by the European Union (PI22/00107); Fundación la Marató de TV3 (202220-30-31); PFIS ISCIII (FI23/00004) co-funded by the ESF+; the Secretaria d’Universitats i Recerca del Departament d’Economia i Coneixement of the Generalitat de Catalunya AGAUR 2021 (SGR 00624); and CIBER of Epidemiology & Public Health (CIBERESP; CB06/02/0046). As part of the application for funding for the project (Era PerMed Joint Transnational Call for Proposals 2022 “Prevention in Personalised Medicine”), the research proposal was reviewed by three peer reviewers.

Author information

Authors and Affiliations



PM initiated the study and coordinated the creation of the PERMANENS consortium; EA, JB, PM, MP, and PQ obtained the funding for the project; all authors contributed to the final protocol and approved submitting the manuscript for publication.

Corresponding author

Correspondence to Philippe Mortier.

Ethics declarations

Ethics approval and consent to participate

All aspects of the research protocol have been approved by the respective Institutional Review Boards at the relevant participating sites. In Catalonia (Spain), the IRB was the Parc de Salut Mar Clinical Research Ethics Committee (study 2022/10681/I and 2022/10325/I); in Sweden, the Swedish Ethical Review Authority (no. 2012/1669-31/5 and no. 2023-02336-02); in Norway, the Regional Ethical Committee for Medical and Health Research (ref. 2013/1620/REK south-east); and in Ireland by the Clinical Research Ethics Committee (Review Reference number- ECM 3 [qq] 06/02/2024).

The processing of electronic registry data (i.e., data harmonization followed by the development of prediction models) does not involve research with human subjects or the joint processing of personal data. The requirement of explicit informed consent for the use of this data has been waived at all sites given that such requirement would make any registry-based research impossible, and given that strict data protection regulations are in place to minimize risk for subject identification (e.g., anonymization and/or registry data access through highly-secure virtual environments).

Implementation research activities will adopt technical and procedural measures to strictly follow the EU and National General Data Protection Regulations, and will be in line with the principles established by national and international regulations, including the Declaration of Helsinki and the Code of Ethics. All personal data will be handled following Regulation (EU) 2016/679 of the European Parliament and of the Council of April 27 on the protection of natural persons with regard to the processing of personal data and on the free movement of such data (General Data Protection Regulation, GDPR) and corresponding national laws from the participating countries. Proposed data collection procedures are relevant and limited to the purposes of the project. Collected data will be stored on secure servers and will be pseudo-anonymised using encrypted identifiers to ensure privacy. All participants in implementation research will give explicit informed consent for participation following GDPR articles 6.1a and 9.2a. Implementation research activities will always occur in setting where high-quality effective treatment is available.

Consent for publication

Not applicable.

Competing interests

The authors declare no competing interests.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit The Creative Commons Public Domain Dedication waiver ( applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Mortier, P., Amigo, F., Bhargav, M. et al. Developing a clinical decision support system software prototype that assists in the management of patients with self-harm in the emergency department: protocol of the PERMANENS project. BMC Psychiatry 24, 220 (2024).

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: