Psychometric properties of the Arabic version of the confusion assessment method for the intensive care unit (CAM-ICU)

Background It is recommended that critically ill patients undergo routine delirium monitoring with a valid and reliable tool such as the Confusion Assessment Method for the Intensive Care Unit (CAM-ICU). However, the validity and reliability of the Arabic version of the CAM-ICU has not been investigated. Here, we test the validity and reliability of the Arabic CAM-ICU. Methods We conducted a psychometric study at ICUs in a tertiary-care hospital in Saudi Arabia. We recruited consecutive adult Arabic-speaking patients, who had stayed in the ICU for at least 24 hours, and had a Richmond Agitation-Sedation Scale (RASS) score ≥ − 2 at examination. Two well-trained examiners (ICU nurse and intensivist) independently assessed delirium in eligible patients with the Arabic CAM-ICU. Evaluations by the two examiners were compared with psychiatrist blind clinical assessment of delirium according to the Diagnostic and Statistical Manual of Mental Disorders, Fifth Edition (DSM-5). Subgroup analyses were conducted for age, invasive mechanical ventilation, and gender. Results We included 108 patients (mean age: 62.6 ± 17.6; male: 51.9%), of whom 37% were on invasive mechanical ventilation. Delirium was diagnosed in 63% of enrolled patients as per the psychiatrist clinical assessment. The Arabic CAM-ICU sensitivity was 74% (95% confidence interval [CI] = 0.63–0.84) and 56% (95%CI = 0.44–0.68) for the ICU nurse and intensivist, respectively. Specificity was 98% (95%CI = 0.93–1.0) and 92% (95%CI = 0.84–1.0), respectively. Sensitivity was greater for mechanically-ventilated patients, women, and those aged ≥65 years. Specificity was greater for those aged < 65 years, non-mechanically-ventilated patients and men. The median duration to complete the Arabic CAM-ICU was 2 min (interquartile range, 2–3) and 4.5 min (IQR, 3–5) for the ICU nurse and intensivist, respectively. Inter-rater reliability (kappa) was 0.66. Conclusions The Arabic CAM-ICU demonstrated acceptable reliability and validity to assess delirium in Arabic-speaking ICU patients.


Background
Delirium is defined as a disturbance in attention and awareness that fluctuates over a short period of time and accompanied by changes in cognition or perceptual disturbances [1]. Delirium is extremely common in critically ill patients, resulting from the acute illness, comorbidities, and use of sedation and analgesia. Furthermore, delirium is influenced by environmental and physical factors in intensive care unit (ICU) settings, such as isolation, limited visiting hours, the use of restraint, and connection to multiple tubes, lines, and catheters [2][3][4][5].
Routine monitoring of delirium is considered as a quality indicator for optimum care in older patients [19], and was recommended in 2013 by clinical practice guidelines for the management of pain, agitation, and delirium in adult patients in the ICU [20]. However, delirium is underdiagnosed and monitoring of delirium tends to be sporadic and underperformed [13,[21][22][23]. Therefore, the routine use of valid and reliable tools to assess for the presence of delirium is recommended at least once per ICU shift for patients with a high risk of delirium [20].
The Confusion Assessment Method for the ICU (CAM-ICU) is a rapid, reliable, and valid tool to assess delirium in the ICU [6]. The CAM-ICU is an adapted version of the bedside assessment tool for non-psychiatrists, the Confusion Assessment Method (CAM), which was based on the Diagnostic and Statistical Manual of Mental Disorders, Third Edition, Revised (DSM-III-R) [24]. The CAM-ICU demonstrates good validity and reliability when administered by ICU nurses and physicians for both ventilated and non-ventilated ICU patients [6,[25][26][27].
The CAM-ICU has been translated into several languages, with high reliability and validity [28][29][30][31][32][33]. Indeed, the CAM-ICU has been translated into Arabic by the ICU Delirium and Cognitive Impairment Study Group at Vanderbilt University Medical Center for Health Services Research [34]. However, the tool has not yet been tested for validity and reliability. The use of a valid and reliable Arabic delirium assessment tool is crucial to diagnose, prevent, and manage delirium among Arabicspeaking ICU patients.
In this study, we assess the psychometric properties of the current Arabic version of the CAM-ICU and its applicability to Arabic-speaking critical care patients' at a tertiary care hospital in Saudi Arabia.

Setting
We conducted this psychometric validation study at the Intensive Care Department (ICD) in King Abdulaziz Medical City (KAMC), Riyadh, Saudi Arabia. KAMC is a 1000-bed teaching tertiary care centre accredited by the Joint Commission International (JCI). The ICD encompasses a 21-bed medical ICU, 9- All study participants or substitute decision-makers (for patients lacking decision-making capacity) signed informed consent before participation. A consecutive sample technique was used to obtain the targeted sample size. Screening was conducted twice weekly during the day shift in the ICD between April 2016 and March 2017.

Study participants
In this study, we included Arabic-speaking critically ill patients at the ICD, aged ≥18 years and with a Richmond Agitation-Sedation Scale (RASS) ≥ − 2, who had stayed for at least 24 hours in the ICU. We excluded patients with the following: pre-existing severe dementia or psychosis; neurologic diseases including traumatic brain injury, acute brain lesion, or spinal cord injury; burns; patients admitted because of drug overdose; patients with blindness or deafness; those with Glasgow Coma Scale < 9 at the time of screening; patients unable to complete assessments as they are medically unstable; and patients/substitute decision-makers who refused to sign informed consent form and participate in the study.

Sample size
Previous studies have reported that delirium occurs in between 21% and 84% of critically ill patients [6][7][8][9][10][11]. Based on previous non-English CAM-ICU validation studies, sensitivity ranges from 73% to 92% and specificity ranges from 72% to 100% [29-31, 33, 35]. Accordingly, we expected values for sensitivity and specificity of 85%. Therefore, with an estimated mid-percentage of delirium prevalence of 50%, and an acceptable width of 95% confidence interval (CI) for sensitivity and specificity to no greater than 10% (75-95%), we calculated a minimum required sample size of 98 [36]. To account for potential dropouts and to compensate for assessments that could not be done within 4 hours by all assessors or missing assessment by any assessor, we anticipated a sample size of 125 patients.

Procedures
The CAM-ICU has already been translated into Arabic by the ICU Delirium and Cognitive Impairment Study Group at Vanderbilt University Medical Center for Health Services Research. The tool (both visual and auditory sections) was translated to suit the cultural and educational background of Arabicspeaking populations. The translations were conducted independently by two members before they discussed it together. The final Arabic version was then back translated to the original language (English) by a professional independent translator [34].
The Arabic CAM-ICU comprises the same four features that are used in the English CAM-ICU to screen patients for delirium. These features include: 1) acute onset of mental status changes or a fluctuating course; 2) inattention; and 3) altered level of consciousness; or 4) disorganized thinking. For inattention, the Arabic CAM-ICU uses mainly number sequences (e.g. 6 3 1 1 4 1 5 7 1 8) instead of letter sequences (e.g. S A V E A H A A R T). However, using alternate sequences of Arabic letters is possible. In addition, some words were changed in items assessing disorganized thinking to suit the Arabic speaking culture (e.g. pounds changed to kilos) [34]. As with the CAM-ICU, a patient is considered delirious if he/she test positive for features 1 and 2, in addition to being positive for either feature 3 or 4 [34]. The Arabic CAM-ICU and the training manual are available from the ICU Delirium and Cognitive Impairment Study Group website [34].
In our validation study, two well-trained examiners (an ICU Clinical Nurse Specialist and Intensivist) independently assessed delirium in eligible ICU patients with the Arabic CAM-ICU. They performed the CAM-ICU in the same group of patients but without any specific order, with an allowed duration of up to 4 hours between two exams for each patient. All assessments were performed during the day shift.
In addition, an experienced psychiatrist specialising in geriatric psychiatry independently assessed the same ICU patients for delirium according to the Diagnostic and Statistical Manual of Mental Disorders, Fifth Edition (DSM-5) criteria [1] within the same 4 hours window of evaluation as the ICU examiners. All assessors were blinded to the results of the others until the end of the study. The results of the psychiatric screening were compared with those of the ICU examiners screening. All assessors are authors in this study.

Data collection
The following data was recorded: date and time of assessments; assessment location; gender; age; use of mechanical ventilation; RASS score [37]; and delirium assessment results (Arabic CAM-ICU and psychiatrist clinical assessment).

Statistical analysis
The psychometric properties of the Arabic CAM-ICU were calculated for the ICU nurse and intensivist separately compared with the psychiatrist clinical assessment of delirium. We calculated the sensitivity, specificity, positive likelihood ratio (LR+), negative likelihood ratio (LR-), positive predictive values, negative predictive values, pretest probability, pretest odds, posttest odds, posttest probability, accuracies, and prevalence. Subgroup analyses were conducted for age, invasive mechanical ventilation status, and gender.
Agreement (inter-rater reliability) was assessed by calculating Cohen's kappa. Continuous data are represented by mean ± standard deviation (SD) or median (quartile Q1, Q2). Calculations were performed with the Statistical Package for the Social Sciences version 24 (SPSS Inc., Chicago, IL, USA).

Patient characteristics
From 129 patients, a total of 125 patients agreed to participate and were enrolled in the study. Of these, 17 patients were excluded from the study analysis for the following reasons: duration of assessments between examiners exceeded the allowed 4 hours (n = 13); diagnosis of chronic schizophrenia (n = 1); diagnosis with drug addiction (n = 1); psychiatrist clinical assessment of delirium not completed (n = 1); and psychiatrist clinical assessment of delirium missing (n = 1).
We included 108 patients in the final analysis, among which the CAM-ICU was conducted by a nurse and the clinical assessment of delirium by psychiatrist in 108 patients, and the CAM-ICU was conducted by an intensivist for 106 patients. Approximately half of the patients were male (51.9%), and the mean age of participants was 62.6 ± 17.6 years with a median of 66 years (interquartile range: 63.8-75 years). The majority of participants were not on invasive mechanical ventilation (n = 68, 63%).
With the psychiatrist clinical assessment, delirium was diagnosed in 68 patients (63%).

Reliability of the Arabic CAM-ICU
We assessed the level of agreement between assessments with the Arabic CAM-ICU conducted by the ICU nurse and intensivist by calculating inter-rater reliability (kappa). Assessments were matched in 88 patients (83%) with a kappa of 0.66 (standard error: 0.07; p < 0.0001).

Discussion
In this study, we demonstrated the psychometric properties of the Arabic CAM-ICU. In this study, patients were assessed for delirium by an intensivist and ICU nurse with the Arabic CAM-ICU, and these assessments were compared with the psychiatrist clinical assessment according to DSM-5. In addition, subgroup analyses were conducted for age, mechanical ventilation status, and gender to determine the validity of the Arabic CAM-ICU in specific groups. Previous studies have reported that delirium occurs in between 21% and 84% of critically ill patients [6][7][8][9][10][11]. In this study, the psychiatrist clinical assessment identified delirium in 63% of critically ill patients. This data suggests that delirium is similarly common among critically ill patients in the Arabic-speaking population. This high rate of delirium highlights the importance of a reliable and valid measure to assess delirium for critically ill patients in the ICU. In addition, it suggests the need for urgent action to establish delirium treatment and prevention measures.
Assessment of the Arabic CAM-ICU by the ICU nurse had a sensitivity of 74%. The sensitivity increased to 75% for mechanically ventilated patients, 77% for female patients, and 79% for patients aged ≥65 years. Although sensitivity in our study is less than that reported in the initial validation study for the English CAM-ICU by Ely et al. (93-100%) [6,27], it was within the ranges reported by other validation studies for non-English versions of the CAM-ICU (73-92%) [29][30][31][32][33]35]. Therefore, we believe that the sensitivity for the Arabic CAM-ICU is acceptable.
Assessment of the Arabic CAM-ICU by the ICU nurse had a specificity of 98%. The specificity increased to 100% for patients aged < 65 years, non-mechanically ventilated patients, and male patients. The specificity of  [6,27], which suggests that the specificity of the Arabic CAM-ICU is acceptable. Taking our results together, the Arabic CAM-ICU appears to be a valid tool for the assessment of delirium among Arabic-speaking critically ill patients. Nonetheless, our results did show some discrepancies between raters. Although comparable values for specificity were achieved between raters (98% vs. 92%), there was greater discrepancy in sensitivity values (74% vs. 56%). These discrepancies have not been reported in other studies that have used nurses and intensivists as CAM-ICU raters [27,32]. In the study by Ely et al., sensitivity values of the CAM-ICU for two nurses and an intensivist were 95%, 96%, and 100%, respectively, and specificity values were 93%, 93%, and 89%, respectively [27]. Moreover, in a Spanish study, sensitivity values of the CAM-ICU for an ICU nurse and an intensivist were 83% and 80%, respectively, and specificity was 96% for both raters [32]. Nonetheless, in our study, the raters' assessments were matched in 83% patients with a kappa of 0.66, which is considered moderate and acceptable interrater reliability [38]. We believe that the accuracy of the CAM-ICU assessment could be further improved by providing proper education about delirium and training for the CAM-ICU [33,39,40]. This is the first study to assess the reliability and validity of the Arabic CAM-ICU as a delirium assessment tool for critically ill patients. Our study has several strengths including a relatively large and powered sample size. In addition, it measured inclusive psychometric features of the Arabic CAM-ICU. Furthermore, we conducted subgroup analyses to measure the psychometric features of the Arabic CAM-ICU in specific groups. This is the first study to highlight delirium rate for Arabicspeaking critically ill patients. The main limitation of the CAM-ICU Confusion Assessment Method for the Intensive Care Unit, CI confidence interval * Specificity value is 1; ** Likelihood Ratio is undefined; *** Post-test odd is undefined study is that it assessed inattention components of the Arabic CAM-ICU with the auditory attention screening exam only. We recommend that for future studies to investigate the reliability and validity of the visual attention screening exam of the Arabic CAM-ICU. In addition, we excluded patients with neurological diseases including traumatic brain injury; such neurological diseases may dispose patients to different neuropsychiatric consequences [41]. The CAM-ICU may not be the optimal tool to assess delirium in such patients [42]. Developing specific tools to assess and differentiate delirium from other neuropsychiatric dysfunctions is recommended for future research [42].

Conclusions
The Arabic CAM-ICU demonstrated acceptable reliability and validity to assess delirium in Arabic-speaking ICU patients. We recommend researchers and clinicians to use the Arabic CAM-ICU as delirium assessment tool for Arabic-speaking ICU patients, taking into consideration the knowledge and training needed of delirium assessment. Further studies including the Arabic CAM-ICU would identify its further strengths and limitations in research and clinical fields.  CAM-ICU Confusion Assessment Method for the Intensive Care Unit, CI confidence interval * Specificity value is 1; ** Likelihood Ratio is undefined; *** Post-test odd is undefined