The screen for cognitive impairment in psychiatry (SCIP) as a routinely applied screening tool: pathology of acute psychiatric inpatients and cluster analysis

Background Cognitive dysfunction has been reported in acute psychiatric patients for a long time. The detection of cognitive deficits is crucial both for clinical treatment and for predicting the psychosocial functional level in the further course of the disease. The SCIP is a well-evaluated screening instrument for the examination of cognitive performance in psychiatric patients. We recently integrated the SCIP into our routine admission and discharge assessments on two inpatient wards, and we examined the cognitive profiles of patients with psychotic and affective disorders over the course of their admission. Methods Shortly after admission, and prior to discharge, patients were routinely referred for examination with the SCIP. A total of 529 assessments were completed on admission, and 227 returned for SCIP at the time of discharge. After standardization of the test results against a normative sample, we examined the normalized test values in terms of percentages of pathological cognitive performance based on the total SCIP score, and each of the SCIP subscale scores. We conducted cluster analysis to identify cognitive subgroups within the clinical sample. Results More than 70% of the SCIP results on admission were pathological. At discharge, improvements were observed, especially on tests with attention and speed components. Cluster analysis identified two groups. The cluster with chronic patients showed poorer results at admission, but greater improvement and reached the level of the others at discharge. Conclusions The SCIP appears to have value in routine diagnostic assessments, and in the quantification of improvements in cognitive performance during an inpatient stay. The greatest benefit was observed in chronically ill patients with many previous stays. Trial registration DRKS00019825 (retrospectively registered on 03.12.2019).


Background
Neuropsychological deficits have been reported in psychosis since Kraepelin [1] and are still of high interest [2]. A meta-analysis of patients with psychotic disorders showed deficits in all cognitive areas, especially in the domains of verbal memory and processing speed [3]. It was also observed that executive functions are impaired in psychotic patients [4]. Deficits in executive function, attentional performance and memory are also described for patients with bipolar disorders [5] and for patients with depressive disorders [6,7]. Comparing the cognitive performance of patients with psychosis and affective disorders, no differences were found [8][9][10][11]. This is in contrast to the results of Bonner-Jackson et al. [12]. The authors had examined 244 patients diagnosed with schizophrenia, other psychosis or non-psychotic depression over a period of 20 years after index treatment. At all seven points of measurement, the cognitive performance of patients with schizophrenia was significantly worse than patients with other disorders. Again, deficits in processing speed were the most common. Inconsistent findings were reported on the stability of the described deficits over the course of relatively brief inpatient treatment, with some reports of no improvement in cognitive deficits between admission and discharge [9,13], and other reports of demonstrated improvements [10].
Cognitive deficits have considerable prognostic impact [14,15]. It is widely accepted that the correlation between cognitive impairment and psychosocial functional level or occupational activity is significantly stronger than between clinical symptoms and functional level or occupational activity [14,15]. The cognitive functional level proved to be a good predictor for the number and quality of social relationships [16]. Tourjman et al. [17] showed for depressed patients that the results of a cognitive screening were associated with severity of illness, self-reported cognitive dysfunction and impairment in daily life. Consequently, the assessment of cognitive performance is relevant for therapy and rehabilitation planning, but also for providing realistic job and employment perspectives [18][19][20].
Screening instruments are economical and easy to apply in everyday clinical practice. Several screening methods are available for the detection of cognitive disturbances in mental disorders. The Screen for Cognitive Impairment in Psychiatry (SCIP, [21]) and the Brief Assessment of Cognition in Schizophrenia (BACS, [22]) were designed for use in psychiatric samples. The SCIP requiring slightly less administration time compared to the BACS and it appears to have greater sensitivity to cognitive deficits associated with psychosis [2] compared to the established Montreal Cognitive Assessment (MoCA, [23]).
Several international studies have offered support for the utility of the SCIP as a screening tool in psychiatry applying the SCIP are already available [2,8,17,[24][25][26]. The primary emphasis of the prior reports concerned the psychometric properties of the SCIP in samples that ranged in size from n = 40 [17] to a high n = 514 [8]. Our goal was to extend these results by examination of the utility of integrating the SCIP into the routine clinical assessments undertaken on admission and discharge within a more typical high volume inpatient environment. To this end, the SCIP was added to the routine evaluations undertaken within two general psychiatric wards in 2012. Since that time we have administered the SCIP to n = 529 patients on admission. The aim of this study was to examine and compare the cognitive profiles of a diverse group of acute psychiatric inpatients at admission, and to assess any changes in profile that may occur between admission and discharge. Both absolute and normalized test values of performance were analyzed. Furthermore, we examined whether different clusters could be found based on the cognitive performance profiles.

Sample and study design
The present study is an evaluation of routinely collected clinical data that included an assessment of cognitive performance with the SCIP. Study participants were inpatients of the two acute psychiatric wards of the Clinic for Psychiatry and Psychotherapy I of the University of Ulm (Germany) at ZfP Suedwuerttemberg Ravensburg-Weissenau. Inclusion criteria were inpatient treatment, age between 18 and 65 years and ICD-10 primary diagnosis of a psychotic (F2) or an affective disorder (F3). Patients with other ICD-10 primary diagnoses, language barriers or who were not able or not willing to participate were excluded. The presence of other disorders as secondary diagnosis was not a reason for exclusion. The patients were examined at admission and again shortly before discharge. On average, admission testing was performed 10.7 days after admission (SD = 16.7) and discharge testing was performed 3.6 days (SD = 12.4) before discharge. Data analysis was performed with the approval of the responsible ethics committee of the University of Ulm (vote of August 21, 2019, no. 228-19). The study was registered at the German Register of Clinical Studies (DRKS00019825).

Screen for cognitive impairment (SCIP)
The SCIP (Screen for Cognitive Impairment in Psychiatry, [21]) is a well-studied screening tool for detecting cognitive impairment in patients with psychotic or affective disorders [2,17,27]. The SCIP consists of five subscales: verbal learning test -immediate (VLT-I), working memory test (WMT), verbal fluency test (VFT), verbal learning test -delayed (VLT-D) and processing speed test (PST). There are three different test forms to facilitate test repetition [27] and therefore reducing learning effect. The SCIP is available as a paper-pencil test, it consists of one page and takes about 15 min to complete [2,[25][26][27]. Subscale scores are calculated for each of the five tests, and a total score is calculated from the sum of the subscale scores. A higher value indicates a better cognitive performance. The SCIP is available in a many languages including English, Spanish, French, Italian, Danish and German [2,11,21,[25][26][27][28][29]. With regard to its psychometric properties, the SCIP has been investigated in various validation studies in healthy and ill people. All versions showed good psychometric properties [2,8,[25][26][27]29]. However, the German version has not yet been examined in this regard.

Pilot: validation of the German version of the SCIP
Initially we undertook a pilot investigation of the SCIP in a small group of patients (n = 31) to determine the feasibility of implementing the screening tool in our hospital, and to begin to assess the external validity of the German translation of the SCIP [30]. To investigate the external validity of the SCIP, the Zahlen-Verbindungs-Test (ZVT, [31]) was also administered to this patient sample. The ZVT [31] is a non-verbal, simple, standardized and well-established measure of cognitive processing speed. The ZVT can be completed in 4-10 min and it requires subjects to draw lines to connect numbers from 1 to 90 which are positioned more or less randomly on a piece of paper. Pearson correlations of the SCIP values with the age-normalized standard value of the ZVT (mean value 100, standard deviation 10) were calculated.
On average, the n = 31 patients in this pilot study were 40.3 years of age (SD = 11.00). The majority of the examined patients (74%) suffered from a psychotic disorder (F2 ICD-10), and only 13% had an affective disorder (F3 ICD-10) as the primary diagnosis. The average administration time for the SCIP was 13.6 min (SD = 2.22). Patients achieved an average total score of 58.55 (SD = 15.67) on the SCIP and a scaled score of M = 85.58 (SD = 11.41) on the ZVT. The SCIP total score correlated significantly with the scaled score from the ZVT (r = .68; p < .001). The subscales of the SCIP also correlated significantly between r = .44 and r = .57 with the ZVT scaled score [30]. The relatively brief administration time, high sensitivity to cognitive impairment, and the reasonable validation against the ZVT together supported the feasibility of our intended widespread adoption of the SCIP for our routine inpatient assessments [32].

Electronic medical file data
Clinically established primary and secondary diagnoses were extracted from the electronic medical files as well as dates of birth, admission and discharge. Sociodemographic data including the Clinical Global Impression Scale (CGI) were used as recorded by the basic documentation as recommended by the German Association of Psychiatry and Psychotherapy (DGPPN) and validated by Jaeger et al. [33].

Statistical analysis
All analyses were conducted using IBM SPSS for windows version 24. Some patients were only tested at admission. These patients were compared with patients tested at admission and discharge to investigate a possible selection bias. Chi 2 -Tests were used for normally scaled data, in all other cases we calculated Mann-Whitney-U Tests. To investigate diagnosis-specific differences in SCIP results at admission, a multivariate analysis of variance was calculated for the total value of the SCIP and its subscales, with diagnosis as an independent variable. Variance homogeneity was checked with the Levene test. Pillai-Bartlett trace was used for the test statistics, because it is robust to violations of the multinormal distribution and variance homogeneity [34]. After Bonferroni adjustment, the significance level was set to p < .008. For parameter estimation, beta was set to zero for the group of patients with affective disorder (F3 ICD-10). Repeated-Measures ANOVAs were calculated to investigate diagnostic and time effects (admission vs. discharge) as well as interaction effects on the SCIP test values. The test statistics were applied as described above. To facilitate the interpretation of the test results, we standardized the SCIP results using the values reported from a prior healthy sample [2]. By convention we assigned observed scores that were lower than normal by one standard deviation to a pathological classification. Changes in normalized test values over time were analyzed with Chi 2 -Tests. Cohen's d was calculated as effect sizes. According to Cohen [35], values between 0.2 and 0.5 indicate a small effect, between 0.5 and 0.8 a medium effect and values greater than 0.8 a strong effect. For calculating effect sizes of nominal scaled variables, the Phi coefficient (φ) was used. Here, values above 0.1 are considered to be small effects, values above 0.3 are medium effects and values above 0.5 are large effects [35]. A hierarchical cluster analysis (Ward's cluster method with squared Euclidean distance measurements) was performed with the SCIP results at admission to identify groups of study subjects with similar cognitive profiles. Afterwards differences between the two clusters were investigated with Chi 2 -Tests and Mann-Whitney-U Tests.

Analysis of the admission test scores of the total sample Sample characteristics
The study included data from a total of 529 SCIP results at admission. The patients examined were on average 43.7 years of age (SD = 11.5), male (56.7%), unmarried (60.1%), and without migration background (74.5%). Not quite a third (30%) lived on state support (unemployment benefit, social welfare, sick pay, etc.). The most common primary diagnosis (82.4%) was a psychotic disorder (F2 ICD-10), and only 17.6% (n = 93) of the sample were diagnosed with an affective disorder (F3 ICD-10). Of the latter, n = 49 patients had a diagnosis of a manic episode or a bipolar affective disorder and n = 44 patients had a diagnosis of a depressive episode or a recurrent depressive disorder. On average, patients had 1.77 (SD = 2.00) comorbid diagnoses. 41.6% had more than 5 previous admissions. In the CGI, 46.7% were assessed as "clearly ill" and 24.6% as "seriously ill". Thus the values in the CGI, the number of pre-treatments and the number of further diagnoses all suggest a relatively high disease severity suffered by the patients examined.
The sample characteristics for the total sample are displayed in Table 1. Table 1 Sample characteristics of participants of the total sample (n = 529) and the subsample with admission and discharge results (n = 227), as well as significant differences between the subsample and drop-outs (without discharge results).

Outcome 2: normalized test values of cognitive performance
Analyzing the rate of impaired SCIP performance in the sample, we found high percentages in the subtests for working memory (WMT), verbal fluency (VFT) and processing speed (PST). In the SCIP total score more than 70% of the patients had impaired performances. Only the cognitive performance in verbal learning was pathological in less than 50% of patients (see Table 2).  Table 2).

Sample characteristics
In n = 227 patients (42.1%) the SCIP was performed at admission and discharge. A possible selection bias between patients with admission and discharge findings and those without discharge findings (drop-outs) was investigated. Drop-outs differed from patients with discharge results in age (U = 30,288.50; p < .05), length of inpatient stay (U = 29,647.00; p < .05), native language (Chi 2 (1)=6.921; p < .01) and migration background (Chi 2 (6)=18.320; p < .01). In other variables, especially those related to disease severity (CGI, number of pretreatments), the two groups did not differ (Table 3).
Thus, a selection bias is more likely with respect to treatment duration and language barrier than in the area of disease severity (see also  Table 3 shows, all effects were small. The remaining two subscales showed no significant time effects (VLT-I: F (1.225) = 3.531; p > .008; VLT-D: F (1.225) = 1.085; p > .008). Table 3 Results of cognitive performance at admission and discharge.

Outcome 2: normalized test values of cognitive performance
Percentages of the sample with pathological SCIP results at the time of admission and at the time of discharge are   Large effect sizes resulted for the SCIP total score (φ = 0.62) and for the processing speed test (φ = 0.58). Otherwise, effects of medium size emerged.

Outcome 3: identification of groups with different cognitive performance profiles
To assess heterogeneity of cognitive profiles within the clinical sample, we undertook a hierarchical cluster analysis based on the SCIP results at admission. A two cluster solution with n = 243 and n = 286 patients yielded the best fit. Patients belonging to cluster 1 were significantly older (U = 21,462.00; p < .001), had more additional diagnoses (U = 28,328.5; p < .001) and had significantly more than 5 previous admissions (Chi 2 (2)= 13.375; p < .001) compared to patients belonging to cluster 2. In the SCIP admission scores, patients of the first cluster scored significantly worse in all test results compared to patients of cluster 2 (see Table 4, see also Fig. 2). All patients of cluster 1 showed pathological SCIP total scores at admission, whereas only 45,8% of the patients of cluster 2 show (Chi 2 (1529) = 186,28; p < .001).
To make statements about possible improvements in cognitive performance during treatment, the subsample with admission and discharge results was analyzed. For those n = 106 and n = 121 patients the discharge results and the improvement between admission and discharge are presented in Table 4. As with the findings at admission, the discharge results showed that cluster 1 had significantly worse results in all scales than cluster 2. In terms of improvement, patients in the first cluster scored significantly higher in the SCIP total score and in the immediate verbal learning test compared to patients in the second cluster. Table 4 Differences in sociodemographic variables and SCIP profiles between the two clusters based on SCIP results at admission.

Discussion
The SCIP is an economical, internationally approved and validated screening tool to assess cognitive performance in psychiatric patients. In this report we provide data of a naturalistic sample of 529 psychiatric inpatients screened with the SCIP at the time of admission to a general psychiatric ward. This is the first paper to report data of such a sample. Results for the SCIP total score and the five subscale scores corresponded to published data from other translations of the SCIP (e.g. [2]). Diagnosis-specific differences between psychotic and affective disorders in our sample did not occur. Our results are therefore consistent with the results of Gómez-Benito et al. [8], Sachs et al. [11], Neu et al. [10] and Hill et al. [9]. In addition to the absolute values, we used the results of the healthy population norms from Murri et al. [2] to investigate the proportion of our sample with pathological results. This revealed a high percentage of cognitive dysfunction in our sample evident in the SCIP total score (over 70%) as well as for the three subtests with significant demands for attention and processing components (working memory, verbal fluency test, psychomotor speed test). The subtests of verbal learning and memory on the other hand were pathological in less than 40% of the patients.
Looking at the cognitive performance profiles over the course of inpatient treatment, there were significant improvements from admission to discharge in the total score and the subtests for verbal fluency, processing speed and working memory. In contrast, verbal learning, both immediate and delayed, showed no changes over time. This is in contrast to previous findings with reported deficits as well as in the domains of verbal memory and processing speed [3,4]. In terms of the normalized values, large effect sizes were observed for the improvement in the SCIP total score and the psychomotor speed test between admission and discharge. Taken together, most cognitive improvements occurred in working memory, processing speed and verbal fluency (which also includes a speed factor). We suppose that social psychiatric inpatient treatment improves most in cognitive domains of attention, working memory and speed components, because in daily ward living, social interaction and communication are key processes within the recovery model. Possible effects of the reapplied psychiatric medication during an inpatient stay must also be taken into account. On the one hand, improvements in cognitive performance are conceivable, particularly if second generation antipsychotics are given [36,37] on the other hand, side effects such as fatigue and loss of attention could also occur [10]. In the present study, the influence of medication effects could not be analyzed.
Following Rodriguez et al. [38] we further investigated whether different patient clusters could be identified based on the results of a screening tool. In our analysis, two clusters were extracted based on SCIP results at admission. Rodriguez et al. [38] extracted three clusters corresponding to a general impairment group, an attention, speed and verbal deficit group, and a third cluster with above average attention but impaired verbal memory. In contrast to Rodriguez et al. [38], the sample reported here separated into either one group with mild impairment in terms of mean value with almost 50% of pathological results on the SCIP (cluster 2) and 1 sec group with severe impairment in terms of mean with 100% of pathological results on the SCIP (cluster 1). Patients of cluster 1 were older, had more comorbid diagnoses and had more previous admissions. However, compared to cluster 2, they improved significantly more between admission and discharge results, both in the overall score and in the verbal learning test. Since the patients of cluster 2 are already considerably less impaired in the admission findings based on the values of Murri et al. [2], a ceiling effect must be considered for this cluster. In conclusion, cluster 1 could be interpreted as a sample of chronic patients, more impaired clinically as well as cognitively, who then benefitted from the social psychiatric inpatient stay. Our study has several limitations. First, our results are based on routinely collected clinical data by a screening tool for cognitive performance. Beside the ecological validity advantages of such a naturalistic clinical dataset, selection processes of the examined patients must be discussed. Only those, who were willing to participate in such a diagnostic procedure, could be included. To investigate possible selection processes concerning repeated measurement, drop-out analyses were performed. Compared to the complete cases with both admission and discharge data, the patients that dropped out prior to the discharge evaluation differed only in sociodemographic variables, and a shorter length of stay. It can therefore be assumed that selection was based more on organizational concerns and less on disease severity. Furthermore it must be noted that patients with cognitive impairment due to other disorders are regularly not treated on the units where the patients were recruited but presence of such conditions in a milder degree had not been predefined as an exclusion criterion. Comorbid disorders should be considered more in future work.
Another limiting factor is the lack of a concurrent healthy control sample for comparison, and the absence of published data with reported SCIP scores from a German translation of the SCIP. We selected the relatively large sample reported by Murri et al. [2] from the Italian translation over similar reports from the Spanish translation [25,26]. When comparing with the healthy sample of Murri et al. [2], the differences in age, gender distribution and educational status compared to our sample must be mentioned as a limitation. With the current demonstration of the sensitivity of the German translation of the SCIP to the cognitive limitations apparent in an inpatient psychiatric sample we are confident that the time and effort required to secure normative data for this version would be very well spent. As in all investigations of cognitive limitations, we were not able to include in our analysis many additional factors that could influence or potentially mediate test results, e.g. medication or learning effects. Moritz et al. [39] were able to show that factors such as motivation and attitude towards testing or medication significantly reduce cognitive abnormalities in schizophrenia sufferers. This all should be considered in future work.

Conclusion
In conclusion, the SCIP has proven to be a suitable instrument for routine use in the detection of cognitive impairment. This is particularly important because of the impact of cognition for psychosocial functioning. Our results further indicate an improvement in cognitive performance during an inpatient stay. This may be particularly relevant to chronically ill patients with many previous admissions because we might anticipate substantial gains during the admission that could be communicated back to the patient and their family to instill some hope for even partial recovery.