A cross-sectional testing of The Iowa Personality Disorder Screen in a psychiatric outpatient setting

Background Patients suspected of personality disorders (PDs) by general practitioners are frequently referred to psychiatric outpatient clinics (POCs). In that setting an effective screening instrument for PDs would be helpful due to resource constraints. This study evaluates the properties of The Iowa Personality Disorder Screen (IPDS) as a screening instrument for PDs at a POC. Methods In a cross-sectional design 145 patients filled in the IPDS and were examined with the SCID-II interview as reference. Various case-findings properties were tested, interference of socio-demographic and other psychopathology were investigated by logistic regression and relationships of the IPDS and the concept of PDs were studied by a latent variable path analysis. Results We found that socio-demographic and psychopathological factors hardly disturbed the IPDS as screening instrument. With a cut-off ≥4 the 11 items IPDS version had sensitivity 0.77 and specificity 0.71. A brief 5 items version showed sensitivity 0.82 and specificity 0.74 with cut-off ≥ 2. With exception for one item, the IPDS variables loaded adequately on their respective first order variables, and the five first order variables loaded in general adequately on their second order variable. Conclusion Our results support the IPDS as a useful screening instrument for PDs present or absent in the POC setting.


Background
Several studies have indicated that the prevalence of personality disorders (PDs) is high in the setting of psychiatric outpatient clinics (POCs). From the United States Zimmerman reported a prevalence of 50% [1], while 80% was found by Alnaes & Torgersen [2] in Norway. The variation in prevalence rate depends in part on practical matters like the referral practice of the general practitioners (GPs), and in part on research matters like the instruments used to assess PDs. Frequent co-morbidity of Axis I disorders and PDs regularly demands extensive diagnostic assessments [3,4], and PD as an influential but unacknowledged factor impedes the referral process [5]. The GPs want a qualified diagnostic assessment and advice for further treatment as feedback of their referrals. A correct diagnosis of PDs is of clinical importance since their presence is associated with longer duration, poorer treatment outcome and recurrence of Axis I disorders [6][7][8]. Identification of such comorbidity is therefore also important for the choice of treatment [9,10]. All these issues make diagnostic evaluation of PDs an important matter at POCs.
Structured interviews are considered as the most reliable and valid method for the diagnostic assessment of PDs [11], but they are time-consuming and demand substantial clinical competence of the interviewer. At POCs in Norway, such clinical competence is a limited resource and the pressure to evaluate patients is considerable, and for efficient and qualified diagnostic assessment of PD a psychometrically valid screening instrument for PDs would be very helpful in the POC setting.
The Iowa Personality Disorder Screen (IPDS) is an 11 items interview-based screening instrument for identification if PD is present or absent, using diagnoses based on the Structured Interview for DSM-III-R Personality Disorders (SIDP-R) as reference [12]. The authors also tested different subsets of five to seven items in order to identify the presence of PDs. An optimal sensitivity of 92% and specificity of 79% were observed for the IPDS in their clinical sample with a PDs base rate of 46%. In a replication study, Trull et al. [13] reported an optimal sensitivity of 69% and a specificity of 91% for the IPDS in a non-clinical sample with a PDs base rate of 35%. The screening properties of a self-report version of the IPDS were evaluated by Morse et al. [14]. They found the optimal sensitivity of 80% and specificity of 55% in their sub-sample of psychiatric patients with a base rate of 84% PDs, and somewhat poorer values in their nonpsychiatric subsample with a base rate of 44%. Recently, Germans et al. [15] tested the IPDS self report version with the The Structured Clinical Interview for DSM-IV Axis II Disorder (SCID-II) [16] as reference in a Dutch sample of psychiatric outpatients (N = 195) with a base rate of 50% PDs. They reported an optimal sensitivity of 77% and specificity of 85%. The IPDS was based on eleven diagnostic criteria defined by DSM-III-R, and ten of these items were retained in DSM-IV.
In the four studies of the IPDS published so far, the sensitivity and specificity of the IPDS have shown somewhat variable results. This may be due to differences between the interview and self-rating formats, as well as small sample sizes and variable base rates of PDs. In this study from Hamar POC, we tested the IPDS selfrated version with the SCID-II as reference (gold standard). We investigated three research questions: 1) Do socio-demographic and other psychopathology influence the screening properties of the IPDS? 2) What are the sensitivities and specificities of the IPDS items alone and in combination, and 3) What are the relationships of the 11 IPDS items and the concept of PDs as studied by latent variable path analysis?

Material
Exclusion criteria for the study were age < 20 years, clinically assessed cognitive impairment, psychosis, severe somatic illness, or problems regarding Norwegian language. Due to the organisation of the POC patients referred with alcohol or drug dependence as main diagnoses were excluded, while abuse diagnoses were accepted for inclusion. Suicidality was assessed by the clinical interviews and patients with severe suicidality in need of immediate hospitalization were excluded, while lower levels of suicidality were not defined as an exclusion criterion. The therapists asked their eligible patients if they were willing to participate in the study. A stratified recruitment procedure was used in order to get a sample of 50% patients diagnosed with PDs and 50% without. From the start of inclusion February 1, 2009 we included both types of patients, however, when the proportion of non-PD patients was filled, only PDs patients were included. The inclusion period ended on May 15, 2010.

Procedure
The IPDS was part of a questionnaire filled in by the patients after they had given informed consent. The SCID-II interviews were done by their therapists who were blind to the patients' questionnaire ratings. The time between the IPDS self-rating and the SCID-II interview varied from 3 days to eight weeks. Preceding the inclusion period, the experienced therapists participated in a two day intensive SCID-II seminar covering theoretical aspects, scoring of video interviews with discussions, organized by experts from the Department for Personality Psychiatry at Ullevaal University Hospital, Oslo.

Self-rated measures
The IPDS contains 11 items which correspond to diagnostic criteria for PDs which showed the best discriminative ability in the study by Langbehn et al. [12]. These items are scored "yes" (1) or "no" (0), and an IPDS sum score ranging from 0 to 11 is calculated. When rating the items, the patients are instructed to look back to their usually self if the ways they have been in recent weeks or months are different from the way they usually are. The IPDS was translated and back-translated into Norwegian by the last author with permission from Bruce Pfohl, MD. Adaption of the IPDS into a selfadministered questionnaire did not require any special procedure. The items are given in Table 1 with their location in DSM-IV.
The Global Severity Index (GSI) is derived from the The Symptom Check-List 90 Revised [17] based on The Hopkins Symptom Checklist [18], and reflects the general symptom level of the individual in the previous seven days. The SCL-90R consists of 90 items which are rated on a five-point Likert scale (0-4) from "not at all" to "extremely". The GSI is the sum of the item scores divided with 90, and a GSI score of ≥0.85 (males) and ≥0.70 (females) separates individuals with caseness of mental distress from those without [19].
Socio-demographic variables: Relationship status was dichotomized into paired and non-paired, and basic level of education was divided into ≤12 years of education (low level) and >12 years (high level). Work status was classified into 'paid work' versus 'not in paid work'. Those who were employed full time, part time or were self-employed belonged to the former category, while others (i.e. unemployed, retired or on disability pension) belonged to the latter. We included two items from the Health Survey of Nord-Trøndelag County (http://www. ntnu.no/hunt/skjema). Self-rated health was rated by the item: "How is your current health?" with a four point Likert-scale ('bad'/'not so good'/'good'/'very good'), which was dichotomized into "good health" and "poor health" with two scale scores in each categories. General satisfaction with life was rated on a seven point Likertscale from one ('very satisfied') to seven ('very dissatisfied') and in the analyses dichotomised into "satisfied" (1-3) and "dissatisfied" (4-7).

Interview-based measures
The SCID-II diagnoses of PDs were the diagnostic references in this study. The SCID-II is a semi structured interview for the assessment of PDs according to DSM IV [20] and covers ten different PDs and in addition PD not otherwise specified (PD-NOS) [16]. We diagnosed PD-NOS if the therapist scored nine or more positive criteria on the SCID-II without reaching the threshold for any specific PDs. We used the official Norwegian SCID-II, revised version 2004.
The MINI International Neuropsychiatric Interview (MINI) is a brief structured diagnostic interview for Axis I diagnoses. The reliability and the validity of the MINI are considered to be good [21]. In this study we used the Norwegian version 5.0.0 of the MINI, revised 2007.
Global Assessment of Functioning (GAF) [20] is a commonly used rating scale for assessing patients' overall mental health reflecting psychological, social and occupational functioning. The GAF-Split version was used in this study, assessing symptom and function scores separately [22].

Statistics
The statistics analyses were performed with SPSS for Windows, version 17.0 and Partial Least Squares Path Modeling (PLSPM) with XlStat version 2010.2.03. The internal consistency of the IPDS was evaluated by Cronbach's coefficient alpha. The receiver operating curve for the IPDS score versus PDs present or absent was produced, and the area under the curve was calculated. We tested if other variables interfered with the associations between the IPDS score as independent variable and PDs present or absent as dependent variable using bivariate and multivariate logistic regression analyses. The strength of the associations was expressed as odds ratios (ORs) with 95% confidence intervals.
We constructed a hierarchical IPDS model consisting of the measured IPDS items, a set of identified first order latent variables and the IPDS as a second order variable using the key steps in PLSPM recommended by Wetzels et al. [23]. In turn the second order IPDS variable, based on the hierarchical IPDS specification, was specified as an exogenous variable in a model with PD as the endogenous variable (cf. Figure 1). In the evaluation of the PLSPM model, a t-value higher or equal to 1.96 represents significant findings (p ≤ 0.05). Hence, the significance level was set at p ≤ 0.05, and all tests beyond the PLSPM were two-sided.

Ethics
The study was approved by The National Committee for Research Ethics of Health Region South-East. All participants gave written informed consent.

Sample description
In total 156 interviews and self-ratings were completed. Individuals with attention deficit/conduct disorders were excluded (N = 11) due to lack of sufficient concentration for completion of the SCID-II interview and the questionnaire. The study sample therefore consisted of 145 patients, 61% (N = 89) women and 39% men, with mean age 37.8 (SD 11.8) years. Based on the SCID-II interview 73 patients had a total of 95 PDs, mainly belonging to cluster C (51% of the PDs diagnoses) with 18% of diagnoses in cluster A, 14% cluster B, and 18% PD-NOS (Table 2). Concerning Axis I disorders based on the MINI, mood disorders were most common (72%, N = 105) followed by anxiety disorders (23%, N = 33) ( Table 3). More than one Axis I disorder was found in 43% (N = 63) of the patients.

Factors associated with PDs diagnoses
In bivariate analyses the IPDS score was significantly associated with PDs present or absent, but so was also the GSI, GAF-S and GAF-F scores (Table 3). In multivariate analysis only the IPDS score showed a persistent significant association with PDs.

IPDS item description
The prevalence of positive criteria varied from 3% of the patients (IPDS-2) to 55% (IPDS-10) ( Table 1). The internal consistency of the IPDS was Cronbach's coefficient alpha 0.72, and the alpha values when one item was omitted varied between 0.68 and 0.72.
Correspondingly the highest specificity was shown by IPDS-2 (0.99) and the lowest by IPDS-1 and IPDS-10 (0.56). The highest positive predictive value was shown by IPDS-7 with 0.82 and the lowest was IPDS-3 with 0.03. Maximum negative predictive value was found for IPDS-2 (0.99) and minimum for IPDS-3 with 0.53. IPDS-5 had the highest proportion of PDs cases correctly classified (0.78) while the lowest proportion (0.52) was found for IPDS-2.

IPDS item combinations
We tried out the screening properties of various IPDS item combinations. If all 11 items were used, a cut-off of ≥4 positive criteria seemed to have the best case-findings properties (Table 4). We found that the various shorter versions of the IPDS introduced by Langbehn et al. [12] had similar diagnostic properties as the full scale. We also introduced a new combination consisting of the five IPDS items that had a correct classification ≥0.66 (items #1, [4][5][6]9), and found a cut-off ≥2 had good screening properties.
The receiver operating analysis of the 11 items version of the IPDS showed an area under the curve of 0.86 for the IPDS in relation to PDs present or absent, and the optimal cut-off value of ≥4, showed a sensitivity of 0.77 and specificity of 0.71.
Among the shorter versions we mention good properties of the IPDS items 4-8 and cut-off ≥ 2 with sensitivity 0.82, specificity 0.74 and area under the curve of     0.84, since this version was used in the Oslo Health Survey [24].

IPDS as a latent second order variable
We specified IPDS as a second order variable utilizing the PLSPM statistics, and the results are shown in Figure 1. As the figure shows, the measured variables loaded in general adequately on their respective first order variables. The exception from this is the item IPDS-2 with a weak (i.e. 0.27) and insignificant (i.e. tvalue 1.69) factor loading. The remaining ten items had significant loadings (i.e. t-value > 1.96) associated with their respective first order variables. Four of five first order variables loaded in general adequately on their second order variable. Histrionic PD (HST) loaded only with 0.34 and we categorize this as a relatively weak loading. All five second order loadings had however tvalues significantly > 1.96.

IPDS in relation to the various PDs
The hit rates in relation to the PDs were examined with a cut-off level ≥4 of all 11 IPDS items ( Table 2). The overall positive hit rate was 83% in relation to 95 PDs diagnoses made. The hit rate was best for PD-NOS (0.94) and cluster C disorders (0.83), but somewhat weaker for cluster A (0.76) and cluster B (0.77). The second order IPDS variable was specified as an antecedent of PDs. As Figure 1 shows, the standardized regression coefficient is 0.64 and the second order IPDS variable explains 41% variation in PDs. Tenenhaus et al. [25] have suggested a global fit measure for PLSPM: Goodness of Fit (0 <GoF < 1), defined as the geometric mean of the average communality and average R 2 (for endogenous constructs). Based on Cohen's [26] recommendation for evaluation of effect sizes, Wetzels et al. [23] recommend the following evaluation criteria for GoF values: small = 0.1, medium = 0.25, and large = 0.36. These values may serve as baseline values for validating the model specified in Figure  1. For the complete model, we obtained a GoF value of 0.53, which exceeds the cut-off value of 0.36 for large effect sizes of R 2 and allows us to conclude that our model performs well compared to the baseline values defined above.

Discussion
In this study we observed: 1) No socio-demographic or psychological variables studied by us are confounding the IPDS as a screener for PDs. 2) The sensitivity and specificity of the IPDS supported the values reported by Germans et al. [15]. 3) The PLSPM analysis of the IPDS showed satisfactory coefficients (cf. standardized regression coefficient and factor loadings) and an adequate fit value.
We found that the GSI and the GAF-S as measures of psychopathology and the GAF-F as a measure of function as well as the IPDS were significantly associated with the presence of PDs in bivariate analysis. A new finding is that only the IPDS score remained significant in the multivariate analysis. Our interpretation of these results is that psychological and functional variables do not seem to interfere to any significant extent on the IPDS as a screener for PDs.
Among the previous studies of the screening properties of the IPDS, comparisons with the study of Germans et al. [15] is the most relevant one since they also studied psychiatric outpatients and had a base rate of 50%. Our findings concerning the IPDS on sensitivity, specificity, positive and negative predictive value, and proportion correctly classified were close to those of Germans et al., and could be considered as a replication. In POC samples with a base rate of 50% for PDs, a sensitivity of 0.82 a specificity of 0.74, seem to the optimal screening ability reached by the IPDS using a brief 5 items version consisting of the IPDS items 4-8 with cutoff ≥ 2 positive items.
What do such figures mean in practical clinical work? In a sample of 100 patients admitted to the POC, 50 have PDs, when the PDs base rate is 50%. A sensitivity of 0.82 tells that 41 (50 * 0.82) of these 50 PDs patients are correctly identified, while 9 are missed as false negatives. Among the 50 patients without PDs 37 (50 * 0.74) are correctly identified without PDs, while 13 are rated as false positive for PDs. Taken together 78 of the 100 patients are correctly classified. Doing 54 (41+13) instead of 100 SCID-II interviews, will miss 9 PDs patients and have 13 negative SCID-II interviews. If this consequence of sparing 46 interviews is considered suboptimal, setting a lower cut-off with higher sensitivity will reduce the number of PDs patients missed, however at a price of performing more negative SCID-II interviews. Therefore the cut-off value of the items, as well as the item combination used should be considered when the price of false negatives and false positives are considered at the local POC.
The PLSPM analysis indicated a relatively strong relation between the IPDS and PDs, i.e. IPDS explains 41 percent of the variation in the PDs. The analysis also supports IPDS as a second order construct with five different sub dimensions. Both a set of satisfactory factor loadings and an adequate fit value support this conceptualization of IPDS. Two factor loadings were, however, relatively weak; cf. the concept histrionic PD in Figure 1 and the low coefficients of 0.27 and 0.34. This may indicate that histrionic PD does not represent a valid dimension of IPDS, but it may as well be a result of setting specific conditions. Our sample was relatively low (N = 145) and it is legitimate to ask if this is large enough for the second order PLSPM analysis. PLSPM is categorized as a "soft modeling technique" if compared with covariance based structure equation modeling technique (such as LISREL). Soft modeling means an approach where no strong assumptions (with respect to the distributions, the sample size and the measurement scale) are required [27], and we therefore conclude that our sample size is adequate for the second order PLSPM analysis. However, further research is clearly needed to address these issues.
The positive hit rate of the 11 item version of the IPDS with cut-off ≥4 varied from 76% for cluster A PDs to 94% for PD-NOS (Table 2). These findings were in accordance with those of Germans et al. [15]. When we compared the distribution of positive ratings of the 11 IPDS items, item #5 (social anxiety) and item #6 (unwilling to get involved) were significantly more common in our sample than in Germans et al., while the distribution of the other 9 items did not differ significantly. The most probable explanation is differences in the diagnostic distribution of the samples, since our sample contained significantly more cluster A and C PDs and significantly fewer cluster B PDs compared to the sample of Germans et al.
We also want to point out the considerable difference between the IPDS items concerning their proportions of correct classification. The two best items (item #5 and #6, with 78% and 74%, respectively) belonged to avoidant PD, while the two poorest ones (#2 and #3 with 52% and 55%, respectively) belonged to histrionic PD. This result confirms the finding from the path analysis, namely that the histrionic items are the weakest ones in relation to the PD concept of the IPDS.
Our results have to be considered in the light of some limitations. The reference diagnoses based on the SCID-II interviews were performed by 22 therapists, that each did from 1 to 15 interviews. In spite of the SCID-II training seminar, there is a definite risk for heterogeneity of the diagnostic practice concerning PDs. Further, we included 145 patients, which could be considered as suboptimal for the power of some of the statistical tests. The exclusion of patients referred with drug and alcohol dependence as main diagnosis might contribute to a selection bias, mostly decreasing the prevalence rate of cluster B PDs. A certain degree of consensus has emerged concerning prevalence rates of PDs in the general population [4,28]. Seeking treatment is however related to a number of clinical and demographical factors [29], and prevalence rates and distribution of PDs in clinical samples in vary considerably with methodological and diagnostic tools used in the assessments [1]. In The Rhode Island Methods to Improve Diagnostic Assessment and Services (MIDAS) project [30] patients referred to a community based POC were diagnosed with reliable and valid procedures. The project found a base rate of 45% for PDs and a 24% prevalence rate Cluster B among those having a PD. Despite our lower prevalence rate of 14% and Germans et al. [15] higher prevalence rate of 48% of Cluster B the sensitivity and specificity of IPDS in the studies are fairly comparable.
Finally, the IPDS was developed using 11 DSM-III-R criteria for PDs. 10 of these criteria were retained in DSM-IV, and one (histrionic PD criterion 7) was omitted. This omission is a minor point in our view since we test to what extent a set of criteria function as a good screening for PDs in DSM-IV. Such a task does demand that the criteria are derived from DSM-IV, although that would have been to some advantage.
Since performing SCID-II interviews are extensive time consuming a screening instrument for PDs is needed in POC due to heavy work burdens and lack of qualified SCID-II interviewers. Taking the limitations of the study into account we regard the short and feasible IPDS in Norwegian as a useful screening instrument in a busy clinical setting until the revision of the DSM-IV is completed.

Conclusions
In conclusion, our results give support to the IPDS as a useful screening instrument for PDs present or absent in the POC setting. Particularly, several of the shorter versions seem to have better case finding abilities than the full version of the IPDS.