Standardised assessment of personality – a study of validity and reliability in substance abusers
© Hesse et al; licensee BioMed Central Ltd. 2008
Received: 16 October 2007
Accepted: 25 January 2008
Published: 25 January 2008
Brief screening instruments for co-morbid personality disorders could potentially have great value in substance abuse treatment settings.
We assessed the psychometric properties of the 8-item Standardised Assessment of Personality – Abbreviated Scale (SAPAS) in a sample of 58 methadone maintenance patients.
Internal consistency was modest, but similar to the original value (alpha = 0.62), and test-retest correlation at four months follow-up was moderately encouraging for a short instrument such as this (n = 31, test retest intraclass correlation = 0.58), and change at the mean level was minimal, but marginally significant (from an average of 3.3 to 3.8, p = 0.06). Analyses of nurse ratings of patients' behaviour at the clinic showed that SAPAS was significantly correlated with nurse ratings of externalizing behaviour (r = 0.42, p = 0.001), and Global Assessment of Functioning (r = -0.36, p = 0.006), but unrelated to intoxication (r = 0.02, NS), or withdrawal (r = 0.20, NS).
There is evidence that the SAPAS is a modestly valid and relatively reliable brief screening measure of personality disorders in patients with ongoing substance abuse undergoing methadone maintenance. It can be used in situations where limited resources are available, and researchers or others wish to get an impression of the degree of personality pathology in a clinical population, as well as for screening purposes.
Personality disorders are among the most common co-morbidities among patients with substance use disorders [1, 2]. Personality disorders complicate treatment in a range of ways, elicit negative emotional reactions in clinicians , and are associated with worse outcome in treatment for substance use disorders [4–7]. Evidence is emerging that substance abuse treatment is more effective, if it addresses personality related issues, including personality disorders [8–10].
However, identifying patients with co-morbid personality disorders remain a challenge for substance abuse treatment services. Self-report inventories tend to diagnose nearly all substance abusers with personality disorders , and often traits improve rapidly with time [12, 13]. Diagnostic interviews are time-consuming, and require substantial training, and are therefore expensive for service providers to carry out. Reducing the number of patients who need a full personality disorder examination by screening out patients without personality disorders could therefore be attractive. However, inexpensive, efficient and valid measures of personality disorder have not been validated for substance abusing populations.
Only three interviewer-administered screens for personality disorder have been published. Langbehn and colleagues  have developed the Iowa Personality Disorder Screen (IPDS) to provide a mini-structured interview that the authors estimate can be completed in five minutes. The IPDS consists of eleven questions that address general personality disorder criteria as well as specific criteria. The instrument has been validated against the Structured Interview for DSM-IV Personality Disorders (SIDP-IV). The authors reported excellent sensitivity (92%) and good specificity (79%), although the validation was a somewhat circular exercise, as the IPDS items were derived from the SIDP-R. A structured patient interview for personality disorders, the Rapid Personality Assessment Schedule, requires staff training and performs moderately well as a screen for personality disorder when compared to the full version of the PAS (sensitivity = 64%, specificity = 82%) . Moran and colleagues developed the Standardised Assessment of Personality, Abbreviated Scale (SAPAS), a brief interviewer-administered screener that could be completed in less than 2 minutes, and reported good sensitivity (0.94) and specificity (0.85) in a sample of psychiatric patients with a range of different disorders .
Such a brief measure could potentially function as a screener, that is, a brief instrument that could be administered in minutes as part of an intake assessment, and could alert clinicians to the possibility of personality disorder, and refer to further assessment. Also, when doing audits of the prevalence of personality disorders in various clinical populations, or screening for research studies on personality disorders, a very brief screener might save costs.
In this study, we assessed the validity and reliability of the SAPAS in patients undergoing opioid substitution treatment in a low-threshold setting. We used nurse ratings of patients' behaviour to indicate the concurrent validity of the SAPAS scores. Given that problems with behavioural inhibition, emotional problems, and problems controlling substance use characterize personality disorders, we chose to use observations of the patients' behaviour at medication pick-up as validation of the instruments.
The setting was a public opioid substitution clinic in the City of Copenhagen. The clinic is one of four clinics providing free substitution treatment for all opioid dependent patients, who are referred for opioid substitution treatment within the clinics uptake area. The staff at the clinic are social workers, physicians, nurses, healthcare assistant, psychologist, and administrative staff. No requirement is made that patients abstain from other drug or alcohol use during treatment, but attempts are made to help patients stabilize drug and alcohol problems during treatment.
Patients who are violent or threatening to an extent that is not manageable within the clinic are referred to a special opioid substitution bus run by a private company. Thus, patients are practically never discharged from treatment or detoxified against their wish. Pharmacotherapy works with flexible and individualized dosing in close collaboration between physician and patient. There is no official maximum dose of methadone or buprenorphine, although generally physicians try to keep doses under 150 millilitres per day. All treatment is free, and is available for patients with legal residence in Denmark who either live in the City of Copenhagen, or are homeless and mainly stay in the Copenhagen area. Tapering of medication is only carried out on patient demand, and is generally neither encouraged nor discouraged. Caseworkers are assigned to clients, and provide counselling and typical case management services (e.g., assessment of problems and needs, planning and setting goals, linking with other services, advocating the patients case with external partners), but are not responsible for monitoring patients' progress, as patients are expected to come for help when they need it.
Patients are offered access to drop-in centres that are placed in various areas of Copenhagen, but no drop-in centre or café is available at the clinics.
Nurses at the Inner City centre do health check-ups with patients, during which they assess needs for medical treatment, plan for treatment, contraception, hepatitis testing, HIV-testing, hepatitis immunization, and the clinic physician can refer patients to testing or treatment at community hospitals.
Patients were approached in the clinic by the first author, and asked whether they were willing to consider participating in a research study concerning their treatment. Following this, they were given written and verbal information about the purposes of the study. The main purpose of the study was a quasi-experimental study of teaching knowledge about personality disorders to case-workers. A secondary purpose was to test psychometric properties of instruments. At follow-up, patients were followed up by in-person (n = 29) or telephone interviews (n = 2). Different interviewers at both points interviewed a total of 23 subjects, and the same interviewer interviewed 8 patients.
At the follow-up interview, approximately 4 months after the first assessment, patients were again administered the SAPAS (see below). When the SAPAS was administered the second time, patients were instructed not to try to remember what they had said the first time, and were told that we wanted to know "if people see themselves in the same way at two different points in time".
At baseline, patients were administered a very brief 5 minute interview.
The Standardised Assessment of Personality – Abbreviated Scale (SAPAS)
Reliability of the SAPAS
Four months test-retest reliability
Kappa with different interviewers
Alpha if deleted
Difficulty making and keeping friends
Usually a loner
Normally loses temper easily
Normally a worrier
Nurse ratings of patient functioning
Nurses rated patients on medication pick-ups using four rating scales developed for the study. The four rating scales represented intoxication (0–4, with 0 representing no intoxication and 4 severe intoxication), withdrawal (0–4, with 0 representing no withdrawal and 4 severe withdrawal), externalizing behaviour (0–4, with 0 representing being polite and friendly and 4 severe aggressiveness), and global assessment of functioning (GAF, 0–100, with 0 representing the extremely psychiatrically ill patient, and 100 the extremely psychiatrically healthy person (adapted from Bodlund ). All rating scales contained anchor points.
We tested inter-rater agreement by having a different rater rate all patients who came to pick up medication on one day (n = 26). Inter-rater agreement was satisfactory for single item instruments (range for intra-class correlation: 0.39 to 0.53). We also had the treating physician at the clinic rate a total of 26 patients using the clinical global impression scale for substance use problems and externalizing behaviour, and rate the GAF, whilst being blind to the nurses ratings. Analyses showed moderate associations between physician ratings and nurses ratings (clinical global impression of current substance use problems strongly associated with nurse-rated withdrawal and intoxication: rho = 0.48 and 0.59, p < 0.02; clinical global impression of externalizing behaviour strongly related with nurse-rated externalizing behaviours: rho = 0.41, p = 0.04; physician rated GAF strongly related with nurse rated GAF: rho = 0.62, p < 0.001)).
While the nurses' knowledge of patients varied, they had health check-ups with patients, and regular contact and did some case management with patients, especially in relationship to patients' medical needs.
The nurses and other staff members were kept blind to the results of the SAPAS and other data from the interviews. A total of 5 different nurses rated the patients. As pick-up frequency varied from five days per week to once per two weeks, the number of ratings differed substantially between patients.
A total of 67 patients were approached, and 87% (58) gave consent to participate in the study. Seven declined, and 2 were deemed unable to give consent due to a mixture of language problems and psychiatric problems.
The sample of 58 patients consisted of 42 men and 16 women. The mean age at consent was 39.9 years (SD = 8.97, range = 23 to 62).
Of the 58 patients who gave consent, 48 were in methadone maintenance treatment, and 10 were in Buprenorphine maintenance treatment. Two accepted participation in the study, but were unwilling to be interviewed. Poly-pharmacy was common in the clinic, with 42% receiving benzodiazepines, 22% receiving antidepressants, and 7% receiving antipsychotics. In total, 65% received other medications than opioid substitution. Methadone-doses were 0–40 mg for 3 patients, 41–80 for 13, and over 80 mg for 32.
Patients were asked about their desire to leave opioid substitution maintenance. No desire to quit substitution treatment was reported by 28%, desire to do so some day in an unknown future by 34%, desire to do so within the next 6 months by 26%, and desire to quit substitution as soon as possible by 7%.
A total of 58% had a personality disorder diagnosis according to the SAPAS . However, the SAPAS does not show which personality disorder patients have, and no further assessment of this question was made. The mean proportion of time since the age of 15 that patients reported having worked was 26%. Only two patients were employed at the time of interview.
The reliability of the SAPAS was estimated using Cronbach's alpha, and test-retest reliability of the full scale was estimated using intra-class correlation. For each item, we calculated Cohen's κ as a chance-corrected measure of agreement.
Criterion validity of the SAPAS was estimated using staff ratings of functioning, including withdrawal, intoxication, externalizing behaviour and GAF. We used the mean of all scores over all observations in a 6-month period at the clinic to assess validity. We calculated regular Spearman correlation coefficients, because there were strong deviations from normality in the rating scales (externalizing behaviour and withdrawal had very strong skew.
Reliability of the SAPAS
The results of the reliability analyses are shown in Table 1.
The internal consistency of the SAPAS was slightly lower in this sample than in the psychiatric sample (α = 0.62). As in the original study by Moran and colleagues, the impulsivity item reduced reliability slightly. The test-retest reliability of individual items ranged from 0.26 to 0.58. When omitting subjects who were re-interviewed by the same interviewer at both points, the test-retest reliability was slightly decreased for 6 of eight items. For the total SAPAS, the test-retest intraclass correlation was 0.58 (asymptotic z = 2.81, p = 0.005). When omitting subjects who were interviewed by the same interviewer, the intraclass correlation was reduced to 0.52 (asymptotic z = 2.19, p = 0.02).
The mean SAPAS score increased from baseline to follow-up. The mean at baseline was 3.3 (standard deviation = 2.0), and the mean at follow-up was 3.8 (standard deviation = 1.7). The increase was marginally significant (t = -1.99, p = 0.06).
Criterion validity of the SAPAS
In order to assess the validity of the SAPAS, we used the nurse ratings of functioning. As pick-up frequency varied between individuals, the number of ratings that could be included varied from four to 48 with a mean of 20.1.
Convergent validity of the SAPAS
Correlation with SAPAS (Spearman Rho)
Global Assessment of Functioning
Nurse ratings by SAPAS scores
Scores on the SAPAS
Global Assessment of Functioning
We conducted a multiple regression analysis to control for age and gender, after rank-order transforming the rating scales. The results were similar: Externalizing behaviour (beta = 0.37, t(52) = 2.94, p = 0.004), and GAF (beta = -0.29, t = 2.19, p = 0.03) remained significantly associated with SAPAS.
As the number of observations differed due to variability in pick-up frequency, we also conducted regression analyses controlling for the number of observations that had been recorded in the clinic. Again, the results were highly similar: again, externalizing behaviour (beta = 0.43, t(53) = 3.49, p = 0.001), and GAF (beta = -0.29, t(53) = -2.52, p = 0.01) remained substantially correlated with SAPAS score.
Other correlates of the SAPAS
No gender difference was found on the SAPAS (men mean: 3.1, SD = 1.9; women mean: 3.7, SD = 2.3, t(54) = -0.99, p = 0.33), and the correlation between age and SAPAS was non-significant (Pearson r = 0.13, NS). Also, no difference was found between patients who were receiving methadone substitution and patients who were receiving buprenorphine substitution (methadone: 2.28, SD = 2.1; buprenorphine: 3.2, SD = 1.7; t = 0.07, p = 0.94), and no association was found between methadone dose and SAPAS score among those in methadone maintenance treatment (r = 0.11, p = 0.46, n = 46).
Total SAPAS score was negatively correlated with the proportion of time since age 15 that had been spent working without interruption (Spearman rho = -0.31, p = 0.02), but unrelated to degree of desire to quit methadone (Spearman rho = -0.07, p = 0.60).
The findings of this study supported the validity of the SAPAS, and gave indications of modest longer-term stability. Even when different interviewers interviewed the same patients 4 months ago, the stability coefficient was reasonable for such a short measure.
Patients who reported more features of personality disorders, were rated as displaying more externalizing behaviour was observed by the nurses (unsatisfied, complaining, angry), and as having lower scores on the Global Assessment of Functioning. In contrast with our expectations, we found no strong indication of higher ongoing substance use during treatment with personality disorders. However, in this clinic, most patients use drugs and alcohol to some extent, and stable abstinence is rare.
We did not use another interview or self-report inventory to assess the validity of the SAPAS. Obviously, this means that we do not know how many of these patients would have been diagnosed with another instrument. Although such a replication of the original findings of Moran and colleagues could potentially be important, this study adds new light to the validity of this very brief instrument: the SAPAS identifies patients with problems that are highly indicative of personality disorder. It identifies patients who show more aggressive behaviour, and who appear to be functioning less well in terms of their mental health.
The SAPAS was constructed based on primary care patients, and its applicability was originally tested on psychiatric patients . Its items includes item that reflect content related primarily to cluster A personality (being a loner), one item relating to both cluster A and B (trusts people, reversed scored), two items relating to cluster B disorders (impulsive and problems controlling temper), and four items relating primarily to cluster C disorders (difficulty getting and keeping friends; usually dependent; usually a worrier; perfectionist), and thus has a reasonably broad coverage of the various personality disorders in the DSM-IV.
In terms of clinical use, the SAPAS may alert clinicians to the possibility of personality disorder in a patient, but will obviously never serve as a substitute for a full assessment of personality disorders.
Several strengths and limitations deserve comment
The first strength of the study is validation through independent and blinded assessment by multiple observers over an extended period of time, in that several different nurses rated the same patients every time they came to pick up medications. Patients may fluctuate from day to day, from week to week, depending on events that occur in their life, and nurses may differ in their views of particular patients. However, when observations are averaged over several observers and several occasions over a long period of time, these sources of error is levelled out.
Limitations include a limited sample size, and the use of a cross-sectional sample, rather than a consecutive sample. Also, the validity correlations are likely to be attenuated as a result of the relative unreliability of the SAPAS.
In conclusion, there is evidence that this very brief measure gives some indication of the presence of personality disorder. It may be useful as an indicator of the presence of personality disorder, either for screening purposes in a clinical setting, or as a tool in a survey.
This study was made possible by the support of head of the clinic Lau H. Petersen, head of the Centre John Bjerregaard, M.D. Leif Skauge, and the work of several nurses and caretakers who participated in the Data collection. The study is part of the research program of the Centre for Alcohol and Drug Research at Aarhus University, and has received no extra-mural support.
- Grant BF, Stinson FS, Dawson DA, Chou PS, June Ruan W, Pickering R: Co-occurrence of 12-month alcohol and drug use disorders and personality disorders in the United States. Results from the National Epidemiologic Survey of Alcohol and Related Conditions. Archives of General Psychiatry. 2004, 61: 361-368. 10.1001/archpsyc.61.4.361.View ArticlePubMedGoogle Scholar
- Verheul R: Co-morbidity of personality disorders in individuals with substance use disorders. European Psychiatry. 2001, 16: 274-282. 10.1016/S0924-9338(01)00578-8.View ArticlePubMedGoogle Scholar
- Betan E, Heim AK, Zittel Conklin C, Westen D: Countertransference phenomena and personality pathology in clinical practice: an empirical investigation. American Journal of Psychiatry. 2005, 162: 890-898. 10.1176/appi.ajp.162.5.890.View ArticlePubMedGoogle Scholar
- Grella CE, Joshi V, Hser YI: Followup of cocaine-dependent men and women with antisocial personality disorder. Journal of Substance Abuse Treatment. 2003, 25: 155-164. 10.1016/S0740-5472(03)00127-2.View ArticlePubMedGoogle Scholar
- Galen LW, Brower KJ, Gillespie BW, Zucker RA: Sociopathy, gender, and treatment outcome among outpatient substance abusers. Drug Alcohol Depend. 2000, 61: 23-33. 10.1016/S0376-8716(00)00125-3.View ArticlePubMedGoogle Scholar
- Kranzler HR, Tennen H, Babor TF, Kadden RM, Rounsaville BJ: Validity of the longitudinal, expert, all data procedure for psychiatric diagnosis in patients with psychoactive substance use disorders. Drug Alcohol Depend. 1997, 45: 93-104. 10.1016/S0376-8716(97)01349-5.View ArticlePubMedGoogle Scholar
- Kokkevi A, Stefanis N, Anastasopoulou E, Kostogianni C: Personality disorders in drug abusers: prevalence and their association with AXIS I disorders as predictors of treatment retention. Addictive Behaviors. 1998, 23: 841-853. 10.1016/S0306-4603(98)00071-9.View ArticlePubMedGoogle Scholar
- Conrod PJ, Stewart SH, Pihl RO, Cote S, Fontaine V, Dongier M: Efficacy of brief coping skills interventions that match different personality profiles of female substance abusers. Psychology of Addictive Behaviors. 2000, 14: 231-242. 10.1037/0893-164X.14.3.231.View ArticlePubMedGoogle Scholar
- Ball SA, Cobb-Richardson P, Connolly AJ, Bujosa CT, O'Neall T W: Substance abuse and personality disorders in homeless drop-in center clients: symptom severity and psychotherapy retention in a randomized clinical trial. Comprehensive Psychiatry. 2005, 46: 371-379. 10.1016/j.comppsych.2004.11.003.View ArticlePubMedGoogle Scholar
- Nielsen P, Røjskjær S, Hesse M: Personality-guided treatment for alcohol dependence: A quasi-randomized experiment. American Journal on Addictions. 2007, 16: 357-364. 10.1080/10550490701525640.View ArticlePubMedGoogle Scholar
- Craig RJ: Prevalence of Personality Disorders among Cocaine and Heroin Addicts. Substance Abuse. 2000, 21: 87-94. 10.1023/A:1007711112790.PubMedGoogle Scholar
- Hesse M, Nielsen P, Røjskjær S: Stability and Change in Millon Clinical Multiaxial Inventory II Personality Disorder Scores in Treated Alcohol Dependent Subjects: Relationship to Post-treatment Abstinence. International Journal of Mental Health and Addiction. 2007Google Scholar
- Calsyn DA, Wells EA, Fleming C, Saxon AJ: Changes in Millon Clinical Multiaxial Inventory scores among opiate addicts as a function of retention in methadone maintenance treatment and recent drug use. American Journal of Drug and Alcohol Abuse. 2000, 26: 297-309. 10.1081/ADA-100100606.View ArticlePubMedGoogle Scholar
- Langbehn DR, Pfohl BM, Reynolds S, Clark LA, Battaglia M, Bellodi L, Cadoret R, Grove W, Pilkonis P, Links P: The Iowa Personality Disorder Screen: development and preliminary validation of a brief screening interview. Journal of Personality Disorder. 1999, 13: 75-89.View ArticleGoogle Scholar
- Van Horn E, Manley C, Leddy D, Cicchetti D, Tyrer P: Problems in developing an instrument for the rapid assessment of personality status. European Psychiatry. 2000, 15 Suppl 1: 29-33. 10.1016/S0924-9338(00)90497-8.View ArticlePubMedGoogle Scholar
- Moran P, Leese M, Lee T, Walters P, Thornicroft G, Mann A: Standardised Assessment of Personality - Abbreviated Scale (SAPAS): preliminary validation of a brief screen for personality disorder. British Journal of Psychiatry. 2003, 183: 228-232. 10.1192/bjp.183.3.228.View ArticlePubMedGoogle Scholar
- Bodlund O, Kullgren G, Ekselius L, Lindstrom E, von Knorring L: Axis V--Global Assessment of Functioning Scale. Evaluation of a self-report version. Acta Psychiatrica Scandinavica. 1994, 90: 342-347. 10.1111/j.1600-0447.1994.tb01604.x.View ArticlePubMedGoogle Scholar
- The pre-publication history for this paper can be accessed here:http://www.biomedcentral.com/1471-244X/8/7/prepub
This article is published under license to BioMed Central Ltd. This is an Open Access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.