The group of participants comprised a reference sample from the general population (ROM reference group) and a ROM sample of psychiatric outpatients (ROM patient group), as previously described in detail .
The ROM reference group consisted of 1294 participants aged 18 to 65 years (62.8% females; mean age=40.3 years; SD=12.6) from the ‘Leiden Routine Outcome Monitoring Study’. The study design, objectives, and methods have been described elsewhere [26, 27]. Participants were randomly selected from registration systems of eight general practitioners (GPs) in the province South-Holland, the Netherlands. In the Netherlands, 99.9% of the general population is registered with a GP . Therefore, non-consulting GP patients are a very good representation of the Dutch general population. The ROM reference group was stratified for gender, age, and urbanization-level (62.3% urban), to make the group demographically comparable to the ROM patient group. Invitations for this study were sent to 4840 persons; 1283 could not be contacted and 67 were not included because of time constraints. Of the remaining 3490 potential participants, 1302 were assessed and 1294 generated complete datasets, resulting in a response rate of 37.1%.
The ROM patient group consisted of 5269 psychiatric outpatients, aged 18 to 65 years (64.6% females; mean age=37.7, SD=12.0). They were diagnosed with and treated for one or more MAS disorders in the Leiden University Medical Center (LUMC) Department of Psychiatry or in the Rivierduinen Psychiatric Institute, the regional provider of specialized mental health care.
Procedures for the web-based ROM program of the LUMC Department of Psychiatry are described elsewhere [27, 29]. The participants in the ROM reference group were assessed in a similar way to the ROM patient group. Subjects from the ROM reference group completed the self-report instruments BSI, MASQ-D30, and SF-36, and due to time constraints, a random sample of 50% completed the DAPP-SF . The BSI, MASQ-D30, and SF-36 were completed by all 5269 subjects from the ROM patient group, while 234 (4.6%) did not complete the DAPP-SF, again due to time constraints. To facilitate diagnoses of psychopathology according to the DSM-IV, the proceduret for the two groups included a standardized diagnostic interview (i.e., the Mini-International Neuropsychiatric Interview plus (MINI-Plus 5.0.0.) [30, 31]). The Medical Ethical Committee of the LUMC approved the general study protocol regarding ROM, in which ROM was organized as part of the treatment process for patients. It involved a comprehensive protocol (titled “Psychiatric Academic Registration Leiden database”) which safeguarded the anonymity of patients and participants and ensured proper handling of the ROM data. All patients gave permission for the use of their ROM data for scientific purposes (written informed consent for this study was not required). In addition, participants of the ROM reference group (non-patients) signed informed consent for the purpose of this study.
The BSI, a short version of the Symptom Checklist (SCL-90) , measures psychopathological symptoms. The BSI consists of 53 items divided into 9 subscales: Somatization (SOM), Obsessive-Compulsive (O-C), Interpersonal Sensitivity (I-S), Depression (DEP), Anxiety (ANX), Hostility (HOS), Phobic Anxiety (PHOB), Paranoid Ideation (PAR), and Psychoticism (PSY). Item scores range from 0 (“not-at-all”) to 4 (“extremely”). The subscale and total scores are calculated as an average of the relevant items, with higher scores indicating more severe psychopathology.
The MASQ-D30 measures the dimensions of Clark and Watson’s tripartite model, covering both shared and distinct symptoms of depression and anxiety [13, 14]. The MASQ-D30 consists of 30 items, divided into three subscales: Negative Affect (NA), associated with both depression and anxiety; lack of Positive Affect (PA), associated with depressive moods; and Somatic Arousal (SA), associated with anxiety. The items are rated on a 5-point Likert scale, with scores ranging from 1 (“not at all”) to 5 (“extremely”). Subscale scores are calculated as the sum of the relevant items, ranging from 10 to 50, with higher scores indicating more severe psychopathology.
The SF-36, derived from the Rand Medical Outcome Study (MOS) [15, 16], measures functional health status and well-being. It can be used as a population-based assessment of quality of life. The SF-36 consists of 36 items divided into eight subscales: Physical Functioning, Role limitations due to Physical health problems (Role-Physical), Bodily Pain, Social Functioning, General Mental Health (Mental Health), Role limitations due to Emotional problems (Role-Emotional), Vitality, General Health Perceptions (General Health) and a question about perceived change of health during the last year (Health Transition). Subscale scores are calculated as the sum of the relevant items, ranging from 0 to 100, with higher scores indicating better functioning.
The DAPP-SF, the short form of the Dimensional Assessment of Personality Pathology – Basic Questionnaire (DAPP-BQ) [17, 18], measures personality pathology. It consists of 136 items divided into 18 subscales: Submissiveness, Cognitive Distortion, Identity Problems, Affective Lability, Stimulus Seeking, Compulsivity, Restricted Expression, Callousness, Oppositionality, Intimacy Problems, Rejection, Anxiousness, Conduct Problems, Suspiciousness, Social Avoidance, Narcissism, Insecure Attachment, and Self-harm. Item scores range between 1 (“very unlike me”) and 5 (“very like me”). Subscale scores are calculated as an average of the relevant items, ranging from 1 to 5, with higher scores indicating more maladaptive personality traits.
The Dutch version of the Mini-International Neuropsychiatric Interview plus (MINIplus 5.0.0.) [30, 31] was used to establish the presence of Axis I diagnoses according to the Diagnostic and Statistical Manual of Mental Disorders (DSM-IV). This standardized diagnostic interview comprises 23 modules for mood, anxiety, psychotic, somatoform, and eating disorders.
Means, standard deviations (SDs), and percentile scores were calculated for the two samples separately, while ROC analyses were performed in the combined groups. In both samples, subjects with 1 or more missing values per subscale were excluded. This allowed us to conduct a robust evaluation of the use of the instruments. The occurrence of missing values is not completely random, and it depends on unobserved predictors. Therefore we decided to use an almost complete-case analysis, as bias due to missing values was likely to be small due to the small percentage (i.e., 0.01%) of cases that needed to be excluded.
A descriptive analysis of sociodemographic and psychopathological variables was performed, using percentages in the case of categorical variables and means and SDs for the continuous variables. Internal consistency was assessed using Cronbach’s alpha, with >0.70 indicating adequate internal consistency. ROC analyses provided cut-off scores, indicating an optimal discrimination threshold between ‘healthy’ (reference population) and ‘psychiatrically ill’ (psychiatric outpatients). The cut-off was chosen at the value representing equal sensitivity and specificity, since this is the point that yields the best compromise between specificity and sensitivity, with the lowest number of false results (false positive plus false negative). The areas under the ROC curve (AUCs) were calculated to indicate the discriminatory power of the instrument (sub)scales, where AUCs over 0.75 were considered clinically useful with 0.85 showing moderate discriminatory power and 0.95 very high discriminatory power . Furthermore, means and SDs were calculated, together with 5th, 25th, 50th, 75th and 95th percentile scores. When instruments merely assess the level of dysfunctionality, and the discriminative power to detect the level of ‘health’ or normal functionality is limited (i.e., no persons can be earmarked as ‘abnormally healthy or good functioning’), the lowest 2.5% is irrelevant. Therefore, the top 5% (or lower 5% in case of SF-36 subscales) was chosen as representing ‘abnormal’. Reference values were also presented for 4 subgroups: young women (aged 18–40 years), older women (aged 41–65 years), young men (aged 18–40 years), and older men (aged 41–65 years). SPSS for Windows version 17.0 (SPSS Inc., Chicago, IL, USA) was used for data analysis. To test our decision not to exclude those individuals in the ROM reference group with a current psychiatric diagnosis, we performed a sensitivity analysis.