Diagnosing comorbidity in psychiatric hospital: challenging the validity of administrative registers
© Øiesvold et al.; licensee BioMed Central Ltd. 2013
Received: 29 February 2012
Accepted: 3 January 2013
Published: 8 January 2013
This study will explore the validity of psychiatric diagnoses in administrative registers with special emphasis on comorbid anxiety and substance use disorders.
All new patients admitted to psychiatric hospital in northern Norway during one year were asked to participate. Of 477 patients found eligible, 272 gave their informed consent. 250 patients (52%) with hospital diagnoses comprised the study sample. Expert diagnoses were given on the basis of a structured diagnostic interview (M.I.N.I.PLUS) together with retrospective checking of the records. The hospital diagnoses were blind to the expert. The agreement between the expert’s and the clinicians’ diagnoses was estimated using Cohen’s kappa statistics.
The expert gave a mean of 3.4 diagnoses per patient, the clinicians gave 1.4. The agreement ranged from poor to good (schizophrenia). For anxiety disorders (F40-41) the agreement is poor (kappa = 0.12). While the expert gave an anxiety disorder diagnosis to 122 patients, the clinicians only gave it to 17. The agreement is fair concerning substance use disorders (F10-19) (kappa = 0.27). Only two out of 76 patients with concurrent anxiety and substance use disorders were identified by the clinicians.
The validity of administrative registers in psychiatry seems dubious for research purposes and even for administrative and clinical purposes. The diagnostic process in the clinic should be more structured and treatment guidelines should include comorbidity.
KeywordsComorbidity Psychiatric hospital Psychiatric case register
Psychiatric comorbidity is a prevalent phenomenon and remains a challenge for the effective delivery of mental health services. Recent community surveys show that among those with a psychiatric disorder the lifetime prevalence of more than one diagnosis is about 50% . The highest rates of comorbidity are observed between anxiety and affective disorders , and affective, anxiety and substance use disorder often occur together [2, 3]. The presence of comorbid disorders is associated with a significantly higher rate of help seeking [4–6].
To what extent psychiatric case registers or administrative registers reflect this comorbidity is not known. Since the 1960s, psychiatric case registers have been regarded as important epidemiological research tools for estimating treated incidence, prevalence and patterns of care . With the development of new and better information and communication technologies, their importance is expected to increase [8, 9]. Much of the utility of a psychiatric case register, however, will depend on the validity of the psychiatric diagnoses . Byrne et al.  in their review conclude that relatively little high-quality work exists into systematically measuring the diagnostic data validity of registers for research purposes. Almost no studies (1 out of 14) performed anything else than case note reviews to assess validity. Only two of the studies reviewed stated that the register diagnoses were blinded to the researchers and inter-rater reliability testing was only performed in three of the studies.
Thus, there is a need for studies using more stringent methodological approaches to estimate the validity of case register diagnoses. In our study a structured diagnostic interview was performed on all new patients consecutively admitted to psychiatric hospital, comparing these diagnoses with those given by the clinicians. In a previous paper we have focused on the underdiagnosing of bipolar disorder . In this paper we will present the results with regard to comorbid anxiety and substance use disorders.
Design and participants
The North-Norwegian study on first-time admitted patients to psychiatric hospital (FINN-study) is a prospective cohort study on treated incidence, utilization, and outcome in a one-year period and a 12-month follow-up period. The University Hospital in Northern Norway in Tromsø, and Nordland Hospital in Bodø, participated. All admissions to psychiatric hospitals in a region with a population of about 500 000 people are administered by these two hospitals. There are 14 community mental health centres in the region. The psychiatric services in Northern Norway are fully described elsewhere .
Included in the study were patients between 18 and 65 years of age who had no previous admissions to the participating hospitals and who gave written informed consent to participate. Exclusion criteria were: Lack of language competency and cognitive impairment such as dementia, serious mental retardation or other mental incapacities preventing the individual from giving an informed written consent. Further, The Regional Ethics Committee for Medical Research required that a patient be given at least 24 hrs after admission to consider participation, and hence patients who were discharged less than 3 days after admission had to be excluded. Thus comprehensive in/out interviews were unfeasible for short-stay patients. Of 674 first-time admitted patients, 477 patients were found eligible for participation. 272 patients gave their informed consent, and of these 250 patients (52%) with hospital diagnoses comprised the study sample.
Diagnoses were assessed by means of the Mini International Neuropsychiatric Interview PLUS (M.I.N.I.PLUS)  Norwegian version 5.0.0. . M.I.N.I. was developed in Europe and USA as a short diagnostic instrument for generating DSM-IV criteria diagnoses convertible to ICD.10 diagnoses . The M.I.N.I.PLUS is an extended version of the M.I.N.I. that includes information on specific phobias and has an expanded psychosis module. The M.I.N.I.PLUS is built up of 15 modules corresponding to diagnostic categories and collects information along 23 axis-I problem areas in relation to past and current symptoms. The interviews were carried out by psychiatric nurses, psychologists, graduate students in psychology, a resident doctor and a psychiatrist. Except for the two students, all had extensive clinical experience and none had therapeutic or other relations to the patients. The interviewers underwent systematic training and consecutive reliability checks using videotaped interviews. The interview was performed as soon as possible after admission when the patient was found eligible to participate in an interview and had given written consent.
An experienced psychologist (I. Skre), who has studied the validity and reliability of psychiatric diagnoses during two decades [17, 18], will in the following will be referred to as the expert. She was not employed at the participating hospitals and she determined the diagnoses on the basis of the M.I.N.I. PLUS interviews and retrospective inspection of the patients’ records. The expert was blind to the hospital diagnoses. First, the M.I.N.I.PLUS schedule, including notes made by the interviewer, was reviewed and scored according to the ICD-10 criteria as they appear in ICD-10 Diagnostic Criteria for Research . In cases where the information given in the interview was meagre, lacking or contradictory, additional information about the patient was sought from the hospital records: (1) the referral letter applying for admission, which accompanies all admissions to psychiatric hospitals in Norway, (2) the notes written by the receiving medical doctor at the hospital, and (3) when involving an involuntary admission, the notes written by the specialist in psychiatry/psychology who did the formal evaluation. In order to keep the expert blind to the hospital diagnoses and the referring physician’s tentative diagnosis, the information was extracted from the patient’s file and read aloud to the expert by an assistant. The assistant was instructed to omit all material concerning diagnostic evaluations. The following information was extracted from these documents: (1) the symptoms and behaviour of the patient in the days and hours immediately prior to hospitalization, (2) the symptoms and behaviours observed and described by the receiving medical doctor and/ or the specialist in psychiatry/psychology at the hospital. In some cases, when suspecting an organic mental disorder, any documentation of results from brain imaging and neuropsychological tests were used.
In accordance with the ICD-10, a diagnostic hierarchy was employed only when exclusion criteria were explicitly given in the diagnostic manual. When assigning more than one diagnosis, the diagnoses were listed in the following order: The first or primary diagnosis was always the disorder from which the behaviours or symptoms stemmed which had resulted in hospitalisation. Following the main diagnosis were additional disorders diagnosed in the patient, most often anxiety or somatoform disorders. Finally, diagnoses for harmful use of or dependence on psychoactive substances were assigned.
Hospital clinicians are obliged to use the ICD-10 criteria and to make a diagnostic evaluation in the discharge letter which is routinely sent to the patient’s GP. The hospital diagnosis is based on clinical interviews and observations made during the hospital stay. Interviews with relatives may be used as well as rating scales and structured interviews, but this is uncommon. The clinician’s diagnoses are given in the discharge letter from the hospital. Usually, what is considered the main disorder causing hospitalization is entered first, as the primary diagnosis, and additional diagnoses, if assigned, are entered subsequently. The discharge letter is the hospital’s summary of diagnostic evaluations, symptoms and behaviours which have been the focus for treatment, and which the hospital finds it important to communicate to the patient’s GP, or other primary or secondary case manager. Thus any clinical significant symptoms observed and addressed during hospital stay should be mentioned at discharge.
The Regional Ethics Committee of Northern Norway approved the study.
Descriptive statistics were used to present sample characteristics and the frequencies of the different diagnoses given by the clinicians as well as by the expert. Kruskall-Wallis and chi-square statistics were used to assess possible bias in the study sample. Cohen’s kappa (к) was used to estimate degree of agreement between expert and clinical diagnoses. According to the guidelines of Landis and Koch , a kappa agreement < .20 is poor, .21 - .40 is fair, .41 - .60 is moderate, .61-.80 is good and >.81 is almost perfect. SPSS 16.00 was used in the statistical analyses.
The study sample
Characteristics and possible biases (Kruskal-Wallis & Chi-square (X 2 )) of the sample (N=250)
X 2 /p
N = 197
N = 227
41.4 (sd 21.0)
44.2 ( sd 19.9)
40.4 (sd 15.2)
> 39 yrs
X 2 = 26.64, p =.000
40 - 59
X2 = 92.34, p = .000
National insurance benefits
X2 = 12.81, p = .012
X2 = 19.89, p = .000
Length of first stay
11.6 (sd 19.6)
31.0 (sd 37.8)
Kruskal Wallis p < .001
Degree of agreement between expert’s diagnoses and clinicians’ diagnoses
The expert gave a mean of 3.4 diagnoses while the clinicians only gave 1.4. The most common diagnoses given by the expert were drug and alcohol abuse (57%), major depression without psychosis (49%) and anxiety disorder (48%). Only major depression without psychosis was common among the diagnoses made by clinicians (51%). The agreement, as estimated by the kappa statistics, between the expert and the clinicians ranges from poor to good for the different diagnostic groups. Generally the agreement was better when comparing all diagnoses given opposed to comparing only the first diagnosis given .
Frequency and degree of agreement (Cohens kappa/κ) between expert and clinician with regard to substance use disorder and anxiety disorder; all diagnoses set and primary diagnoses only (N=250)
Substance use disorder (F10-19)
Anxiety disorder (F40-41)
Anxiety disorder diagnoses (F40-41) set by the clinicians (N=122)
The same diagnosis as the expert
Social phobia F40.1
Specific phobias F40.2
Panic disorder F41.0
Concurrent comorbidity of both anxiety disorder (F40-41) and substance use disorder (F10-19)
Concurrent comorbidity of anxiety (F40-41) and substance use disorder (F10-19) diagnosed by the clinicians (N=76)
F10-19 and F40-41
Affective disorder F30-39
Bipolar disorder F31
Sciz. spectrum F20(1) & F25(4)
This study casts doubt upon the validity of administrative registers concerning comorbidity of anxiety and substance use disorder. To diagnose anxiety in particular, seems to be neglected; the clinicians gave such a diagnosis to only 17% of those identified with an anxiety disorder (F40-41) by the expert. Concerning substance use disorders (F10-19) the picture is somewhat better with 31% identified by the clinicians. Regarding concurrent comorbidity of anxiety and substance use disorder, this was to a very small degree identified by the clinicians; only in two of 76 patients. Almost all these patients had a primary affective disorder.
Not many studies have looked at the quality of administrative registers with regard to comorbidity. Only one of the studies reviewed by Byrne et al.  did so, finding that comorbidity was under-reported as in our study . In an investigation of the Danish Psychiatric Register, Hansen et al.  reported underdiagnosing of substance use disorder by nearly 50%. Further, several studies reported that comorbid anxiety are being missed in clinical practice [23, 24].
The question is raised whether comorbidity actually is an artefact of current diagnostic systems . There is a marked symptom overlap between at first hand affective and anxiety disorders which could be a source of diagnostic unreliability and a dimensional classification system based on shared features of anxiety and mood disorders have been proposed . In clinical practice a dimensional approach to treatment is common and a formal diagnosis is as a rule not given until discharge. It could be that the clinicians are aware of comorbid anxiety and substance use without diagnosing it formally . One could imagine a kind of ethical considerations of not stigmatizing the patient with too many diagnoses giving only those who primarily brought the patient into hospital. However, Zimmerman and Chelminski  report underrecognition of anxiety disorders, for which the patients want treatment, in psychiatric outpatients with a principal diagnosis of major depression.
There could be several other reasons for this underdiagnosing of comorbidity. Treatment guidelines commonly only relate to specific diagnostic groups like affective disorders or anxiety disorders, and not to comorbid conditions . This could make the clinicians less aware of comorbid conditions. The relation between co-occuring substance and anxiety disorders has not received much attention and is generally poorly understood. The diagnostic challenge in relation to individuals with current substance use disorders has been to devise diagnostic criteria and measurement techniques that differentiate between intoxication and withdrawal symptoms and the symptoms of psychiatric disorders. Many of the symptoms of intoxication and withdrawal from alcohol and other substances resemble the symptoms of mood and anxiety disorders . Diagnostician using a structured interview routinely asks control questions that delineates between transient anxiety caused by substance withdrawal, and chronic anxiety symptoms. The results could also be due to the general phenomenon that clinicians rely on a limited number of heuristic principles which in some instances may lead to severe and systematic errors [30, 31]. We believe clinicians are more apt to use a heuristic top-down approach when they diagnose patients, i. e. not asking about other symptoms when the patient presents with depression. The expert employing data from a structured clinical interview, however, employs a bottom-up approach in the diagnostic process, i. e. asking questions which at first seem irrelevant. The risk of misclassification is supposed to be higher using the top-down diagnostic approach and relying on the diagnostic manual to confirm a clinical impression rather than to openly screen for alternative or additional diagnoses. Lack of relevant information in the patients’ records is shown to be a general phenomenon affecting all diagnostic groups .
The importance of correctly diagnosing comorbidity in clinical practice should be emphasized. Several studies have shown that psychiatric comorbidity is associated with a significantly increased probability of treatment and that comorbidity can be regarded as an index of a more severe course and outcome of mental disorders . Comorbidity is associated with more severe psychiatric symptoms, more functional disability, longer illness duration, less social competence, and higher service utilization . Furthermore, patients with affective disorder and comorbid anxiety and substance abuse, show less adherence to pharmacological treatment  and need more specialized treatment . It is shown that such comorbidity is associated with suicidality in mood disorders . This underpins the importance of correctly diagnosing comorbid conditions in the clinic. The psychiatric evaluation and diagnoses given at discharge from psychiatric hospital will follow the patient and is guiding for the treatment the patient will receive from GP’s and psychiatric personnel in the community.
Concerning administrative consequences, not diagnosing comorbidity represents an undercommunication of the burden these patients represent to the health care system and consequently gives the wrong signals concerning how to develop necessary services. Misleading medical statistics may cause spurious comparisons during the planning and evaluation of treatment for patients . Further, our findings suggest that register diagnoses are dubious for research purposes when it comes to comorbid psychiatric diagnoses. This is in accordance with the investigations of Baca-Garcia et al. [37, 38] and McConville et al. .
Our study comprises only 250 patients, and only first time admissions, so the generalizability of the findings could be questioned. It could be argued that new patients are more difficult to diagnose making the diagnostic validity of registers including all patients, better. Compared with the studies reviewed by Byrne et al.  on the diagnostic validity of administrative registers in psychiatric research, our study has some advantages strengthening the validity of the results. First, a structured diagnostic interview was performed, adding information from the records when necessary, and the clinical diagnoses were blind to the expert. On the other hand, the expert never actually saw the patient. Thus the observations, scorings and case notes could have been evaluated otherwise if the expert had observed the patient directly. The greatest possible caveat here is that signs and symptoms may have been missed or misinterpreted. However the expert only scored a symptom as present if there was given a description of overt behaviour or citations from the patient in either the interview protocol or in the hospital records. Furthermore there is always a risk that diagnoses based on an interview which screens for all psychiatric symptoms may be overinclusive. This possible bias may result both from a “yes-saying” response style of the patient, and from a tendency of the interviewer to put weight on positive answers about signs and symptoms that are not clinically significant. Thus, there is a risk that the high number of diagnoses given by the expert is a result of response bias and scoring bias. However, we do not believe that this bias will disturb the main findings. Structured interviews are shown to be better than unstructured traditional diagnostic assessment [40, 41], and combining structured interviewing with a review of the medical records appears to produce more accurate primary diagnoses and to identify more secondary diagnoses than routine clinical methods or a structured interview alone . The studies reviewed by Byrne et al. , where only case notes were checked and no new information added, should be regarded more as reliability studies than validity studies. Second, in our study the clinicians’ diagnoses were blind to the expert thus avoiding bias in either direction. A weakness in this study may be that formal inter-rater reliability testing was not done among the interviewers, however, there were organized discussions among them, also on selected videotaped cases. Inter-rater reliability can be low even if diagnoses are determined by researchers as found by Cheniaux et al. . However, to counter this, diagnoses were not formulated by the interviewers, but by one experienced researcher.
Even if the nosological status of the comorbidity concept is by no means clear [25, 44] the importance of correctly diagnosing comorbidity should be emphasized and the results from this study tells us that much work needs to be done in the clinic in structuring the diagnostic process, and in developing treatment guidelines for comorbid conditions [41, 45, 46]. The validity of psychiatric administrative registers concerning comorbidity, seems dubious for research purposes as well as for administrative and clinical purposes.
In a previous paper we have reported that this is even the case especially for bipolar disorder . When it comes to psychotic disorders such as substance use induced psychosis and schizophrenia, the validity seems satisfactory.
The study is supported by a grant from the Psychiatric Research Centre of Northern Norway, Tromsø and Bodø, grants nr. P16-03FPTF, P03-02FPTF, P15-01.
- Merikangas KR, Kalaydjian A: Magnitude and impact of comorbidity of mental disorders from epidemiological surveys. Curr Opin Psychiatr. 2007, 20: 353-58. 10.1097/YCO.0b013e3281c61dc5.View Article
- Burns L, Teesson M: Alcohol use disorders comorbid with anxiety, depression and drug use disorders. Findings from the Australian national survey of mental health and wellbeing. Drug Alcohol Depend. 2002, 68: 299-307. 10.1016/S0376-8716(02)00220-X.View ArticlePubMed
- Teesson M, Slade T, Mills K: Comorbidity in Australia: findings of the 2007 national survey of mental health and wellbeing. Aust N Z J Psychiatr. 2009, 43: 606-14. 10.1080/00048670902970908.View Article
- Kessler RC, McGonagle KA, Zhao S, Nelson CB, Hughes M, Eshleman S, Wittchen HU, Kendler KS: Lifetime and 12-month prevalence of DSM-III-R psychiatric disorders in the United States, results from the national comorbidity survey. Arch Gen Psychiatry. 1994, 51: 8-19. 10.1001/archpsyc.1994.03950010008002.View ArticlePubMed
- Merikangas KR, Angst J, Eaton W, Canino G, Rubio-Stipec M, Wacker H, Wittchen HU, Andrade L, Essau C, Whitaker A, Kraemer H, Robins LN, Kupfer DJ: Comorbidity and boundaries of affective disorders with anxiety disorders and substance misuse: results of an international task force. Br J Psychiatr Suppl. 1996, 30: 58-67.
- Jacobi F, Wittchen HU, Holting C, Höfler M, Pfister H, Müller N, Lieb R: Prevalence, co-morbidity and correlates of mental disorders in the general population: results from the German Health Interview and Examination Survey (GHS). Psychol Med. 2004, 34: 597-611. 10.1017/S0033291703001399.View ArticlePubMed
- Wierdsma AI, Sytema S, van Os JJ, Mulder CL: Case registers in psychiatry: do they still have a role for research and service monitoring?. Curr Opin Psychiatr. 2008, 21: 379-384. 10.1097/YCO.0b013e328304d99b.View Article
- Perera G, Soremekun M, Breen G, Stewart R: The psychiatric case register: noble past, challenging present, but exiting future. Br J Psychiatry. 2009, 195: 191-193. 10.1192/bjp.bp.109.068452.View ArticlePubMed
- Stewart R, Soremekun M, Perera G, Broadbent M, Callard F, Denis M, Hotopf M, Thornicroft G, Lovestone S: The South London and Maudsley NHS Foundation Trust Biomedical Research Centre (SLAM BRC) case register: development and descriptive data. BMC Psychiatr. 2009, 9: 51-10.1186/1471-244X-9-51.View Article
- Morgan VA, Jablensky AV: From inventory to benchmark: quality of psychiatric case registers in research. Br J Psychiatry. 2010, 197: 8-10. 10.1192/bjp.bp.109.076588.View ArticlePubMed
- Byrne N, Regan C, Howard L: Administrative registers in psychiatric research: a systematic review of validity studies. Acta Psychiatr Scand. 2005, 112: 409-414. 10.1111/j.1600-0447.2005.00663.x.View ArticlePubMed
- Øiesvold T, Nivison M, Østensen L, Hansen V, Sørgaard KW, Skre I: Classification of bipolar disorder in psychiatric hospital, a prospective cohort study. BMC Psychiatr. 2012, 12: 14-10.1186/1471-244X-12-14.View Article
- Rezvy G, Øiesvold T, Parniakov A, Ponomarev O, Lazurko O, Olstad R: The Barents project in psychiatry: a systematic comparative mental health services study between Northern Norway and Archangelsk County. Soc Psychiatr Psychiatr Epidemiol. 2007, 42: 31-139.View Article
- Sheehan D, Janavs J, Baker R, Harnett-Shehaan K, Knapp E, Sheehan M: Mini International Neuropsychiatric Interview. 1994, Tampa: University of South Florida
- Leiknes KA, Leganger S, Malt EA, Malt U: M.I.N.I. PLUS. 1999, Oslo: Psykosomatisk avd., Rikshospitalet
- World Health Organization: The ICD-10 Classification of mental and behavioural disorders: clinical descriptions and diagnostic guidelines. 1992, Geneva: World Health Organization, 10th
- Skre I, Onstad S, Torgersen S, Kringlen E: High interrater reliability for the Structured Clinical Interview for DSM-III-R Axis I (SCID-I). Acta Psychiatr Scand. 1991, 84: 167-173. 10.1111/j.1600-0447.1991.tb03123.x.View ArticlePubMed
- Rossi A, Morgan V, Amaddeo F, Sandri M, Grigoletti L, Maggioni F, Ferro A, Rigon E, Donisi V, Venturi VV, Goria F, Skre I, Tansella M, Jablensky A: Diagnosing psychotic disorders: validity, reliability and application of the Diagnostic Interview for Psychosis (DIP), Italian version. Epidemiol Psichiatr Soc. 2010, 19 (1): 33-43.PubMed
- World Health Organization: The ICD-10 Classification of mental and behavioural disorders. 1993, Geneva: Diagnostic criteria for research
- Landis RJ, Koch GG: The measurement of observer agreement for categorical data. Biometrics. 1997, 33: 159-174.View Article
- Keskimäki I, Aro S: Accuracy of data and diagnoses, procedures and accidents in the Finnish Hospital Discharge Register. Int J Health Sci. 1991, 2 (1): 15-21.
- Hansen SS, Munk-Jørgensen P, Guldbæk B, Solgård T, Lauszus KS, Albrechtsen N, Borg L, Egander A, Faurholdt K, Gilberg A, Godsen NP, Lorentzen J, Richelsen B, Weischer K, Bertelsen A: Psychoactive substance use diagnoses among psychiatric in-patients. Acta Psychiatr Scand. 2000, 102: 432-438. 10.1034/j.1600-0447.2000.102006432.x.View ArticlePubMed
- Zimmerman M, Mattia JI: Psychiatric diagnosis in clinical practice: Is comorbidity being missed. Compr Psychiatr. 1999, 40 (3): 1-9.View Article
- Zimmerman M, Chelminski I: Clinical recognition of anxiety disorders in depressed outpatients. J Psych Res. 2003, 37: 325-333. 10.1016/S0022-3956(03)00020-7.View Article
- Maj M: Psychiatric comorbidity’: an artefact of current diagnostic systems. Br J Psychiatry. 2005, 186: 182-184. 10.1192/bjp.186.3.182.View ArticlePubMed
- Brown TA, Barlow DH: A proposal for a dimensional classification system based on the shared features of the DSM-IV anxiety and mood disorders: Implications for assessment and treatment. Psychol Assess. 2009, 21 (3): 256-271.PubMed CentralView ArticlePubMed
- Milling RN, Faulkner LR, Craig JM: Problems in the recognition and treatment of patients with dual diagnoses. J Subst Abuse Treat. 1994, 11 (3): 267-271. 10.1016/0740-5472(94)90085-X.View ArticlePubMed
- Schoevers RA, Van HL, Koppelmans V, Kool S, Dekker JJ: Managing the patient with co-morbid depression and anxiety disorder. Drugs. 2008, 68: 1621-1634. 10.2165/00003495-200868120-00002.View ArticlePubMed
- Grant BF, Stinson FS, Dawson DA, Chou SP, Dufour MC, Compton W, Pickering RP, Kaplan K: Prevalence and co-occurence of substance use disorders and independent mood and anxiety disorders: results from the National Epidemiological Survey on Alcohol and Related Conditions. Arch Gen Psychiatry. 2004, 61: 807-816. 10.1001/archpsyc.61.8.807.View ArticlePubMed
- Tversky A, Kahneman D: Judgement under uncertainty: heuristics and biases. Science. 1974, 185: 1124-1131. 10.1126/science.185.4157.1124.View ArticlePubMed
- Elstein AS, Schwartz A: Clinical problem solving and diagnostic decision making: selective review of the cognitive literature. BMJ. 2002, 324: 729-732. 10.1136/bmj.324.7339.729.PubMed CentralView ArticlePubMed
- Miller PR: Inpatient diagnostic assessment: 3. Causes and effects of diagnostic imprecision. Psychiatr Res. 2002, 111: 191-197. 10.1016/S0165-1781(02)00147-6.View Article
- Cerdá M, Sagdeo A, Galea S: Comorbid forms of psychopathology: key patterns and future research directions. Epidemiol Rev. 2008, 30: 155-77. 10.1093/epirev/mxn003.View ArticlePubMed
- Perlis RH, Ostacher MJ, Miklowitz DJ, Hay A, Nierenberg AA, Thase ME, Sachs GS: Clinical features associated with poor pharmacological adherence in bipolar disorder: results from the STEP-BD study. J Clin Psychiatr. 2010, 71 (3): 296-303. 10.4088/JCP.09m05514yel.View Article
- Howland RH, Rush AJ, Wisniewski SR, Trivedi MH, Warden D, Fava M, Davis LL, Balusubramani GK, McGrath PJ, Berman SR: Concurrent anxiety and substance use disorders among outpatients with major depression: Clinical features and effect on treatment outcome. Drug Alcohol Depend. 2009, 99: 248-260. 10.1016/j.drugalcdep.2008.08.010.View ArticlePubMed
- Uwake R, Gureje O: The relationship of comorbidity of mental and substance use disorders with suicidal behaviours in the Nigerian Survey of Mental Health and Wellbeing. Soc Psychiatr Psychiatr Epidemiol. 2011, 2011 (46): 173-180.View Article
- Baca-Garcia E, Perez-Rodriguez MM, Basurte-Villamor I, del Moral AL F, Jimenez-Arriero MA, de Rivera JL G, Saiz-Ruiz J, Oquendo MA: Diagnostic stability of psychiatric disorders in clinical practice. Br J Psychiatry. 2007, 190: 210-216. 10.1192/bjp.bp.106.024026.View ArticlePubMed
- Baca-Garcia E, Perez-Rodriguez MM, Basurte-Villamor I, Lopez-Castroman J, del Moral AL F, Jiminez-Arriero MA, de Rivera JL G, Saiz-Ruiz J, Leiva-Murillo JM, de Prado-Cumplido M, Santiago-Mozos R, Artes-Rodriguez A, Oquendo MA, de Leon J: Diagnostic stability and evolution of bipolar disorder in clinical practice: a prospective cohort study. Acta Psychiatr Scand. 2007, 115: 473-480. 10.1111/j.1600-0447.2006.00984.x.View ArticlePubMed
- McConville P, Walker NP: The reliability of case register diagnoses: a birth cohort analysis. Soc Psychiatr Psychiatr Epidemiol. 2000, 35: 121-127. 10.1007/s001270050194.View Article
- Miller PR, Dasher R, Collins R, Griffiths P, Brown F: Inpatient diagnostic assessments: 1. Accuracy of structured vs. unstructured interviews. Psychiatr Res. 2001, 105: 255-264. 10.1016/S0165-1781(01)00317-1.View Article
- Rogers R: Standardizing DSM-IV diagnoses: the clinical application of structured interviews. J Pers Assess. 2003, 8 (83): 220-225.View Article
- Basco MR, Bortic JQ, Davies D, Rush AJ, Witte B, Hendrickse W, Barnett V: Methods to improve diagnostic accuracy in a community mental health setting. Am J Psychiatry. 2000, 157 (10): 1599-1605. 10.1176/appi.ajp.157.10.1599.View Article
- Cheniaux E, Landeira-Fernandez J, Versiani M: The diagnoses of schizophrenia, schizoaffective disorder, bipolar disorder and unipolar depression: interrater reliability and congruence between DSM-IV and ICD-10. Psychopathology. 2009, 42: 293-298. 10.1159/000228838.View ArticlePubMed
- Jorm AF: National surveys of mental disorders: are they researching scientific facts or constructing useful myths?. Aust N Z J Psychiatr. 2006, 40: 830-834. 10.1080/j.1440-1614.2006.01901.x.View Article
- Shear MK, Greeno C, Kang JK, Ludewig D, Frank E, Swartz HA, Hanekamp M: Diagnosis of nonpsychotic patients in community clinics. Am J Psychiatry. 2000, 157 (4): 581-587. 10.1176/appi.ajp.157.4.581.View ArticlePubMed
- Wüsthoff LE, Waal H, Ruud T, Røislien J, Gråwe RW: Identifying co-occuring substance use disorders in community mental health centres. Tailored approaches are needed. Nord J Psychiatr. 2011, 65 (1): 58-63. 10.3109/08039488.2010.489954.View Article
- The pre-publication history for this paper can be accessed here:http://www.biomedcentral.com/1471-244X/13/13/prepub
This article is published under license to BioMed Central Ltd. This is an Open Access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.