Using three independent and nationally representative data-sets, we found that the diagnosis of insomnia was virtually non-existent in patients with mental disorders. Only 34 of 42507 patients had a diagnosis of insomnia as either a primary or a comorbid diagnosis. However, about 40% of all patients experienced severe sleep disturbance and at least 22% reported this to be one of their most prominent problems. Although they did not diagnose it, clinicians indicated that 24% of their patients had sleep disturbance as a prominent problem. We found a notable discrepancy between the clinicians' report and the patients' report of sleep disturbance. The patients and clinicians agreed in less than 30% of the cases where either the patients or clinicians had reported sleep disturbance. The probability of the patients themselves experiencing sleep disturbance when the clinician reported this, was less than 62%. This probability was surprisingly even lower when the patient experienced their sleep disturbance to be one of their most prominent symptoms, with a predictive value of less than 37%. Thus, the precision of clinician-rated sleep disturbance was close-to-chance when compared to the patient report. Of the patients who used hypnotics, 55% reported having sleep disturbance whereas 23% had neither clinician nor patient reported sleep disturbance. These findings were replicated with only minor differences in the different samples.
The diagnostic findings are at odds with findings from epidemiological studies where the prevalence has been reported to be about 100 times higher [18–20]. However, our findings are in line with two previous studies where large discrepancies between findings from epidemiological studies and diagnostic practice in clinical settings have been reported [21, 22]. Although the diagnostic recommendation in the ICD-10 is to code insomnia as a comorbid disorder if it is one of the most prominent symptoms, our findings indicate that clinicians are reluctant to use these recommendations.
Underlying this may be a long-standing issue of whether insomnia should be regarded as a symptom of other disorders or as a disorder in it self. Clinicians may not find it relevant to code insomnia as a comorbid condition as they regard it as a part of the primary mental disorder. If clinicians do not find the diagnosis of insomnia relevant to their practice, this may raise concerns about the usefulness of the diagnosis in its current form. In this respect it is interesting to note that the online draft of the DSM-5 proposes to remove the diagnosis of Insomnia Related to Another Mental Disturbance and only operate with the diagnosis of Insomnia. The proposed change in the DSM reflects a change in paradigm from thinking of insomnia as a symptom to thinking of insomnia as a disorder. It recommends coding Insomnia Disorder if the criteria are fulfilled regardless of meeting criteria for other mental disorders, because determining the cause or consequence of the problem is clinically difficult, if not impossible .
Patient report of sleep disturbance was higher than what is found in the general population using similar measures , but in agreement with other studies from patients with mental disorders . Hypnotic medication was, relative to previous studies [24, 25], seldom prescribed to the patients in the current study. Less than 9% of all patients were prescribed such medication compared to 7% of the general population in Norway in the same period . Patients with mental disorders are likely to be prescribed other medications that have effects on sleep, such as some anti-depressants or anti-psychotic medication that could make hypnotic medication less needed. Still, it is curious that at least 23% of the patients who were prescribed hypnotic medication had neither self-reported nor clinician-reported sleep disturbance. This could indicate that patients who are being prescribed hypnotics stay on hypnotics after the sleep disturbance has been treated.
The low recognition of sleep disturbance raises questions concerning education and training of health care professionals. Although a call for improved education about sleep was made in the 1980s when similar findings were reported , the situation does not seem to have improved. Physician training in the recognition of sleep specific symptoms has been reported to be minimal . Similarly, a survey of 212 directors of graduate and internship programs of clinical psychology revealed that only 6% of the programs offered courses in sleep and most reported that their institution was ineffective in providing sleep education . Moreover, doctors seldom ask about sleep in clinical interviews , and patients with sleep disturbance infrequently report this to their clinicians [29, 30]. This makes it important for health care professionals to have knowledge about the features of sleep.
There is a possibility of sampling bias in data-sets 2 and 3 as about two-thirds of the original sample of patients did not agree to have their self-report linked to the clinician-report. This could be an artifact of the procedures. The patients had to specifically indicate that they wanted to have their scores linked to their clinicians' ratings, rather than having to indicate if they did not want to have their scores linked. This difference could have had a large impact on patient participation .
Clinicians in data-set 3 were asked to only indicate the most prominent problem the patient experienced compared to the three most prominent problems reported by clinicians in data-set 2. This gave a lower prevalence of clinician-rated sleep disturbance in data-set 3 compared to data-set 2. In both data-sets clinicians have therefore only indicated the patients who they regard as having sleep disturbance as one of their most prominent problems. The study was not primarily designed to measure sleep disturbance in mental health care and we might have found a higher prevalence of clinician-rated sleep disturbance if we had utilized a survey directly inquiring about sleep symptoms. This could have resulted in higher agreement and different predictive values for clinician's evaluations. However, our finding that even when clinicians evaluate sleep disturbance to be one of the most prominent problems, there is still a very low agreement between patients and clinicians. Especially as the predictive value of the clinicians' report was less than chance for the patients who reported sleep disturbance as one of their most prominent problems.
The use of a single item measuring sleep disturbance is another limitation. This means that it is not possible to discern if there were differences in agreement between sleep onset or sleep maintenance problems or if the patients experienced other kinds of sleep disturbance.
Finally, although we have assessed patients who received hypnotic medication, there might be a proportion of the patients who received alternate medication or non-pharmacological interventions for their sleep disturbance.