- Research article
- Open Access
- Open Peer Review
The development and psychometric properties of a new scale to measure mental illness related stigma by health care providers: The opening minds scale for Health Care Providers (OMS-HC)
- Aliya Kassam1Email author,
- Andriyka Papish2,
- Geeta Modgill3 and
- Scott Patten4
https://doi.org/10.1186/1471-244X-12-62
© Kassam et al.; licensee BioMed Central Ltd. 2012
- Received: 21 December 2011
- Accepted: 13 June 2012
- Published: 13 June 2012
Abstract
Background
Research on the attitudes of health care providers towards people with mental illness has repeatedly shown that they may be stigmatizing. Many scales used to measure attitudes towards people with mental illness that exist today are not adequate because they do not have items that relate specifically to the role of the health care provider.
Methods
We developed and tested a new scale called the Opening Minds Scale for Health Care Providers (OMS-HC). After item-pool generation, stakeholder consultations and content validation, focus groups were held with 64 health care providers/trainees and six people with lived experience of mental illness to develop the scale. The OMS-HC was then tested with 787 health care providers/trainees across Canada to determine its psychometric properties.
Results
The initial testing OMS-HC scale showed good internal consistency, Cronbach’s alpha = 0.82 and satisfactory test-retest reliability, intraclass correlation = 0.66 (95% CI 0.54 to 0.75). The OMC-HC was only weakly correlated with social desirability, indicating that the social desirability bias was not likely to be a major determinant of OMS-HC scores. A factor analysis favoured a two-factor structure which accounted for 45% of the variance using 12 of the 20 items tested.
Conclusions
The OMS–HC provides a good starting point for further validation as well as a tool that could be used in the evaluation of programs aimed at reducing mental illness related stigma by health care providers. The OMS-HC incorporates various dimensions of stigma with a modest number of items that can be used with busy health care providers.
Keywords
- Focus Group
- Mental Illness
- Medical Student
- Health Care Provider
- Social Desirability
Background
Mental illness related stigma can lead to low rates of seeking help, lack of access to care, under-treatment [1], and social marginalization [2]. Ultimately this leads to the inability of a person with mental illness to recover. Recovery is a process which occurs when people with mental illness discover, or rediscover, their strengths and abilities for pursuing personal goals and develop a sense of identity that allows them to grow beyond their mental illness [3, 4].
Studies have varied in the dimensions of stigma examined, but the most common ones are: ‘perceived stigma’, which refers to one’s belief that others perceive an individual as socially unacceptable [4–7] and ‘self-stigma’, which refers to a similar, internalized perception of oneself leading to the fear of seeking help or disclosing one’s mental illness due to the stigma associated with mental illness [8]. Other dimensions of stigma that have been examined include social distance, which refers to one’s desire to maintain distance from people with mental illness [5, 9], ‘dangerousness”, which refers to one’s belief that the individual is dangerous [6], recovery, which refers to one’s belief that people with mental illness can recover [3]. Emotional reactions [10] such as a lack of social responsibility as well as a lack of empathy or compassion towards people with mental illness are also dimensions of stigma [10]. While some may consider that compassion and social responsibility may be paternalistic and stigmatizing towards people with mental illness [11], these dimensions can be seen as important indicators of non-stigmatizing attitudes and are acknowledged widely as significant competencies of health care providers [4, 12].
Evidence for an association between various dimensions of stigma is either lacking or largely inconsistent. For example, perceived stigma has been found to be unrelated to help seeking in some studies [13, 14]. It must also be noted that most of the models that currently describe the dimensions related to mental illness stigma are based on individual-level rather than a sociological-level of structural indicators of stigma which can occur at the institutional or government level. For the purposes of this paper, we will focus on individual-level of stigma, specifically by health care providers.
Stigma has been criticized as being too vaguely defined and individually focused [15] and thus existing models have defined stigma as a dynamic interrelationship of components. This interrelationship involves cognitive, affective and behavioural components. In order to understand and measure stigma it is important to conceptualize the term within a more detailed model.
A model by Link and Phelan which has been widely used in the literature describes mental illness stigma as four components which are: labelling, stereotyping, separating and status loss/discrimination. Throughout these components, it has been suggested that emotional reactions can occur [15]. There are three cognitive processes; labelling begins when personal characteristics are signalled to show an important difference between the person who stigmatises and the stigmatised. Stereotyping then occurs when the labelled differences are associated with undesirable characteristics. This is followed by categorically distinguishing or separating between the mainstream group and the labelled group, perceiving the labelled group as fundamentally different. Ultimately, the labelled group is then devalued, rejected and excluded through the process of status loss or discrimination [15].
Another model by Corrigan, contains the components of stereotype, prejudice and discrimination that are seen as causally related. For example, a person who believes (cognition) a person with mental illness is dangerous (stereotype) might negatively evaluate or fear (affect) the person with mental illness as dangerous, leading to prejudice. This would then lead to discrimination (behaviour) when the person is treated inappropriately for their mental illness by receiving sub-standard care [16].
What can be drawn from these models so far is that they have commonly embedded within them the cognitive, affective and behavioural components. While they are useful in describing mental illness related stigma as a social psychological phenomenon, they may not be valid or useful for measuring outcomes in mental health promotion and health education interventions aimed to reduce the stigma of mental illness.
A final conceptualization of stigma is the tri-partite model, which proposes that stigma is an overarching term including three core elements: knowledge (misinformation/differences in understanding due to culture or religion), attitudes (prejudice) and behaviour (discrimination) [17]. The knowledge, attitudes and behaviour framework allows clear intervention targets and units of measurement [17, 18]. The importance of knowledge, attitudes and behaviour has been established in medical education with medical students, nurses, and other health care providers [17–20]. The knowledge, attitudes and behaviour framework is also one that is widely used in health promotion [21, 22]. Similar to the Corrigan model, the tri-partite model focuses on the problem of attitudes in the form of prejudice which can be elicited as common stereotypes or emotional reactions rather than separating them like the previous models. The tri-partite model is adaptable in that it allows for attitudes towards people with mental illness to be comprised of the various dimensions of stigma [17].
In the literature, it has been shown that attitudes towards people with mental illness can be measured using stereotypes such as: ‘people with mental illness are dangerous,’ and ‘people with mental illness do not recover’ [23, 24] as well as a desire for social distance because of the aforementioned stereotypes [7]. Stigmatizing attitudes can also be measured in the form of emotional reactions towards people with mental illness. Finally, disclosing that one has a mental illness, because of the dimensions described above, can lead to self stigma and may also be an indicator of mental illness related stigma [24, 25]. While it has been shown that self stigma is different than holding stigmatizing attitudes towards people with mental illness [25], we believed it was important to measure in health care providers because, we saw disclosure as a dimension of stigma that would also indicate whether the respondent held stigmatizing attitudes towards mental illness. For example, those who would disclose that they had a mental illness may not think that mental illness is something to be ashamed of and may therefore be less stigmatizing. This has been described in the literature where some refuse to be diminished by stigma and becoming more active participants of change in health care [7]. Also, potential users of the instrument, such as professional organizations, are likely to be interested in the issue of disclosure because of a desire to see their members receive appropriate treatment and support for mental health issues.
Through extensive review of the academic literature on surveys used to measure attitudes towards people with mental illness, a large gap was shown in the area of surveys used to measure the attitudes of health care providers [26]. A new measure of stigma intended for healthcare providers is pertinent because stigma among health care providers differs from other kinds of stigmas held by various other groups. For example, it has been reported that people with mental illness have poorer physical health in part because medical professionals wrongly associate the physical symptoms experienced by the person with mental illness to the mental illness itself [27–30]. This could be due to a phenomenon called ‘diagnostic overshadowing’ [17, 30].
Diagnostic overshadowing can be defined as the process by which the physical problems of a patient are over-shadowed by their psychiatric diagnosis [17, 30]. It is important to note that diagnostic overshadowing is not unique to primary care and may occur in other areas of health services [17, 26]. An investigation of the physical healthcare of patients with schizophrenia in primary care [31] showed that people with schizophrenia were no more likely than the general population to be targeted for physical health checks despite increased physical health risks. Furthermore, people with schizophrenia were significantly less likely to receive important basic health checks such as blood pressure and cholesterol measurement [31].
Medical practitioners also diagnose and treat people with mental illness differently. For example, people with mental illness are “substantially less likely to undergo coronary re-vascularisation procedures” compared to people without mental illness [32]. Similarly, people with co-morbid mental illness and diabetes are less likely to be admitted to hospital for diabetic complications than those with no mental illness [33].
Furthermore, people with mental illness may have less access to medical care [34, 35] such as obtaining a primary care physician [34, 35] as there is a need for community mental health centres to address barriers to primary medical care [36]. People with mental illness may also feel unwelcome in certain medical settings because of staff attitudes [36–42].
Although health care providers are thought to hold attitudes that are positive, compassionate and encouraging towards people with mental illness, this is often not the case. Health care providers may be ignorant about the possible outcomes of people with mental illness. Often this may be due to inadequate training [36]. It has been reported that 68% of the mental health professionals surveyed thought that most clinicians do not receive appropriate training in dealing with people with severe mental illness [37].
A large body of research on the attitudes of health care providers has repeatedly shown negative attitudes towards people with mental illness. This has been a problem and continues to be a problem in primary care, mental health services and within the education of health care providers [37–42].
As part of its 10-year mandate, The Mental Health Commission of Canada (MHCC) has embarked on an anti-stigma initiative called Opening Minds (OM) to change the attitudes and behaviours of Canadian health care providers towards people with a mental illness. OM is the largest systematic effort undertaken in Canadian history to reduce the stigma and discrimination associated with mental illness. OM’s philosophy is to build on the strengths of existing programs from across the county, and to scientifically evaluate their effectiveness. A key component of programs being evaluated is contact-based educational sessions, where target audiences hear personal stories from and interact with individuals who have experience with mental illness and have recovered or are managing their illness. OM’s goal is to replicate effective programs nationally, develop new interventions to address gaps in existing programs and add other target groups over time. (http://www.mentalhealthcommission.ca/English/Pages/OpeningMinds.aspx)
Evaluation describes and explains the practice of participants to determine their effectiveness [43, 44]. Because evidence-based educational interventions are given priority, evaluation drives undergraduate, post graduate and continuing education curriculum development [44]. In order to achieve its goal with healthcare providers, the Opening Minds initiative required a current, reliable and valid tool to evaluate best practices to reduce the stigma of people with mental illness. Many of the evaluative tools that exist today are not adequate for our purpose because they do not relate specifically to the role of the health care provider.
The aim of this study was to develop, and test the Opening Minds Scale for Healthcare Providers (OMS-HC).
Methods
Research carried out in this study is in compliance with the Helsinki Declaration (http://www.wma.net/e/policy/b3.htm). This study was approved by the Conjoint Health Research Ethics Board of the University of Calgary, Ethics ID E22724.
Development phase
The development of the OMS-HC began by generating an item pool of items pertaining to the dimensions of stigma described earlier from existing scales as well as new items that were developed by the researcher (AK) from consultations with people working with programs to reduce the stigma of mental illness by health care providers. What resulted was a pool of 50 items with five dimensions to measure stigma which were 1.) Recovery, 2.) Social responsibility, 3.) Social distance 4.) Other concepts (dangerousness, diagnostic overshadowing) 5.) Disclosure. The initial items and the scales from which they were derived are provided (see Additional file 1). These questions were then sent to three people who had a mental illness for their feedback. The questions were also reviewed by a research team which consisted of a sociologist, psychiatrist, anti-stigma program manager, and research associate. From this collective feedback, survey questions were eliminated and amended resulting in a 42 question scale.
Cognitive interviewing was conducted with the 42 questions and based on this feedback the scale was reduced to 26 questions. The individual interviews were 20 minutes in length. Cognitive interviewing is a method of evaluating sources of response error in survey questionnaires that focuses mainly on the questionnaire and the cognitive processes that respondents use to answer the survey [45]. For the OMS-HC scale, cognitive interviews were conducted by one of the authors (AK). For the conduct of the cognitive interviews, six volunteers were recruited using a convenience sampling method. Volunteers all worked in the health care field and were female. Questions were asked around what the volunteer felt each item on the questionnaire was referring to, whether they felt pressured to respond to the item in a certain manner (positively or negatively) and whether the item made sense and was measuring attitudes of health care providers towards people with mental illness [45].
The questions were then reviewed again by the research team and one question was eliminated because it was not judged to measure stigma, leading to an initial version of the OMS-HC. The initial OMS-HC scale had 25 questions as well as additional demographic questions. The scale used definitions of mental health problems/illnesses, recovery and health care system drawn from the MHCC framework document [46].
Seven focus groups of health care providers were then held to ask about the wording of the scale, suggest new items as well as to explore their attitudes towards people with mental illness. A focus group of people with mental illness was conducted to ensure the language and tone of the scale was appropriate and that no concepts were missing. For this part of the study, we invited volunteer participants who were health care providers or health care providers in training in Calgary to participate in the focus groups. Focus groups consisted of 6–10 people. For the focus groups, we aimed to recruit participants for each of the following disciplines:
· General practitioners
· Other physicians including psychiatrists
· Surgeons
· Nurses
· Psychiatric nurses
· Medical students
· Social workers/social worker students
· Occupational therapists/occupational therapy students
· Psychologists
· Pharmacists
Focus groups were selected using a convenience sampling method. Emails were sent to local mental health community organizations as well as hospitals and clinics. Medical students and doctors were recruited through the author’s (AP) contacts. Several concepts that were explored in the focus groups were the use of the term “mental illness” as opposed to a specific diagnosis of mental illness such as depression, the definition of mental illness, the definition of recovery, the social desirability bias and the relevance of the items to health care providers. The focus groups took place one after another and the scale presented to each focus group was amended based on the previous focus group’s feedback so that the scale would be improved each time. Focus group data was then transcribed and analyzed for each focus group transcript. The researcher (AK) looked for consensus in qualitative responses around the term “mental illness” the definition of mental illness, the definition of recovery, social desirability bias and the relevance of the items to health care providers [47]. If the majority (greater than 50% of the focus group members agreed on a concept, then the researcher made changes to the final scale accordingly. These changes resulted in a reduction from 25 to 20 items.
Testing phase
In this part of the study, we identified participants by searching for health regions, professional organizations and academic departments affiliated with health care providers or health care providers in training to participate in completing the survey.
Participants were contacted locally, provincially and nationally by e-mail for the surveys which were conducted online. Faculty deans and department heads at the Universities for the areas of medicine, nursing, social work, occupational therapy and psychology were contacted in order to access students in each group. Professional associations were also contacted across Canada to request members who are health care providers to complete the questionnaire. The questionnaire was available online for respondents to complete. Any health care provider or trainee was eligible to participate in the study if they provided informed consent to participate.
We also sought to determine the relationship between the scale items and social desirability items. The social desirability bias can be defined as the tendency of individuals to report favourable impressions of themselves on measures that ascertain sociological and psychological variables of interest such as attitudes to mental illness, ethnic minorities and gender. If items of the scale correlate highly with scores of social desirability, the scale could be influenced by social desirability bias [47].
All statistical analysis was carried out using STATA version 12.0. The Spearman's rank correlation coefficient or Spearman's rho (ρ) was used to determine the relationship between total scores and the Marlow-Crowne Social Desirability scale and the total scores of 669 health care providers. This particular correlation coefficient is considered to be more robust and does not make restrictive assumptions about the frequency distribution of the variables [48].
Test-retest reliability of the 20-item scale was conducted with a sub-sample of health care providers who consented to be contacted four to six weeks after the first time they completed the survey. The intraclass correlation comparing the total scores at the two time periods was determined.
An item analysis was conducted and the Cronbach’s alpha was computed to determine the overall consistency of the scale. Item-total correlations were calculated along with a factor analysis of the items. Those items that did not correlate strongly with the total score were eliminated. Factor analysis was conducted using principal component analysis. The methods used for retaining factors were keeping those factors that had Eigen values above 1.0, and had more than four items that loaded onto a single factor.
Results
The development and testing of the OMS-HC.
Development
Seven health care provider focus groups with 64 health care providers/trainees were held and consisted of 16 males and 48 females. Focus group participants were 22 to 65 years of age. In total, there were 14 medical students, 2 psychiatrists, 8 psychiatry residents, 4 physiotherapists, 9 psychologists, 6 recreational therapists, 10 social workers, 2 nurses and 9 occupational therapists. In the focus group of people with lived experience of mental illness, there were 5 males and 1 female and all were currently diagnosed with a mental illness.
Major discussion topics in the focus groups circled around whether or not to use the term “mental illness”, whether or not to include a definition of the term “mental illness”, the wording of each of the items, whether or not to include a specific diagnosis or to specify the severity of mental illness. Overall, there was no consensus on any of these topics and ultimately the main value of the focus groups was helpful feedback on individual items in terms of wording and retention. A suggestion for an item that came from the focus group of people with mental illness was to determine whether health care providers felt that medication was the best treatment of mental illness. This is because members of this focus group reported feeling stigmatized by health care providers because they were viewed as being more likely to over-medicate patients and less likely to listen or provide other forms of therapy. The use of medication was also perceived as a means of increasing social distance.
From a 25-item scale, a 20-item scale resulted after the focus groups. Regarding severity, the term “managed” was added to one item to imply appropriate treatment was being received, even if the illness was severe. The term “managed” was included so as not to evoke responses which would lead to ceiling or floor effects. The testing version of the scale and the item sources are also provided (see Additional file 2). It must be noted that the items shown may differ in how they are worded from the original item because these were discussed in focus groups and changed according to suggestions made in the focus groups.
Testing
Demographic characteristics of healthcare providers with total and subscale scores b
Demographic variables | Health care providers including medical students, Frequency (%) n = 787 | Total score | Attitudes subscale | Disclosure subscale |
|---|---|---|---|---|
(95% C.I.) | (95% C.I.) | (95% C.I.) | ||
Age (years) | ||||
18–25 years | 106 (13.5) | 60.7 | 14.9 | 16.3 |
(59.6–61.8) | (14.1–15.7) | (15.8–16.8) | ||
26–44 years | 349 (44.4) | 57.7 | 12.8 | 15.1 |
(57.2–58.2) | (11.8–12.5) | (14.9–15.4) | ||
45–64 years | 319 (40.5) | 56.3 | 11.3 | 14.4 |
(55.8–56.8) | (10.9–11.6) | (14.2–14.7) | ||
65 years or older | 13 (1.6) | 55.5 | 10.1 | 13.8 |
(53.4–57.7) | (8.7–11.5) | (12.3–15.2) | ||
Sex | ||||
Female | 631 (80.2) | 57.3 | 11.9 | 14.9 |
(56.9–57.6) | (11.6–12.1) | (14.7–15.1) | ||
Male | 156 (19.8) | 58.5 | 13.2 | 15.3 |
(57.6–59.5) | (12.5–13.9) | (14.8–15.7) | ||
Do you know a close friend or family member with mental illness? | ||||
Yes | 634 (80.6) | 57.3 | 11.9 | 14.9 |
(56.9–57.7) | (11.6–12.1) | (14.7–15.1) | ||
No | 130 (16.5) | 58.4 | 13.2 | 15.5 |
(57.6–59.1) | (12.6–13.8) | (15.1–15.9) | ||
Don’t know | 23 (2.9) | 58.9 | 13.9 | 15.5 |
(57.3–60.4) | (12.5–15.2) | (14.5–16.6) | ||
Have you ever treated a person with a mental illness? | ||||
Yes | 657 (83.5) | 57.2 | 11.7 | 14.9 |
(56.8–57.5) | (11.5–12.0) | (14.7–15.0) | ||
No | 113 (14.3) | 59 | 14 | 15.7 |
(58.1–59.9) | (13.3–14.7) | (15.3–16.1) | ||
Don’t know | 17 (2.2) | 60.8 | 15 | 15.4 |
(56.8–64.7) | (11.9–18.1) | (13.9–16.8) | ||
Ethnicity | ||||
White | 683 (86.8) | 57.3 | 11.8 | 14.9 |
(56.9–57.6) | (11.6–12.1) | (14.7–15.1) | ||
Asian | 43 (5.5) | 59.4 | 15 | 15.8 |
(58.1–60.8) | (14.0–16.1) | (15.0–16.6) | ||
South East Asian | 32 (4.1) | 58.5 | 13.8 | 15.7 |
(56.8–60.2) | (12.6–15.0) | (15.0–16.4) | ||
Aboriginal | 12 (1.5) | 58.5 | 14.3 | 14.3 |
(52.1–64.9) | (9.6–18.9) | (12.4–16.3) | ||
Black | 8 (1.0) | 56.6 | 11.8 | 14.5 |
(52.9–60.3) | (9.9–13.6) | (12.1–16.9) | ||
Arab/West Asian | 7 (0.9) | 62.7 | 15 | 17.1 |
(60.0–65.5) | (11.8–18.2) | (14.1–20.2) | ||
Latin American | 2 (0.2) | 58 | 11 | 17 |
(45.3–70.7) | (-27.1–49.1) | (-8.4–42.4) | ||
Province | ||||
Ontario | 204 (25.9) | 56.6 | 11.2 | 14.8 |
(56.1–57.2) | (10.8–11.6) | (14.4–15.1) | ||
Manitoba | 203 (25.8) | 56.8 | 11.7 | 14.4 |
(56.3–57.4) | (11.2–12.1) | (14.1–14.7) | ||
Alberta | 195 (24.8) | 59.5 | 13.8 | 15.8 |
(58.8–60.2) | (13.3–14.3) | (15.5–16.2) | ||
British Columbia | 153 (19.4) | 57 | 11.9 | 15 |
(56.2–57.8) | (11.3–12.5) | (14.6–15.4) | ||
Atlantic provinces | 21 (2.7) | 57 | 11.2 | 14.2 |
(55.3–58.6) | (9.9–12.4) | (13.1–15.4) | ||
Northwest Territories and Yukon | 6 (0.8) | 62 | 14.8 | 16.7 |
(44.3–79.7) | (4.1–25.6) | (13.1–20.2) | ||
Quebec | 4 (0.5) | 58 | 12.8 | 15.3 |
(56.2–59.8) | (7.2–18.3) | (10.0–20.5) | ||
Saskatchewan | 1 (0.1) | - | - | - |
Professional group | ||||
Social workers/social work students | 167 (21.2) | 56.6 | 11 | 14.8 |
(56.0–57.3) | (10.6–11.5) | (14.4–15.2) | ||
Nurses/nursing students | 138 (17.5) | 57.3 | 12.3 | 14.8 |
(56.5–58.1) | (11.7–12.9) | (14.4–15.2) | ||
Physicians and medical students | 129 (16.4) | 61 | 15.1 | 16.4 |
(60.2–61.8) | (14.5–15.7) | (16.0–16.8) | ||
Psychiatric nurses/psychiatric nursing students | 108 (13.7) | 55.7 | 11 | 14.4 |
(55.0–56.5) | (10.4–11.5) | (13.9–14.8) | ||
Other | 102 (13.0) | 57.2 | 11.8 | 14.5 |
(56.2–58.1) | (11.1–12.6) | (14.0–15.0) | ||
Psychologists/graduate students in psychology | 95 (12.1) | 57.1 | 11.8 | 14.9 |
(56.0–58.2) | (11.1–12.5) | (14.4–15.4) | ||
Occupational therapists/ occupational therapy students | 48 (6.1) | 57.4 | 11.4 | 14.9 |
(56.3–58.5) | (10.6–12.3) | (14.2–15.5) |
Scoring the scale
The testing version of the OMS-HC contained 20 items. A 5-point Likert scale was used and response options were 1 = Strongly disagree, 2 = Disagree, 3 = Neither agree nor disagree, 4 = Agree and 5 = Strongly agree. Scores could range from 20 to 100 and a lower score indicated less stigma. Items 3, 8, 9, 10, 11, 15, 19 required reverse scoring. The 12 item scale had scores ranging from 12 to 60, and only one of the 12-items was reverse scored. Regarding the subscales, (see results on factor analysis), the scoring of the attitudes of healthcare providers towards people with mental illness subscale may range from 7 (least stigmatizing) to 35 (most stigmatizing) while the scoring for the attitudes towards disclosure of a mental illness subscale may range from 5 (least stigmatizing) to 25 (most stigmatizing).
The mean total score for the 20 item scale for all health care providers (see Table 1) was 57.5 (95% C.I. 57.2 – 57.9). Scores ranged from 41.0 to 96.0 and the standard deviation was 4.8.
The mean total score for the 12 item scale was 27.0 (95% C.I. 26.7 - 27.4), scores ranged from 16.0 to 67.0 and the standard deviation was 5.0. The mean total score for the attitudes of healthcare providers towards people with mental illness subscale was 12.1 (95% C.I. 11.9 to 12.4) and for disclosure was 15.0 (95% C.I. 14.8 to 15.2). Scores ranged from 7.0 to 35.0 (SD = 3.6) for the attitudes of healthcare providers towards people with mental illness and from 8.0 to 23.0 (SD = 2.5) for attitudes towards disclosure of a mental illness.
Within each demographic group, there did not appear to be differences in scores except that among the age groups, the 18–25 year age group score was slightly higher than that of the other groups as was the score of health care providers from Alberta and the physicians/medical student group. This may be explained by the fact that the physician/medical student group consisted largely of medical students aged between 18–25 who were from Alberta. These 2nd year medical students (n = 112) were recruited as part of another study which involved the evaluation of an educational intervention aimed at reducing stigma and will be described elsewhere. The medical students completed the 20-item OMS-HC as a baseline assessment prior to a psychiatry course. The Cronbach’s alpha reliabilities for the medical student sample remained internally consistent (0.78 for the 20-item scale and 0.71 for the 12 item scale) and the subscales were responsive to change.
Item total correlations
Item-total correlations of each scale item and total score (20 items on OMS-HC)
1. I am more comfortable helping a person who has a physical illness than I am helping a person who has a mental illness. | .514 ** |
2. If a person with a mental illness complains of physical symptoms (e.g. nausea, back pain or headache), I would likely attribute this to their mental illness. | .422** |
3. If a colleague with whom I work told me they had a managed mental illness, I would be as willing to work with him/her. | -.072 * |
4. If I were under treatment for a mental illness I would not disclose this to any of my colleagues. | .426** |
5. I would be more inclined to seek help for a mental illness if my treating healthcare provider was not associated with my workplace. | .350** |
6. I would see myself as weak if I had a mental illness and could not fix it myself. | .572** |
7. I would be reluctant to seek help if I had a mental illness. | .516** |
8. Employers should hire a person with a managed mental illness if he/she is the best person for the job. | -.016 |
9. I would still go to a physician if I knew that the physician had been treated for a mental illness. | -.078 * |
10. If I had a mental illness, I would tell my friends. | -.205** |
11. It is the responsibility of health care providers to inspire hope in people with mental illness. | .011 |
12. Despite my professional beliefs, I have negative reactions towards people who have mental illness. | .475** |
13. There is little I can do to help people with mental illness. | .466** |
14. More than half of people with mental illness don’t try hard enough to get better. | .407** |
15. People with mental illness seldom pose a risk to the public. | -.072 * |
16. The best treatment for mental illness is medication. | .232** |
17. I would not want a person with a mental illness, even if it were appropriately managed, to work with children. | .317** |
18. Healthcare providers do not need to be advocates for people with mental illness. | .289** |
19. I would not mind if a person with a mental illness lived next door to me. | -.129 ** |
20. I struggle to feel compassion for a person with a mental illness. | .400** |
Test-retest reliability
A subset of n = 112 participants from the original sample of participants consented to complete the survey again. This subset of n =112 is different from the n =112 medical students discussed later. The intra-class correlation coefficient was 0.66 (95% CI 0.54 to 0.75, p < 0.001) for the total scores indicating near satisfactory test-retest stability as an intra-class coefficient of 0.7 or higher is considered satisfactory [50, 51].
The demographic characteristics of the sample (n = 112) that responded to the OMS-HC for the test-retest reliability were similar to the entire sample. Again, the majority were white, female, nurses or social workers aged 26–64 years from British Columbia, Alberta, Manitoba and Ontario.
Social desirability bias testing
The total score on the OMS-HC was correlated with the total score of the Marlowe-Crowne Social Desirability scale (short form) for a sample of health care providers (n = 669). There was a small but significant correlation, ρ = 0.10, (95% CI 0.08 to 0.13, p = 0.01).
Factor analysis
Factor analysis of remaining 12 items of the OMS-HC
Cronbach’s alpha = 0.78 | Factor Loading |
|---|---|
Attitudes of health care providers towards people with mental illness, Cronbach’s alpha = 0.75 | |
· I am more comfortable helping a person who has a physical illness than I am helping a person who has a mental illness. (Item 1 of original scale) | 0.61 |
· If a person with a mental illness complains of physical symptoms (e.g. nausea, back pain or headache), I would likely attribute this to their mental illness. (Item 2 of original scale) | 0.51 |
· Despite my professional beliefs, I have negative reactions towards people who have mental illness. (Item 12 of original scale) | 0.67 |
· There is little I can do to help people with mental illness. (Item 13 of original scale) | 0.75 |
· More than half of people with mental illness don’t try hard enough to get better. (Item 14 of original scale) | 0.62 |
· Healthcare providers do not need to be advocates for people with mental illness. (Item 18 of original scale) | 0.56 |
· I struggle to feel compassion for a person with a mental illness. (Item 20 of original scale) | 0.64 |
Attitudes of health care providers towards disclosure of a mental illness, Cronbach’s alpha = 0.72 | |
· If I were under treatment for a mental illness I would not disclose this to any of my colleagues. (Item 4 of original scale) | 0.76 |
· I would be more inclined to seek help for a mental illness if my treating healthcare provider was not associated with my workplace. (Item 5 of original scale) | 0.61 |
· I would see myself as weak if I had a mental illness and could not fix it myself. (Item 6 of original scale) | 0.60 |
· I would be reluctant to seek help if I had a mental illness. (Item 7 of original scale) | 0.67 |
· If I had a mental illness, I would tell my friends. (Item 10 of original scale) | 0.70 |
Internal consistency
The Cronbach’s alpha of the 20-item scale was 0.82. After removal of the items during factor analysis cronbach’s alpha was 0.78 (0.75 and 0.72 for the attitudes towards people with mental illness subscale and attitudes towards disclosure of a mental illness subscale respectively). An alpha greater than 0.70 was considered acceptable [52]. When correlating the subscales, the attitudes towards mental illness subscale was moderately correlated with the subscale pertaining to attitudes towards disclosure, ρ = 0.30, (95% C.I. 0.23 to 0.36, p <0.001).
Discussion
Until this study, a scale had not been developed in Canada to assess the attitudes of health care providers towards people with mental illness. We have developed and tested a scale that we subsequently found to have good internal consistency and adequate test-retest reliability during its testing phase. The development of the scale was guided by the tri-partite model of stigma arising from literature reviews focusing on the stigma and how attitudes towards people with mental illness can be measured. We chose to adopt the tri-partite model because this allows for the measurement of clear outcomes such as attitudes which can be measured using the various dimensions of stigma.
Extensive stakeholder consultations were held with regards to the items to be included in the scale and items included the dimensions of recovery, social responsibility and social distance. Other dimensions of stigma such as dangerousness, treatment of people with mental illness and disclosure of a mental illness were also seen as important and items pertaining to them were included in the scale.
The scale did not correlate highly with a short form of the Social Desirability scale. This indicates that while there may be some relationship between measuring stigma and social desirability (i.e. people responding to the instrument may attempt to provide socially desirable responses), the scale responses were not strongly related to social desirability. Social desirability can be a threat to validity of scales such as this one.
As with any scale, consensus on what to measure and the wording of items is difficult to obtain especially in the area of mental illness where there are different schools of thought in treatment and recovery. Through our study, which used thorough methods from developing to testing the OMS-HC, we believe that we obtained views from several types of health care providers from the focus groups and tested the scale across Canada to obtain a wide range of professions, age groups as well as representation of sex and province. Although we did not achieve fully comprehensive representation of professions such as medical doctors, provinces such as the northern, central, French speaking, and maritime parts of Canada, our sample was in many respects broadly representative of diverse health care provider groups. This was the first attempt to develop a tool to assess attitudes for health care providers in Canada.
The total scores were slightly higher in the 18–25 age group, those in the physician/medical student group and those in Alberta. This requires further exploration and future research can look into closely examining other variables that may influence scores on the OMS-HC such as socio-demographic variables. The factor analysis showed two subscales which measured attitudes towards people with mental illness using the various dimensions of stigma and attitudes towards disclosure and help seeking. While the 12-item scale accounts for only 45% of the variance, this scale is succinct in its number of items which would offset response burden and increase the feasibility of its use in busy health care providers. Future research could also examine testing more items along with those tested in this study to determine if these change the variance accounted for by the two factors or if other factors emerge with sufficient items for additional subscales. Factor analysis in each health care provider group may also provide insight into subscales which may be unique to certain health care provider groups.
We adopted the idea that stigmatizing attitudes can also be measured in the form of disclosing whether one has a mental illness and/or seeking help for it, because self stigma may also be an indicator of mental illness related stigma [24, 25]. Likewise, those who would disclose that they had a mental illness may not think that mental illness is something to be ashamed of and may therefore be less stigmatizing to other people with mental illness. In support of the literature that supports self stigma as distinct from holding stigmatizing attitudes towards people with mental illness [25], we found that while health care provider’s attitudes towards disclosure were related to their attitudes towards people of mental illness, there was only a moderate correlation between these two constructs. Evidence for this was also shown by our factor analysis which yielded a two factor solution for the OMS-HC in which we had a 7 item subscale measuring attitudes towards people with mental illness and a 5 item subscale measuring attitudes towards disclosure of a mental illness.
As such, the overall scale should be used with caution given the relatively low between factor correlation. Nonetheless, there is a possibility that the OMS-HC may behave differently in different groups of health care providers so we advise using the overall scale as well as subscale scores to gain further insight into participant responses. Attitudes towards disclosure and help seeking are important to measure in health care providers as this provides an indication of the stigma that they believe exists due to having a mental illness and how this would impact seeking help. As health care providers are not immune to mental illness themselves, this is an important dimension of stigma to measure in relation to health care provider well being [53].
As there is no gold standard for assessing attitudes of Canadian health care providers towards people with mental illness, we hope that this scale provides a good starting point for further testing and development as well as a pilot scale that can be used in the evaluation of programs aimed at reducing the stigma of mental illness in health care providers. This scale can be used both in the realms of education such as evaluating courses in health care programs from undergraduate to postgraduate areas of health care providers, continuing professional development as well as part of quality improvement and quality assurance initiative within hospitals and health care regions.
Future research with the OMS-HC should also include its use in evaluating future mental illness anti-stigma related educational interventions, validating it in French so that it can be used in French speaking areas of Canada and validating it with physicians across Canada. Other areas of future research could incorporate additional instruments for external validity such as those that measure knowledge [54] intended behaviour [55] or behavior such as Objective Structured Clinical Exams (OSCEs) used in teaching health care providers to determine the relationship between the tri-partite model components of knowledge, attitudes and behavior. Further research in determining minimally important change (MIC) [56] (the meaning of changes in scores on the OMS-HC over time) is also warranted. This would provide valuable information to detect clinically important changes also known as the responsiveness of an instrument which is an aspect of validity [22].
Conclusions
The OMS–HC provides a good basis for further validation as well as a tool that could be used in the evaluation of programs aimed at reducing mental illness related stigma by health care providers. Future research and testing is required for the OMS-HC pertaining to additional items and other variables that may influence responses of health care providers such as demographic variables. The OMS-HC incorporates various dimensions of stigma with a modest number of items that can be used with busy health care providers. While this project is supported by a national commission, it may have international applicability.
Endnotes
aAs consensus could not be reached, we decided to keep the term mental illness because we thought the term itself would evoke stigma. Where applicable in terms of severity, we added the word “managed” to the item pertaining to hiring a person with mental illness since they may be the best person for the job if their condition was managed. We felt that the term “mental illness” in itself would evoke a response regardless of severity.
Declarations
Acknowledgement
This project was made possible through funding from the Mental Health Commission of Canada. The work of the Mental Health Commission of Canada is supported by a grant from Health Canada. Dr. Patten is a Senior Health Scholar with Alberta Innovates, Health Solutions.
Authors’ Affiliations
References
- Oliver MI, Pearson N, Coe N, Gunnell D: Help-seeking behaviour in men and women with common mental health problems: cross-sectional study. Br J Psychiatry. 2005, 186: 297-301. 10.1192/bjp.186.4.297.View ArticlePubMedGoogle Scholar
- Thornicroft G, Brohan E, Kassam A, Lewis-Holmes E: Reducing stigma and discrimination: Candidate interventions. Int J Ment Health Syst. 2008, 2: 1-7. 10.1186/1752-4458-2-1.View ArticleGoogle Scholar
- Mueser KT, Corrigan PW, Hilton DW, et al: Illness Management and Recovery: A Review of the Research. Psychiatr Serv. 2002, 53: 1272-1284. 10.1176/appi.ps.53.10.1272.View ArticlePubMedGoogle Scholar
- Corrigan PW: How stigma interferes with mental health care. Am Psychol. 2004, 59: 614-625.View ArticlePubMedGoogle Scholar
- Link BG, Cullen FT, Frank J, Wozniak J: The social rejection of ex-mental patients: understanding why labels matter. Am J Sociol. 1987, 92: 1461-1500. 10.1086/228672.View ArticleGoogle Scholar
- Link BG, Cullen FT: Contact with the mentally ill and perceptions of how dangerous they are. J Health Soc Behav. 1986, 27: 289-302. 10.2307/2136945.View ArticlePubMedGoogle Scholar
- Corrigan P, Markowitz FE, Watson A, Rowan D, Kubiak MA: An attribution model of public discrimination towards persons with mental illness. J Health Soc Behav. 2003, 44: 162-179. 10.2307/1519806.View ArticlePubMedGoogle Scholar
- Corrigan PW, Watson AC: The Paradox of Self-Stigma and Mental Illness. Clinical Psychology: Science and Practice. 2002, 9 (1): 35-53. 10.1093/clipsy.9.1.35.Google Scholar
- Bentz WK, Hollister WG, Kherlopian M: Attitudes of Social Distance and Social Responsibility for Mental Illness: A Comparison of Teachers and the General Public. Psychol Sch. 1970, 7 (2): 198-203. 10.1002/1520-6807(197004)7:2<198::AID-PITS2310070218>3.0.CO;2-D.View ArticleGoogle Scholar
- Angermeyer MC, Holzinger A, Matschinger H: Emotional reactions to people with mental illness. Epidemiol Psichiatr Soc. 2010, 19 (1): 26-32.View ArticlePubMedGoogle Scholar
- Jacoby A: Felt versus enacted stigma: a concept revisited. Evidence from a study of people with epilepsy in remission. Soc Sci Med. 1994, 38 (2): 269-274. 10.1016/0277-9536(94)90396-4.View ArticlePubMedGoogle Scholar
- Spandler H, Stickley T: No hope without compassion: the importance of compassion in recovery-focused mental health services. J Ment Health. 2011, 20 (6): 555-566. 10.3109/09638237.2011.583949.View ArticlePubMedGoogle Scholar
- Golberstein E, Eisenberg D, Gollust SE: Perceived stigma and help-seeking behavior: longitudinal evidence from the Healthy Minds study. Psychiatr Serv. 2009, 60: 1254-1256. 10.1176/appi.ps.60.9.1254.View ArticlePubMedPubMed CentralGoogle Scholar
- Schomerus G, Matschinger H, Angermeyer MC: The stigma of psychiatric treatment and help-seeking intentions for depression. Eur Arch Psychi Clin Neurosci. 2009, 259: 298-306. 10.1007/s00406-009-0870-y.View ArticleGoogle Scholar
- Link BG, Phelan JC: Conceptualizing stigma. Annu Rev Sociol. 2001, 27: 363-385. 10.1146/annurev.soc.27.1.363.View ArticleGoogle Scholar
- Corrigan P: On the Stigma of Mental Illness. 2005, American Psychological Association, Washington, D.CGoogle Scholar
- Thornicroft G: Shunned: Discrimination against people with mental illness. 2006, Oxford University Press, New YorkGoogle Scholar
- Johnston JM, Leung GM, Fielding R, Tin KY, Ho LM: The development and validation of a knowledge, attitude and behaviour questionnaire to assess undergraduate evidence-based practice teaching and learning. Medical Education. 2003, 37: 992-1000. 10.1046/j.1365-2923.2003.01678.x.View ArticlePubMedGoogle Scholar
- Hoke MM, Robbins LK: Continuing the cultural competency journey through exploration of knowledge, attitudes, and skills with advanced practice psychiatric nursing students: an exemplar. Nurs Clin North Am. 2011, 46 (2): 201-205. 10.1016/j.cnur.2011.02.004.View ArticlePubMedGoogle Scholar
- Kajermo KN, Tyni-Lenné R, Guidetti S, Samuelsson M, Andersson IL, Wengström Y: Evidence-based practice: attitudes, knowledge and behaviour among allied health care professionals. Int J Qual Health Care. 2011, 23 (2): 198-209. 10.1093/intqhc/mzq083.View ArticlePubMedGoogle Scholar
- La Torre G, Semyonov L, Mannocci A, Boccia A: Knowledge, attitude, and behaviour of public health doctors towards pandemic influenza compared to the general population in Italy. Scand J Public Health. 2011, 0: 1-7.Google Scholar
- National Institute for Health and Clinical Excellence: Behaviour change at population, community and individual levels. London. 2007, : -Google Scholar
- Corrigan PW, Rowan D, Green A, Lundin R, River P, Uphoff-Wasowski K, White K, Kubiak MA: Challenging two mental illness stigmas: personal responsibility and dangerousness. Schizophr Bull. 2002, 28 (2): 293-309. 10.1093/oxfordjournals.schbul.a006939.View ArticlePubMedGoogle Scholar
- Corrigan PW, Watson AC: At issue: Stop the stigma: call mental illness a brain disease. Schizophr Bull. 2004, 30 (3): 477-9. 10.1093/oxfordjournals.schbul.a007095.View ArticlePubMedGoogle Scholar
- Hocking B: Reducing mental illness stigma and discrimination - everybody's business. Med J. 2003, 178 (9): S47-S48.Google Scholar
- Kassam A, Glozier N, Leese M, et al: Development and responsiveness of a scale to measure clinicians' attitudes to people with mental illness (medical student version). Acta Psychiatr Scand. 2010, 122: 153-61. 10.1111/j.1600-0447.2010.01562.x.View ArticlePubMedGoogle Scholar
- Disability Rights Commission: Equal treatment-Closing the gap (Report No. 1). Great Britain. 2006.Google Scholar
- Felker B, Yazel JJ, Short D: Mortality and medical co-morbidity among psychiatric patients: a review. Psychiatr Serv. 1996, 47: 1356-1363.View ArticlePubMedGoogle Scholar
- Harris EC, Barraclough B: Excess mortality of mental disorder. Br J Psychiatry. 1998, 173: 11-53. 10.1192/bjp.173.1.11.View ArticlePubMedGoogle Scholar
- Jones S, Howard L, Thornicroft G: ‘diagnostic overshadowing’: worse physical health care for people with mental illness. Acta Psychiatr Scand. 2008, 118: 169-173. 10.1111/j.1600-0447.2008.01211.x.View ArticlePubMedGoogle Scholar
- Roberts L, Roalfe A, Wilson S, Lester H: Physical health care of patients with schizophrenia in primary care: A comparative study. Fam Pract. 2007, 24 (1): 34-40.View ArticlePubMedGoogle Scholar
- Druss BG, Bradford DW, Rosenheck RA, Radford MJ, Krumholz HM: Mental disorders and use of cardiovascular procedures after myocardial infarction. JAMA. 2000, 283: 506-511. 10.1001/jama.283.4.506.View ArticlePubMedGoogle Scholar
- Sullivan G, Han X, Moore S, Kotrla K: Disparities in hospitalization for diabetes among persons with and without co-occurring mental disorders. Psychiatr Serv. 2006, 57: 1126-1131. 10.1176/appi.ps.57.8.1126.View ArticlePubMedGoogle Scholar
- Druss BG, Rosenheck RA: Mental disorders and access to medical care in the United States. Am J Psychiatry. 1998, 155: 1775-1777.View ArticlePubMedGoogle Scholar
- Bradford DW, Kim MM, Braxton LE, Marx CE, Butterfield M, Elbogen EB: Access to medical care among persons with psychotic and major affective disorders. Psychiatr Serv. 2008, 59: 847-852. 10.1176/appi.ps.59.8.847.View ArticlePubMedGoogle Scholar
- Hodges B, Inch C, Silver I: Improving the psychiatric knowledge, skills, and attitudes of primary care physicians, 1950–2000: a review. Am J Psychiatry. 2001, 158: 1579-1586. 10.1176/appi.ajp.158.10.1579.View ArticlePubMedGoogle Scholar
- Mirabi M, Weinman ML, Magnetti SM, Keppler KN: Professional attitudes toward the chronic mentally ill. Psychiatr Serv. 1985, 36: 404-405.View ArticleGoogle Scholar
- Ucok A: Other people stigmatize but, what about us? Attitudes of mental health professionals towards patients with schizophrenia. Archives of Neuropsychiatry. 2007, 44: 108-116.Google Scholar
- Ucok A, Polat A, Sartorius N, Erkoc S, Atakli C: Attitudes of psychiatrists toward patients with schizophrenia. Psychiatry Clin Neurosci. 2004, 58: 89-91. 10.1111/j.1440-1819.2004.01198.x.View ArticlePubMedGoogle Scholar
- Lauber C, Nordt C, Braunschweig C, Rossler W: Do mental health professionals stigmatize their patients?. Acta Psychiatr Scand Suppl. 2006, 113: 51-59. 10.1111/j.1600-0447.2005.00718.x.View ArticleGoogle Scholar
- Lauber C, Anthony M, Ajdacic-Gross V, Rossler W: What about psychiatrists' attitude to mentally ill people?. European Psychiatry: the Journal of the Association of European Psychiatrists. 2004, 19: 423-427. 10.1016/j.eurpsy.2004.06.019.View ArticleGoogle Scholar
- Rowe J: Great expectations: a systematic review of the literature on the role of family carers in severe mental illness, and their relationships and engagement with professionals. J Psychiatr Ment Health Nurs. 2012, 19 (1): 70-82. 10.1111/j.1365-2850.2011.01756.View ArticlePubMedGoogle Scholar
- Wilkes M, Bligh J: Evaluating educational interventions. BMJ. 1999, 318: 1269-1272. 10.1136/bmj.318.7193.1269.View ArticlePubMedPubMed CentralGoogle Scholar
- Hutchinson L: Evaluating and researching the effectiveness of educational interventions. BMJ. 1999, 318: 1267-1269. 10.1136/bmj.318.7193.1267.View ArticlePubMedPubMed CentralGoogle Scholar
- Campanelli P, Martin E, Rothgeb JM: The use of respondent and interviewer debriefing studies as a way to study response error in survey data. The Statistician. 1991, 40: 253-264. 10.2307/2348278.View ArticleGoogle Scholar
- Mental Health Commission of Canada: Toward recovery & well-being: A framework for a mental health strategy for Canada. Ottawa. 2009.Google Scholar
- Reynolds RW: Development of reliable and valid short forms of the Marlowe-Crowne social desirability scale. J Clin Psychol. 1982, 38: 119-125. 10.1002/1097-4679(198201)38:1<119::AID-JCLP2270380118>3.0.CO;2-I.View ArticleGoogle Scholar
- Kendall MG: Rank correlation methods. 1962, Griffin, LondonGoogle Scholar
- Christison GW, Haviland MG, Riggs ML: The medical condition regard scale: measuring reactions to diagnoses. Acad Med. 2002, 77: 257-262. 10.1097/00001888-200203000-00017.View ArticlePubMedGoogle Scholar
- Portney LG, Watkins MP: Foundations of clinical research applications to practice. 2000, Prentice Hall Inc, New JerseyGoogle Scholar
- Netemeyer RG, Bearden W, Sharma SC: Scaling Procedures: Issues and Applications. 2003, Sage, Thousand OaksView ArticleGoogle Scholar
- Streiner DL: Starting at the beginning: an introduction to coefficient alpha and internal consistency. J Pers Assess. 2003, 80: 99-103. 10.1207/S15327752JPA8001_18.View ArticlePubMedGoogle Scholar
- Voltmer E, Kieschke U, Schwappach DL, Wirsching M, Spahn C: Psychosocial health risk factors and resources of medical students and physicians: a cross-sectional study. BMC Med Educ. 2008, 8: 46-10.1186/1472-6920-8-46.View ArticlePubMedPubMed CentralGoogle Scholar
- Evans-Lacko S, Rose D, Little K, Flach C, Rhydderch D, Henderson C, Thornicroft G: Development and Psychometric Properties of the Reported and Intended Behaviour Scale (RIBS): A Stigma Related Behaviour Measure. Epidemiol Psychiatr Sci. 2011, 20: 263-271. 10.1017/S2045796011000308.View ArticlePubMedGoogle Scholar
- Evans-Lacko S, Little K, Meltzer H, Rose D, Rhydderch D, Henderson C, Thornicroft G: Development and Psychometric Properties of the Mental Health Knowledge Schedule (MAKS). Can J Psychiatry. 2010, 55 (7): 440-8.PubMedGoogle Scholar
- de Vet HC, Terwee CB, Ostelo RW, Beckerman H, Knol DL, Bouter LM: Minimal changes in health status questionnaires: distinction between minimally detectable change and minimally important change. Health Qual Life Outcomes. 2006, 4: 54-10.1186/1477-7525-4-54.View ArticlePubMedPubMed CentralGoogle Scholar
- The pre-publication history for this paper can be accessed here:http://www.biomedcentral.com/1471-244X/12/62/prepub
Pre-publication history
Copyright
This article is published under license to BioMed Central Ltd. This is an Open Access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.
