Skip to main content

Development and validity of the Value-based Stigma Inventory (VASI): a value-sensitive questionnaire for the assessment of mental health stigma

Abstract

Background

It has been hypothesized that mental illness stigma differs according to what matters most to people, and that this results in value-based differences in stigma within societies. However, there is a lack of stigma measures that account for a broad range of values, including modern and liberal values.

Methods

For the development of the Value-based Stigma Inventory (VASI) a preliminary item-pool of 68 VASI-items was assembled by mental health and stigma experts. For psychometric evaluation, we tested the VASI in an online sample of the general population (n = 4983).

Results

Based on item-characteristics as well as explorative and confirmatory factor analyses, a final version of the VASI was developed, comprising 15 items and 5 subscales. The VASI shows good psychometric properties (item difficulty = 0.34 to 0.67; mean inter-item correlation r = 0.326; Cronbach’s α = 0.879). Medium to high correlations with established stigma scales (SDS, SSMI), medium associations with instruments assessing personal values (PVQ, KSA-3) and small to no associations with a social desirability scale (KSE-G) attest to good convergent and discriminatory validity of the new instrument. Normative values for the VASI subscales are presented.

Conclusions

The developed VASI can be used to assess public stigma of mental illness including personal stigma-relevant value orientations.

Peer Review reports

Background

The stigma of mental illness has been shown to vary between different cultures [1]. To conceptualize cultural differences in mental illness stigma, Yang and co-workers [2] have introduced the theory of What Matters Most (WMM), predicting that stigma is most visible when threatening the loss or diminution of what matters most within a cultural context. They argue that culture specific measures of stigma are needed to measure stigma in those cultural interactions that are most relevant to the respondents of a specific cultural background.

Not only between, but also within cultures there are differences of WMM, visible for example in the growing polarization of Western societies. Inglehart and Norris [3] found that it is not so much socioeconomic status but primarily opposing cultural world views that play a central role in the formation of the Left-Right cleavage. They state that the shift to post-materialistic and cosmopolitan values is accompanied by a protesting departure from liberal values of tolerance and self-realization. Differentiations between the value dimension of the progressive, cosmopolitan milieu and the traditionalist, communitarian milieu are made by various authors [4,5,6].

For understanding such differences in mental illness stigma within cultures, Schomerus and Angermeyer [7] apply the WMM theory to the formation of stigma within Western societies. The authors hypothesize that, although conservative/authoritarian values have traditionally been found to be associated with mental illness stigma, liberal/modern values can similarly lead to stigma, but have not been thoroughly examined in this respect. Assuming a ‘blind spot in stigma research’, Schomerus and Angermeyer [7] hypothesize that liberal-minded people might also stigmatize people with mental illness, if these are perceived as a potential threat to their liberal values.

The theory that individuals of various outgroups are more likely prejudiced and discriminated if they are perceived as a threat to personal values is empirically supported [2, 8,9,10,11,12,13]. Brandt and coworkers [14] showed in three independent studies intolerance of both conservatives and liberals of groups whose values and worldviews contradict with their ideological worldview. To counter the assumed blind spot among mental illness stigma research, instruments are needed that measure mental illness stigma with regard to situations relevant both to people with conservative/authoritarian and liberal/modern values.

Aims & research hypotheses

The present study reports on the development and validation of a Value-based Stigma Inventory (VASI). To ensure the validity of the instrument, we examine aspects of discriminatory and convergent validity, setting up several assumptions. According to Cohen [15], correlation coefficients between the new instrument and established stigma scales should be moderate to high (≥0.5) and moderate (> 0.3) between the new instrument and established value scales. Both authoritarian and liberal values should be associated with increased stigmatization in respective subscales. Finally, correlation coefficients between the new instrument and an established social desirability scale should be small.

Methods

Creation of a preliminary item pool (VASI-68)

First, based on the heuristic model of party competition in Western societies by Inglehart and Norris [3] and the theory of basic values of Schwartz [16] we distinguished six possible value orientations: liberal, authoritarian, community welfare, individual well-being, performance and hedonism. Second, items were generated by a team of mental health stigma experts and a person with lived experience (MA, EP, SR, GS) relating stigma and exclusion to potential conflicts with these value orientations. For example, the item If you are living with a person with mental illness, it is difficult to lead a life according to your own ideas was created to illustrate a potential conflict with individualism and hedonism. Third, by means of a rating by an extended team of experts (including MA, SR, CS, SS and GS), items were reformulated and reduced from 156 to 68, taking into account criteria such as comprehensibility, unambiguity, everyday relevance, face validity and possible response tendencies.

Data collection

A sample of the German adult population who were registered in a professional online-access panel (KANTAR) responded to an online survey containing 68 VASI items and other questionnaires related to mental health stigma and personal values. The study was approved by the Ethics Committee of the University of Leipzig (239/20-ek) and conducted in accordance to the declaration of Helsinki. All participants provided informed consent. Data collection was carried out in cooperation with a social and market research institute (USUMA).

Sampling was stratified for age, gender and place of residence of the respondents, resulting in a sample matching the general population for these characteristics. A total of 8136 people responded to the invitation to the survey. Of these, 199 (2.4%) did not agree to participate, 681 (8.4%) did not complete the questionnaire and 754 (9.3%) could not participate since the predefined quota for age, gender or place of residence were already accomplished.

After pretesting the questionnaire, a minimum time of 11 min to respond to all items had been determined to reduce the number of rapid guessers [17]. A total of 1.480 (18.2%) responders fell below this time limit and were therefore excluded. A further 10 data sets had to be excluded due to a high proportion of item non-response. In total, 5012 unique datasets were obtained.

Sample

From the dataset of 5012 cases, 29 datasets (0.6%) were excluded due to missing values in at least one of the 68 VASI items. Therefore, the final sample comprises of 4983 data sets. Of these, 2373 (47.6%) were from male and 2604 (52.3%) from female participants, while 6 participants (0.1%) identified as divers. The average age of the sample was 47.4 years (SD = 14.387, range 18-74 years). Further socio-demographic data can be found in Table 1.

Table 1. Characteristics of the study population

Survey

The survey was subdivided in order to allow the use of different instruments, while at the same time ensuring a tolerable survey length for the individual participant. All survey participants first answered the preliminary VASI items. The items were presented in randomized order to avoid sequence effects. Participants were asked to respond to each item on a 5-point Likert scale ranging from 1 (strongly disagree) to 5 (strongly agree).

After answering the VASI-items, participants were randomly assigned to one of two groups differing with respect to the applied value measures (see Fig. 1). Subgroup A (N = 2489) processed several items on socio-political opinions that are not used for this publication. Subgroup B (N = 2494) answered the Portrait Values Questionnaire (PVQ, [18]). The 21-item version (PVQ-21) was used as applied in the European Social Survey (ESS). For a better representation of the subscale universalism, an additional item from the long version representing the subscale universalism was added (He wants everyone to be treated justly, even people he doesn’t know. It is important to him to protect the weak in society, [19]). Both subgroups answered the Authoritarianism Short Scale (KSA-3 [20]).

Fig. 1
figure 1

Survey structure

Afterwards, both groups were again divided randomly (ratio 4:1) into two subgroups, which answered different stigma scales. The first subgroup (N = 3986) processed the subscales Aware and Agree of the Self-Stigma in People with Mental Illness scale (SSMI-short form [21], German version: [22]) and the Social Distance Scale (SDS [23], German version: [24]). The latter was used in a modified version with two additional items for a more contemporary recording of social distance (To what extent would you invite such a person to your home?, To what extent would you give such a person your phone number?). The second subgroup (N = 997) answered the Prejudice against people with mental Illness scale (PPMI [25], results not used for this publication). Finally, all participants completed the Social Desirability–Gamma Short Scale (KSE-G [26]).

Statistical analyses

For psychometric analysis, item distributions were tested for normality using the Shapiro-Wilks test. Item difficulty was calculated as pm = sum of item scores / (N * maximum item score) and item-total correlations (rit) were calculated as measure of item discriminatory power. According to Bortz and Döring [27] discriminatory power should be > 0.5, while items with discriminatory power < 0.3 therefore require a scale revision, e.g. by deleting the affected items.

Exploratory and confirmatory analyses of the factorial structure were carried out in two independent subgroups to which the survey participants were randomly assigned during the study. First, in subsample A exploratory factor analyses were carried out using a principal component analysis (PCA) approach, including all items of the preliminary VASI item pool with satisfactory item discriminatory power (rit > 0.3). The Kaiser-Meyer-Olkin (KMO) measure and the Bartlett test were used to check if requirements for factor analysis were met. Eigenvalues (> 1) and the scree plot are used as criteria for determining the optimal number of factors. To obtain factors that are as uncorrelated as possible, an orthogonal rotation technique was used. According to Gorsuch [28], a varimax rotation is inappropriate because a general factor (mental health stigma) can be expected. Therefore, an Equamax rotation was chosen, which would distribute the variance of the factor loadings as evenly as possible over the factors. Factor loadings < 0.3 were not considered, as such low factor loadings indicate that a variable has less than 10% common variance with the factor [29, 30].

Subsequently, confirmatory factor analyses (CFA) using maximum likelihood estimations were performed in subsample B to confirm the factorial structure. Normality of the data was examined using the Mardia test. The adequacy of the model fit was assessed using goodness-of-fit indices recommended by Beauducel and Wittmann [31] and DiStefano and Hess [32], with cutoffs specified in various publications [33,34,35]: Chi2-test (with Bollen-Stine-Bootstrap in case of multivariate non-normality); Comparative Fit Index (CFI, cutoff ≥0.95), Tucker-Lewis Index (TLI, cutoff > 0.95), Root Mean Square Error of Approximation (RMSEA, cutoff < 0.06) and Standardized Root Mean Residual (SRMR, cutoff < 0.08).

Scores of the resulting VASI subscales were analysed using descriptive statistics: mean, standard deviations as well as quartiles and selected percentiles. Cronbach’s α was computed as a measure of internal consistency. To investigate discriminative and convergent validity of the VASI, Spearman’s rank correlations coefficients were calculated.

The significance level was set at α = 0.05 (two-tailed). All statistical analyses were conducted using the statistical software SPSS 24 (IBM SPSS Statistics for Windows, version 24). For the confirmatory factor analyses, IBM SPSS AMOS 26.0 was used.

Results

Table 1 depicts the sociodemographic characteristics of the total sample (compared to the German population). The overall sample corresponds to the German population in terms of gender distribution, but comprises fewer participants in the age segment > 65 years, as these population strata can generally hardly be reached by online surveys. With regard to the educational level, our sample shows a higher proportion of respondents with high educational levels, which is also typical of online studies. The two subgroups A and B were created by random assignment and accordingly do not differ in terms of the socio-demographic characteristics.

VASI-68

Item analysis

The 68 preliminary VASI-items were first subjected to an item analysis (results not shown). The Shapiro-Wilks test confirmed that the distribution of all items differed significantly from a normal distribution (all p < .01). Out of the 68 items, 53 showed a significantly positive skewness (maximum 1.47), while 10 items were negatively skewed (minimum − 0.42). Kurtosis was significantly negative in 53 items (minimum − 0.93) and positive in 11 items (maximum 1.65). Difficulty indices were of medium size (range 0.34 to 0.84). The corrected item-total-correlations (rit) ranged from 0.11 to 0.74 with rit < 0.3 in 6 items, which were therefore removed from the VASI item pool for the next evaluations.

Exploratory factor analysis and item reduction

In order to determine the factorial structure of the VASI and to reduce the number of items, an explorative factor analysis was carried out on the remaining 62 VASI items in subsample A (N = 2989). The Kaiser-Meyer-Olkin (KMO) measure of 0.980 and the Bartlett test (p < .001) showed that the requirements for factor analysis were met. A Principal Component Analysis (PCA) resulted in seven components with eigenvalues (EV) > 1, explaining a cumulative 53.3% of the variance. Based on the screeplot a strong general factor (EV = 21.018, explained variance 33.9%) could be identified on which all items had factor loadings > 0.3 with polarities corresponding to the direction of the respective item wording.

Since item compilation had been based on six topics and the EV of the 7th component (1.001) was only slightly above the usual cutoff, in the next step a PCA with Equamax-rotation was performed with a fixed amount of six factors, which still explained a large amount of variance (51.6%). All items had factor loadings >.300 on at least one component, yet several items showed cross loadings > 0.3 on at least one additional component.

Based on the item loadings, components could be described meaningfully in terms of content, although some deviations from the original six topics of the item compilation were noted. In the sense of the intended item reduction, for each component, the three items with the highest factor loadings were selected from all items that could be clearly assigned to a respective component (i.e. without cross-loading > 0.3 on other components). In a final revision, in order to avoid redundancies in the item wording, some items were exchanged with items of lower loadings but more relevant content. However, only two items could be allocated to the 6th component, as any possible 3rd item charged on at least one of the other five components. Therefore, the final item selection comprised of 15 items allocated to five components. These 15 items were again subjected to a PCA with five fixed components and Equamax-rotation to check whether the item reduction had an impact on the allocation of items to components (KMO = 0.899; Bartlett: p < .001; explained variance: 68.7%). Factor loadings on the respective components were high (range .656 to .853) and no item had loadings > 0.3 on more than one component (see Table 2).

Table 2. Final VASI-15 Items and factor loadings

Confirmatory factor analysis (CFA)

To verify the validity of the exploratory factor analysis (see Fig. 2), a CFA was performed on the 15 selected items using data from subsample B (N = 2494). A significant Mardia test (z = 58.244, p < .001) indicated the absence of a multivariate normal distribution, although skewness and excess of all items were below the limits proposed by West and colleges [36]. Despite the implementation of a Bollen-Stine-Bootstrap, the Chi2 test reached significance (X2(80) = 502.902; p = .001), which questioned the fit of the model. However, other goodness-of-fit indices consistently attested a good model fit (CFI = 0.969; TLI = 0.959; RMSEA = 0.046; SRMR = .0368). Factor loadings were again high for all items (range .614 to .842, see Table 2). A description of the VASI scales from a content perspective is given in the next section.

Fig. 2
figure 2

CFA of the five-factor model of the VASI-15

VASI-15

Description of VASI-subscales and item analysis

Table 3 shows the 15 final VASI items, with their assignment to the five VASI subscales, which are described as follows:

Table 3. Item characteristics of the final 15 VASI-items

Subscale 1 Self-realization (SR) focusses on the extent to which the presence of people with mental illness in the immediate environment is perceived as restricting one’s well-being and enjoyment of life as well as the ability to achieve one’s individual life goals.

Subscale 2 Personal Enrichment (PE) includes the extent to which dealing with mentally ill persons is perceived as an enrichment for one’s life and represents an openness to diversity. Underlying are humanistic values. In contrast to the other VASI scales, these values are not threatened but rather promoted by dealing with the mentally ill persons. Accordingly, the items of this scale are inversely coded and need to be inverted before they can be included in the calculation of an overall VASI score.

Subscale 3 Reputation (RE) deals with the loss of social status associated with mental illness. It addresses the extent to which being associated with mentally ill people would be considered detrimental to one’s reputation.

Subscale 4 Meritocratic Values (MV) deals with two aspects of individualistic values. First, an underlying belief that too much consideration is given to mentally ill people, which is perceived as unfair against those who are not mentally ill. Mentally ill people are perceived as receiving benefits for which the respondent feels more entitled. Second, the scale includes the attitude of personal responsibility in conjunction with a high level of self-improvement, according to the principle that everyone is responsible for themselves. Both aspects can be summarized under the label Meritocratic Values, where people achieve and earn what they deserve.

Subscale 5 Security (SE) deals with the perceived dangerousness of mentally ill people, to whom a propensity to violence and a willingness to commit particularly serious crimes are attributed. The attitudes recorded on this scale correspond most closely to the stigmatising attitudes associated with authoritarianism attitudes.

The VASI subscales Self-realization and Security relate to the similarly titled subscales of the basic values of Schwartz [16], whereas Personal Enrichment is comparable to Schwartz´ Self-Transcendence. Reputation and Meritocratic Values are factors which are not to be equated with the basic values of Schwartz in their importance.

Item characteristics of the 15 VASI items are shown in Table 3. The difficulty indices ranged from 0.34 to 0.67, thus attesting to a medium difficulty. The mean inter-item correlation was r = 0.326, thus attesting to an optimal item homogeneity. Corrected item-total correlations ranged from 0.40 to 0.65 and the internal consistency was good for the total VASI scale (Cronach’s α = 0.879) and acceptable for the five VASI subscales (SR: α = 0.805; PE: α = 0.729; RE: α = 0.756; MV: α = 0.721; SE: α = 0.769). Mean Scores of the VASI scales separated by gender and age-groups are shown in Table 4. The final VASI questionnaire and notes on its evaluation can be found in Additional file 1.

Table 4. VASI-15 Mean Scores

Discriminant & convergent validity

Correlations of the VASI-scores and other scales are depicted in Table 5. As expected, the highest correlations were found between the overall VASI score and the individual VASI subscales (R = .636 to .787). Intercorrelations between the five VASI-subscales were of medium size, with highest correlations between subscale Security and subscale Meritocratic Values (R = .522) as well as with subscale Reputation (R = .518) and lowest correlations between subscale Personal Enrichment and subscales Reputation (R = .307), Meritocratic Values (R = .355) and Security (R = .367).

Table 5. Rank-Correlation coefficients (Spearman) between VASI scores and instruments assessing stigmatization and personal values

Concerning associations with other stigma related scales, a strong positive correlation between the VASI total score and the SDS total score was shown (R = .676). Of the VASI subscales, the strongest correlations with social distance were found for Security (R = .575) and Personal Enrichment (R = −.582). The two sub-scales of the SSMI show very different association patterns with the VASI. Strong positive correlations were found for the SSMI-subscale Stereotype Agreement, which measures individual endorsement of the common public stereotypes, and the VASI total score (R = .670) as well as the VASI-subscale Security (R = .625) and as well as moderate associations for the other VASI-subscales. Neither the VASI total score, nor any VASI-subscale showed considerable correlations (R > .02) for the SSMI subscale Stigma Awareness, which assesses whether people know existing stereotypes about mental illnesses.

Associations between VASI scores and value-specific scales were in most cases significant but of smaller magnitude. Small positive correlations were found between the VASI total score (R = .334) and all VASI subscales (R = .190 to .297). We observed a recurring pattern across all VASI subscales mainly regarding two values. The Self-Transcendence scale (PVQ) showed a negative correlation with all VASI subscales, whereas the Authoritarianism scale (KSA-3) showed an overall positive correlation (the inversely coded Personal Enrichment subscale showing opposite relations). Authoritarianism was most strongly related to the VASI subscales Security and Meritocratic Values.

Associations with the KSE-G scores assessing social desirability were also significant, although arguably only due to the large sample size, as most correlations coefficients were well below 0.2. Only the VASI subscale Reputation showed small correlations with the KSE-G social desirability score (R = −.228). Looking at the two KSE-G subscales, one can assume that this correlation is mainly based on an association with the subscale Minimizing negative properties (R = .218).

Discussion

This study reports the development and the validity of the Value-based Stigma Inventory (VASI). It is the first questionnaire measuring mental illness stigma referring to a value-based concept. In addition to conservative/authoritarian value attitudes, an attempt was made to measure stigmatization in connection with liberal/modern values.

During the VASI development a preliminary pool of 68 items was generated by a mental health stigma expert team and a person with lived experience. These were surveyed with established stigma and value scales via an online panel. One strength is the questionnaire development by means of a large sample (N = 4983), which allows to capture the various value orientations prevalent among the German population. For the subscale development, an exploratory factor analysis (EFA) was carried out within one half of the sample, in which five factors were identified. Taking factor loadings and item contents into account, three items per factor were selected. This model could be verified by a confirmatory factor analysis (CFA) within the other half of the sample. The Bollen-Stine corrected Chi2 test did reach significance, thus questioning the good model fit, however, the chi-square value is known to be susceptible to large sample sizes [32, 37], therefore it cannot be excluded that the statistical significance is mainly due to the large N of our sample. Accordingly, other goodness-of-fit indices consistently met the required cutoffs [33,34,35], thus attesting a good model fit.

A five-factor structure of value-based stigma is proposed, forming the subscales Self-Realization, Personal Enrichment, Reputation, Meritocratic Values and Security, which are medium to high interrelated and represent a spectrum from liberal/modern to conservative/authoritarian stigma-associated values. Alternatively to this recommended calculation of 5 subscale scores, a total score can also be calculated (see Additional file 1).

The validity of the new instrument was investigated using established scales for the assessment of mental health stigma (SDS; SSMI), personal values (PVQ-21; KSA-3), as well as social desirability (KSE-G). The findings suggest a good convergent validity, with medium to high correlation coefficients between the VASI subscales and the SDS- and SSMIAgree-scores corresponding to our assumptions. At the VASI subscale level the correlation coefficients in both stigma scales are highest in Security and lowest in Personal Enrichment. As could be expected, people who are more likely to feel threatened and endangered by people with mental illness tend to have a greater need for social distance, while people who experience dealing with the mentally ill as enriching are more open for social contacts. In accordance with the recommendations by Cohen [15], we found correlations between the VASI scales and value scales to be of medium range. Correlations of the VASI subscales with the value scales approve that Security is associated with an authoritarian value orientation and Personal Enrichment is related to values of self-transcendence. This indicates that people with authoritarian values are stigmatizing to a higher degree than others, whereas values of self-transcendence function as a protective factor against stigmatization. This trend appears constantly on all VASI subscales. Contrary to our assumptions, the subscales do not significantly differ in the correlations with different values. This indication for the existence of a single stigma relevant value dimension (authoritarian values vs. self-transcendence) should nevertheless be viewed critically. It can be assumed that people with liberal values are more in conflict with their values (tolerance vs. individualism) than people with authoritarian values. Therefore, they may quickly see through the liberal stigma items and tend to respond in terms of social desirability. This could be considered in more subtle item formulations or the use of implicit measurement methods [38] in the future. Furthermore, a response shift due to an assumed higher education level among participants with a liberal basic attitude is possible.

An expansion of the liberal/modern concept of values, which was viewed primarily in connection with openness and tolerance, with the aspect of individualism should be considered as a possibility based on the data and should be further investigated in the future. The factor Self-Realization reflect aspects of personal development and self-optimization, which are associated with stigmatization of people with mental illness when presenting as a potential threat to relevant individualistic values. These results point to a field of tension between an anti-stigmatizing attitude and liberal stigmatization. For example, Wetherell and coworkers [39] found that both conservatives and liberals discriminate against ideologically divergent groups if those violate their core values. Thereby abstract values can mitigate (universalism, egalitarianism) or reinforce (traditionalism) discrimination. This suggests that discrimination takes place among liberal-minded people, but in contrast to conservative persons, it might be mitigated by their liberal values.

Concerning discriminatory validity, low correlations to the subscale SSMIAware are shown. In distinction to SSMIAgree it is not measuring stigma per se [40], but rather the awareness of stereotype in the public. The VASI asks for agreement with stereotypical opinions of mental illness, therefore the differences in correlations between the two SSMI subscales are as to be expected. The correlation with the social desirability scale are within an acceptable small range except for the subscale Reputation and the KSE-G subscale Negative Qualities. It can be argued that people who hide own negative characteristics in order to appear socially desirable are likely to associate mental illness with loss of reputation and prestige.

With the VASI scale, we introduce a valid instrument that integrates public stigma of mental health with individual value preferences. The scale includes items pertaining to stigma relevant values beyond authoritarianism, as well as to stigma protective values. This extension of the content of the stigma assessment by a multifaceted value concept is an important strength of this scale and distinguishes it from other already established stigma scales.

Limitations and perspectives

Although the sample of this study is still comparatively large, there are two limitations inherent in data collection via online panel. First, the sample cannot be regarded as representative of the German-speaking resident population, as no statistical random selection was carried out. Participants with older age and a higher education were overrepresented. However, it can be assumed that the size of the sample ensures that people with the most diverse attitudes and values were included. Second, a relatively high percentage of survey participants (18.2%) were not included because they answered the survey below the pre-determined minimum response time of 11 min. When determining the time limit, we were guided by the processing times of independent consultants with questionnaire experience, the distribution of the pretest processing times and the recommendations of a visual inspection of the frequency distribution of response times by Wise and DeMars [41]. Although this time limit was intended to exclude participants with rapid guessing behavior, nevertheless an exclusion of participants with solution behavior but a high processing time cannot be excluded.

Reliability was determined by calculating the internal consistencies. However, since we conducted a one-time, anonymous online survey, no information on the test-retest reliability of the VASI can be derived from this study. The investigation of retest reliability needs to be considered in future use of the instrument.

Conclusions

In total the study offers sufficient data to substantiate the validity of the VASI. The questionnaire showed a clear factor structure and convergent and discriminant validity. The use of the VASI can be suggested in further studies to measure public stigma of mental illness including personal stigma relevant value orientations. The VASI forms an interface between psychiatric, psychological and sociological fields of research. One possible field of application could be the detection of stigma in different socio-economic milieus in order to identify different patterns of discrimination against persons with mental illness.

Availability of data and materials

The datasets obtained and/or analysed during the current study are available from the corresponding author on reasonable request.

Abbreviations

CFA:

Confirmatory Factor Analysis

CFI:

Comparative Fit Index

EFA:

Exploratory Factor Analysis

ESS:

European Social Survey

EV:

Eigenvalues

KMO:

Kaiser-Meyer-Olkin

KSA-3:

Authoritarianism Short Scale

KSE-G:

Social Desirability–Gamma Short Scale

PCA:

Principal Component Analysis

PPMI:

Prejudice against People with Mental Illness Scale

PVQ:

Portrait Values Questionnaire

RMSEA:

Root Mean Square Error of Approximation

SDS:

Social Distance Scale

SRMR:

Standardized Root Mean Residual

SSMI:

Self-Stigma in People with Mental Illness scale

TLI:

Tucker-Lewis Index

VASI:

Value-based Stigma Inventory

WMM:

What Matters Most

References

  1. Abdullah T, Brown TL. Mental illness stigma and ethnocultural beliefs, values, and norms: an integrative review. Clin Psychol Rev. 2011;31(6):934–48. https://doi.org/10.1016/j.cpr.2011.05.003.

    Article  PubMed  Google Scholar 

  2. Yang LH, Kleinman A, Link BG, Phelan JC, Lee S, Good B. Culture and stigma: adding moral experience to stigma theory. Soc Sci Med. 2007;64(7):1524–35. https://doi.org/10.1016/j.socscimed.2006.11.013.

    Article  PubMed  Google Scholar 

  3. Inglehart RF, Norris P. Trump, Brexit, and the Rise of Populism: Economic Have-Nots and Cultural Backlash [Faculty Research Working Paper Series]. Havard: Havard Kennedy School; 2016.

    Google Scholar 

  4. Merkel W, Zürn M. Kosmopolitismus, Kommunitarismus und die Demokratie. In: Nida-Rümelin J, Daniels D, Wloka N, editors. Internationale Gerechtigkeit und institutionelle Verantwortung. Berlin, Boston: De Gruyter; 2019. p. 67–102. https://doi.org/10.1515/9783110615876-007.

    Chapter  Google Scholar 

  5. Goodhart D. The road to somewhere: the populist revolt and the future of politics. New York: Oxford University Press; 2017.

    Google Scholar 

  6. Koppetsch C. Die Gesellschaft des Zorns: Rechtspopulismus im globalen Zeitalter. Bielefeld: transcript-Verlag; 2019.

    Book  Google Scholar 

  7. Schomerus G, Angermeyer MC. Blind spots in stigma research? Broadening our perspective on mental illness stigma by exploring 'what matters most' in modern Western societies. Epidemiol Psychiatr Sci. 2021;30:e26. https://doi.org/10.1017/S2045796021000111.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  8. Feather NT, McKee IR. Values and prejudice: predictors of attitudes towards Australian aborigines. Aust J Psychol. 2008;60(2):80–90. https://doi.org/10.1080/00049530701449513.

    Article  Google Scholar 

  9. Crandall CS, Schaller M, editors. Social Psychology of Prejudice: Historical and Contemporyry Issues. Lawrence, Kansas 66045: LEWINIAN PRESS; 2004.

    Google Scholar 

  10. Biernat M, Vescio TK. Theno sA. Violating American values: a “value congruence” approach to understanding outgroup attitudes. J Exp Soc Psychol. 1996;32(4):387–410. https://doi.org/10.1006/jesp.1996.0018.

    Article  Google Scholar 

  11. Duckitt J. Differential effects of right wing authoritarianism and social dominance orientation on outgroup attitudes and their mediation by threat from and competitiveness to outgroups. Personal Soc Psychol Bull. 2006;32(5):684–96. https://doi.org/10.1177/0146167205284282.

    Article  Google Scholar 

  12. Stephan WG, Stephan CW. An integrated threat theory of prejudice. In: Oskamp S, editor. Reducing prejudice and discrimination. Hillsdale, NJ: Lawrence Erlbaum; 2000. p. 23–46.

    Google Scholar 

  13. Biernat M, Vescio TK. Values and prejudice. In: Crandall CS, Schaller M, editors. Social psychology of prejudice: historical and contemporary issues. Kansas, Lawrence: Lewian Press; 2004. p. 191–216.

    Google Scholar 

  14. Brandt MJ, Reyna C, Chambers JR, Crawford JT, Wetherell G. The ideological-conflict hypothesis. Curr Dir Psychol Sci. 2014;23(1):27–34. https://doi.org/10.1177/0963721413510932.

    Article  Google Scholar 

  15. Cohen J. Statistical power analysis for the behavioral sciences. Hillsdale, NJ: L. Erlbaum Associates; 1988.

    Google Scholar 

  16. Schwartz SH. An overview of the Schwartz theory of basic values. Online Readings Psychol Culture. 2012;2(1). https://doi.org/10.9707/2307-0919.1116.

  17. Wise SL, Ma L. Setting response time thresholds for a CAT Item Pool: The Normative Threshold Method; Annual meeting of the National Council on Measurement in Education, Vancouver, Canada. 2012: 163-183.

  18. Schwartz SH, Melech G, Lehmann A, Burgess S, Harris M, Owens V. Extending the cross-cultural validity of the theory of basic human values with a different method of measurement. J Cross-Cult Psychol. 2001;32(5):519–42. https://doi.org/10.1177/0022022101032005001.

    Article  Google Scholar 

  19. Cieciuch J, Davidov E. A comparison of the invariance properties of the PVQ-40 and the PVQ-21 to measure human values across German and Polish Samples; 2012. https://doi.org/10.18148/srm/2012.v6i1.5091.

    Book  Google Scholar 

  20. Beierlein C, Asbrock F, Kauff M, Schmidt P. Die Kurzskala Autoritarismus (KSA-3): Ein ökonomisches Messinstrument zur Erfassung dreier Subdimensionen autoritärer Einstellungen. 2014. https://www.gesis.org/fileadmin/upload/forschung/publikationen/gesis_reihen/gesis_arbeitsberichte/WorkingPapers_2014-35.pdf.

    Google Scholar 

  21. Corrigan PW, Michaels PJ, Vega E, Gause M, Watson AC, Rüsch N. Self-stigma of mental illness scale--short form: reliability and validity. Psychiatry Res. 2012;199(1):65–9. https://doi.org/10.1016/j.psychres.2012.04.009.

    Article  PubMed  PubMed Central  Google Scholar 

  22. Rüsch N, Hölzer A, Hermann C, Schramm E, Jacob GA, Bohus M, et al. Self-stigma in women with borderline personality disorder and women with social phobia. J Nerv Ment Dis. 2006;194(10):766–73. https://doi.org/10.1097/01.nmd.0000239898.48701.dc.

    Article  PubMed  Google Scholar 

  23. Link BG, Cullen FT, Frank J, Wozniak F. The Social Rejection of Former Mental Patients: Understanding Why Labels Matter. Am J Sociol. 1987;92(6):1461–500. https://doi.org/10.1086/228672.

    Article  Google Scholar 

  24. Angermeyer MC, Matschinger H, Schomerus G. Attitudes towards psychiatric treatment and people with mental illness: changes over two decades. Br J Psychiatry. 2013;203(2):146–51. https://doi.org/10.1192/bjp.bp.112.122978.

    Article  PubMed  Google Scholar 

  25. Kenny A, Bizumic B, Griffiths KM. The prejudice towards people with mental illness (PPMI) scale: structure and validity. BMC Psychiatry. 2018;18(1):293. https://doi.org/10.1186/s12888-018-1871-z.

    Article  PubMed  PubMed Central  Google Scholar 

  26. Kemper CJ, Beierlein C, Bensch D, Kovaleva A, Rammstedt B. Eine Kurzskala zur Erfassung des Gamma-Faktors sozial erwünschten Antwortverhaltens: Die Kurzskala Soziale Erwünschtheit-Gamma (KSE-G). GESIS-Working Papers 2012|25. Köln: GESIS2012.

  27. Bortz J, Döring N. Forschungsmethoden und Evaluation für Human- und Sozialwissenschaftler. 4th ed. Berlin: Springer; 2006. https://doi.org/10.1007/978-3-540-33306-7.

    Book  Google Scholar 

  28. Gorsuch RL. Factor analysis. 2nd ed. Hillsdale, NJ: Erlbaum; 1983.

    Google Scholar 

  29. Kline P. An easy guide to factor analysis. London: Routledge. London: Routledge; 1994.

    Google Scholar 

  30. Bühner M. Einführung in die Test- und Fragebogenkonstruktion. 3rd ed. Munich: Pearson Education Deutschland GmbH; 2011.

    Google Scholar 

  31. Beauducel A, Wittmann WW. Simulation study on fit indexes in CFA based on data with slightly distorted simple structure. Struct Equ Model Multidiscip J. 2005;12(1):41–75. https://doi.org/10.1207/s15328007sem1201_3.

    Article  Google Scholar 

  32. DiStefano C, Hess B. Using confirmatory factor analysis for construct validation: an empirical review. J Psychoeduc Assess. 2005;23(3):225–41. https://doi.org/10.1177/073428290502300303.

    Article  Google Scholar 

  33. Prinsen CAC, Mokkink LB, Bouter LM, Alonso J, Patrick DL, de Vet HCW, et al. COSMIN guideline for systematic reviews of patient-reported outcome measures. Qual Life Res. 2018;27(5):1147–57. https://doi.org/10.1007/s11136-018-1798-3.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  34. Hu L, Bentler PM. Cutoff criteria for fit indexes in covariance structure analysis: conventional vriteria versus new alternatives. Struct Equ Model. 1999;6(1):1–55. https://doi.org/10.1080/10705519909540118.

    Article  Google Scholar 

  35. Schermelleh-Engel K, Moosbrugger H. Evaluating the fit of structural equation models: tests of significance and descriptive goodness-of-fit measures. Methods Psychol Res Online. 2003;8:23–74.

    Google Scholar 

  36. West SG, Finch JF, Curran PJ. Structural Equatation models with nonormal variables: problems and remedies. In: Hoyle RH, editor. Structural Equatation modeling: concepts, issues, and applications. Thousand Oaks: Sage Publications; 1995. p. 56–75.

    Google Scholar 

  37. Bollen KA. Structural models with latent variables. New York: Wiley; 1989. https://doi.org/10.1002/9781118619179.

    Book  Google Scholar 

  38. Greenwald AG, Banaji MR. Implicit social cognition: attitudes, self-esteem, and stereotypes. Psychol Rev. 1995;102(1):4–27. https://doi.org/10.1037/0033-295X.102.1.4.

    Article  CAS  PubMed  Google Scholar 

  39. Wetherell GA, Brandt MJ, Reyna C. Discrimination across the ideological divide. Soc Psychol Personal Sci. 2013;4(6):658–67. https://doi.org/10.1177/1948550613476096.

    Article  Google Scholar 

  40. Corrigan PW, Watson AC, Barr L. The self–stigma of mental illness: implications for self–esteem and self–efficacy. J Soc Clin Psychol. 2006;25(8):875–84. https://doi.org/10.1521/jscp.2006.25.8.875.

    Article  Google Scholar 

  41. Wise SL, DeMars CE. An application of item response time: the effort-moderated IRT model. J Educ Meas. 2006;43(1):19–38. https://doi.org/10.1111/j.1745-3984.2006.00002.x.

    Article  Google Scholar 

Download references

Acknowledgments

Not applicable.

Funding

The study was funded through the research budget of the Department of Psychiatry and Psychotherapy at the University of Leipzig Medical Center. We acknowledge support from Leipzig University for Open Access Publishing. Open Access funding enabled and organized by Projekt DEAL.

Author information

Authors and Affiliations

Authors

Contributions

SR, GS and MA contributed to the conceptualization and the design of the work, items were generated by MA, EP, SR and revised by MA, SR, CS, SS and GS. CS and SR did the statistical analysis, interpretation and wrote the first draft. SS, EP, MA and GS revised the paper. All authors read and approved the final manuscript.

Corresponding author

Correspondence to Sophia Rieckhof.

Ethics declarations

Ethics approval and consent to participate

Ethical approval was given by the medical ethics committee the University of Leipzig with the following reference number: 239/20-ek. All participants gave written informed consent. The study was conducted in accordance to the declaration of Helsinki.

Consent for publication

Not applicable.

Competing interests

All authors declare no conflicts of interest.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary Information

Additional file 1.

Value-based Stigma Inventory (VASI).

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Rieckhof, S., Sander, C., Speerforck, S. et al. Development and validity of the Value-based Stigma Inventory (VASI): a value-sensitive questionnaire for the assessment of mental health stigma. BMC Psychiatry 21, 570 (2021). https://doi.org/10.1186/s12888-021-03427-4

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1186/s12888-021-03427-4

Keywords