Comparing the validity of the self reporting questionnaire and the Afghan symptom checklist: dysphoria, aggression, and gender in transcultural assessment of mental health
BMC Psychiatry volume 14, Article number: 206 (2014)
The relative performance of local and international assessment instruments is subject to ongoing discussion in transcultural research on mental health and psychosocial support. We examined the construct and external validity of two instruments, one developed for use in Afghanistan, the other developed by the World Health Organization for use in resource-poor settings.
We used data collected on 1003 Afghan adults (500 men, 503 women) randomly sampled at three sites in Afghanistan. We compared the 22-item Afghan Symptom Checklist (ASCL), a culturally-grounded assessment of psychosocial wellbeing, with Pashto and Dari versions of the 20-item Self-Reporting Questionnaire (SRQ-20). We derived subscales using exploratory and confirmatory factor analyses (EFA and CFA) and tested total and subscale scores for external validity with respect to lifetime trauma and household wealth using block model regressions.
EFA suggested a three-factor structure for SRQ-20 - somatic complaints, negative affect, and emotional numbing - and a two-factor structure for ASCL - jigar khun (dysphoria) and aggression. Both factor models were supported by CFA in separate subsamples. Women had higher scores for each of the five subscales than men (p < 0.001), and larger bivariate associations with trauma (rs .24 to .29, and .10 to .19, women and men respectively) and household wealth (rs -.27 to -.39, and .05 to -.22, respectively). The three SRQ-20 subscales and the ASCL jigar khun subscale were equally associated with variance in trauma exposures. However, interactions between gender and jigar khun suggested that, relative to SRQ-20, the jigar khun subscale was more strongly associated with household wealth for women; similarly, gender interactions with aggression indicated that the aggression subscale was more strongly associated with trauma and wealth.
Two central elements of Afghan conceptualizations of mental distress - aggression and the syndrome jigar khun – were captured by the ASCL and not by the SRQ-20. The appropriateness of the culturally-grounded instrument was more salient for women, indicating that the validity of instruments may be gender-differentiated. Transcultural validation processes for tools measuring mental distress need to explicitly take gender into account. Culturally relevant measures are worth developing for long-term psychosocial programming.
Expert guidelines advocate the inclusion of local perspectives of distress in the assessment of mental health in conflict-affected populations [1–3] to address issues of cultural relevance for psychosocial support programming [4–8]. Context-insensitive questionnaires may lead to inaccurate inferences concerning the needs of conflict-affected populations, whose members may or may not respond to mental health instruments in ways comparable to populations living in relatively secure communities . Simply translating questionnaires into relevant languages is insufficient to address local conceptual issues [10–13]. Developing contextually valid emic (i.e., culturally-specific) instruments involves combining qualitative and quantitative approaches, marrying the ethnographic concern for local meanings with the epidemiological focus on replicable surveys [7, 13, 14]. In postconflict settings, mixed-methods techniques to develop and implement local measures have been applied to genocide-affected communities in Rwanda [15, 16], refugees from Darfur in Chad , postpartum women in the Democratic Republic of Congo , violence-affected adults in Timor-Leste [19, 20], internally displaced persons in Sri Lanka , and war-affected youth in several settings [22, 23].
The Afghan Symptom Checklist (ASCL) was developed using ethnographic interviews and thematic coding of wellness and illness narratives to provide a culturally-grounded assessment of psychosocial wellbeing in Afghanistan . In their study, Miller and colleagues described several Afghan idioms of distress: asabi, an anxiety-like state; fishar, or blood pressure abnormalities; and jigar khun, literally “liver blood,” an extreme and persistent dysphoria explained in humoral medicine as resulting from an overabundance of blood in the liver. The ASCL had good psychometric properties in a sample of 324 adults in Kabul.
While developing local measures like the ASCL has clear advantages in serving specific linguistic and cultural groups, it is not without costs. Considerable time and resources are needed to complete the essential steps of transcultural measure development: translate and back-translate content, evaluate content validity with international and local experts, and pilot instruments to evaluate psychometric properties . Even when such steps are taken, there remain important questions concerning conceptual comparability. Results derived from measures specifically designed for one setting are often difficult to compare to results from others. In the face of such challenges, psychosocial programs often select screening instruments that have been previously implemented and proved reliable in other international settings. Such etic (i.e., culturally neutral) global mental health instruments are usually deployed to assess a range of symptoms indicative of common mental disorders, positing common factors in psychological distress. Typically these measures are used as screening instruments, and are not designed to be diagnostic of mental health disorders [26, 27] or to provide within-population prevalence rates .
The Self-Reporting Questionnaire (SRQ-20) is one such instrument. The SRQ-20 was developed by the World Health Organization  to measure symptoms of common mental disorders across different cultures in low-income settings [28, 29]. Designed for ease of administration, it consists of 20 items covering symptoms of anxiety and mood disorders with a binary (yes/no) response scale. The SRQ-20 has been assessed across a wide range of contexts, including conflict and refugee settings in Colombia , Guinea Bissau , Mozambique , Rwanda , Uganda , Iran , Pakistan , and Afghanistan [37, 38]. In Afghanistan, the psychometric properties of a Pashto SRQ-20 were examined by Ventevogel and colleagues , who reported reasonable psychometric properties but large differences across gender. Some concern has been voiced that internationally-developed instruments such as the SRQ-20 may be insensitive to culturally-salient idioms of distress and thus may not differentiate between clinically-relevant and non-clinical distress .
We compared the ASCL and the SRQ-20 in a dataset from Afghanistan to examine the value of using a locally-developed emic measure versus using an internationally-developed etic measure. We approached this question methodologically in two stages: first we examined the construct validity of each measure by comparing items’ face validity and using factor analysis. Second, we compared the two measures’ external validity using regression models. In the latter we asked whether the measures were comparably associated with two third variables that have a well-known impact on mental health in low-income and conflict settings: exposure to traumatic events and socioeconomic status. Trauma and poverty are reliably associated with anxiety disorders, depression, and general emotional distress in conflict-affected adults [41–43]. Comparing how the ASCL and SRQ-20 were associated with these variables would provide an estimate of their external validity, a critical component of measurement validity . A similar approach was used by Jayawickreme and colleagues to compare the PTSD Symptom Scale-Self Report  and the Beck Depression Inventory (BDI) to a locally-developed measure in a conflict affected sample in Sri Lanka ; the latter was found to explain additional variance in functional impairment.
In order to address the conceptual ambiguity inherent in using total scores calculated from multidimensional questionnaires, we combined construct and external validity goals to examine how empirically identified subscales on the ASCL and the SRQ-20 were differentially associated with trauma exposure and socioeconomic status. Our hypotheses were that certain subscales would draw on similar variance, but that the ASCL would include dimensions that the SRQ-20 did not, and these dimensions would account for more variance in external criteria. However, given that there has been little examination of the factor structure of either measure, we did not have specific hypotheses about the content of specific subscales or which subscale would be more associated with which criteria. Given the known gender segregation and disparities in mental health in Afghanistan [45–48], we tested hypotheses that these effects would be moderated by gender.
We drew our data from a gender-balanced community-based mental health survey undertaken in 2006 [37, 38, 49, 50] in three research sites in central and northern Afghanistan (Kabul, Bamyan, and Mazar-e-Sharif). The study featured a school-based stratified random sample, beginning with a random sample of government schools with probability sampling proportional to size, and then a random sample of child-adult dyads (students and adult caregivers) targeting 320 dyads per site (on the basis of power calculations from pilot work). The research protocol was formally approved by the Ministry of Education in Afghanistan, the United Nations High Commissioner for Refugees in Peshawar, the ethics committees of the University of Peshawar and of Durham University, where the principal investigator received Welcome Trust funding for this research. In order to eliminate any risk that documented participation in research would result in danger to participants, informed consent was obtained verbally from all participants, and in writing only from school directors; only one potential participant refused to be interviewed.
Interview and instrument preparation
We trained a small field team of male and female local researchers to interview participants in their preferred language, Dari or Pashto. The study featured two years of prior preparation in three sites (Wardak province, Kabul, and Peshawar) to achieve reliable translations, establish sampling and recruitment logistics, and conduct extensive pilots including test-retest reliability across sites. Translations from English to Dari and Pashto were undertaken by an Afghan clinical psychologist with professional experience in both the UK and Afghanistan, and independently checked by an Afghan psychologist at the University of Peshawar. We commissioned a professional translator and a linguist to undertake independent back-translations, which were then systematically reviewed for content validity by a panel comprised of fieldworkers and academics with expertise in anthropology, Middle Eastern studies, social work, child psychiatry, and cross-cultural psychology .
Measures of trauma and socioeconomic status
Lifetime trauma exposures were measured using a checklist (20 yes/no items) adapted from the Harvard Trauma Questionnaire  previously implemented with adults in Afghanistan . Our analyses focused on exposure severity (the total number of lifetime trauma reports) rather than specific types of traumatic experiences (detailed elsewhere ). To assess socioeconomic status, we asked participants to enumerate the total number of household goods from a specified list of 15 household items including cooking facilities, piped water, toilet, generator, sawdust-fuelled heater, mobile phone, radio, satellite dish, bicycle, and car. Such dimensional indices of household wealth are useful in contexts where self-reported income and expenditure data are likely unreliable.
The self-reporting questionnaire
The SRQ-20 was developed as a mental health screening instrument for people presenting to general health services. Ventevogel and colleagues  reported that a Pashto version of SRQ-20, implemented in the eastern Afghan province of Nangarhar, revealed a two-factor structure consisting of “common disorders” and “social disability.” Compared with a structured psychiatric interview, the SRQ-20 had moderate sensitivity and specificity for predicting the presence of common mental health disorders.
The Afghan symptom checklist
The ASCL consists of 22 items drawn from thematic coding of transcripts of ethnographic interviews undertaken with adult men and women in Kabul . Among items are three Dari terms representing Afghan idioms of distress: jigar khun, a term describing “a form of sadness that includes grief following interpersonal loss but that may also be a reaction to any deeply disappointing or painful experience” (p. 425); asabi, a term for “feeling nervous or highly stressed” (p. 425), used to describe a state of nervous agitation when overwhelmed by stress; and fishar, or “pressure” locally thought to map onto blood pressure variation. While developing the ASCL, Miller and colleagues combined the idioms fishar-e-bala (high blood pressure) and fishar-e-payin (low blood pressure) into one item, but after factor analysis recommended that they be separated (footnote 3, p.426). The ASCL had adequate reliability and a latent variable structure with three interpretable factors: “sadness with social withdrawal and somatic distress” (p. 425); “ruminative sadness without social isolation and somatic distress” (p. 425); and stress-induced reactivity, consisting of quarreling, beating one’s children, and asabi.
To examine face validity, we compared SRQ-20 items with ASCL items qualitatively, identifying parallel content and item differences. To examine construct validity, we followed a sequence of fitting exploratory and confirmatory factor models. We split the full sample into three subsamples, using simple random selection to select a random third of cases for an exploratory subsample followed by a random half of the unselected subsample to select two confirmatory subsamples. All selections were made using the “select cases” option in the Data menu of SPSS. We then used EFA with varimax rotation to suggest confirmatory models, and confirmatory factor analysis (CFA) to test these models in the remaining two subsamples. Because of the arbitrariness of using eigenvalues greater than 1.0 to suggest the number of latent factors, we examined scree plots and rotated factor plots in EFA to build interpretable confirmatory models. We operationalized meaningful factor loadings as > 0.400. We ran the EFA using both SPSS and Mplus, using the former for continuous responses on the ASCL and the latter for binary responses on the SRQ-20 ; we estimated CFA models using MPlus only. We qualitatively compared the content of factors across measures, with interpretation aided by reference to extant global mental health literature, including previous work in Afghanistan [24, 51].
We tested external validity by examining the value of adding ASCL scores following SRQ-20 scores to regression block models predicting trauma and wealth indices. We conducted these analyses for (1) total scores as a dimensional measure of mental health problems, and (2) subscale scores for each ASCL subscale following each SRQ-20 subscale. Relative advantages in external validity of the ASCL over the SRQ-20 were operationalized as statistically significant beta coefficients of ASCL scores in regression models that included SRQ-20 scores in previous blocks. We also examined how these associations might be moderated by gender by including interactions between gender and ASCL subscales. We entered gender in the first block, SRQ-20 scores in the second block, ASCL scores in the third, and interactions between gender and ASCL scores in the fourth.
Our sample included 1003 adults (500 men, 503 women), averaging 36.44 years (SD 12.37) of age. A third were from the Kabul area (n = 364, 36.3%), a third from Bamyan (n = 327, 32.6%), and a third from Mazar-e-Sharif (n = 311, 31.0%). The sample was largely Tajik (n = 429, 42.8%) and Hazara (n = 424, 42.3%), with small minorities of Pashtun (n = 86, 8.6%), Uzbek (n = 39, 3.9%) and other ethnicities (n = 24, 2.4%). Half (n = 501, 50.0%) were uneducated, just over a third (n = 352, 35.1%) had some primary or secondary education, and ten percent (n = 101, 10.1%) had some post-secondary education (n = 48, 4.8% were missing education data). Just under half (n = 495, 49.4%) were employed (49, 49% were missing employment data). The three subsamples used for exploratory and confirmatory factor analyses did not differ by gender, age, ethnicity, region, education or employment. The mean score for the SRQ-20 was 13.03 (SD 2.52); there were large differences between women (9.45, SD 4.51) and men (5.51, SD 3.75; p < 0.0001; d 0.95). The mean score for the ASCL was 35.21 (SD 5.72), with higher scores for women (52.44, SD 15.73) than men (38.04, SD 10.99; p < 0.0001; d 1.06).
Review of SRQ-20 and ASCL
We identified five comparable items on the SRQ-20 and the ASCL. These included SRQ1, “headaches” and ASCL11, “had a headache”; SRQ2, “poor appetite” and ASCL2 “lack of appetite”; SRQ3, “bad sleep” and ASCL3, “difficulty falling asleep”; SRQ9, “unhappy” and ASCL9, “felt sad”; and SRQ10, “cry more than usual” and ASCL1, “cried”. Three other SRQ items were broadly comparable to ASCL items: SRQ4, “frightened easily” and ASCL14, “felt startled”; SRQ6, “nervous, tense, worried” and ASCL17, “experienced asabi”; and SRQ8, “trouble thinking clearly” and both ASCL23, “trouble concentrating” and ASCL16, “thinking too much”. The comparison of other items showed that SRQ-20 items included more somatic complaints, while the ASCL included items related to aggression. Following comparison of item content, we removed the ASCL item “difficulty meeting responsibilities because of jigar khun” from subsequent analyses in order to avoid conflating emotional distress with functional impairment.
SRQ-20 factor analyses
Table 1 presents the 3-factor EFA for the SRQ-20 items from the exploratory sample. The first factor consisted primarily of somatic complaints, and was dominated (loading > .800) by the item “poor appetite”. The second factor was best characterized as general negative affect; it was dominated by “cry more than usual” and notably included “thoughts of ending your life” and four items that also loaded on the first factor. The third factor consisted of difficulty making decisions and loss of pleasure, suggesting emotional numbing. The 3-factor model somatic complaints, negative affect, and emotional numbing fit the two confirmatory samples reasonably well (CFI = .918, TLI = .946, RMSEA = .061 in confirmatory sample 1; CFI = .929, TLI = .947, RMSEA = .056 in confirmatory sample 2).
ASCL factor analyses
During EFA we found that the ASCL item “quarreling with a neighbor or friend” was weakly associated with all other items, constituted its own factor in initial EFA solutions, and was a clear outlier in rotated factor plots. We dropped this item from subsequent analyses. Table 2 presents the 3-factor EFA model for ASCL items. The first factor (16 items) consisted of symptoms of negative affect (e.g., “felt sad”), depressed mood (e.g., “felt hopeless”) and nervous agitation (“asabi”). Because of the dominance of “became jigar khun” (loading > .800) and item concordance with Miller and colleagues’ description of the syndrome jigar khun, we labeled the first factor jigar khun. The second factor (four items) was comprised of aggressive behavior toward family members and self; it also included “cried,” which also loaded with the first factor. The third factor consisted of just two inversely-related items: “fishar-e-bala” (loading positively) and “fishar-e-payin” (loading negatively). We named this factor fishar. The 3-factor model suggested by the EFA fit the two confirmatory samples adequately (CFI = .943, TLI = .974, RMSEA = .087 in confirmatory sample 1; CFI = .920, TLI = .959, RMSEA = .099 in confirmatory sample 2). Although RMSEA statistics were higher than the recommended .06, both CFI and TLI suggested good fit . Varying the model slightly (e.g., not allowing for double-loading items) did not result in improved fit.
Comparing SRQ-20 and ASCL subscales in the full sample
Because of comparable fit across subsamples we used the full sample for subsequent analyses. The SRQ-20 subscales were constructed based on the 3-factor solution. They attained good Cronbach’s α reliability (mathematically identical to the Kruder-Richardson’s reliability coefficient for binary responses ), as follows: somatic complaints (8 items) α = .78, negative affect (11 items) α = .77, and emotional numbing (7 items) α = .74. For ASCL, we retained two subscales, jigar khun and aggression, and eliminated the third, fishar. As a subscale, the two-item fishar would have been unstable , as shown by the sum of the item variances being greater than the scale variance, resulting in a negative Cronbach’s α of -.23. The jigar khun and aggression subscales attained satisfactory Cronbach’s α reliability coefficients: jigar khun (16 items) α = .91, and aggression (4 items) α = .66.
Mean scores for SRQ-20 subscales were: somatic complaints 3.93 (SD 2.46), negative affect 4.38 (SD 2.76), and emotional numbing 1.68 (SD 1.83). On all SRQ-20 subscales, women scored higher than men (p < 0.001) and two of the three differences were large: women’s somatic complaints score averaged 4.90 (SD 2.21) and men’s 2.95 (SD 2.30), d 0.86; their negative affect score averaged 5.71 (SD 2.62) and men’s 3.04 (SD 2.18), d 1.11; their emotional numbing score averaged 1.96 (SD 2.09) and men’s 1.40 (SD 1.47), d .31. Mean scores for ASCL subscales were as follows: jigar khun, 32.49 (SD 12.12), and aggression, 5.72 (SD 2.52). On both subscales, women showed higher symptom scores (p < 0.001) and effect size differences were large (d > .7): women’s jigar khun score averaged 38.17 (SD 12.27) and men’s 26.77 (SD 8.83), d 1.07; their aggression score averaged 7.03 (SD 2.86) and men’s 4.41 (SD 1.03), d 1.22.
Table 3 shows the associations between SRQ-20 and ASCL subscales and variables of interest. Correlations between subscales in the full sample were large (range .39 to .80); removing double-loading items from subscales did not substantially alter correlations (range .38 to .72). We report full subscales (i.e., with double-loading items allowed) here. Both trauma exposures and wealth were significantly correlated across the ASCL and SRQ-20 subscales for women, but inconsistently so for men.
Comparing external validity of ASCL and SRQ-20 scores
We found strong and consistent associations with gender and SRQ-20 scores (Total scores, Table 4). For trauma, we found no independent associations with ASCL scores or with the gender-by-ASCL interaction (total R2 = 0.077). For household wealth, associations with ASCL scores and gender-by-ASCL were significant, over and above associations with SRQ-20 (total R2 = 0.117).
Results for the jigar khun ASCL subscale are shown in Table 5. For trauma, neither jigar khun nor the gender-by-jigar khun interactions were associated with variance beyond the SRQ-20 subscales of somatic complaints, negative affect, or emotional numbing. For wealth, both jigar khun and gender-by-jigar khun were associated with variance with respect to the somatic complaints SRQ-20 subscale, but not with the negative affect or emotional numbing SRQ-20 subscales. The interaction between gender and jigar khun scores is presented graphically in Figure 1. Jigar khun was negatively associated with wealth only for women, with women above median jigar khun scores reporting fewer household items.
We repeated this analysis for the aggression ASCL subscale (Table 6). Gender and SRQ-20 subscales showed strong and consistent associations with trauma and wealth. ASCL aggression and the gender-by-aggression interaction were weakly significant (.05 > p > .01) in these regression models. In contrast, they were strongly and consistently significant for wealth. ASCL aggression and gender-by-aggression were associated with variance in wealth above and beyond the SRQ-20 subscales somatic complaints, negative affect, and emotional numbing. The interaction between gender and ASCL aggression scores is presented graphically in Figure 2. Similar to results for jigar khun, women in poorer households reported more aggressive behaviors.
Strengths of the current study include an explicit focus on the cultural relevance of instruments, two years of preparation and pilot surveys, recruitment across multiple sites, random sampling, and gender balance. We feel that because of these strengths this study avoids many of the limitations common in the field. However, like all research projects, this study is not without its limitations. These include not measuring functional impairment, relatively low levels of associations between distress scores and external criterion variables, double-loading items in EFA models, and suboptimal RMSEA statistics in CFA models for the ASCL. Nevertheless, we believe our analyses comparing the external validity of the SRQ-20 and the ASCL are clinically meaningful, and we encourage researchers to build upon these findings to improve mental health assessment in Afghanistan and other conflict-affected settings.
Our findings suggest that the ASCL was a better measure of distress than the SRQ-20 for women, while the two measures were similar for men. Given the strict gender demarcation of Afghan society [24, 48, 51, 55, 56], we expected to find striking gender differences. Men and women inhabit different social and emotional spaces in Afghanistan, with downstream implications for psychosocial [37, 46–48] and physiological  wellbeing. The large mean differences across ASCL and SRQ-20 subscale scores are thus consistent with the literature. Why the cultural appropriateness of the ASCL seemed more salient for women is less clear. Being involved in the public sphere, men may be more likely to recognize the more generalized modes of expression codified in the SRQ-20, while women, who have limited social interactions outside the home, may respond to more culturally-grounded expressions of distress. It may also be that responses follow the script of gender-differentiated display rules (i.e., culture of emotions). Permissible ways to express distress are gendered, such that Afghan men may publicly express anger, jealousy, and hate, but not fear, grief, or doubt – emotions that might bring shame to families’ honor . Thus in our data, men’s scores may have been constrained by a ceiling effect on particular item responses.
The hypothesis that using total scores would mask important variability with implications for external validity was supported by our findings. The internal structures we found for the SRQ-20 and the ASCL were somewhat distinct from previous descriptions. The latent structure of the SRQ-20 included three factors: somatic complaints, negative affect, and emotional numbing. Although different from the two-factor solution identified by Ventevogel and colleagues  using a different Afghan dataset, this structure still largely mirrors symptom clusters of common mental disorders – i.e., anxiety and depression. Unlike the factor structure reported by Miller and colleagues , the ASCL in our data was comprised of a latent variable structure consisting of jigar khun, aggression, and fishar. Jigar khun was by far the dominant latent variable in exploratory and confirmatory models of ASCL scores, suggesting that it represents a specific cultural syndrome rather than simply an idiom of distress. Idioms of distress are more general ways of experiencing and expressing distress within specific local contexts [58–60], whereas cultural syndromes (as described by Nichter) are “widely recognized prototypical cultural ailment[s] that encompasses a fuzzy set of associations coalescing around one or more core cultural symbols” (p.407) . In prior development of the ASCL , the item jigar khun consisted primarily as grief. In our study, the inclusion of asabi and other items representing states of arousal within the jigar khun subscale seems to suggest that jigar khun is a multidimensional construct, combining several symptoms of common mental disorders. In other words jigar khun is more than a general expression of distress, and we feel that it meets criteria as an Afghan cultural syndrome.
Jigar khun was not associated with variance in trauma events beyond negative affect and emotional numbing as measured by the SRQ-20. This suggests that jigar khun pulls on similar sources of variance as somatic complaints, negative affect, and emotional numbing for Afghans with respect to trauma. With respect to socioeconomic status, however, the models for jigar khun and SRQ-20 somatic subscale suggest a different conclusion – namely, that jigar khun is associated with household wealth above and beyond somatic complaints for women. That dysphoria is associated with poverty in populations affected by armed conflict is well-known [41, 42]. What our findings seem to suggest is that this association may be gender-dependent. Further research should examine the gendered nature of features of distress, especially in regions with strict gender segregation such as Afghanistan.
In contrast to jigar khun, we found that aggression was associated with both trauma and socioeconomic status above and beyond the SRQ-20 subscales, especially for women. This suggests that the ASCL is a better measure of Afghan distress in large part because it includes aggression items. Aggression has been for the most part ignored in standard North American and European psychiatric concepts of common mental health disorders. Its prominence here is consistent with reports that behavioral disturbances are common in other taxonomies of mental distress. Explosive anger is a critical element of psychosocial distress among West Papuan refugees in Australia . In Timor-Leste, explosive anger is associated with ongoing socio-economic disadvantage  and past trauma exposure . Rees and colleagues  have emphasized the risk of explosive anger for women in particular, who experience the disproportionate impact of poverty hardships and human rights abuses. In Afghanistan, Catani, Schauer and Neuner  have linked war exposure for adults and increased levels of abusive behavior toward their children. Our findings show that aggression is an important dimension of Afghan expressions of distress, and that it can take the form of self-directed as well as interpersonal violence. In content analyses of qualitative interview data collected concurrently with the present study  and in other work conducted in Afghanistan , women with intense emotional distress reported hitting themselves, beating their children, and causing injury to others. Indeed, the connection between aggressive behavior and psychological distress in Afghanistan is reflected in language used to describe distress. In Pashto and Dari, the word khapa means angry and annoyed as well as sad, anxious, and worried. The word khapgan (lit: hardship) refers to either sadness or anger, depending on social context .
Our findings are specific to comparing measurement across constructs, and do not imply equivalence of meaning. In other words, that the subscales of the SRQ-20 and the jigar khun subscale of the ASCL drew upon a similar pool of variance psychometrically does not necessarily imply that the SRQ-20 subscales represent culturally meaningful concepts. Being associated with similar variance in external criteria may imply comparable content and external validity, but does not imply similar construct validity. This is not to undermine those who would argue that the SRQ-20 is useful for screening in humanitarian response efforts; indeed, our findings suggest that it is, insofar as it is associated with important variance in Afghan emotional distress. However, presenting evidence of psychometric association is not the same as presenting evidence of culturally-grounded parallel meanings. Symptom checklists and screening inventories are situated within the cultures of their development; useful in comparative monitoring and evaluation, they are rarely designed to capture the subtleties of local explanatory models.
Studies that have compared locally-developed, emic mental health measures to globally-deployed, etic measures suggest considerable psychometric overlap between instruments, but highlight that local development does provide some additional value in terms of both content and external validity [12, 21]. Findings from the current study echo this conclusion. Although the body of literature is still relatively small, the consistent advantage of locally-developed measures suggests that efforts to develop them are indeed worthwhile. However, the equally consistent finding regarding the large overlap in content and external validity suggests that decisions not to develop culturally-grounded instruments may be justified, depending on project scope and resources. We are reminded of the emphasis placed by Kohrt and colleagues’  on the consideration of research purpose as key to tool selection. Where the goals of psychosocial programming in emergency contexts are short-term and relatively limited in scope, resources dedicated to developing local measures to screen for mental health are perhaps better spent elsewhere. Over the long term, however, developing culturally-grounded and locally validated emic measures will significantly improve the quality of psychosocial programming, as the small but consistent advantages of emic measures observed in our findings and others’ are likely magnified over time. For sustainable programs it is particularly important that case-identification and monitoring and evaluation tools capture salient expressions of distress . In terms of research, the development of emic instruments is critical because the goals of methodological excellence and cultural specificity overshadow many other possible concerns. Using such scales alongside etic scales may open up a range of strategies to examine multiple health outcomes with multiple, complementary tools. Psychosocial programs deciding whether or not to devote the resources necessary to develop local measures of distress where there are none should weigh the purposes of the tool and the potential for long-term engagement.
Comparing the ASCL and SRQ-20 revealed substantial overlap in construct and external validity, but the ASCL captured more variance associated with trauma and poverty for women. These findings highlight the advantages of locally-developed measures of mental health and the importance of considering gender in transcultural validation. In addition, practitioners should not ignore the role of aggression in emotional distress in cross-cultural setting. Culturally relevant measures are worth developing for long-term psychosocial programming.
Inter-Agency Standing Committee (IASC): IASC Guidelines on Mental Health and Psychosocial Support in Emergency Settings. 2007, Geneva: United Nations
MHPSS IASC-Reference: IASC Reference Group Mental Health and Pscyhosocial Support Assessment Guide. IASC Reference Group on Mental Health and Psychosocial Support in Emergency Settings. 2012, Geneva: United Nations
WHO-UNHCR: Assessing Mental Health and Psychosocial Needs and Resources: Toolkit for Humanitarian Settings. 2013, Geneva: World Health Organization
Tol WA, Patel V, Tomlinson M, Baingana F, Galappatti A, Silove D, Sondorp E, van Ommeren M, Wessells MG, Panter-Brick C: Relevance or excellence? Setting research priorities for mental health and psychosocial support in humanitarian settings. Harv Rev Psychiatry. 2012, 20 (1): 25-36.
Tol W, Rees S, Silove D: Broadening the scope of epidemiology in conflict-affected settings: opportunities for mental health prevention and promotion. Epidimiol Psychiatr Sci. 2013, 22 (03): 197-203.
Bolton P, Tol W, Bass J: Introduction to Special Issue: combining qualitative and quantitative research methods to support psychosocial and mental health programmes. Intervention. 2009, 7 (3): 181-186.
Kohrt BA, Rasmussen A, Kaiser BN, Haroz EE, Maharjan SM, Mutamba BB, de Jong JT, Hinton DE: Cultural concepts of distress and psychiatric disorders: literature review and research recommendations for global mental health epidemiology. Int J Epidemiol. 2014, 43 (2): 365-406.
Kohrt BA, Jordans MJ, Tol WA, Luitel NP, Maharjan SM, Upadhaya N: Validation of cross-cultural child mental health and psychosocial research instruments: adapting the Depression Self-Rating Scale and Child PTSD Symptom Scale in Nepal. BMC Psychiatry. 2011, 11 (1): 127-
Rodin D, van Ommeren M: Commentary: explaining enormous variations in rates of disorder in trauma-focused psychiatric epidemiology after major emergencies. Int J Epidemiol. 2009, 38: 1045-1048.
Rahman A, Iqbal Z, Waheed W, Hussain N: Translation and cultural adaptation of health questionnaires. J Pak Med Assoc. 2003, 53 (4): 142-146.
Syed H, Zachrisson H, Dalgard O, Dalen I, Ahlberg N: Concordance between Hopkins Symptom Checklist (HSCL-10) and Pakistan Anxiety and Depression Questionnaire (PADQ), in a rural self-motivated population in Pakistan. BMC Psychiatry. 2008, 8: 59-
Sweetland AC, Belkin GS, Verdeli H: Measuring depression and anxiety in Sub-Saharan Africa. Depress Anxiety. 2014, 31 (3): 223-232.
Jayawickreme N, Jayawickreme E, Goonasekera MA, Foa EB: Distress, wellbeing and war: qualitative analyses of civilian interviews from north eastern Sri Lanka. Intervention. 2009, 7 (3): 204-222.
Patel V, Simunyu E, Gwanzura F, Lewis G, Mann A: The Shona symptom questionnaire: the development of an indigenous measure of common mental disorders in Harare. Acta Psychiatr Scand. 1997, 95 (6): 469-475.
Bolton P, Neugebauer R, Ndogoni L: Prevalence of depression in rural Rwanda based on symptom and functional criteria. J Nerv Ment Dis. 2002, 190 (9): 631-637.
Bolton P, Tang AM: An alternative approach to cross-cultural function assessment. Soc Psychiatry Psychiatr Epidemiol. 2002, 37 (11): 537-543.
Rasmussen A, Katoni B, Keller AS, Wilkinson J: Posttraumatic idioms of distress among Darfur refugees: Hozun and majnun. Transcult Psychiatry. 2011, 48 (4): 392-415.
Bass JK, Ryder RW, Lammers MC, Mukaba TN, Bolton PA: Post–partum depression in Kinshasa, Democratic Republic of Congo: validation of a concept using a mixed–methods cross–cultural approach. Trop Med Int Health. 2008, 13 (12): 1534-1542.
Liddell B, Silove D, Tay K, Tama N, Nickerson A, Brooks R, Rees S, Zwi A, Steel Z: Achieving convergence between a community-based measure of explosive anger and a clinical interview for intermittent explosive disorder in Timor-Leste. J Affect Disord. 2013, 150: 1242-1246.
Silove D, Brooks R, Steel C, Steel Z, Hewage K, Rodger J, Soosay I: Explosive anger as a response to human rights violations in post-conflict Timor-Leste. Soc Sci Med. 2009, 69: 670-677.
Jayawickreme N, Jayawickreme E, Atanasov P, Goonasekera MA, Foa EB: Are culturally specific measures of trauma-related anxiety and depression needed? The case of Sri Lanka. Psychol Assess. 2012, 24 (4): 791-800.
Betancourt TS, Borisova I, Williams TP, Meyers-Ohki SE, Rubin-Smith JE, Annan J, Kohrt BA: Research review: psychosocial adjustment and mental health in former child soldiers - a systematic review of the literature and recommendations for future research. J Child Psychol Psychiatry. 2013, 54 (1): 17-36.
Tol WA, Song S, Jordans MJ: Annual research review: resilience and mental health in children and adolescents living in areas of armed conflict–a systematic review of findings in low- and middle-income countries. J Child Psychol Psychiatry. 2013, 54 (4): 445-460.
Miller K, Omidian P, Yaqubi A, Quraishy A, Naziry M, Quraishy N, Nasiry S, Karyar N: The Afghan symptom checklist: a culturally grounded approach to mental health assessment in a conflict zone. Am J Orthopsychiatry. 2006, 76: 423-433.
van Ommeren M, Sharma B, Thapa S, Makaju R, Prasain D, Bhattarai R, de Jong J: Preparing instruments for transcultural research: use of the translation monitoring form with Nepali-speaking Bhutanese refugees. Transcult Psychiatry. 1999, 36: 285-301.
Goldberg DP: The Detection of Psychiatric Illness by Questionnaire: A Technique for the Identification and Assessment of non-Psychotic Psychiatric Illness. 1972, Oxford: Oxford University Press
Goldberg DP, Huxley P: Common Mental Disorders: A Bio-social Model. 1992, London: Tavistock/Routledge
Sartorius N, Janca A: Psychiatric assessment instruments developed by the World Health Organization. Soc Psychiatry Psychiatr Epidemiol. 1996, 31 (2): 55-69.
Harding T, De Arango M, Baltazar J, Climent C, Ibrahim H, Ladrigo-Ignacio L, Murthy RS, Wig NN: Mental disorders in primary health care: a study of their frequency and diagnosis in four developing countries. Psychol Med. 1980, 10: 231-241.
Lima BR, Pai S, Santacruz H, Lozano J, Luna J: Screening for the psychological consequences of a major disaster in a developing country: Armero, Colombia. Acta Psychiatr Scand. 1987, 76 (5): 561-567.
de Jong J: A Descent into African Psychiatry. 1987, Amsterdam: Royal Tropical Institute
Igreja V, Kleijn WC, Schreuder BJ, Van Dijk JA, Verschuur M: Testimony method to ameliorate post-traumatic stress symptoms: community-based intervention study with Mozambican civil war survivors. Br J Psychiatry. 2004, 184 (3): 251-257.
Scholte WF, Verduin F, van Lammeren A, Rutayisire T, Kamperman AM: Psychometric properties and longitudinal validation of the self-reporting questionnaire (SRQ-20) in a Rwandan community setting: a validation study. BMC Med Res Methodol. 2011, 11 (1): 116-
Neuner F, Schauer M, Klaschik C, Karunakara U, Elbert T: A comparison of narrative exposure therapy, supportive counseling, and psychoeducation for treating posttraumatic stress disorder in an African refugee settlement. J Consult Clin Psychol. 2004, 72 (4): 579-
Mollica RF, Caspi-Yavin Y, Bollini P, Truong T, Tor S, Lavelle J: The Harvard Trauma Questionnaire: validating a cross-cultural instrument for measuring torture, trauma, and posttraumatic stress disorder in Indochinese refugees. J Nerv Ment Dis. 1992, 180 (2): 111-116.
Muthén Linda K, Muthén Bengt O: Mplus User’s Guide. 2007, Muthén & Muthén: Los Angeles, CA
Panter-Brick C, Eggerman M, Gonzalez V, Saftar S: Violence, suffering, and mental health in Afghanistan: a school-based survey. Lancet. 2009, 374 (21 August 2009): 807-886.
Panter-Brick C, Goodman A, Tol W, Eggerman M: Mental health and childhood adversities: a longitudinal study in Kabul, Afghanistan. J Am Acad Child Adolesc Psychiatry. 2011, 50 (4): 349-363.
Ventevogel P, De Vries G, Scholte W, Shinwari NR, Faiz H, Nassery R, van den Brink W, Olff M: Properties of the Hopkins Symptom Checklist-25 (HSCL-25) and the Self-Reporting Questionnaire (SRQ-20) as screening instruments used in primary care in Afghanistan. Soc Psychiatry Psychiatr Epidemiol. 2007, 42: 328-335.
Ventevogel P: The psychiatric epidemiological studies in Afghanistan: a critical review of the literature and future directions. J Pak Psychiatr Soc. 2005, 2 (1): 9-12.
Ayazi T, Lien L, Eide A, Swartz L, Hauff E: Association between exposure to traumatic events and anxiety disorders in a post-conflict setting: a cross-sectional community study in South Sudan. BMC Psychiatry. 2014, 14: 6-
Roberts B, Browne J: A systematic review of factors influencing the psychological health of conflict-affected populations in low- and middle-income countries. Glob Public Health. 2011, 6 (8): 814-829.
Brooks R, Silove D, Steel Z, Steel C, Rees S: Explosive anger in postconflict Timor Leste: interaction of socio-economic disadvantage and past human rights-related trauma. J Affect Disord. 2011, 131 (1-3): 268-276.
van Ommeren M: Validity issues in transcultural epidemiology. Br J Psychiatry. 2003, 182 (5): 376-378.
Omidian P, Miller KE: Adressing the psychosocial needs of women in Afghanistan. Critical Half. 2006, 17-21. Summer
Cardozo BL, Bilukha OO, Gotway CA, Wolfe MI, Gerber ML, Anderson M: Report from the CDC: mental health of women in postwar Afghanistan. J Womens Health. 2005, 14 (4): 285-293.
Scholte WF, Olff M, Ventevogel P, de Vries GJ, Jansveld E, Cardozo BL, Crawford CAG: Mental health symptoms following war and repression in eastern Afghanistan. JAMA. 2004, 292 (5): 585-593.
Eggerman M, Panter-Brick C: Suffering, hope, and entrapment: resilience and cultural values in Afghanistan. Soc Sci Med. 2010, 71: 71-83.
Panter-Brick C, Eggerman M: Understanding Culture, Resilience, and Mental Health: The Production of Hope. The Social Ecology of Resilience: A Handbook of Theory and Practice. Edited by: Ungar M. 2012, New York: Springer, 369-386.
Panter-Brick C, Eggerman M, Mojadidi A, McDade TW: Social stressors, mental health, and physiological stress in an urban elite of young Afghans in Kabul. Am J Hum Biol. 2008, 20 (6): 627-641.
Ventevogel P, Jordans M, Reis R, de Jong J: Madness or sadness? Local concepts of mental illness in four conflict-affected African communities. Conflict Health. 2013, 7 (1): 3-
L-t H, Bentler PM: Fit indices in covariance structure modeling: sensitivity to underparameterized model misspecification. Psychol Methods. 1998, 3 (4): 424-
McDonald R: Test Theory: A Unified Treatment. 1999, Mahwah NJ: LEA
Pedhazur EJ, Schmelkin LP: Measurement, Design, and Analysis. 1991, New York, NY: Psychology Press, Taylor & Francis Group
Afghan Independent Human Rights Commission (AIHRC): Report on the Situation of Economic and Social Rights in Afghanistan IV. 2009, Kabul: AIHRC
Dupree NH: The family during crisis in Afghanistan. J Comp Fam Stud. 2004, 35: 311-331.
Omidian P, Panter-Brick C: Dignity under Extreme Duress: Local Humanitarian Work in the Afghan-Pakistan Border Areas. Medical Humanitarianism: Ethnographies of Practice. Edited by: Abramowitz S, Panter-Brick C. Pennsylvania: Pennsylvania University Press, In press
Nichter M: Idioms of distress revisited. Cult Med Psychiatry. 2010, 34 (2): 401-416.
de Jong JT, Reis R: Kiyang-yang, a West-African postwar idiom of distress. Cult Med Psychiatry. 2010, 34 (2): 301-321.
de Jong J, Reis R: Collective trauma processing: dissociation as a way of processing postwar traumatic stress in Guinea Bissau. Transcult Psychiatry. 2013, 50 (5): 644-661.
Rees S, Silove D: Sakit Hati: a state of chronic mental distress related to resentment and anger amongst West Papuan refugees exposed to persecution. Soc Sci Med. 2011, 73 (1): 103-110.
Rees S, Silove D, Verdial T, Tam N, Savio E, Fonseca Z, Thorpe R, Liddell B, Zwi A, Tay K, Brooks R, Steel Z: Intermittent explosive disorder amongst women in conflict affected timor-leste: associations with human rights trauma, ongoing violence, poverty, and injustice. PLoS One. 2013, 8 (8):
Catani C, Schauer E, Neuner F: Beyond individual war trauma: domestic violence against children in Afghanistan and Sri Lanka. J Marital Fam Ther. 2008, 34 (2): 165-176.
Grima B: The Role of Suffering in Women’s Performance of Paxto. Gender, Genre, and Power in South Asian Expressive Traditions. 1991, Philadelphia, PA: University of Pennsylvania Press, 81-101.
Clarke K, Saville N, Bhandari B, Giri K, Ghising M, Jha M, Jha S, Magar J, Roy R, Shrestha B: Understanding psychological distress among mothers in rural Nepal: a qualitative grounded theory exploration. BMC Psychiatry. 2014, 14 (1): 60-
The pre-publication history for this paper can be accessed here:http://www.biomedcentral.com/1471-244X/14/206/prepub
The first author’s work is supported in part by a K23 Career Development Award (K23HD059075) from the Eunice Kennedy Shriver National Institute of Child & Human Development (NIH/NICHD), and in part by a Young Scholars Award from the Foundation for Child Development.
The authors declare that they have no competing interests.
AR was responsible for the analytic plan, with help from PV and AS. CPB was the primary investigator on the original survey, undertaken with ME and local collaborators (no other authors were involved in data collection). All authors contributed to authorship of this paper.
About this article
Cite this article
Rasmussen, A., Ventevogel, P., Sancilio, A. et al. Comparing the validity of the self reporting questionnaire and the Afghan symptom checklist: dysphoria, aggression, and gender in transcultural assessment of mental health. BMC Psychiatry 14, 206 (2014). https://doi.org/10.1186/1471-244X-14-206
- Cultural concepts
- Transcultural research
- Idioms of distress