Psychometric properties of the perceived stress scale in a community sample of Chinese

Background The Perceived Stress Scale (PSS) is a globally used and self-report scale measuring perceived stress. Three versions of PSS (PSS-14, PSS-10 and PSS-4) are available which comprise 14, 10 and 4 items respectively. However, the Chinese version of the PSS has not yet been validated in a large community-based general population. The aims of this study were to evaluate the psychometric properties of the Chinese PSS in a large community-based general population and to compare the appropriateness of the three versions of PSS. Methods A total of 9507 adults with at least a junior high school education and completed PSS-14 from the China Health and Nutrition Survey were involved in this study. The internal consistency reliability of PSS was assessed using Cronbach’s alpha coefficient and confirmatory factor analysis was employed to test the construct validity. Modification index was used for model extension and the critical ratio was used for model restriction. Results The internal consistency coefficients were satisfactory for PSS-14 and PSS-10, but not for PSS-4. The corresponding Cronbach’s alpha were 0.830, 0.754 and 0.473 respectively. A 2-factor structure was confirmed for the PSS-14 and PSS-10, and all items’ standardized factor loadings exceeded 0.4 for either negative or positive factors. Given that item 12 loaded on both negative and positive factors for PSS-14 and the goodness of fit for PSS-14 was not acceptable, PSS-13 (PSS-14 excluding item 12) was studied. The construct validities of PSS-13 and PSS-10 were satisfactory, but the goodness of fit for PSS-10 were better than that for PSS-13. Conclusions PSS-13 (PSS-14 excluding item 12) and PSS-10 have satisfactory psychometric properties. PSS-10 are more applicable to measure the perceived stress than PSS-13 in a large community-based general population in China.


Background
The concept of stress can be classified approximately into three perspectives which are environmental, psychological and biological stress [1]. Previous studies have shown that psychological stress is associated with eating behavior, smoking, physical activity, waist circumference, BMI and other health outcomes [2][3][4][5][6]. Perceived Stress Scale (PSS), developed by Cohen, Kamarck and Mermelstein [7], is one of the most widely used tools to measure psychological stress in the world. Instead of focusing on a particular event, the PSS appraises the extent that the participants feel unpredictable, uncontrollable or overloaded in their lives [8]. The original PSS comprises 14 items (PSS- 14). Two shorten versions (PSS-10 and PSS-4) are also available which comprise 10 and 4 items selected from the PSS-14 respectively [7,8].

Participants
The participants were from the China Health and Nutrition Survey co-operated by the National Institute for Nutrition and Health of the Chinese Center for Disease Control and Prevention and the University of North Carolina at Chapel Hill in the United States [27]. The ongoing open cohort began in 1989 and drew a sample using a multistage, random cluster sampling method. There were eight diverse provinces and autonomous regions from 1989 to 1997, nine from 2000 to 2009, three municipalities were added in 2011, three provinces were added in 2015. The 2015 survey is the first survey to incorporate Perceived Stress Scale (PSS). The most recent database in 2018 has not been released, hence the present study uses data from the 2015 wave.
The PSS was designed for use with community samples with at least a junior high school education [7]. Therefore, we included participants with age ≥ 18 years and education ≥ junior high school, and excluded those who did not complete the PSS. There were 10,798 individuals with age ≥ 18 years and education ≥ junior high school. 1291 individuals with uncompleted PSS were excluded. There were 9507 individuals at 5296 households living in 361 communities involved in this study eventually. The institutional review board of the University of North Carolina at Chapel Hill and the National Institute for Nutrition and Health, Chinese Center for Disease Control and Prevention approved the study protocol (ethics approval code 201524). All of the participants signed the informed consents.

Measures
The original PSS consists of 14 items (PSS-14) which was translated from English into Chinese, and subsequently back into English to ensure the accuracy of translation.
Each item is rated on a 5-point Likert-type scale, ranging from 0 = 'never' to 4 = 'very often'. The scale can cluster into two subscales: negative subscale (items 1,2, 3,8,11,12 and 14) and positive subscale (items 4,5,6,7,9,10 and 13). The negative subscale negatively states items(e.g., In the last month, how often have you been upset because of something that happened unexpectedly?), and is intended to assess lack of control and negative reactions (perceived distress), while the positive subscale positively states items(e.g., In the last month, how often have you dealt successfully with irritating life hassles?), and measures the degree of ability to cope with existing stressors (coping capacity) [7,9,26]. PSS-10, a shorter version of PSS-14, comprises six negative (items 1,2,3,8,11 and 14) and four positive items (items 6,7,9 and 10). PSS-4, designed for telephone interviews, has four items (items 2,6,7 and 14). The total score of PSS is obtained by reversing the scores on the positive items and then summing across all the items, with a higher score indicating higher perceived stress. Possible total scores for PSS-14, PSS-10 and PSS-4 range from 0 to 56, 0 to 40 and 0 to 16 respectively. In our study, we asked the participants to answer the PSS-14, and then calculate the total scores of PSS-14, PSS-10 and PSS-4 respectively according to the corresponding items.

Statistical analysis
The internal consistency reliability of the three versions of PSS was examined by Cronbach's alpha and the reasonable acceptability criterion of which is ≥0.70.
The construct validity was examined by confirmatory factor analysis (CFA). Using the generalized least squares method, the two-factor models were fitted for different versions of PSS respectively to assess the goodness-of-fit of the factor structure. Models with goodness-of-fit index (GFI) > 0.9, adjust goodness-of-fit index (AGFI) > 0.9, comparative fit index (CFI) > 0.9, standardized root mean square residual (SRMR) < 0.08, and root mean square error of approximation (RMSEA) < 0.08 were regarded as a good fit. Little suggested that investigators ought not rely too heavily on chi-square test for comparing competing models, but rather on the indices mentioned above to determine the overall adequacy of a fitted model, for the chi-square value was an overly sensitive index of fit when working with large measurement models. Therefore, we reported chi-square value, freedom degree and corresponding P value to ensure the results' completeness.
The CFA analyses were performed by Amos 24.0. Cronbach's alpha was obtained using SPSS 21.0. All statistical tests were two-tailed and employed a significance level at p < 0.05.

Model modifications
Two kinds of model modification indices were used, of which the modification index(M.I.) was used for model extension and the critical ratio(C.R.) was used for model restriction. Modification priority was given to the path with the maximum M.I. value or C.R. value.

Results
The sample demographics The sample consisted of 9507 individuals with a mean age of 47.5 years (SD = 14.1) and 51.1% of the sample were men. The majority (88.4%) of the participants were married. The demographics are presented in Table 1.

Confirmatory factor analysis
The goodness-of-fit indices of confirmatory factor analysis (Table 2) presented that the 2-factor model did not fit well with PSS-14 (GFI = 0.923, AGFI = 0.894, CFI = 0.548, RMR = 0.107, SRMR = 0.092 and RMSEA = 0.083). After adding the path from positive factor to item 12 in the model (see modified PSS-14-a), the fitness was acceptable and AIC decreased from 5155.516 to 4503.156. After adding the two-way path between error 4 and error 5 based on the modified PSS-14-a (see modified PSS-14-b), all of the goodness-of-fit indices improved (GFI = 0.947, AGFI = 0.925, CFI = 0.697, RMR = 0.060, SRMR = 0.064 and RMSEA = 0.070) and AIC decreased from 4503.156 to 3579.504 again. The 2-factor model fitted marginally with PSS-13 in which the item 12 was deleted. After adding the two-way path between error 4 and error 5 in the model (see modified PSS-13), the fitness greatly improved. As for PSS-10, the 2-factor model was satisfactory (GFI = 0.959, AGFI = 0.936, CFI = 0.778, RMR = 0.054, SRMR = 0.055 and RMSEA = 0.076) and did not need to be modified. Although in all models, the ratio of chi-square value to degrees of freedom was beyond the range of 1-3, it did not matter heavily in such a large sample as this study. Figure 1 visualized the models in order to clearly understand their structure. Table 3 revealed that all of the standardized factor loadings were statistically significant in PSS-14, modified  Comparison of stress level by characteristics respectively. The mean scores for men and women didn't significantly differ (P = 0.169). The mean score significantly decreased with age from 19.6 in the 18-44 age group to 18.6 in the 60-94 age group (P < 0.001). In addition, the PSS-10 score of participants who were employed was the highest.

Discussion
This study verified the reliability and construct validity of the Chinese version of the Perceived Stress Scale (PSS-14, PSS-10 and PSS-4). To our knowledge, this study is the first to evaluate the psychometric properties of the PSS in a large general community-based population in China.
The results presented that the PSS-10 and modified PSS-14 were suitable for this population, while PSS-14 and PSS-4 did not have adequate psychometric properties. Cronbach's alpha values in this current study revealed that not only the overall PSS-14 and PSS-10, but also each of the two subscales of PSS-14 and PSS-10 were internally reliable, but PSS-4 was not. These findings were in line with previous studies in different countries, such as China [23], Japan [14], Vietnam [20], Korea [28], Thailand [22], Arabia [17], America [29,30], Brazil [21], Greece [11,31], Mexico [12], Germany [19], Sweden [9,15] and Serbia [18]. Few study showed acceptable reliability of PSS-4, such as the United Kingdom [32], French workers [10] and American survivors of suicide [13]. Two studies found that the Cronbach's alpha did not meet the Kline's criteria, but the authors believed that the PSS-4 was reliable for some other reasons, for example the item-total correlations and split-half coefficient were high [16], and a reliability coefficient as low as 0.5 should not seriously attenuate validity [26]. We thought that PSS-4 when applied in our population was not reliable, hence we did not analyze its validity.
Our study supported a two-factor structure of the 14-, 13-and 10-item versions of PSS which was confirmed by most previous studies [18-20, 23, 26, 28, 29]. As expected, the two factors in our study also represented negative and positive feelings, because all the negatively worded items loaded together and all the positively worded items loaded together. In line with some studies [9,26,29,31], our results showed that item 12 (In the last month, how often have you found yourself thinking about things that you have to accomplish?) had relatively low factor loading and loaded approximately equally on both negative and positive factors. This might be due to the translation or the potential interpretation by the subjects, but the possibility needed to be verified in further studies utilizing the Chinese versions of PSS. Given the item 12 was not a good measure for either of the subscale for PSS-14, some researchers suggested delete this item when calculating the total score or subscale scores in future studies [9,29]. We compared the modified PSS-14-a, which had one more path from positive factor to item 12 than PSS-14 to PSS-13, and found that the fitness of PSS-13 was better. Therefore, we also proposed that item 12 be deleted. With regard to the PSS-10, all of items highly loaded on their designated factors. Although most of the previous studies confirmed the twofactor model of PSS, it was controversial whether using the full scale as a whole or using the two sub-scales separately. Considering the correlation between the two factors, some  researchers recommended using the scale as a whole [8,21,23], while others suggested using the two factors as separate indicators of stress although which were weakly correlated [29]. In our study, the two factors were weakly correlated for PSS-13(r s = 0.120), and were not correlated for PSS-10, we believed that it was acceptable no matter using the subscales as a whole or using them separately. By confirmatory factor analysis, our study found covariance between error terms of items 4 and 5 indicating a systematic error in the response. The existence of error covariance may be due to the high degree of overlap in item content, but it was unclear. However, it was seemingly unlikely due to the subjects' misunderstanding of item 4 ("In the last month, how often have you dealt successfully with irritating life hassles?") and item 5("In the last month, how often have you felt that you were effectively coping with important changes that were occurring in your life?"), because "irritating life hassles" was totally different from "important changes". More studies were need to analyze the error covariance. Although the psychometric properties of both PSS-13 and PSS-10 were satisfactory, the reliability and validity of PSS-10 were the best when compared to PSS-13. Moreover, it was critical to complete the questionnaire in a shorter time in a large survey with abundant multiple measures. Therefore, we recommended measurement of perceived stress utilizing the PSS-10 among the community-based general population in China.
The first advantage of this study is our utilization of a large community-based general Chinese population. The second advantage is that we excluded those with education lower than junior high school, to whom the PSS is not applicable. To our knowledge, no authors have mentioned this in their manuscripts. There are a few limitations. First, the China Nutrition Transition Cohort Study does not involve other psychological investigation. We can only verify the structure validity of PSS, but cannot verify the concurrent validity or other validity.

Conclusions
Comprehensively, the results of our study reveal that PSS-13 (PSS-14 excluding item 12) and PSS-10 have satisfactory psychometric properties. PSS-10 are more applicable to measure the perceived stress than PSS-13 in a large community-based general population in China.   Wilcoxon rank sum test used for comparing mean differences in the total score by gender a,b,c results of LSD test; different letters indicate significant differences between groups JGZ, WWD, CS, YFOY, YW, LL and HRJ conducted the survey. All authors read and approved the final manuscript.

Funding
Financial support was provided by the Carolina Population Center, University of North Carolina at Chapel Hill (No. 5R24 HD050924), the National Institutes of Health (No. R01-HD30880, DK056350, R24 HD050924 and R01-HD38700) and the Fogarty International Center, National Institutes of Health (No. 5D43TW007709 and 5D43TW009077). The funders had no role in the design of the study and collection, analysis, and interpretation of data and in writing the manuscript.

Availability of data and materials
The datasets generated and analyzed during the current study are available in the Carolina Population Center repository, http://www.cpc.unc.edu/ projects/china/data.

Ethics approval and consent to participate
The institutional review board of the University of North Carolina at Chapel Hill and the National Institute for Nutrition and Health, Chinese Center for Disease Control and Prevention approved the study protocol (ethics approval code 201524). All of the participants signed the informed consents.

Consent for publication
Not applicable.