A psychometric approach to assessments of problematic use of online pornography and social networking sites based on the conceptualizations of internet gaming disorder

Background The problematic use of online gaming, social networking sites (SNS) and online pornography (OP) is an evolving problem. Contrary to the problematic use of SNS and OP, Internet gaming disorder (IGD) was included in the new edition of the Diagnostic and statistical manual of mental disorders (DSM-5) as a condition for further study. The present study adapted the criteria for IGD to the problematic use of SNS and OP by modifying a validated questionnaire for IGD (Internet Gaming Disorder Questionnaire: IGDQ) and investigating the psychometric properties of the modified versions, SNSDQ and OPDQ. Methods Two online samples (SNS: n = 700, 25.6 ± 8.4 years, 76.4% female; OP: n = 700, 32.9 ± 12.6 years, 76.7% male) completed the SNSDQ/OPDQ, the Brief Symptom Inventory (BSI) and the short Internet Addiction Test (sIAT) and provided information on their SNS/OP use. Standard item and reliability analyses, exploratory and confirmatory factor analyses and correlations with the sIAT were calculated. Problematic and non-problematic users were compared. Results The internal consistencies were ωordinal = 0.89 (SNS) and ωordinal = 0.88 (OP). The exploratory factor analyses extracted one factor for both questionnaires. Confirmatory factor analyses confirmed the results. The SNSDQ/OPDQ scores correlated highly with the sIAT scores and moderately with SNS/OP usage time. Of the users, 3.4% (SNS) and 7.1% (OP) lay above the cutoff for problematic use. Problematic users had higher sIAT scores, used the applications for longer and experienced more psychological distress. Conclusion Overall, the results of the study indicate that the adaption of the IGD criteria is a promising approach for measuring problematic SNS/OP use.


Assessment of problematic SNS and OP use
There are a number of different diagnostic instruments to assess a problematic use of SNS and OP. Most of them are either based on the diagnostic criteria for behavioural addictions (SNS: e.g. Bergen Social Media Addiction Scale [16] | OP: e.g. Problematic Pornography Consumption Scale [17]) or the Internet Addiction Test [18] (SNS: e.g. Addictive Tendencies Towards SNSs Scale [19] | OP: sIAT-sex [20]). Note, that this is by no means an exhaustive enumeration of all diagnostic instruments. For a detailed overview see Andreassen (2015) [2] for SNS and Wéry & Billieux (2017) [21] for OP. There is no shortage of well-validated instruments, but the following problems still remain: (i) different theoretical conceptualisations of problematic SNS and OP use with the consequence (ii) that no unified, standardized criteria are available to assess problematic use of the three most important specific online applications (Gaming, SNS, OP) in a comparative manner.
The most recent theoretical model for specific Internet-use disorders is the I-PACE model [22]. It is based on empirical findings and integrates previous theoretical considerations from other models in the field of behavioral addictions, like the Syndrome Model [23] or the Components Model of Addiction [24]. The I-PACE model hypothesizes that the aetiology of problematic use is similar for different Internet applications. Therefore, it suggests the application of uniform diagnostic criteria to all applications, thereby standardizing the diagnostic criteria and allowing comparisons of their prevalence rates. Since the American Psychiatric Association already proposed standardized criteria for IGD, it suggests itself to apply these criteria to the problematic use of other internet applications and there are several researchers who agree with this approach [25][26][27]. Some studies have already used this approach to develop psychometric tools to assess problematic internet use [26,28,29] However, to the best of the authors' knowledge, there is only one study that used this approach for the problematic use of SNS [27] and none for the problematic use of OP.

Aim of the present study
Therefore the aim of this study was to examine to which extent the conceptualization of the Internet Gaming Disorder can be adapted to the problematic use of SNS and OP. Petry et al. (2014) [30] who were members of the Substance Use Disorder Work group that recommend to include IGD in the DSM-5 -published a questionnaire (Internet Gaming Disorder Questionnaire: IGDQ) to assess IGD. For this study, we used the German version, which was validated by Jeromin, Barke and Rief (2016) [31] and adapted it for problematic SNS and OP use by rephrasing the items (for details see "Measures" section). In order to assess and evaluate to what degree the concept of the IGD can furnish a useful starting point for the assessment of problematic use of SNS and OP, we investigated the psychometric properties of the two modified versions, the SNSDQ and OPDQ.

Participants and procedure
The data were collected via an online survey (October 2017 -January 2018). The link to the questionnaire was posted to general (e.g. reddit) and application-specific Internet forums (e.g. facebook groups), SNS and mailing lists. At the outset, the participants specified whether they mainly use SNS or OP and were redirected to the corresponding questionnaire (SNS/OP). As an incentive, participants could win one of five gift vouchers for an online store (voucher value: €20). The inclusion criteria were: informed consent, age ≥ 18 years. Exclusion criteria were: no native speaker (German), percentage of online time spent using SNSs/OP ≤5%.

SNS subsample
A total of 939 participants fulfilled the inclusion criteria. Of these, 239 (25.45%) had to be excluded: 228 because they had missing data for the SNSDQ, 7 because they failed to provide serious information (e.g. Klingon as their native language) and 4 because they had an unrealistically fast answering time (2 SDs below the mean time). In the end, data from 700 participants were analysed (Table 1).

OP subsample
A total of 1858 participants met the inclusion criteria. Of these, 669 (36.01%) had to be excluded: 630 because they had missing data for the OPDQ, 25 because they provided obviously false information, 9 because of an unrealistically fast answering time and 5 due to comments suggesting that they had failed to understand the survey. To increase the statistical comparability of the two subsamples (SNS/OP), a random sample of 700 participants was drawn from the remaining 1189. Finally, data from 700 participants were analysed (Table 1).

Socio-demographic information
Information regarding gender, age, education, employment and relationship status was collected.

Information regarding general and specific internet use
The participants reported how much time (hours) they spend online in a typical week. In addition, they provided specific information regarding their SNS or OP use, such as which SNS/OP sites they mostly use and how long they use SNSs or OP (hours/week).

Problematic use
The tendency of problematic SNS or OP use was assessed with the German versions of the SNSDQ and OPDQ. These questionnaires are modified versions of the IGDQ. The IGDQ consists of nine items, which reflect the corresponding DSM-5 criteria for IGD. It has a dichotomous response format consisting of 'no' (0) and 'yes' (1). The score is obtained by adding the responses (score range: 0-9). A score of ≥ 5 was defined as the cutoff for receiving a diagnosis of IGD [30]. For its adaptation regarding SNS and OP, the original items were rephrased by replacing all references to online gaming with references to SNS or OP. For example, 'Do you feel restless, irritable, moody, angry, anxious or sad when attempting to cut down or stop using SNS or when you are unable to use SNS?' instead of 'Do you feel restless, irritable, moody, angry, anxious or sad when attempting

Short internet addiction test
The sIAT is a short version of the Internet Addiction Test and consists of 12 statements expressing possible symptoms of problematic Internet use (e.g. 'How often do you find yourself saying "just a few more minutes" when online?') [18]. For our study, we used the validated German version and rephrased the items for SNS and OP use (e.g. 'How often do you try to cut down the amount of time you spend watching online pornography and fail?') [32]. The participants have to rate the frequency with which they experienced each symptom in the last week on a 5-point scale ranging from 1 ('never') to 5 ('very often'). In the resulting sum score (12-60 points), higher scores indicate more problematic use. The internal consistencies of the adapted scales in the present study were good (SNS: ω = 0.88 | OP: ω = 0.88).

Brief symptom inventory
The German version of the Brief Symptom Inventory (BSI) was used to identify clinically relevant symptoms of the participants [33,34]. The BSI consists of 53 statements expressing symptoms of psychological distress (e.g. 'In the last 7 days, how much were you distressed by feeling tense or keyed up?'). The items are answered on a 5-point scale ranging from 0 ('not at all') to 4 ('extremely'). The total score ranges between 0 and 212, with higher scores indicating a higher level of distress. The internal consistency in the present samples was excellent, with ω = 0.96 (SNS) and ω = 0.96 (OP).

Data analysis
Statistical analyses were conducted using SPSS 24 (IBM SPSS Statistics), SPSS Amos, R version 3.5.1 [35] and FACTOR for the exploratory factor analysis (EFA) [36]. For the standard item analyses for each questionnaire, the SNSDQ and the OPDQ, item difficulties and item-total correlations were calculated. As a measure of reliability, coefficient omega or ordinal omega (in case of binominal data) were computed. These coefficients are recommended as a more accurate alternative to Cronbach's alpha, especially when the assumption of tau-equivalence is violated [37][38][39][40].
Regarding validity, we investigated the factor structures by conducting EFAs and confirmatory factor analyses (CFA). For these, each sample (SNS and OP) was randomly divided into two subsamples (SNS1, SNS2 and OP1, OP2; each subsample: n = 350  [31] and used tetrachoric correlations as input and unweighted least squares as the estimation method [41]. The number of factors to be extracted was determined using Velicer's MAP test [42]. A CFA was performed on SNS2 and OP2 to test the factor solution. The model parameters were estimated using maximum likelihood estimations. Due to the violation of the normality assumption Bollen-Stine Bootstrapping was applied [43]. To evaluate the model fit, the comparative fit index (CFI), root mean square error of approximation (RMSEA) and standardized root mean square residual (SRMR) were calculated. According to Hu and Bentler (1999) [44], the cutoff criteria for an acceptable model fit are a CFI of > 0.95, an RMSEA between 0.06 and 0.08 and an SRMR of < 0.08.
Bivariate relationships between the SNSDQ and OPDG scores and the time spent using the Internet in general, the time spent using the preferred application (SNS/OP) and the sIAT scores were tested with Pearson correlations.
To give a first indication of diagnostic validity, we compared problematic users with non-problematic users. Analogously to the IGDQ, users with a score of ≥ 5 points were categorized as problematic users and all other users as non-problematic [30,31]. Independent t tests (in the case of unequal variances: Welch's tests) were computed to compare the groups regarding age, time spent using the Internet, time spent using their preferred application and sIAT and BSI scores. Due to the unequal group sizes, Hedges' g is reported as a measure of effect size [45]. An effect of g = 0.20 is regarded as small, g = 0.50 as medium and g = 0.80 as large [45].

SNS, OP and Internet use SNS
The participants used the Internet on average for 20.9 ± 14.8 h/week and SNSs for 9.4 ± 10 h/week (44% of the total online time), with Facebook being the most popular SNS (n = 355; 50.7%), followed by Instagram (n = 196; 28%) and YouTube (n = 74; 10.6%). The mean SNSDQ and sIAT scores were 1.2 ± 1.5 and 23.6 ± 7.3 points. Overall, 24 participants (3.4%) had an SNSDQ score of ≥5 points and thus lay above the cut-off for problematic use (see Fig. 1 for details). The mean BSI total score across all the participants was 9.8 ± 16.7.

OP
The participants used the Internet on average for 21.9 ± 15.6 h/week and consumed OP for 3.9 ± 6.1 h/ week (18.9% of the total online time). The most popular form of OP was videos (n = 351; 50.1%), followed by pictures (n = 275; 39.3%) and webcams (n = 71; 10.1%). The mean OPDG and sIAT scores were 1.5 ± 1.7 and 22.3 ± 7.9. A total of 50 participants (7.1%) achieved an OPDQ score above the cutoff of ≥ 5 points (see Fig. 1 for details). The mean BSI score across all the participants was 25.6 ± 27.6.

Item analysis and internal consistency
The results of the item analyses are presented in Tables 2 and 3.

SNS
For the SNS version, item 7 had the lowest endorsement (number of affirmative answers (naa) = 21), while item 6 had the highest (naa = 247). This translates into an item difficulty of p i = 0.03 (item 7) and p i = 0.35 (item 6), with a mean difficulty across all items of p i = 0.13. The corrected item-total

OP
In the OP version of the questionnaire, item 9 (naa = 24) had the lowest endorsement rate, whereas item 7 had the highest (naa = 286). The mean item difficulty was p i = .17, with item 9 being the most (p i = 0.03) and item   Fig. 2).

OP
The OPDQ scores correlated with the OP usage time (r = 0.22, p < 0.01) and very weakly with the Internet usage time per week (r = 0.08, p < 0.05). The highest correlation was found with the sIAT scores (r = 0.72, p < 0.01).

Comparison of persons with problematic and nonproblematic SNS/OP use SNS
Compared with unproblematic users, the problematic SNS users used SNS much more and had higher sIAT scores. They also seemed to experience more psychopathological distress, but, despite the effect size of the difference, this was merely a tendency (p = 0.13). For details see Table 4.

OP
Compared with unproblematic users, participants identified as problematic OP users spent more time on the Internet in general and more time using OP, had much higher sIAT scores and experienced more psychopathological distress (Table 4).

Discussion
In the present study, we adapted the German version of the IGDQ to the use of SNSs and OP and evaluated the psychometric properties of the modified versions in order to investigate to what extent the IGD criteria are suitable for assessing problematic use of SNS and OP.

Item analysis
The average endorsement of the items was low for both questionnaires, which is expected and desirable given that the checklists assess criteria of problematic use in a non-clinical sample. For SNS, the most endorsed item, item 6, concerns procrastination. This appears plausible, since SNS are often used to procrastinate [46,47]. Item 7 (deceive/cover up) received the lowest endorsement, which also seems reasonable given that many people use SNS on a daily basis and in a socially accepted manner, making lying about it unnecessary [12]. For OP, item 7 (deceive/cover up) had the highest endorsement. This is possibly the case because the social acceptance of OP is rather low even if it is used casually and many people may feel embarrassed about it [48]. The lowest endorsement was for item 9, which seems reasonable, since it implies severe consequences (risk/loss of relationships/opportunities). The corrected item-total correlations were medium for both questionnaires and above the threshold of r itc = 0.30 [43]. The only exceptions were item 3 for SNS and item 7 for OP. Item 3 refers to tolerance, a criterion that is typical of substance abuse but seems to be harder to apply in the context of SNSs [49]. The low corrected item-total correlation for item 7 (OP) seems reasonable, since, as discussed, the use of OP may generally be associated with embarrassment, so deceiving others about one's use does not discriminate well between problematic and unproblematic users.

Validity
In the course of the EFAs, a single factor was extracted for the SNS as well as the OP version of the questionnaire. This is in line with the result for the original IGDQ [31]. Item 3 had the lowest factor loading in both versions, probably because the tolerance criterion does not fit very well with the context of SNS and OP. Ultimately, the tolerance criterion originated with substance-based addictions. In that context, its meaning was much more clearly defined than with regard to the problematic use of OP, SNS or, indeed, online gaming, for which its usefulness is also discussed controversially (pro: [30,50] | contra: [51,52]). In the OP version, item 7 (deceive/cover up) also had a lower factor loading than the other items. This reflects the above argument regarding why the item is not so useful for differentiating between problematic and non-problematic users (37.4% of the non-problematic and 86% of the problematic users endorsed it). This indicates that the covering-up behaviour is not explicitly associated with problematic over-use measured by the OPDG but probably with social attitudes towards OP in general. Overall, the results for the CFAs suggested that the one-factor solutions for both questionnaires are questionable and do not represent a good fit. While the SRMR was good for both models, the CFI and RMSEA were below and respectively above the cutoffs. As in the EFA, Item 6 for SNS and Item 7 for OP had particularly low factor loadings. This implies that their correlation with the respective overall scale is low and, accordingly, that their correlation with problematic usage behaviour is low. While this does not necessarily pose a problem, it is important that subsequent studies check whether these items should be revised, weighted differently or even removed. Both questionnaires correlated strongly with the corresponding sIAT versions, indicating good convergent validity. The SNS version showed small to medium correlations with the general Internet usage and SNS usage time (per week). The OP version also showed a small correlation with the OP usage time (per week). The size of the correlations of problematic use with time spent using the respective application is in the range of those consistently reported [53][54][55].
To evaluate the diagnostic validity of the SNSDQ and OPDQ, we first compared observed prevalence rates with those found in other studies. For SNSs, 3.4% of the participants exceeded the cutoff, and, with regard to OP, 7.1% met the criteria for problematic use. Although comparing prevalence rates is difficult due to the multitude of different diagnostic instruments, the rates found here are comparable to some in the existing literature. In their study of a national representative sample of Hungarian adolescents, Bányai et al. (2017) [3] found a prevalence rate of 4.5% for problematic SNS use. Regarding the problematic use of OP, Giordano and Cashwell (2017) [55] reported a prevalence rate of 10.3% in a sample of American college students and Ross and colleagues (2012) [15] found a rate of 7.6% in a sample of Swedish adults.
It is important to note that no diagnosis can be made using these instruments. Firstly, neither the DSM-5 nor the ICD-11 contain any diagnoses for the problematic use of OP or SNS. Secondly, even if they did, a clinical interview by an expert would be necessary to verify the presence of clinically significant distress and functional impairment and the absence of any the exclusion criteria for the individual case, which are a requirement for a psychiatric diagnosis. Such an independent clinical judgement was not collected in the present study, so we cannot determine whether persons above the cutoff would warrant any diagnosis. However, we would consider them as possible candidates for such a diagnosis. To further investigate the diagnostic validity, we compared the users above and below the cutoff and found marked differences. Problematic users spent more time online per week (only for OP) and used their preferred application for longer. Although an increased usage time is not a sufficient criterion to infer a problematic use, several studies have found an -albeit weak -correlation between usage time and problematic use [53][54][55]. In addition, problematic users had much higher sIAT scores and seemed to experience a higher level of psychological distress (only for OP). Overall, these resultsparticularly the very large difference between the BSI total scores in the case of the problematic OP usersmay be regarded as first indicators of the criterion validity of the instruments and suggest that the IGD criteria might be suitable to identify individuals with a problematic use of SNS or OP [56].

Limitations
The study has to be considered in the light of its limitations. One limitation is that only adult participants were tested, although SNS particularly are also frequently used by adolescents [3]. A further limitation is that not all participants answered all questionnaires regarding problematic use (SNS, OP and IGD). This would have allowed a more detailed investigation of the overlapping between the problematic use of the respective applications. Moreover, only self-reported data were collected, which are prone to bias effects, like social desirability or common method variance. In addition, they did not include a clinical judgement. Considering that the aim of the self-report checklists is to identify problematic users, further studies should investigate their validity with samples of persons who are judged by clinicians to show problematic use in a clinically relevant sense. Furthermore, it is important to note that neither the criteria for a diagnosis, nor the number of items or any cut-off have been agreed. We do not intend to propose any arguments as to whether these behavioural patterns would warrant the status of a "disorder". We rather aim to promote research into the identification of the problematic use of SNS and OP by providing a common instrument that may help with a comparative assessment and suggest using this instrument as a common starting point for such investigations, amending them as further research suggests this.

Conclusion
As some psychometric parameters of the tested questionnaires are not satisfactory, it seems that the IGD criteria cannot simply be transferred to the problematic use of SNS/OP. Nevertheless, our overall results indicate that this is a promising starting point and support the viability of using adapted IGD criteria as a framework to assess problematic SNS/OP use. This study contributes to the research regarding measuring aspects of problematic SNS and OP use and might be a first step towards a standardized assessment and contribute to investigations of these emerging constructs. Future research should further investigate the usefulness of the DSM-5 criteria for IGD in the context of SNS/OP use.