Table 6 Quality criteria of measurement properties [18–20]
From: Measurement properties of tools measuring mental health knowledge: a systematic review
Property | Quality criteria | Rating |
|---|---|---|
Reliability | ||
Internal consistency | (Sub)scale unidimensional AND Cronbach’s alpha(s) ≥ $0.70 | + |
Dimensionality not known OR Cronbach’s alpha not determined | ? | |
(Sub)scale not unidimensional OR Cronbach’s alpha(s),0.70 | - | |
Reliability | ICC/weighted Kappa ≥ $0.70 OR Pearson’s r ≥ 0.80 | + |
Neither ICC/weighted Kappa, nor Pearson’s r determined | ? | |
ICC/weighted Kappa ≤ 0.70 OR Pearson’s r ≤ 0.80 | - | |
Measurement error | MIC > SDC OR MIC outside the LOA | + |
MIC not defined | ? | |
MIC ≤ SDC OR MIC equals or inside LOA | - | |
Validity | ||
Content validity | The target population considers all items in the questionnaire to be relevant AND considers the questionnaire to be complete | + |
No target population involvement | ? | |
The target population considers items in the questionnaire to be irrelevant OR considers the questionnaire to be incomplete | - | |
Structural validity | Factors should explain at least 50 % of the variance | + |
Explained variance not mentioned | ? | |
Factors explain < 50 % of the variance | - | |
Hypothesis testing (construct validity) | Correlation with an instrument measuring the same construct ≥0.50 OR at least 75 % of the results are in accordance with the hypotheses AND correlation with related constructs is higher than with unrelated constructs | + |
Solely correlations determined with unrelated constructs | ? | |
Correlation with an instrument measuring the same construct <0.50 OR <75 % of the results are in accordance with the hypotheses OR correlation with related constructs is lower than with unrelated constructs | - | |
Criterion validity | Correlations with the gold standard is ≥0.70 | + |
Correlations with the gold standard is unknown | ? | |
Correlations with the gold standard is <0.70 | - | |
Responsiveness | ||
Responsiveness | (Correlation with an instrument measuring the same construct ≥0.50 OR at least 75 % of the results are in accordance with the hypotheses OR AUC ≥0.70) AND correlation with related constructs is higher than with unrelated constructs | + |
Solely correlations determined with unrelated constructs | ? | |
Correlation with an instrument measuring the same construct <0.50 OR <75 % of the results are in accordance with the hypotheses OR AUC <0.70 OR correlation with related constructs is lower than with unrelated constructs | - | |