Developing a clinical decision tool based on electroretinogram to monitor the risk of severe mental illness

BMC Psychiatry

Table 2 Model performance, discriminative ability, and internal and external validation

	Internal validation in the training data (n = 401)			External validation in the testing data (n = 100)
	Full model^a with all variables	Best model^b	LOOCV	Best model^b
Goodness of the fit
Pseudo-R2 (Nagelkerkes) score	0.46	0.45	0.41	0.48
Brier Score	0.15	0.16	0.16	0.15
Hosmer–Lemeshow p value*	0.67	0.74	0.66	0.72
AIC score	399.03	392.51	–	–
Discriminative ability
Accuracy	0.77	0.77	0.76	0.81
AUC [CI]	0.85 [0.81–0.89]	0.85 [0.81–0.88]	0.83 [0.79–0.87]	0.87 [0.80–0.94]

Note. – Model performance, discriminative ability, and internal validation including the full model and best model using leave-one-out cross validation (LOOCV) on the training dataset (n=401) and external validation including the best model using the testing dataset (n=100)
^a: Full model: a-wave amp fixed + a-wave lat fixed + b-wave amp fixed + b-wave lat fixed + b-wave lat Vmax + a-wave amp Vmax + a-wave lat Vmax + b-wave amp Vmax + age at ERG + sex +pupil size
^b: Best model: predictor variables were selected using the backward and forward stepwise method. The model selected: a-wave amp fixed + b-wave lat fixed + b-wave lat Vmax + a-wave lat Vmax + age at ERG + sex
*Significance levels set at 0. 05, p value >0.05 indicates no evidence of poor fit
LOOCV Leave-one-out cross validation with the variables of the best model. AIC Akaike information criterion, AUC Area under the ROC curve, CI Confidence Interval, amp amplitude, lat latency

ISSN: 1471-244X