- Study protocol
- Open Access
The impact of video gaming on cognitive functioning of people with schizophrenia (GAME-S): study protocol of a randomised controlled trial
BMC Psychiatry volume 21, Article number: 46 (2021)
Video gaming is a promising intervention for cognitive and social impairment in patients with schizophrenia. A number of gaming interventions have been evaluated in small-scale studies with various patient groups, but studies on patients with schizophrenia remain scarce and rarely include the evaluation of both clinical and neurocognitive outcomes. In this study, we will test the effectiveness of two interventions with gaming elements to improve cognitive and clinical outcomes among persons with schizophrenia.
The participants will be recruited from different outpatient units (e.g., outpatient psychiatric units, day hospitals, residential care homes). The controlled clinical trial will follow a three-arm parallel-group design: 1) cognitive training (experimental group, CogniFit), 2) entertainment gaming (active control group, SIMS 4), and 3) treatment as usual. The primary outcomes are working memory function at 3-month and 6-month follow-ups. The secondary outcomes are patients’ other cognitive and social functioning, the ability to experience pleasure, self-efficacy, and negative symptoms at 3-month and 6-month follow-ups. We will also test the effectiveness of gaming interventions on neurocognitive outcomes (EEG and 3 T MRI plus rs-fMRI) at a 3-month follow-up as an additional secondary outcome. Data will be collected in outpatient psychiatric services in Hong Kong. Participants will have a formal diagnosis of schizophrenia and be between 18 and 60 years old. We aim to have a total of 234 participants, randomly allocated to the three arms. A sub-sample of patients (N = 150) will be recruited to undergo an EEG. For neuroimaging assessment, patients will be randomly allocated to a subset of patients (N=126). We will estimate the efficacy of the interventions on the primary and secondary outcomes based on the intention-to-treat principle. Behavioural and EEG data will be analysed separately.
The study will characterise benefits of gaming on patients’ health and well-being, and contribute towards the development of new treatment approaches for patients with schizophrenia.
ClinicalTrials.gov NCT03133143. Registered on April 28, 2017.
More than 21 million people worldwide suffer from schizophrenia . Schizophrenia is associated with severe disabilities in cognition and global functioning [2, 3]. Deficits in attention, executive function and working memory are the core challenges for this patient group [4, 5]. Having a working memory is a crucial part of daily functioning and holding and manipulating information . Poor working memory is associated with poor functional outcome  and limits for social relationships . Disabling cognitive symptoms in schizophrenia are associated with prefrontal dysfunction .
Cognitive deficits can be improved with specialised cognitive training programmes. For example, remediation therapy is designed to improve attention, memory, and problem solving . Various types of computer-based applications for cognitive remediation have also been established, as well as serious games, which are specialised for training cognitive deficits. Fisher et al. tested a set of computerised exercises in patients with schizophrenia and showed that those who made the most progress in the basic psychophysical auditory exercise also exhibited the most improvement in verbal working memory and global cognition . More recently, video gaming in general (on a computer, console, online platform, or mobile device) and sole entertainment games (action, sport, role-playing, adventure, fighting, racing, family entertainment, casual games)  have opened up an avenue for new remedial interventions targeting attention, problem-solving, emotional expression, and socialization . This approach is potentially more effective than direct instructional methods, such as coaching, which yields limited benefits for patients with schizophrenia . Kühn et al., for example, reported that video gaming increases grey matter volume in the brains of healthy adults, and they recommended gaming for patients with schizophrenia .
Functional magnetic resonance imaging (fMRI) studies have recently been used in working memory tests for patients with schizophrenia . Some studies have reported increased activation of the dorsolateral prefrontal cortex (DLPFC) in patients with schizophrenia during their working memory performance . Imaging methods have also been used to assess the effectiveness of gaming interventions for neurocognitive outcomes. One fMRI study suggested that persons who have played a shooter game improved their attentional resources more efficiently than non-gamers . A systematic review further suggests action video games as a promising method in mental health treatment, as asserting that specific types of video games can alter brain structure or improve certain aspects of cognitive functioning . Video gaming may be particularly useful for persons who have difficulties in treatment compliance or engagement because gaming itself may offer positive experiences and emotions [19, 20], a sense of self-efficacy, relatedness, and social interaction [21, 22]. These effects are expected to benefit patients with schizophrenia [2,3,4,5]. Further, video gaming is cost effective and allows personalised targeting of specific brain abnormalities by ameliorating the dysfunction in specific brain circuits and normalizing network dynamics. When suitably scaled, gaming may be highly effective in countering the impact of risk factors associated with prodromal states of various mental disorders [14, 22]. This suggestion is in keeping with speculation that some commercially available video games may already have the potential to cause changes in human behaviour . However, the influences of video gaming on brain function and structures, in relation to its potential benefits for cognitive functioning, social relationships , pleasure and self-efficacy, remains poorly characterised . In addition, the most useful setting for the implementation of gaming interventions is still unclear .
We hypothesise that playing video games  may alter brain function, cognition and behaviour  for patients with schizophrenia . Benefits for prospective memory impairments and emotion-behaviour decoupling related hedonia have been demonstrated in schizophrenia patients . One neuroimaging study outside of the field of mental health shows that both early stimulus processing and late retention period activities associated with video gaming could improve visual working memory function, and suggests that video gaming might be suitable for patients with schizophrenia . Indeed, there is some evidence that entertainment games can be useful in mental health in, for example, improving moods or stress levels [28, 29]. However, these are studies with small sample sizes and narrow scopes in age groups .
Based on the current role of video gaming , its rapid expansion in the market, and its promises in treatment for persons with schizophrenia, it is important to conduct a larger-scale investigation to fully characterise its potential clinical profile and efficacy . Here, we will test the effectiveness of gaming embedded within two interventions in a longitudinal study on relevant cognitive and clinical outcomes, as well as EEG and neuroimaging assessments (structural MRI, resting-state fMRI, and diffusion tensor imaging) as a form of brain physiological readout.
This study aims to compare the effectiveness of cognitive training using computerized gaming exercises to that of entertainment video gaming and a non-gaming, passive control group, looking at a range of outcomes. The primary objective is to test the effects of the interventions on patients’ cognition, especially on working memory at 3-month and 6-month follow-ups. The secondary objectives are to test the effects of the interventions on patients’ other cognitive and social functioning, the ability to experience pleasure, self-efficacy, and negative symptoms at 3-month and 6-month follow-ups. Additional secondary outcomes are EEG brain activity and structure related to working memory function at a 3-month follow-up.
Cognitive training with computerised exercises is more effective than entertainment video gaming or the non-gaming control in improving patients’ working memory at a 3-month and/or 6-month follow-up.
Cognitive training with computerised exercises is more effective than entertainment video gaming and the non-gaming control in improving patients’ cognitive and social functioning, experience of pleasure, and self-efficacy at a 3-month and/or 6-month follow-up.
Cognitive training with computerised exercises is more effective than entertainment video gaming and the non-gaming control in improving negative symptoms in schizophrenia at a 3-month and/or 6-month follow-up.
Cognitive training with computerised exercises is more effective than entertainment video gaming and the non-gaming control in improving neurocognitive outcomes (EEG signals, brain activity and structure) at a 3-month follow-up.
The effectiveness of the intervention will be assessed using a controlled clinical trial with a pragmatic, three-arm parallel-group design. The three-arm design is chosen to explicitly test the superiority of the experimental cognitive training with computerised exercises (CogniFit) compared to the active control (entertainment games) and treatment as usual (non-gaming passive control) . Since the objective of the study is to test the efficacy of the type of gaming (cognitive training with computerised exercises vs. entertainment gaming), having a three-arm trial is necessary to assess the effects of the gaming types separately.
To assess fidelity, we will use gaming diaries to verify whether the interventions have been delivered as intended (intervention fidelity) and confirm patient gaming activity (gaming frequency [number of gaming sessions per week], the length of each session [minutes], number of drop-outs). We will also analyse any information on the perceived strengths and limitations of interventions written in the patients’ diaries.
Study setting and participant characteristics
The data collection, subject recruitment and gaming will be carried out in outpatient services (e.g., outpatient psychiatric units, day hospitals, residential care homes). Specific inclusion criteria mandate that patients should have a formal diagnosis of schizophrenia spectrum disorder (Diagnostic and Statistical Manual of Mental Disorders, fifth edition, DSM-5), and be between 18 and 60 years old. We will intentionally recruit people who are non-active, less-experienced gamers . Participants will be required to be able to speak Cantonese, be able to participate based on their own free will and have the ability to provide written informed consent. Participants should have a cognitive status deemed suitable for study participation based on ‘the judgement standard’, i.e., be approved by the staff responsible of the treatment based on their clinical expertise.
The exclusion criteria are: 1) meeting the diagnostic criteria for a current major depressive, manic or hypomanic episode (DSM-5), or mental retardation, 2) having severe visual impairment, 3) being an active gamer (i.e. gaming > 5 h/week ), 4) displaying a lack of ability to decide one’s own participation, 5) substance abuse (other than nicotine dependence), 6) head injury, hemiplegia, or other neurological disorder, 7) electroconvulsive therapy (ECT) in the past six months, 8) having a lack of Magnetic Resonance Imaging (MRI) compatibility (e.g., cardiac pacemakers, metallic implants, restless behaviour, claustrophobia), and 9) pregnancy.
Power analysis and sample size
We have calculated the sample size based on the primary outcome as follows: (1) the number of actual pairwise tests to be made for the efficacy of the primary outcome and (2) the two-level modelling approach in the final data analysis, in which the type I error has been adjusted. The statistical efficiency will be ensured using the multivariate analysis of variance (MANOVA) method for multiple group comparison. First, given that video gaming is a fairly novel strategy, we will base the sample size calculation (a priori)  on a cognitive-efficacy meta-analysis for patients with schizophrenia  showing an overall effect size (ES, Cohen’s d) of 0.58 on verbal working memory (a primary outcome). Based on our hypothesis, the primary endpoints will be the effects on verbal working memory after 3 and 6 months of cognitive training group (CogniFit), compared with the other two groups: cognitive training vs. entertainment gaming, cognitive training vs. non-gaming control group, and entertainment gaming vs. non-gaming control group. Four pairwise interactions between the contrast of the two comparisons and the two time points will be tested. For multiple comparison tests of four, for a type I error level of 5% (two-sided), an adjusted significant level should be α/2 = (1- (1–0.05)4) /2 = 0.01274 /2 = 0.0064, and the corresponding z score for a one-tailed test should be 2.49. Given the effect size 0.58, assuming equal sample size of the three groups, with a statistical power of 0.8 and α= 0.01274, we will require at least 198 subjects (66 per group at follow-up phase) by applying the equation, 2(Z1-β + Z1-α/2)2 / ES2. According to a meta-analysis , the total sample size in previous cognitive training studies was around 50 (range 10–138).
Using evidence-based rationale for patient flow in this study, we can assume that about 60% of patients screened will not be eligible for the study, due to age or lack of capacity to participate in the study . Based on existing literature, we anticipate that about 45% of patients with schizophrenia will refuse to participate in the RCT studies , about 60% of patients screened will not be eligible for the study, and around 16% will drop out during the course of intervention . Thus, we need a total of 1073 subjects to be approached and 237 participants to be randomly allocated to three study groups (79 patients/group) to ensure a final sample of 198 participants in the follow-ups (66 patients/group). These numbers are realistic given that the total number of patients with schizophrenia in our study sites is about 5500, and the total number of schizophrenia patients in Hong Kong is 40,000.
For the EEG analysis, a sub-sample of 50 patients in each group will be recruited (N = 150). For neuroimaging assessment using 3 T MRI and rs-fMRI, a sub-sample of 126 participants from our total sample (N = 198) will need (63%) to be randomised at baseline. Randomisation will be based on a list of computer-generated random numbers. We assume that 30% will drop-out between baseline and the 3-month follow-up assessment, which leaves us with 29 patients in each group (a total of 87 participants at baseline). The sample size will be appropriate for our neuroimaging assessment; the average number of participants in RCT studies assessing patient cognition  or changes in brain structure has been about 20 . The flow-chart of the study is presented in Fig. 1.
For privacy purposes, the medical records in each service organisation will be screened by the clinical staff to determine the eligibility of patients to participate in the study. To ensure that patients have the capacity to make the decision to participate or not, an extended informed consent process will be used . Eligible participants will first receive a short leaflet about the study from the staff or during a short information session organised for the participants to consider their availability. If an eligible patient shows interest, more detailed written and oral information will be given. It will be ensured that a patient understands the information and they will have the opportunity to ask any questions. If a patient is willing to participate in the study, they will sign an informed consent form. To ensure accuracy of the written information to be provided to subjects, its content will be revised whenever new information relevant to the subject’s consent becomes available. Prior to participation in the trial, the study participants will receive a copy of the signed and dated written informed consent form and any other written information provided to the subjects. If any updates occur, the participant will receive a copy of the signed and dated consent form updates and a copy of any amendments to the written information provided to subjects during their participation in the trial.
After signing the informed consent form, background information of each respondent, their Intelligence Quotient (IQ, if available in medical records) and Mini Mental State Examination (MMSE) will be collected . Their medication dosages (chlorpromazine equivalence) will also be collected as medications currently used in schizophrenia may affect the response to cognitive training strategies [9, 37]. A simple practical test to assess the participants’ computer skills will be carried out (starting up a computer, using a mouse). If any doubts of computer use arise, our intervention is long enough (12 weeks) for participants to learn basic computer and gaming skills. The recruitment will end as soon as the planned total numbers of participants are reached for the analysis.
Randomisation, allocation concealment and blinding (masking)
After baseline data collection, the trial manager will be notified of new participants. Each participant will be allocated to one of the three arms based on a list of computer-generated random numbers provided by an external clinical trial randomisation service (blocks of 6 consecutive patients, a 1:1:1 ratio). Allocation will be masked to the trial statistician, but it cannot be masked to the research assistants (RAs) who will supervise the gaming sessions and collect follow-up outcome data, or from the study unit staff who will work with the patients. Furthermore, a sub-sample of 150 participants will be recruited for EEG analysis, and 126 participants from our total sample will be randomly selected and assigned to neuroimaging assessment at baseline (fMRI). Again, the participants for neuroimaging assessment will be allocated based on a separate list of computer-generated random numbers provided by an external clinical trial randomisation service (blocks of 6 consecutive patients, a 1:1:1 ratio). The trial manager will inform the RAs the identities of participants allocated to fMRI assessment.
Participants will be allocated into three groups: 1) cognitive training with computerised exercises (G1, CogniFit, N=78), 2) entertainment video gaming (G2, Entertainment Gaming, N=78), and 3) treatment as usual (N=78). CogniFit and entertainment games (SIMS 4) represent two distinct gaming genres: CogniFit is a cognitive training package with computerised exercises aiming to improve cognitive abilities based on a personalised brain training regimen. SIMS 4 is a pure entertainment game without any known health impact. The administration of both interventions will include 60 pre-scheduled gaming sessions  at the study settings as part of the normal day programme, lasting from 45 to 60 min each, at least 5 days/week , 60 h in total. The gaming schedule can also be tailored based on the participants’ needs (working, studying, family issues) as long as the gaming hours are in line with the total hours planned. Individual gaming sessions on separate computers will be conducted in small groups (about 3–6 per group) and closely monitored by the trained research assistant. Monitoring of gaming sessions is crucial to ensure intervention fidelity. This was exemplified in our feasibility study, which showed that, out of all possible 60 pre-scheduled sessions, 97% of sessions were realised as planned, and all participants enjoyed gaming. Gaming hours (diaries, automatic gaming logs) for each gaming session will be recorded to provide evidence of the intervention fidelity; a minimum of 50 gaming hours will be acquired to ensure observable neuroplasticity after gaming . To report any negative side effects of gaming (e.g., tiredness of eyes), feedback and any concerns will be reported in a gaming diary after each gaming session .
Cognitive training with computerised exercises (an experimental intervention group, CogniFit)
The supposed mechanism of CogniFit is derived from evidence that computerised exercises focusing on auditory and verbal processing are likely to yield improved verbal learning and memory , and activate the brain reward system that drives brain plasticity in adults with schizophrenia [40, 41]. Exercises include the elements needed for an effective cognitive intervention: a large number of specific repetitive learning trials driven by maximally enduring, neurologically reliable cognitive gains, and individually adapted content based on each learner’s needs . To match the training regimen for each participant’s unique cognitive needs (deficits), a short assessment using CogniFit exercises will be done at the beginning of the intervention. During each training session, the participants will be instructed to play memory games and at least one exercise from each of the three categories: memory, spatial perception, and mental planning. Otherwise, they will be free to choose which exercises they wish to play.
During the first gaming session, a game will be introduced to the participant, and its use will be tested together with an RA. The participant’s ability to play digital games will be explored to ensure that they have the basic gaming skills required for active gaming (how to start the computer, how to play the game, how to change game options, and so on). During each gaming session, an RA will be available for gaming support. The RA will also provide assistance, if required, based on each gamer’s individual needs.
Entertainment video gaming (an active control group)
The video game offers entertainment without any known cognitive or health-related outcome. We will use SIMS 4, which includes a variety of personalities portrayed as cartoon characters that be customised based on gamers’ needs and wishes. For example, a gamer can change the clothing of characters, develop their stories, build homes, have the characters travel and build relationships between characters. As in the intervention group, an RA will be available during each gaming session to guide and support gaming skills and initiatives.
Treatment as usual (a passive control group)
No specific gaming intervention will be offered to those who receive treatment as usual. To minimise the risk of intervention contamination, the participants will be encouraged not to play video games during the study period and will be asked to report any video gaming with which they may have engaged.
Outcome evaluation (primary and secondary outcomes)
Verbal working memory will be measured with the Letter-Number-Sequencing task (Letter-Number-Span-Test, LNST) from WMS III (Wechsler Memory Scale, the 3rd Edition, simplified/traditional Chinese versions). The instrument includes an NIMH-MATRICS (US National of Institute of Mental Health, the Measurement and Treatment Research to Improve Cognition in Schizophrenia) battery .
A battery of cognitive tests will be used to measure the processing speed [Trail Making Test, TMT, A ], attention and vigilance [Sustained Attention to Response Task, SART], visuospatial working memory [Spatial Span from WMS III ], and reasoning and problem solving [Wisconsin Card Sorting Test ]. The instruments to be used are well tested, reliable, recommended, and available in simplified/traditional Chinese, either in paper or digital format .
A Chinese version of the Brief Social Phobia Scale, BSPS, will be used for assessing the severity of and treatment response to social phobia . The Cronbach’s alpha for the Chinese version is 0.88, which states a high level of internal consistency .
Experience of pleasure
The Temporal Experience of Pleasure Scale (TEPS, Chinese version) covers anticipatory and consummatory components (Cronbach’s alpha=0.83; test-retest reliability r=0.79) .
The Chinese version of the General Self-Efficacy Scale, GSE, a self-reporting measurement of self-efficacy with good internal consistency (Cronbach’s alpha = 0.91), will be used .
Negative symptoms of schizophrenia
The Chinese version of the Clinical Assessment Interview for Negative Symptoms (CAINS)  will be administered by qualified psychiatrists who have received proper training for this assessment tool. The item-total score correlation will also be presented for each item, and can range from 0.58 to 0.81. The internal consistency (Cronbach’s alpha) for the scale is high (0.91) .
Depressive symptoms will be measured with the Calgary Depression Scale for Schizophrenia (CDS-C) , an observer-rated Likert-scale to measure depressive symptoms (Cronbach’s alpha=0.80; inter-rater reliability Kappa coefficient > 0.79; test-retest reliability r=0.927), the Simpson-Angus Rating Scale (SAS) , the Barnes Akathisia Rating Scale (BARS), and the Abnormal Involuntary Movement Scale (AIMS). Participants’ IQs will be collected from medical records (if available) and the Mini-MMSE will be used to verify each participant’s capacity to participate in the study. If, at baseline, it is obvious that the participant suffers from a condition that will interfere with participation in the study (IQ, MMSE), the participant will be excluded from the study. Permission to use each instrument has been obtained and license costs have been paid as necessary.
The impact of gaming on brain functional networks and changes in neuronal dynamics can be revealed by an EEG during the execution of a visual working memory task. Resting-state EEG will be collected to evaluate changes in spontaneous activities in the brain for comparison with the resting-state MRI. The EEG has been extensively utilised to assess the effectiveness of daily cognitive exercises to improve sensory information processing, from attention to working memory processes, in people with schizophrenia. Its superior temporal resolution is most apt in identifying relevant markers in neuronal network dynamics that underlie the observed improvement, that might help in further identifying subsets of patients most likely to benefit from cognitive exercises .
In resting-state functional (rsfMRI) and structural MRIs, the most often reported neurocognition measurement for regions includes that of the nucleus accumbens, amygdala, striatum, anterior cingulate cortex, orbitofrontal cortex, and dorsolateral prefrontal cortex , the dorsal lateral prefrontal cortex and the anterior cingulate Testgyrus . During the rsfMRI, the subject will not perform any tasks while being scanned. The rsfMRI is based on the principle that, even when the brain is at rest, there are still low fluctuations of neural activity. While the resting-state neural activity is at a much lower rate than when the brain is active, it can be used as a biomarker to study the underlying neuronal networks.
Comparing pre- and post-training records will reveal the impact of cognitive training with computerised exercises on the neuronal networks underlying the physiological basis of working memory ability. We will assess how the training influences inter-areal functional and structural (measured with MRI/Diffusion tensor imaging [DTI]) connectivity and the quantitative indices of cortical criticality and excitation/inhibition balance.
Data collection and follow-up
To avoid burdening the patients, data will be collected in separate phases (see Table 1):
A preparatory data collection (estimated 45 min) will be collected only once, at the beginning of the study. It aims to collate the background information of the participants to determine their capacity for informed consent and any other potential factors that might restrict participation in the intervention (background information, IQ, medication dosage [based on medical records], MMSE, CDS-C, SAS, BARS, AIMS).
For the primary (verbal working memory) and secondary outcomes (estimated 75 min) (cognitive and social functioning, experience of pleasure, self-efficacy, negative symptoms), the data will be collected at three points in time: at baseline, after the intervention (3-month follow up) and at a six-month follow-up (6 months after baseline).
The EEG measures taken during the working memory test will be collected at baseline and 3 months after baseline only (EEG 1 h; rsfMRI and structural MRI, 45 min).
First, for the primary and secondary outcomes (cognitive and social functioning, experience of pleasure, self-efficacy, negative symptoms), the data will be collected in paper format or digital format  by a trained psychologist or RA.
Second, the EEG will be recorded during resting state and a multi-objects visual working memory task (VWM). The task, modified from our prior studies , will be performed so that the neuronal mechanisms limiting VWM performance and capacity can be identified during the memory retention period. Subjects will attend either hemifield, depending on the pre-stimulus cue, and memorise a visual display containing one to six objects. After the retention period, a second stimulus will appear, and the subject will answer whether the stimulus presented in the second display matches part of the first display or not. We will correlate both local and large-scale neuronal dynamics with the task performance, both in pre-training and post-training neuroimaging sessions.
Third, fMRI scans will be acquired using a Philips 3 T MRI scanner. During scanning, a painless procedure, patients will rest on a table, which will be slid into a large tunnel-shaped scanner. As there will be a constant drumming noise during scanning, earplugs or earphones will be provided. During the examination, the patient will be able to freely communicate with staff via an intercom. Resting-state fMRI data will be acquired using T2-weighted echo planar pulse sequence imaging (300 whole-brain volumes were collected with slice thickness=4 mm, TE=30 ms, TR= 2000 ms, flip angle=90 degrees, spatial resolution 3x3x4 mm isotropic voxels, transverse orientation, 32 slices fully covering the cerebral cortex and the cerebellum, acquisition time = 10 min). Structural MRIs will be acquired with the MP-RAGE T1 sequence. A DTI of the axonal tracts in cerebral white matter will be performed with a protocol validated in the Human Connectome Project. We will acquire a 15-min session of resting-state data to obtain high-quality measurements of intrinsic functional connectivity of BOLD signals (fMRI)  and critical dynamics (fMRI) .
Cognitive and clinical outcomes
The demographic information and outcome measurements of the participants in the three study groups will be compared for similarities at baseline. Based on the types of variables, Chi Square tests and analyses of variance (ANOVA) will be used for categorical and continuous variables, respectively. Variables found significantly different between study groups will be controlled in the main analysis as covariates. The normality assumption of the outcome scores will be tested (Kolmogorov-Simonov tests). Outcome measures of non-normal distribution will be transformed to normal in the main analysis.
In the main analysis, we will estimate and test efficacy of interventions on the primary and secondary outcomes. Based on the intention-to-treat principle, we will first assess intervention effects on the primary outcome, verbal working memory, by means of multilevel modelling analysis. A two-level model can be built with repeated time points at level 1, and patients at level 2. Two indicator variables, G2 and G3, will be included in the fixed part of the model to estimate and test mean differences between the cognitive training group with computerised exercises (G1) and the entertainment gaming group (G2), as well as between G1 and the non-gaming control group (G3). The baseline difference between G2 and G3 can be tested using a Ward test, based on the regression coefficient estimates of the two indicators. For changes in the outcome over time, two more indicators, variables T2 and T3, in contrast to the baseline time point, T1, will be included in the fixed part of the model to estimate changes from baseline to 3 months, and from baseline to 6 months. To assess the effects of interventions over time, interaction terms between the treatment group indicators, G2 and G3, and time indicators, T2 and T3, will be included in the fixed part of the model. The significantly different demographics between groups identified in the first step of the analysis and other possible confounders will be adjusted by including them in the main effect model. The within-patient random effects of the outcome measurements will naturally be taken into account by the level two variance in the model. A pairwise comparison and other hypotheses on the intervention effects can be tested using a generalised Ward test with an adequate degree of freedom from the modelling analysis. This analysis approach has two main advantages: a statistically highly efficiency and taking missing data into account. The effects of interventions on other secondary outcome measures, normally distributed and measured at three time points, such as patients’ cognitive and social functioning, experience of pleasure, and self-efficacy, will be analysed using the same model, but a separate model for each outcome. A multilevel logistic model will be used for outcomes in binary form, and multilevel multinomial models will be used for ordinal or nominal outcomes. For outcomes measured at two time points only, such as neurocognitive outcomes, the model will have only one time indicator with two interaction terms to test the efficacy of cognitive training with computerised exercises three months into the trial.
SPSS (the Statistical Package for the Social Sciences) Missing Values will be used to identify the amount and pattern of missing data. If missing values are few and randomly allocated, the last observation carried forward (LOCF) will be used to replace the missing data. If missing values are not randomly distributed, multiple imputation approaches will be used. A sensitivity analysis will be performed by comparing results from analysis datasets without missing data imputation and with imputation.
In all tests, significant differences between groups will be detected with a 95% confidence level, and P values of < 0.05 will be considered significant. Statistical analyses will be carried out with SAS System for Windows, version 9.4 (SAS Institute Inc.) or R statistical software (R Core Team, 2016).
Behavioural data analysis for neuroimaging measurement
Accuracy (%) and reaction times (ms [milliseconds]) will be recorded for the cognitive tasks  and additional signal detection indices will be calculated for the multi-object visual working memory task. Change in behavioural performance will be analysed for all three groups using a multilevel regression/repeated measures ANOVA, testing the task/group and by time interaction. Within this framework, group differences in change will be evaluated using contrast analyses.
For EEG signals associated with specific defined events, ERP data analysis will be conducted in Curry 8 (Compumedics, Charlotte, NC, USA). Raw continuous EEG will be re-referenced to linked mastoids and then bandpass filtered from 0.1 to 30 Hz. Eye blink artifacts will be corrected by electrooculography (EOG) regression method. Stimulus-locked epochs will be extracted from − 200 ms before to 1000 ms after the onsets of stimulus. Only trials with correct behavioural responses will be included in the subsequent ERP analyses. Epochs of the same experimental condition will be averaged to produce the ERP. The mean voltage between − 200 and 0 ms will be used for baseline correction. Electrodes chosen for analyses cover the two hemispheres and midline cortical regions. The six lateral electrodes that cover two hemispheres include Fp1 and Fp2 for prefrontal regions, F3 and F4 for the frontal regions, C3 and C4 for the central regions, T7 and T8 for the temporal regions, P3 and P4 for the parietal regions, and PO3 and PO4 for the parieto-occipital regions. The four electrodes that cover the midline regions are Fz, Cz, Pz, and POz. ERP components (P1, N1, N2 and P3), and their corresponding time windows will be identified from the grand averaged waveforms. The mean amplitude of each ERP component will be calculated for further statistical analysis.
MRI data analysis
The outcomes will be analysed at two points only (baseline and after 3 months). A time-series statistical analysis will be carried out using FILM (FMRIB S Improved Linear Model) with local autocorrelation correction . To increase the power to detect group differences, to reduce the number of voxel-wise statistical comparisons, and to avoid risk of bias for subsequent analyses, the data from all subjects at both points of data collection (54 sessions) will be pooled to identify functional regions of interest (ROIs) for each task. Z-statistic images will be thresholded using a cluster correction method to account for multiple comparisons, as determined by Z42.3, to provide a brain-wise cluster significance threshold of p¼0.05.
The WMA Declaration of Helsinki  and its ethical principles for medical research involving human subjects has guided our study design and protocol. The data collection will be voluntary based; written informed consent forms will be signed by patients. Prior to enrolment, participants will be made aware of all potential risks and benefits of participation (oral and written information). All steps will be taken to avoid any harm to participants. First, to minimise extra burden due to gaming interventions, playing intensity will be set at a maximum of 5 h per week, or it will be tailored to participants’ individual needs. Second, we have chosen video games that are purely fictional and do not include intentional violence. Third, possible changes in a patient’s health status will be continually monitored throughout the project. Fourth, players in this study will play the game under supervision only. Fifth, patients’ participation may be biased toward those who are interested in gaming, which will be taken into account in the conclusions. Any extra costs to the patients as a result of participation (treatment or travel costs, gaming licenses) will be covered.
This study will be the first clinical trial in Hong Kong to test the effectiveness of video gaming on improving cognitive and neurocognitive functioning in people with schizophrenia. The study will offer new insights into, and characterization of, the impact of using video gaming on patients’ health and well-being. The study will also contribute towards the development of new approaches to patient care in mental health services in Hong Kong. The results may also have implications for other health conditions in which motivational problems are related to health outcomes. The topic of the study is justified by the need to develop more innovative and engaging interventions for patients with schizophrenia . However, before making recommendations for patients and especially decision makers, we should have an understanding of what the clinical outcomes of gaming interventions on health and well-being are. If ICT (Information and communications technology)-based interventions are to be taken as part of routine care without a clear understanding of how they work, there is the danger of using new interventions that may be non-effective and costly. If successful, the findings of this study will have the potential for remarkable societal and clinical impact, particularly in relation to changing treatment approaches and cultures in psychiatric services. In the long run, the findings could inform whether the gaming intervention could potentially reduce service costs and if our interventions are effective. It might also be possible that the study could result in a paradigm shift for non-medicinal treatment for schizophrenia, as this type of treatment is low-cost, easy to use and effective for cognitive symptoms of schizophrenia.
Patient recruitment is ongoing.
Availability of data and materials
Abnormal Movement Involuntary Scale
Barnes Akathisia Rating Scale
Brief Social Phobia Scale
Clinical Assessment Interview for Negative Symptoms
Calgary Depression Scale for Schizophrenia
Diagnostic and Statistical Manual of Mental Disorders, fifth edition
FMRIB S Improved Linear Model
Functional magnetic resonance imaging
General Self-Efficacy Scale
Information and communications technology
Last observation carried forward
Multivariate analysis of variance
Mini Mental State Examination
Magnetic Resonance Imaging
US National of Institute of Mental Health, the Measurement and Treatment Research to Improve Cognition in Schizophrenia
Randomised controlled trial
Regions of interest
Simpson-Angus Rating Scale
Statistical Package for the Social Sciences
Temporal Experience of Pleasure Scale
Trail Making Test
Wechsler Memory Scale
World Health Organization: Schizophrenia. http://www.who.int/mental_health/management/schizophrenia/en/ (2017). Accessed 14 Oct 2020.
Knapp M, Mangalore R, Simon J. The global costs of schizophrenia. Schizophr Bull. 2004;30(2):279–93. https://doi.org/10.1093/oxfordjournals.schbul.a007078.
Heinrichs RW, Zakzanis KK. Neurocognitive deficit in schizophrenia: a quantitative review of the evidence. Neuropsychology. 1998;12(3):426–45. https://doi.org/10.1037//0894-418.104.22.1686.
Green MF, Kern RS, Braff DL, Mintz J. Neurocognitive deficits and functional outcome in schizophrenia: are we measuring the "right stuff"? Schizophr Bull. 2000;26(1):119–36. https://doi.org/10.1093/oxfordjournals.schbul.a033430.
Elvevåg B, Goldberg TE. Cognitive impairment in schizophrenia is the core of the disorder. Crit Rev Neurobiol. 2000;14(1):1–21. https://doi.org/10.1615/CritRevNeurobiol.v14.i1.10.
Keefe RS, Vinogradov S, Medalia A, Buckley PF, Caroff SN, D'Souza DC, Harvey PD, Graham KA, Hamer RM, Marder SM, Miller DD, Olson SJ, Patel JK, Velligan D, Walker TM, Haim AJ, Stroup TS. Feasibility and pilot efficacy results from the multisite cognitive remediation in the schizophrenia trials network (CRSTN) randomized controlled trial. J Clin Psychiatry. 2012;73(7):1016–22. https://doi.org/10.4088/JCP.11m07100.
Brekke JS, Hoe M, Green MF. Neurocognitive change, functional change and service intensity during community-based psychosocial rehabilitation for schizophrenia. Psychol Med. 2009;39(10):1637–47. https://doi.org/10.1017/S003329170900539X.
Manoach DS. Prefrontal cortex dysfunction during working memory performance in schizophrenia: reconciling discrepant findings. Schizophr Res. 2003;60(2–3):285–98. https://doi.org/10.1016/s0920-9964(02)00294-3.
Wykes T, Reeder C, Landau S, Everitt B, Knapp M, Patel A, Romeo R. Cognitive remediation therapy in schizophrenia: randomised controlled trial. Br J Psychiatry. 2007;190:421–7. https://doi.org/10.1192/bjp.bp.106.026575.
Fisher M, Holland C, Merzenich MM, Vinogradov S. Using neuroplasticity-based auditory training to improve verbal memory in schizophrenia. Am J Psychiatry. 2009;166(7):805–11. https://doi.org/10.1176/appi.ajp.2009.08050757.
Statista Research Department: Genre breakdown of video game sales in the United States in 2015. https://www.statista.com/statistics/189660/breakdown-of-us-computer-game-sales-2009-by-genre/ (2016). Accessed 14 Oct 2020.
Horne-Moyer HL, Moyer BH, Messer DC, Messer ES. The use of electronic games in therapy: a review with clinical implications. Curr Psychiatry Rep. 2014;16(12):520. https://doi.org/10.1007/s11920-014-0520-6.
Genevsky A, Garrett CT, Alexander PP, Vinogradov S. Cognitive training in schizophrenia: a neuroscience-based approach. Dialogues Clin Neurosci. 2010;12(3):416–21.
Kühn S, Gleich T, Lorenz RC, Lindenberger U, Gallinat J. Playing super Mario induces structural brain plasticity: gray matter changes resulting from training with a commercial video game. Mol Psychiatry. 2014;19(2):265–71. https://doi.org/10.1038/mp.2013.120.
Honey GD, Bullmore ET, Sharma T. De-coupling of cognitive performance and cerebral functional response during working memory in schizophrenia. Schizophr Res. 2002;53(1–2):45–56. https://doi.org/10.1016/s0920-9964(01)00154-2.
Manoach DS, Gollub RL, Benson ES, Searl MM, Goff DC, Halpern E, Saper CB, Rauch SL. Schizophrenic subjects show aberrant fMRI activation of dorsolateral prefrontal cortex and basal ganglia during working memory performance. Biol Psychiatry. 2000;48(2):99–109. https://doi.org/10.1016/s0006-3223(00)00227-4.
Bavelier D, Achtman RL, Mani M, Föcker J. Neural bases of selective attention in action video game players. Vis Res. 2012;61:132–43. https://doi.org/10.1016/j.visres.2011.08.007.
Shams TA, Foussias G, Zawadzki JA, Marshe VS, Siddiqui I, Müller DJ, Wong AH. The effects of video games on cognition and brain structure: potential implications for neuropsychiatric disorders. Curr Psychiatry Rep. 2015;17(9):71. https://doi.org/10.1007/s11920-015-0609-6.
Sherry JL, Lucas K, Greenberg BS, Lachlan K. Video game uses and gratifications as predictors of use and game preference. In: Vorderer P, Bryant J, editors. Playing video games: motives, responses, and consequences. London: Routledge Taylor & Francis group; 2006. p. 248–62.
Nahum M, Fisher M, Loewy R, Poelke G, Ventura J, Nuechterlein KH, Hooker CI, Green MF, Merzenich M, Vinogradov S. A novel, online social cognitive training program for young adults with schizophrenia: a pilot study. Schizophr Res Cogn. 2014;1(1):e11–9. https://doi.org/10.1016/j.scog.2014.01.003.
Przybylski A, Rigby CS, Ryan RM. A motivational model of video game engagement. Rev Gen Psychol. 2010;14(2):154–66.
Kimhy D, Khan S, Ayanrouh L, Chang RW, Hansen MC, Lister A, Ballon JS, Vakhrusheva J, Armstrong HF, Bartels MN, Sloan RP. Use of active-play video games to enhance aerobic fitness in schizophrenia: feasibility, safety, and adherence. Psychiatr Serv. 2016;67(2):240–3. https://doi.org/10.1176/appi.ps.201400523.
Green CS, Seitz AR. The impacts of video games on cognition (and how the government can guide the industry). Policy Insights Behav Brain Sci. 2015;2(1):101–10.
Granic I, Lobel A, Engels RC. The benefits of playing video games. Am Psychol. 2014;69(1):66–78. https://doi.org/10.1037/a0034857.
Sahakian B. ‘Brain training’ app may improve memory and daily functioning in schizophrenia. University of Cambridge. 2015. https://www.cam.ac.uk/research/news/brain-training-app-may-improve-memory-and-daily-functioning-in-schizophrenia. Accessed 14 Oct 2020.
Ho KK, Lui SS, Hung KS, et al. Theory of mind impairments in patients with first-episode schizophrenia and their unaffected siblings. Schizophr Res. 2015;166(1–3):1–8. https://doi.org/10.1016/j.schres.2015.05.033.
Palva S, Kulashekhar S, Hämäläinen M, Palva JM. Localization of cortical phase and amplitude dynamics during visual working memory encoding and retention. J Neurosci. 2011;31(13):5013–25. https://doi.org/10.1523/JNEUROSCI.5592-10.2011.
Bowman ND, Tamborini R. Task demand and mood repair: the intervention potential of computer games. New Media Soc. 2012;14:1339–157.
Bouchard S, Bernier F, Boivin E, Morin B, Robillard G. Using biofeedback while immersed in a stressful videogame increases the effectiveness of stress management skills in soldiers. PLoS One. 2012;7(4):e36169. https://doi.org/10.1371/journal.pone.0036169.
Shah A, Kraemer KR, Won CR, Black S, Hasenbein W. Developing digital intervention games for mental disorders: a review. Games Health J. 2018;7(4):213–24. https://doi.org/10.1089/g4h.2017.0150.
Spencer BWJ, Shields G, Gergel T, Hotopf M, Owen GS. Diversity or disarray? A systematic review of decision-making capacity for treatment and research in schizophrenia and other non-affective psychoses. Psychol Med. 2017;47(11):1906–22. https://doi.org/10.1017/S0033291717000502.
Baron G, Perrodeau E, Boutron I, Ravaud P. Reporting of analyses from randomized controlled trials with multiple arms: a systematic review. BMC Med. 2013;11:84. https://doi.org/10.1186/1741-7015-11-84.
McGurk SR, Twamley EW, Sitzer DI, McHugo GJ, Mueser KT. A meta-analysis of cognitive remediation in schizophrenia. Am J Psychiatry. 2007;164(12):1791–802. https://doi.org/10.1176/appi.ajp.2007.07060906.
Jin H, Folsom DP, Lindamer L, Bailey A, Hawthorne W, Garcia P, Jeste DV. Patterns of public mental health service use by age in patients with schizophrenia. Am J Geriatr Psychiatry. 2003;11(5):525–33.
Kannisto KA, Korhonen J, Adams CE, Koivunen MH, Vahlberg T, Välimäki MA. Factors Associated With Dropout During Recruitment and Follow-Up Periods of a mHealth-Based Randomized Controlled Trial for Mobile. Net to Encourage Treatment Adherence for People With Serious Mental Health Problems. J Med Internet Res. 2017;19(2):e46. https://doi.org/10.2196/jmir.6417.
Wong JG, Cheung EP, Chen EY. Decision-making capacity of inpatients with schizophrenia in Hong Kong. J Nerv Ment Dis. 2005;193(5):316–22. https://doi.org/10.1097/01.nmd.0000161685.54077.e4.
Wu BJ, Liao HY, Chen HK, Lan TH. Psychopathology, psychopharmacological properties, decision-making capacity to consent to clinical research and the willingness to participate among long-term hospitalized patients with schizophrenia. Psychiatry Res. 2016;237:323–30. https://doi.org/10.1016/j.psychres.2016.01.020.
Wang P, Liu HH, Zhu XT, Meng T, Li HJ, Zuo XN. Action video game training for healthy adults: a meta-analytic study. Front Psychol. 2016;7:907. https://doi.org/10.3389/fpsyg.2016.00907.
Välimäki M, Korkeila J, Kauppi K, Kaakinen JK, Holm S, Vahlo J, Tenovuo O, Hämäläinen H, Sarajuuri J, Rantanen P, Orenius T, Koponen A. Digital gaming for improving the functioning of people with traumatic brain injury: protocol of a feasibility study. JMIR Res Protoc. 2016;5(1):e6. https://doi.org/10.2196/resprot.4841.
Bao S, Chan VT, Merzenich MM. Cortical remodelling induced by activity of ventral tegmental dopamine neurons. Nature. 2001;412(6842):79–83. https://doi.org/10.1038/35083586.
Green CS, Bavelier D. Learning, attentional control, and action video games. Curr Biol. 2012;22(6):R197–206. https://doi.org/10.1016/j.cub.2012.02.012.
Marder SR. The NIMH-MATRICS project for developing cognition-enhancing agents for schizophrenia. Dialogues Clin Neurosci. 2006;8(1):109–13.
Cao J, Yang J, Zhou Y, Chu F, Zhao X, Wang W, Wang Y, Peng T. The effect of interaction anxiousness scale and brief social phobia scale for screening social anxiety disorder in college students: a study on discriminative validity. J Ment Health. 2016;25(6):500–5. https://doi.org/10.3109/09638237.2015.1124391.
Chan RC, Shi YF, Lai MK, Wang YN, Wang Y, Kring AM. The temporal experience of pleasure scale (TEPS): exploration and confirmation of factor structure in a healthy Chinese sample. PLoS One. 2012;7(4):e35352. https://doi.org/10.1371/journal.pone.0035352.
Zhang JX, Schwarzer R. Measuring optimistic self-beliefs: a Chinese adaptation of the general self-efficacy scale. Psychologia. 1995;38(3):174–81.
Chan RC, Shi C, Lui SS, Ho KK, Hung KS, Lam JW, Wang Y, Cheung EF, Yu X. Validation of the Chinese version of the clinical assessment interview for negative symptoms (CAINS): a preliminary report. Front Psychol. 2015;6:7. https://doi.org/10.3389/fpsyg.2015.00007.
Xiao W, Liu H, Zhang H, Liu Q, Fu P, Chen J, Wang X, Wang G, Li L, Shu L. Reliability and validity of the Chinese version of the Calgary depression scale for schizophrenia. Aust N Z J Psychiatry. 2009;43(6):548–53. https://doi.org/10.1080/00048670902873672.
Simpson GM, Angus JW. A rating scale for extrapyramidal side effects. Acta Psychiatr Scand Suppl. 1970;212:11–9. https://doi.org/10.1111/j.1600-0447.1970.tb02066.x.
US San Diego Health: EEG Test to Help Understand and Treat Schizophrenia. https://health.ucsd.edu/news/releases/pages/2014-10-29-eeg-to-understand-treat-schizophrenia.aspx (2014). Accessed 14 Oct 2020.
Wilson SJ, Sayette MA, Fiez JA. Prefrontal responses to drug cues: a neurocognitive analysis. Nat Neurosci. 2004;7(3):211–4. https://doi.org/10.1038/nn1200.
Ko JH, Ptito A, Monchi O, Cho SS, Van Eimeren T, Pellecchia G, Ballanger B, Rusjan P, Houle S, Strafella AP. Increased dopamine release in the right anterior cingulate cortex during the performance of a sorting task: a [11C] FLB 457 PET study. Neuroimage. 2009;46(2):516–21. https://doi.org/10.1016/j.neuroimage.2009.02.031.
Honkanen R, Rouhinen S, Wang SH, Palva JM, Palva S. Gamma oscillations underlie the maintenance of feature-specific information and the contents of visual working memory. Cereb Cortex. 2015;25(10):3788–801. https://doi.org/10.1093/cercor/bhu263.
Sheffield JM, Barch DM. Cognition and resting-state functional connectivity in schizophrenia. Neurosci Biobehav Rev. 2016;61:108–20. https://doi.org/10.1016/j.neubiorev.2015.12.007.
Nikulin VV, Jönsson EG, Brismar T. Attenuation of long-range temporal correlations in the amplitude dynamics of alpha and beta neuronal oscillations in patients with schizophrenia. Neuroimage. 2012;61(1):162–9. https://doi.org/10.1016/j.neuroimage.2012.03.008.
Coombs CH, Dawes RM, Tversky A. The theory of signal detectability. In: Mathematical psychology; an elementary introduction. Prentice-Hall: Englewood Cliffs; 1970. p. 165–201.
The World Medical Association, Inc: The WMA Declaration of Helsinki. Ethical Principles for Medical Research Involving Human Subjects. https://www.wma.net/policies-post/wma-declaration-of-helsinki-ethical-principles-for-medical-research-involving-human-subjects/ (2020). Accessed 14 Oct 2020.
We would like to thank the Research Grant Council in Hong Kong (General Research Fund) for making this study possible (project 15600418). We also want to thank hospitals and mental health associations and organisations in Hong Kong for their assistance in helping in the recruitment process of the study. We would especially like to thank all the participants and the staff who have so far offered their kind assistance to our study. Further, we appreciate our research assistants and research associates Wong Lai Fong, Fong Kwan Hin and Law Kai Chun Johnson, a number of bachelor’s and master’s students for ongoing patient recruitment and supporting gaming sessions, and doctoral candidate MNSc Jaakko Varpula for his valuable contribution in patient allocation.
This study has been funded by the Research Grant Council in Hong Kong (General Research Fund, project 15600418), the Hong Kong Polytechnic University, Internal Funding (Code YBYB, ID P0009721) and the University of Turku in Finland (26003424). The publication costs of this manuscript will be covered from the 26003424 grant. The funding bodies will not gain or lose financially from the publication of this manuscript now or in the future. The views described in this manuscript are based on authors’ opinions. The funding bodies have not participated in the design of the study or writing the current manuscript and will not participate in collection, analysis, or interpretations of the data.
Ethics approval and consent to participate
This trial has been approved by the Hong Kong Polytechnic University (Ethical Review for Teaching/Research Involving Human Subjects, Faculty Research Committee (HSEARS20161228002–2) and the Clinical Research Ethics Approval (Clinical Research Ethics Sub-Committee (CRES, CRESC201808–01). Ethical assessment for MRI experiments has been conducted (16th April 2019, MR Imaging Unit, University of Hong Kong, the Scientific Committee). The University’s Master Clinical Trial Insurance for the study participants has been approved (ASI19903139A). Ethical assessment for the study has been conducted at two study settings (KCC/KEC-2019-0063; HKECREC–2019-041). Licences for the data collection and games have been purchased, and MRI services have been obtained (tender process).
Participants will be informed that they have the possibility to withdraw their consent, and they will be made aware that data collected before withdrawal may be used for research purposes. Participants will also be informed about confidentiality of the data collected and that refusal or withdrawal from the study will not affect their status in service organisations.
Consent for publication
Not applicable. Details, images, photos, videos or game results relating to individual participants or study organisations will not be published.
Authors declare that they have no competing interests.
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated in a credit line to the data.
About this article
Cite this article
Välimäki, M., Yang, M., Lam, Y.T.J. et al. The impact of video gaming on cognitive functioning of people with schizophrenia (GAME-S): study protocol of a randomised controlled trial. BMC Psychiatry 21, 46 (2021). https://doi.org/10.1186/s12888-020-03031-y
- Randomised controlled trial