- Study protocol
- Open Access
Cognitive Enhancement Therapy vs social skills training in schizophrenia: a cluster randomized comparative effectiveness evaluation
BMC Psychiatry volume 22, Article number: 583 (2022)
Schizophrenia and related disorders are highly disabling and create substantial burdens for families, communities, and health care systems. Although pharmacological treatments can often lessen the psychotic symptoms that are a hallmark of schizophrenia, they do not lessen the social and cognitive deficits that create the greatest impediments to community engagement and functional recovery. This study builds on prior research on psychosocial rehabilitation by comparing the effectiveness of two treatments demonstrated as efficacious in improving social and community functioning, Cognitive Enhancement Therapy (CET) and a version of Social Skills Training (HOPES/SST).
The study uses a randomized cluster design in which a pair of clinicians at community- and hospital-based mental service centers deliver either CET or HOPES to at least one group of 6-8 eligible clients for 12 months. Clinicians are trained and then supervised weekly, with ongoing process measurement of treatment fidelity, attendance, satisfaction, and retention, and use of other services. Measures administered at baseline and at 6 and 12 months while in treatment, and then at 18 and 24 months after treatment include social adjustment, quality of life, social skills, positive and negative symptoms, and neuro- and social cognition. We hypothesize that CET will be associated with greater improvements than SST in both the primary outcome of community functioning and the secondary outcomes of neuro- and social cognition and social skills. Secondarily, we hypothesize that more cognitive impairment at baseline and younger age will predict more benefit from CET compared to HOPES.
Resource shortages endemic in mental health services and exacerbated by the pandemic highlight the importance of identifying the most effective approach to improving social and community functioning. We aim to improve understanding of the impact of two efficacious psychosocial treatments and to improve clinicians’ ability to refer to both treatments the individuals who are most likely to benefit from them. We expect the result to be programmatic improvements that improve the magnitude and durability of gains in community functioning.
ClinicalTrial.gov NCT04321759, registered March 25, 2020.
Schizophrenia (SZ) and related disorders occur in between .4-1% of the population across the world—about 2 million individuals in the United States, with 100,000 new cases diagnosed each year. They are highly disabling, create substantial burdens for their families and communities [1,2,3], and cost the US more than $100 billion annually [4, 5]. Although psychotic symptoms are the most florid manifestations of SZ and related psychotic disorders , associated impairments in social and cognitive functioning are the most disabling features of this diagnosis [7, 8]. While pharmacological treatments can lessen positive symptoms for many individuals [9,10,11], they do not improve social or cognitive functioning [7, 12,13,14,15,16,17,18]. As many as two-thirds of people with SZ have social skills impairments [19, 20], over 90% are impaired in one cognitive domain, and 75% are impaired in at least two domains [21,22,23,24]. Impaired cognitive and social cognitive functioning are also rate-limiting factors in response to psychosocial treatment and constrain prospects for community engagement and functional recovery [19, 20, 23, 25,26,27,28,29,30].
An array of psychosocial programs to improve social skills and cognitive functioning have been developed in recent decades. Social skills training (SST) is a common approach to improving social functioning, most often provided in a group format, aimed at teaching more effective interpersonal skills. Skills are taught based on the systematic application of social learning theory, and include the following steps: breaking complex skills down into smaller steps or components, demonstrating (modeling) the skill for participants in a role play, engaging each participant in practicing the skill in role plays with each one followed by positive feedback and suggestions for improvement, collaboratively developing home assignments for participants to practice the skills on their own, and in vivo community trips to practice the skill. Considerable evidence supports the efficacy of SST in improving social and community functioning as well as reducing negative symptoms [23, 24].
Cognitive remediation (CR) programs take a different approach to improving functioning, with a focus on enhancing cognition through practice of computer cognitive exercises, coaching in strategies designed to improve performance on the cognitive exercises, and teaching self-management (or compensatory) skills to improve cognition in day-to-day situations. Although most cognitive remediation programs have produced moderate gains in overall cognitive performance [31, 32], by themselves they have had limited lasting, positive effects on more ecologically valid abilities that shape community functioning . It is when cognitive remediation programs also include strategic training and more comprehensive integration with other psychosocial interventions that they have more effects on functioning [32, 34,35,36,37]. Meta-analyses suggest that such comprehensive programs have stronger effects on community functioning than either treatment administered separately [37, 38]. However, these more comprehensive treatments have not been compared directly to traditional SST programs that do not include a CR component.
The Patient-Centered Outcomes Research Institute (PCORI) funded Project SUCCESS, Schizophrenia: Understanding and Comparing Cognitive Enhancement and Social Skills training to fill this knowledge gap and thus inform the treatment decisions of clinicians, individuals with SZ, and other stakeholders. Meta-analyses and independent randomized controlled trials (RCTs) of comprehensive CR treatment, including Cognitive Enhancement Therapy (the one we will provide) indicate moderate to large effect sizes (Cohen’s d between .4- 1.0, averaging .8) on cognition, social cognition, and community functioning in both early course and multi-episode patients with SZ (see Table 1) [21, 22, 32, 37, 39, 43]. By comparison, meta-analyses of RCTs testing SST treatments show small to moderate effect sizes (about .2-.5) on social skills and community functioning [23, 24, 44]. Improvements in community functioning are sustained after both treatments for at least 1-2 years [41, 42, 45,46,47,48].
Aims and hypotheses
Our primary study aim is to test the hypothesis that CET will be associated with greater improvements than SST in both the primary outcome—community functioning—and the secondary outcomes of neuro- and social cognition and social skills (see Fig. 1).
Our secondary aim is to explore the differential effectiveness of the two interventions by baseline cognitive functioning and age. Because integrated treatment programs like CET were designed to reduce the moderating effects of impaired cognition on response to social skills training [20, 23], we predicted that poorer baseline cognitive functioning will be associated with greater treatment response for patients who receive CET as compared to those who receive SST (which doesn’t seek to increase cognitive functioning) [32, 37, 49,50,51,52,53,54,55,56]. Given that the younger brain retains neuroplastic reserves that may make it uniquely capable of benefiting from cognitive remediation [57,58,59], we also predicted that younger participants would benefit more from CET than older ones, while the same interaction with age would not occur for SST recipients [60,61,62]. Variation across our treatment delivery sites (approximately 24) in treatment dose may also influence the effectiveness of both interventions, with individuals attending more treatment sessions—even when it is due to challenges at the site rather than orientations of the participants—potentially benefiting more from the treatment.
Project SUCCESS is a comparative effectiveness trial of Cognitive Enhancement Therapy (CET), a combined treatment offering both CR and features designed to improve social functioning, with Helping Ourselves Pursue and Experience Success (HOPES), a treatment that offers only SST. The study uses a cluster-randomized design to allocate participating treatment sites to provide either CET or HOPES in groups of 6-8 eligible participants for 12 months of treatment, with post-treatment assessments of functioning conducted up to 12 months post-treatment. Most sites will offer two successive treatment groups.
Study setting and sample
Treatment sites are approximately 24 community mental health centers and hospital outpatient clinics in Massachusetts and eight other states that offer programs for individuals with schizophrenia-spectrum disorders (final list on project website, project-success.net). These sites have been paired by geographic proximity, organizational type, and by race and ethnic proportions in the local community and then randomly assigned to the two study arms, so that the number of participants per arm is approximately equal and the distribution of sites is comparable geographically and unbiased with respect to participant and site characteristics.
Cluster randomization was used to reduce challenges created by the group-based nature of both treatments. We also had several other reasons to reject person-level randomization: increased resources required by participating sites to train two sets of clinicians to implement CET and SST, potential cross-contamination of the two interventions offered within the same setting, and slower accrual of groups in each treatment due to randomization to two different programs.
Cognitive Enhancement Therapy is a comprehensive cognitive remediation program designed to maximize gains in social functioning by integrating computer-based training to enhance neurocognition with group-based exercises to improve social cognition [22, 43, 45, 57]. CET facilitates the development of adult social-cognitive milestones (e.g., perspective-taking, social context appraisal) by shifting thinking from reliance on effortful, serial processing to a “gistful” and spontaneous abstraction of social themes. All procedures are described in a step-by-step manual followed by CET trainers . CET’s group-based exercises are delivered for 1.5 hours each week in a group of 6-8 participants led by two clinicians. During each of three modules (basic concepts, social cognition, CET applications), the group focuses on acquisition of adult social milestones in perspective-taking, social context appraisal, and other aspects of social functioning, with psychoeducational lectures on topics ranging from the neurobiology of schizophrenia to the concept of self-defeating thinking, homework assignments, and in-group exercises. CET’s computer-based training is supervised by a clinician who meets weekly for one hour with a pair of participants who make progress on an attention, memory, or problem-solving exercise in each session. The paired participants take turns practicing exercises on a computer, providing encouragement and advice to each other, as the clinician provides strategy coaching and support.
Social Skills Training: HOPES is a manualized SST rehabilitation program that is delivered over a 12-month period, with weekly group skills classes and monthly community practice trips and engagement of natural supports to facilitate skill use in real-world situations. HOPES classes are conducted using the principles of SST (modeling, role playing, positive and corrective feedback, in vivo assignments), some of it adapted from the Bellack et al.  curriculum. The curriculum is organized into eight topic areas or “modules”: Living Well in the Community, Communicating Effectively, Making and Keeping Friends, Making the Most of Leisure Time, Using Medications Effectively, Healthy Living, Intimacy and Dating, and Making the Most of a Doctor’s Visit. Each module includes 6–8 component skills, with one skill covered each week. HOPES is standardized in a manual and a participant workbook with handouts for each skill .
Two modifications have been made in the interventions and accompanying manuals to adapt CET and HOPES for the unique needs of this multi-site comparative effectiveness study.
Although the early studies of CET provided treatment for 24 or 18 months, CET has been adapted to be delivered as a 12-month intervention to enhance feasibility of delivery; it has been shown to be efficacious even when limited to 9 months (effect size of .74 for role functioning after 9 months of CET) . Most SST programs are 6-9 months in length , but HOPES was designed as a 12-month intervention. We offer both CET and SST (HOPES) for 12 months to enhance the likelihood that participants will complete the entire program while allowing recruitment of new cohorts, and to standardize the length of treatment across groups. Evidence supports the effectiveness of both interventions by 12 months of treatment, with durability of effects at one-year follow-up for both CET [45, 46] and SST —including for HOPES .
For CET, participants will be engaged in both the neurocognitive training and group-based social cognition training concurrently, upon enrollment. Although the CR component of CET has typically been initiated 3 months prior to the start of the group therapy sessions, we have found that sequential timing of CR exercises and group interventions is both impractical and unnecessary. Additionally, increasing evidence suggests that near-term transfer (immediate improvement in a test situation) of skills learned in cognitive remediation is likely to occur simultaneously with generalization to daily tasks and improve long-term transfer [58, 64, 65]. This has proven to be successful both in our own research programs and related implementation efforts . This strategy also assures similar treatment lengths, without limiting our ability to answer the main study questions. We have developed a supplement to the CET manual describing adaptations for the 12-month intervention.
Replacement of dropouts in first month (partial rolling admission) for CET groups
HOPES typically uses an open enrollment format where participants can join the group at any point in the curriculum, which is taught on a revolving basis for as long as needed with the last module followed by the first module, and then remain in the program for a full year. An open enrollment format can minimize the time participants have to wait to join a new group, and is feasible in HOPES because the skills taught in the different modules are not cumulative, and thus can be learned in any order. Fully open enrollment is not feasible in CET, since some of the later modules (such as CET application to everyday life situations) require prior practice of the cognitive skills. The standard CET approach instead has been to allow replacement of dropouts in the first month of treatment, with some preliminary coaching of those who enter in weeks 2-4. We therefore have also imposed a first-month limit for enrollment to standardize this aspect of both treatments.
Adaptations for the COVID-19 pandemic
The COVID-19 pandemic required an abrupt and massive shift of community mental health services to telehealth modes of service delivery. Prior research indicated the potential for remote delivery of psychosocial interventions, but mostly in relation to individual therapy rather than group-based treatments [67,68,69,70,71,72,73,74,75,76]. We adapted our intervention delivery and research procedures to maintain equivalent service experiences and assessment protocols even when services and assessments must be delivered remotely. Monthly group community trips in HOPES were changed to virtual monthly meetings of each participant with an identified support person. Virtual group (and individual) sessions in both interventions and computer sessions in CET were offered on tablets or laptops. We have tested these telehealth adaptations for acceptability and feasibility in a 3-month pilot study at four service sites, two of which were assigned to CET and two to HOPES, with one in each condition delivering the intervention via telehealth and the other delivering the intervention in-person, as allowed by site policy and with adherence to site-specific public health requirements (e.g., social distancing, masking). Data on treatment enrollment, retention, participant satisfaction, and treatment fidelity indicate the feasibility and acceptability of the telehealth approach.
Eligibility and randomization
Potential participants are eligible if the referring clinician reports a DSM-5 diagnosis of schizophrenia, schizoaffective disorder, or schizophreniform disorder that is confirmed by a Mini International Neuropsychiatric Interview (MINI) for Psychotic Disorders Studies  administered via video conference by the project research coordinator. Participants must also be 18-65 years of age and have an estimated premorbid IQ > 70 (determined with the Wechsler Test of Adult Reading) and have no known organic neurological disease or intellectual disability (DSM-5). Prior to assessment, on-site research assistants review the project with potential participants and invite their participation if they sign a site- and treatment-specific Informed Consent Form.
Assessments of symptom severity, social cognition, and community functioning will be conducted via video conference by trained research assistants at the coordinating site, Beth Israel Deaconess Medical Center (BIDMC), who are blind to treatment condition. Treatment fidelity will be assessed by coding at least 10 recorded group therapy sessions each year for each treatment site, with 6 sessions in the first 3 months used for feedback and training and 4 or more at random intervals after that. Site research assistants will record participant attendance and engagement during group sessions.
Sample size and power
We expect that each treatment site will enroll on average about 16 participants, yielding a total of 378 participants (see Fig. 2). For all power analyses conducted, based on previous experience, we assumed 20% attrition by 12 months. We further assumed four assessment points (baseline, 6-, 12-, 18- and, for 75% of the sample, 24-month), an alpha level of 0.05, a two-tailed test, with a cross-time correlation of 0.5. We will minimize treatment dropout by reaching out to participants whenever they miss treatment sessions (through the site RA) and by providing makeup sessions (through the site clinicians) that allow those who have missed up to eight sessions to reengage in the treatment without missing content. We will minimize attrition from the research procedures by identifying multiple contact persons when enrolling each participant and by using multiple forms of outreach when a participant is not available for scheduling an assessment or misses a scheduled assessment. Based on our prior experience conducting clinical trials evaluating CET and HOPES (reviewed above), combined with our methods for minimizing attrition, we also believe it is realistic to estimate that the attrition rate for the 1-year durability follow-up will not exceed 20%. However, to be conservative we have estimated our power to detect significant differences between the groups based on a 25% attrition rate.
To estimate effect sizes, we compared effect sizes of CET and SST/HOPES from our own data and the SST literature using randomized controlled designs in schizophrenia, bearing in mind the diversity of design and treatment length issues (see Table 1). Given that CET is a composite intervention that includes both cognition, social cognition and social skills interventions, the effect sizes are not surprisingly higher (from .7 after 1 year to > 1.0 after 2 years); SST effect sizes are mostly under .4 in meta-analyses of SST of lengths from 3 weeks to 2 years (and in the HOPES trial did not consistently lessen for one or even 2 years after active treatment). Based on the available studies outlined in the table, we estimated effect sizes of 0.34 (for ICC due to clustering at site level = .01) and 0.37 (for ICC = .02), which justifies our proposed full sample size. With 4 assessments for the full and 5 assessments for three-fourths of the sample, our durability assessment is only somewhat less powerful, with medium effect sizes (from 0.41 to 0.49).
Accounting for clustering and correlated data. There are multiple forms of clustering due to participants being nested in sites and repeated observations being nested in participants. Therapeutic groups within a site can also be considered as another level of clustering, but because there are no more than 3 therapeutic groups at any site and they are expected to be led by the same clinicians at the site, we expect little between-therapeutic group heterogeneity within sites and so this level of clustering is unlikely to affect estimates. Nonetheless, we conducted two sets of power analyses, with and without considering therapeutic groups as another level of clustering.
We first only consider clustering effects due to sites and repeated measurement and handle clustering effects as follows: for the correlated nature of data due to repeated measurement, we assumed within-person cross-time correlation to be 0.50 and incorporated this correlation into both analytic modeling procedures (mixed-effects model with random effects) and power calculations. To account for site level clustering effects, we adjusted for sample sizes of 378 individuals by assuming a range of ICC values: 0.01, 0.02 and 0.05. We note the literature reports that in public health and medicine, ICCs in group- or cluster-randomized trials are often small, usually ranging from 0.01 - 0.05 (https://researchmethodsresources.nih.gov/FAQs.aspx). Some statisticians have stated that values of ICCs are usually between 0.01 and 0.02 in cluster randomized clinical studies [78, 79]. Given the multi-determined nature of our health-related outcomes, we believe that the effects due to clustering by site in our study will be small, and thus we adjusted our sample size and power calculations based on two scenarios: assuming an ICC of 0.01 and an ICC of 0.02. After adjustment, our original sample size of 378 reduced to an effective sample size (ESS) of 318 (for an ICC of 0.01) and 274 (for an ICC of 0.02), respectively. Second, for the 3-level clustering effects adjustment, we took account of the 51 therapeutic groups (approximating) as another level of clustering and assumed an ICC due to therapeutic group of 0.01, and adjusted sample sizes by both site and therapeutic group down to 302 and 252, respectively.
Power for aim 1
Power analyses for Aim 1 were conducted using Diggle et al.  and Lu et al.’s  approaches implemented in the LongPower package  in R statistical software (version 3.52). Power calculations were conducted to determine minimum detectable differences between arms (CET vs. SST) in changes over time on the primary outcomes. The outcomes for primary hypotheses in Aim 1 are continuous. To detect significantly more improvement for participants in CET than SST with 80% power in the 2-level analysis, the adjusted sample size of N of 318 (with 20% attrition by 12 months) will detect a minimum effect size of 0.34, and the adjusted N of 274 will detect a minimum effect size of 0.37, respectively. Both detectable effect sizes are between small and medium according to Cohen’s d metric for effect size (0.2 = small, 0.5 = medium and 0.8 = large). For the 3-level analysis that includes adjustment for clustering at the therapeutic group level and the adjusted sample sizes of 302 and 252 (everything else is the same), the minimum effect sizes that can be detected are 0.35 and 0.38, respectively.
The full sample of 378 at baseline (thus ICC adjusted sample sizes of 318 and 274) will have 4 assessments (baseline, 6-, 12 and 18 months), but we will also collect data for 284 participants (75% of the original sample of 378) for one extra time point (baseline, 6-, 12-, 18 and 24 months) (thus an ICC adjusted effective sample size for N = 284 participants becomes 250—for an ICC of 0.01—and 222—for an ICC of 0.02—respectively, and so the detectable effect sizes with N = 250 and 222 with 20% attrition are 0.38 and 0.40, respectively. For the analysis with 3-level clustering (both site and therapeutic group, as well as participants over time), the adjusted sample sizes become N = 240 and 208 and the detectable effect sizes with 20% attrition are 0.39 and 0.42, respectively.
All interview ratings and cognitive testing will be conducted by trained raters and administered over the web by a project research assistant at the project’s central site, with an on-site RA managing arrangements so that the interviewer is blind to treatment assignment. Participants will be assessed at Baseline, 6 months into the program, 12 months (at completion of the program), 18 months (6 months after program completion), as well as 24 months (12 months after program completion for at least 75% of participants—those whose groups have completed in sufficient time). All data are recorded at the time of assessment in BIDMC’s secure and HIPAA-compliant REDCap system and are subsequently checked for completeness by the project research coordinator.
The outcomes we have chosen include the primary and secondary outcomes identified in prior research on CET and SST/HOPES and represented in our hypothesized causal model (see Table 2). The measures are easy to administer at the community level, endorsed by the stakeholders, relatively brief (total < 2 hours for the overall assessment), and include engaging qualitative questioning as well as fixed choice questions. Since all assessment procedures were adapted for videoconferencing due to the pandemic, several measures from the Penn CNB that can be administered remotely replace corresponding measures in the NIH Toolbox that have not been adapted for remote delivery.
Primary outcome measures
Community functioning is measured with the Social Adjustment Scale (SAS-II), which has previously been validated as an interview-based measure (utilizing self-reports) of social and community functioning . The SAS-II quantifies adjustment in different roles (e.g., work, school), social and leisure activities, intimate relationships, and overall adjustment. Quality of life is assessed with the Quality of Life Scale . This 21-question interview-based, interviewer-rated measure assesses social functioning, role functioning, and motivation, as indicated by items such as sense of purpose, curiosity, and inner drive . To measure symptom severity, we will use a short, validated version of the Positive and Negative Syndrome Scale (PANSS-6, 6 items) [86, 87].
Secondary outcome measures
Neurocognition. We use a combination of measures from the NIH Toolbox and the Penn CNB that can be administered remotely and that match the 7 cognitive domains in the MATRICS Consensus Cognitive Battery. From the NIH Toolbox: Auditory Verbal Learning Test (verbal memory and learning), List Sorting Working Memory (working memory), Picture Sequence Memory (visual episodic memory and learning), Picture Vocabulary (language), and Oral Reading Recognition (language). From the Penn CNB: Continuous Performance Test (attention/vigilance), Mouse Practice Task (speed of processing), Digit Symbol (speed of processing), Conditional Exclusion Test (executive functioning/reasoning and problem solving). Social cognition. We use the Managing Emotions subtest of the Meyer-Salovey-Caruso Emotional Intelligence Test (MSCEIT) which is sensitive to treatment-related change in social cognition  and social functioning in SZ . Social skills will be assessed using the Social Skills Performance Assessment (SSPA), a role play test that is audio-recorded and then rated for different dimensions of social skills .
We will conduct a process evaluation using mixed methods to assess both intervention delivery and participant experience in each intervention (see Table 3). Treatment delivery will be monitored with three types of data. (1) Treatment fidelity: Group sessions are recorded at each site throughout the study and recordings are uploaded to a REDCap database. The recordings are monitored by supervising clinicians (study Co-Investigators) and used for feedback and training; at least six sessions are reviewed during the first 3 months and at least four sessions are reviewed thereafter through the remainder of the twelve-month treatment period. Supervising clinicians will rate at least ten sessions per year for fidelity with respect to each CET and HOPES group, using standard fidelity scales for both treatments that have been adapted to a common 4-point metric [40,41,42, 45]. (2) Treatment engagement: Treating clinicians and site RAs at each site create careful records of all individual and group sessions delivered, patient attendance, and session highlights. These records also include homework completion (CET, HOPES) community trips (HOPES), and makeup sessions (CET, HOPES). The clinicians and site RAs record this information and the site RA enters it in a REDCap database. (3) Treatment experience: The supervising clinicians of CET and SST summarize their impression of treatment delivery at each site at 6 and 12 months in qualitative comments; these summaries are captured in a REDCap database.
Participant experience is monitored with both quantitative and qualitative data. (4) Participant satisfaction: Participants are sent invitations to complete the Client Satisfaction Questionnaire (CSQ-8), a brief eight item self-report survey, via their mobile device, tablet, or computer once per month, during each month that they are enrolled in treatment . (5) Participant assessment: Two randomly selected participants are selected during month-2 of treatment delivery at each site for a qualitative interview using an open-ended interview designed to elicit information about the treatment experience. This process is repeated using a similar open-ended interview schedule for two randomly selected participants in the 11th month of their treatment. (6) Service utilization: Participants continue to participate in regular services throughout the intervention, excepting other SST-based or CR-based therapies. A modified version of the Service Use and Resource Form (SURF) pertaining to services received in the past month is completed by participants at baseline and follow-up assessments . (Treatment interruptions due to hospitalization or other events are recorded as part of the treatment engagement record).
We will monitor treatment delivery throughout the project and the project leadership will contact site PIs or other stakeholders as necessary to understand the bases for any indications of problems with treatment quality or quantity. Supplementary training will be provided to clinicians at sites as needed to ensure treatment fidelity and the project leadership group-- including engagement coordinators and staff—will review a summary report based on process measures each quarter to ensure identification of processes in need of improvement. If necessary, we will refine our recruitment and retention strategies based on our quarterly review of patient engagement.
Analysis for aim 1
All analyses will be conducted on the full sample regardless of exposure to treatment (intent-to-treat). Our primary hypothesis in Aim 1 is that CET will be associated with greater improvements than SST in both the primary outcome: community functioning (SAS, QLS), and the secondary outcomes of neuro- and social cognition (BACS and MSCEIT) and social skills (SSPA). To test this hypothesis, we will fit generalized linear mixed-effects models (GLMM) with identity link functions and a normal distribution specification for both the primary outcome and the secondary outcomes (because both outcomes are measured with interval-level scales). Because patients are nested within site, and repeated observations are nested within participants, we will fit a 3-level mixed-effects model: treatment arm (CET vs SST), time (baseline, 6-, 12- and 18-month) and arm-by-time interactions will be specified as fixed effects. The primary hypothesis of interest will be examined by testing the significance of the difference from zero in the rate of improvement over 18 months (arm-by-time interaction effect). To consider possible variance inflation due to clustering at the site and patient level, site will be treated as a random effect, and the intercept and slope for time at patient level will be specified as random effects. Randomization of sites to implement the CET or SST intervention will be stratified by geographic proximity, such that pairs of sites closest to each other will be randomized to different treatment conditions, and by organizational type and demographic characteristics of the surrounding area. This stratification plan will minimize the impact of any potentially confounding effects on the findings related to the bases of stratification. Because use of stratification may induce correlation between treatment arms, as a recommended cautionary measure, we will adjust for this stratification by including it as a random effect .
Because the interventions will be delivered in a group format sequentially for both arms, therapeutic groups could be considered as another level of clustering to be adjusted. We therefore will also test for effects with a 3-level model that includes the 51 treatment groups as another level of clustering. If variability (random effects) at the therapeutic group level is not significant, or if there is no improvement in model fit, we will remove therapeutic group as another level of clustering and retain the 2-level model. Whether a 2- or 3-level model is the final one chosen, we will specify heterogeneity of group effects (treatment arms) in the model to account for any potential variation between the treatment groups.
Analysis for Aim2 (HTE or moderation effect)
We will explore differential effectiveness of the two interventions by baseline cognitive functioning and age. Prior research suggests that these factors may influence effectiveness of CET and SST in different ways, but the evidence is not strong, and no studies have been conducted of both treatments. Therefore, we consider these hypotheses exploratory rather than confirmatory in nature. Prior to modeling, two moderators, baseline cognitive impairment and age will be recoded into two dichotomous variables using a median split, respectively.
In this aim, we test heterogeneity of treatment effects (HTE) or moderation effects by fitting the interaction models and conducting sub-group analyses. To test HTE, we will use the same analytic models—GLMM for Aim 1—but we will include 3-way interaction terms (arm*time*group) in the model. A significant 3-way interaction effect will indicate the existence of the heterogeneity of treatment effects between subgroups. Following this step, we will conduct simple effect analysis to estimate treatment effect differences (a difference in slopes by time effect between arms) within each subgroup.
Conceptually, the secondary outcomes of neurocognition and social cognition can be viewed as an intermediated outcome (mediator, M). The intervention (X) may affect the primary outcomes (Y) indirectly through a pathway of the mediator (M) as illustrated in Fig. 1. To test the mediated effect (or mechanism of change), we will conduct causal mediation analysis within the counterfactual framework [93, 94]. The moderation hypothesis in Aim 2 may also apply to the mediation effect. That is, the magnitude of the mediated treatment effect may also vary depending on age category (young vs. old) and pretreatment cognitive functioning (high vs. low), thus we will also conduct moderated causal mediation analysis by treating age and baseline cognitive function as moderators (MO).
Specifically, we will compute change scores (amount of improvement) from baseline to 18 months (for all subjects), and from baseline to 24 months (for 3/4 of subjects) for both primary outcomes and mediators for mediation analysis. For primary (or marginal) mediation analyses, we will estimate total effect of the treatment (TE), which can be decomposed into direct treatment effect (called natural direct effect, NDE) and indirect (or mediated) effect (called natural indirect effect, NIE); that is, TE = NDE + NIE with two-way decomposition. Bootstrap methods will be used to estimate standard errors and confidence intervals for mediation effects. A major advantage of the causal mediation method is that it allows testing and controlling for potential confounding effects due to non-random assignments of the mediators. With this approach, the percentage mediated effect can also be calculated. To test mediated effects potentially moderated by baseline cognitive function (high vs. low) and age category (young vs. old), moderated causal mediation analysis will be conducted to examine whether mediation patterns are the same for subgroups defined by moderators. Magnitude and percent of mediated effect between the subgroups (conditional causal mediation effect) will be compared.
Variation between our patients and across our treatment delivery sites in patient engagement may also influence the effectiveness of both interventions. We expect that greater treatment dose (sessions attended) will multiply gains in treatment impact on community functioning and quality of life for both CET and SST, thus moderating the impact of treatment along both primary causal pathways, but not changing predictions of the relative effectiveness of CET compared to SST.
As our HTE analyses are not based on testing explanatory hypotheses, we will report the corresponding p-values for hypothesis testing for these subgroup analyses. However, rather than relying on statistical significance testing alone, we will evaluate HTE using a combination of statistical significance and clinically meaningful difference with emphasis on the magnitude of treatment effects with standard errors (clinically meaningful effect sizes) as outlined in PCORI Methodology Standards  and recommended by statisticians .
Adjusting for multiple subgroup comparisons
Subgroup analysis for treatment effects (i.e., estimate of the treatment effect for each subgroup) is a common approach for HTE analysis, but it is susceptible to the issue of multiple post hoc analyses. For the analyses of primary and secondary hypotheses, it is necessary to control for possible Type I error due to multiple tests. However, our moderation analyses will test exploratory hypotheses, and thus we will report the results of statistical tests including both corrected and uncorrected probability levels. To this end, we will adjust the p-value to account for multiple tests for each subgroup analysis using the false discovery rate method  implemented in SAS software . Subgroup analysis may also increase the likelihood of Type II error due to small sample sizes. Therefore, we will not rely on statistical tests and p-values alone to evaluate HTE; we will also examine estimated effect sizes (estimate divided by the standard deviation) and compute confidence intervals around these effect sizes.
Missing data modeling
We will use generalized linear mixed modeling (GLMM) to test all hypotheses. The GLMM uses all available data, thus allowing intermediate missing data and attrition in the outcome vector. GLMM yields valid inferences, assuming data are missing at random (MAR). However, it is possible that some outcome data may not be missing at random. Thus, it is important to test the MAR assumption. Because the distinction between MAR and not missing at random (NMAR) involves unobserved data, we cannot test whether the missing pattern is MAR or NMAR using the collected data . We will use multiple empirical ways to check the MAR assumption, and to ensure stability of our estimates and inference. In addition, we will conduct sensitivity analyses using several NMAR models (e.g., selection model and pattern mixture model)  to see if the results from different NMAR models are consistent from our MAR model. At a minimum, we will check to see if the patterns and rates of attrition are different across intervention arms; if we find this to be true, we will use a pattern mixture model to refit our models and to evaluate potential biases associated with missing data.
Throughout, we used the SPIRIT reporting guidelines .
The project leadership structure includes four groups: The Project Leadership Group, the combined Project Steering Committee, the Stakeholder Committee, and the Training and Implementation Team (CET and SST) (see Fig. 3). Local stakeholder representation is provided by the preexisting Consumer Advisory Board (CAB) at the Massachusetts Mental Health Center (MMHC)—where the study is headquartered within the BIDMC system—and by local stakeholder groups at other sites.
The Leadership Group is responsible for general project oversight, project planning, and management of implementation, training, treatment, engagement, and assessment, as well as planning study-related publications. Throughout the study, weekly meetings of the co-PIs and the entire co-investigator group and monthly meetings of the site PI group ensure coordination of all project activities and timely response to any challenges. Our lead consumer consultant will provide guidance as a cultural broker to ensure effective communication with participants and to maintain an appropriate balance of power throughout the project.
The Project Steering Committee provides overall project guidance in a collaborative structure that maximizes shared learning and site-specific lessons. This Committee’s annual meeting brings together the Project Leadership Group, the site PIs, and the Stakeholder Committee representatives to ensure multiple perspectives on project activities and critical engagement about project challenges. Many of the site PIs are very experienced researchers and all are experts in service design and delivery, while members of our Stakeholder Committee represent a wealth of experience in different service delivery systems and with different roles in the treatment system. At the project’s conclusion, a combined Stakeholder and Steering Committee meeting will present main research findings, provide breakout groups for discussion and interpretation, and plan publications.
The co-PIs will lead monthly virtual meetings of the site PIs to ensure flexible and timely response to site-specific challenges in implementation, management, and engagement as well as to maximize sharing of strategies and perspectives and communication of results.
A broad spectrum of stakeholders representing lived experience, advocacy, and organization and delivery of services will provide advice and feedback to guide the direction of the project so that the most effective interventions can be implemented widely following conclusion of the PCORI-sponsored project. Our Stakeholder Committee begins with 13 members representing 11 different locations and the perspectives of people with lived experience of mental illness (7 members), family members (2 members), and professionals (4 members). The Stakeholder Committee members will have two annual virtual meetings to review project progress, seek feedback about challenges to recruitment, data collection and other project activities, and discuss what we have learned from treatment delivery and preliminary analyses. Meetings will also address any site-level barriers to conducting the study.
The MMHC Consumer Advisory Board (CAB) represents a partnership between people with research, clinical and/or lived experience with SMI. The CAB at our project headquarters provides a unique local opportunity to elicit and receive feedback from individuals with lived experience who are not involved in the conduct of the study and are therefore able to offer suggestions and advice that may not occur to those who are project participants. These individuals already have deep experience in participatory clinical research. The MMHC CAB will review project reports at one meeting each quarter to provide regular and frequent input into recruitment, retention, intervention delivery, questions for qualitative interviews on treatment process, and suggestions for data analysis.
Patient representatives and advocates on the Steering Committee will hold semiannual meetings with the local stakeholder groups to review and inform treatment and research plans. We consider these meetings to be a vital part of our plan to manage our project in a way that reflects the values of social connection and peer support that are reflected in the interventions themselves. The local stakeholder groups represent the strategies developed at unique sites to engage stakeholders and so will provide diverse perspectives on project issues that in turn will result in a more holistic understanding of project progress.
The process of synthesizing stakeholder feedback will be dynamic, responsive, and person-centered. Stakeholder committee members will present reports about local stakeholder groups (including the CAB) at the biennial Stakeholder Steering Committee meetings, as well as at the annual project Steering Committee meeting. An ongoing working group of Stakeholder Committee members will periodically review process evaluation and assessment data and suggest improvements in procedures and directions for analyses. Collaboration and partnership among these groups is facilitated by leadership provided by our consumer consultant, Dr. Jon Delman, and a Stakeholder Leadership Team that also includes one of the two co-PIs and two co-investigators .
When stakeholder feedback suggests or the Steering Committee participants recommend changes, the leadership team will propose actions and then seek review by the stakeholder groups. After taking feedback into account, the leadership team will publicize and implement the resulting recommendations. Any indications of adverse events and any protocol modifications are reported to (and must be approved by) the BIDMC CCI (IRB) and trial conduct is also monitored by PCORI in monthly meetings and semi-annual reports.
All co-investigators are expected to be co-authors of major project papers and will serve as an approval board to grant authorship to study staff, site-level principal investigators, and stakeholder committee members upon request and when warranted by expert contributions to specific articles.
Since the advent of deinstitutionalization beginning in the 1960s, leaders in psychosocial rehabilitation have recognized the problems for community functioning caused by deficient social skills and have developed and refined training programs to overcome these deficits. The efficacy of multiple SST programs has now been established. It is only more recently, over the last three decades, that the cognitive impairments associated with SZ have become the focus of systematic remediation efforts . The resulting body of research makes it clear that adding cognitive remediation to a psychosocial intervention improves functional outcomes more than providing psychosocial intervention alone  and that cognitive remediation programs added to psychosocial interventions have stronger effects on functional outcomes than “stand-alone” cognitive remediation interventions that only target cognitive functioning [32, 36, 37]. What has remained unknown has been whether these two different psychosocial treatments are equally effective on average for the improvement of community functioning and whether one or the other is advantageous for some affected individuals. As a result, clinicians have no empirical basis for deciding which treatment to recommend nor determining which individuals may benefit most from which treatment. Our PCORI-funded comparative effectiveness trial will achieve a head-to-head comparison of SST and CET and so will provide guidance to permit choice of intervention.
The resource shortages endemic in mental health services and exacerbated by the pandemic heighten the importance of this investigation . Organizations that use the more comprehensive approach to psychosocial rehabilitation, which includes remediation of deficits in both neurocognition and social cognition, require additional training of personnel and cost more to deliver. It is imperative to determine whether the potential benefits justify the additional costs. In an era when facility with and interest in computer-based exercises varies with age and cognitive ability, the efficiency of resource allocation will also be improved by our test of whether cognitive remediation is more beneficial for younger persons and/or for those more cognitively impaired.
The experimental design of our research allows a well-powered, internally valid test of the two interventions as well as systematic exploration of several key hypothesized interactions between the interventions and participant characteristics. Our comprehensive assessment battery, with multiple assessments during and after the intervention experience, allows us to contrast the two interventions in terms of different dimensions of social, cognitive, and community functioning, as well as to evaluate mechanisms hypothesized to underlie the primary effects of each treatment.
Inclusion of community stakeholders is essential in community-based research to develop implementation strategies that reflect the specific opportunities and constraints in different settings as well as to build support for and publicize the value of the interventions. Our leadership structure creates a bidirectional process of exchange between research personnel and stakeholder groups that include persons with lived experience, advocates, and clinicians. By engaging stakeholders in a project-wide advisory committee as well as in site-specific groups, we will be able to maximize the input we receive and refine the guidelines we develop.
The pandemic created special challenges for delivery of socially engaging mental health services. We conducted a pilot study to adapt our treatments and methodology to these changed circumstances and will report the results in a separate article]. These adaptations do not change the treatment elements or research methods described in this article.
Overall, we believe that the proposed comparative effectiveness trial addresses a key knowledge gap for the field and a decisional dilemma facing clinicians seeking to improve community functioning in schizophrenia. Given that schizophrenia is one of the most disabling illnesses in all of medicine, the public health significance of the research is substantial.
Availability of data and materials
The datasets during and/or analysed during the current study as well as the full protocol and model consent form will be available from the corresponding author on reasonable request.
Patient-Centered Outcomes Research Institute
Cognitive Enhancement Therapy
Helping Ourselves Pursue and Experience Success
Social Skills Training
Randomized clinical trial
Cognition for Learning and for Understanding Everyday Social Situations
Mini International Neuropsychiatric Interview
Diagnostic and Statistical Manual 5
Beth Israel Deaconess Medical Center
Intra-cluster correlation coefficient
Health Insurance Portability and Accountability Act
Social Adjustment Scale
Positive and Negative Syndrome Scale
- Penn CNB:
Penn Computerized Neurocognitive Battery
Meyer-Salovey-Caruso Emotional Intelligence Test
Social Skills Performance Assessment
Client Satisfaction Questionnaire
National Institutes of Health
Service Use and Resource Form
Brief Assessment of Cognition in Schizophrenia
Generalized linear mixed-effects models
Heterogeneity of treatment effects
Statistical Analysis System
Missing at random
Not missing at random
Consumer Advisory Board
Massachusetts Mental Health Center
Standard Protocol Items: Recommendations for Interventional Trials
Serious mental illness
Olfson M, Gerhard T, Huang C, Crystal S, Stroup TS. Premature mortality among adults with schizophrenia in the United States. JAMA Psychiatry. 2015;72(12):1172–81.
Saha S, Chant D, Welham J, McGrath J. A systematic review of the prevalence of schizophrenia. PLoS Med. 2005;2(5):e141.
Tandon R, Keshavan MS, Nasrallah HA. Schizophrenia,“just the facts” what we know in 2008. 2. Epidemiology and etiology. Schizophr Res. 2008;102(1-3):1–18.
Dixon LB, Goldman HH, Srihari VH, Kane JM. Transforming the treatment of schizophrenia in the United States: the RAISE initiative. Annu Rev Clin Psychol. 2018;14:237–58.
Cloutier M, Aigbogun MS, Guerin A, et al. The economic burden of schizophrenia in the United States in 2013. J Clin Psychiatry. 2016;77(6):764–71.
Tandon R, Nasrallah HA, Keshavan MS. Schizophrenia,“just the facts” 4. Clinical features and conceptualization. Schizophr Res. 2009;110(1-3):1–23.
McGurk SR, Mueser KT. Cognitive functioning, symptoms, and work in supported employment: a review and heuristic model. Schizophr Res. 2004;70(2-3):147–73.
Bora E, Pantelis CJSr. Social cognition in schizophrenia in comparison to bipolar disorder: a meta-analysis. Schizophr Res. 2016;175(1-3):72–8.
Collins PY, Patel V, Joestl SS, et al. Grand challenges in global mental health. Nature. 2011;475(7354):27.
Kahn R, Sommer I, Murray R, et al. Schizophrenia. Nat Rev Dis Primers. 2015;1(11-12):15067.
Kane JM, Robinson DG, Schooler NR, et al. Comprehensive versus usual Community Care for First-Episode Psychosis: 2-year outcomes from the NIMH RAISE early treatment program. Am J Psychiatry. 2016;173(4):362–72.
Sergi MJ, Rassovsky Y, Widmark C, et al. Social cognition in schizophrenia: relationships with neurocognition and negative symptoms. Schizophr Res. 2007;90(1-3):316–24.
Van Hooren S, Versmissen D, Janssen I, et al. Social cognition and neurocognition as independent domains in psychosis. Schizophr Res. 2008;103(1-3):257–65.
Savla GN, Vella L, Armstrong CC, Penn DL, Twamley EW. Deficits in domains of social cognition in schizophrenia: a meta-analysis of the empirical evidence. Schizophr Bull. 2013;39(5):979–92.
Mehta UM, Thirthalli J, Subbakrishna D, Gangadhar BN, Eack SM, Keshavan MS. Social and neuro-cognition as distinct cognitive factors in schizophrenia: a systematic review. Schizophr Res. 2013;148(1-3):3–11.
Dickinson D, Bellack AS, Gold JMJSB. Social/communication skills, cognition, and vocational functioning in schizophrenia. Schizophr Bull. 2007;33(5):1213–20.
Schutt RK, Seidman LJ, Caplan B, Martsinkiv A, Goldfinger SM. The role of neurocognition and social context in predicting community functioning among formerly homeless seriously mentally ill persons. Schizophr Bull. 2007;33(6):1388–96.
Goldberg RW, Rollins AL, Lehman AFJPRJ. Social network correlates among people with psychiatric disabilities. Psychiatr Rehabil J. 2003;26(4):393.
Mueser KT, Bellack AS, Douglas MS, Wade JH. Prediction of social skill acquisition in schizophrenic and major affective disorder patients from memory and symptomatology. Psychiatry Res. 1991;37(3):281–96.
Smith TE, Bellack AS, Liberman RPJCPR. Social skills training for schizophrenia: review and future directions. Clin Psychol Rev. 1996;16(7):599–617.
Keshavan MS, Vinogradov S, Rumsey J, Sherrill J, Wagner A. Cognitive training in mental disorders: update and future directions. Am J Psychiatry. 2014;171(5):510–22.
Keshavan MS, Eack SM. Cognitive enhancement in SZ and related disorders. New York: Cambridge University Press; 2019.
Kurtz MM, Mueser KT. A meta-analysis of controlled research on social skills training for schizophrenia. J Consult Clin Psychol. 2008;76(3):491–504.
Turner DT, McGlanaghy E, Cuijpers P, van der Gaag M, Karyotaki E, MacBeth A. A meta-analysis of social skills training and related interventions for psychosis. Schizophr Bull. 2018;44(3):475–91.
Green MF, Kern RS, Braff DL, Mintz J. Neurocognitive deficits and functional outcome in schizophrenia: are we measuring the “right stuff”? Schizophr Bull. 2000;26(1):119–36.
Smith TE, Hull JW, Romanelli S, Fertuck E, Weiss KA. Symptoms and neurocognition as rate limiters in skills training for psychotic patients. Am J Psychiatry. 1999;156(11):1817–8.
Kern RS, Green MF, Satz P. Neuropsychological predictors of skills training for chronic psychiatric patients. Psychiatry Res. 1992;43(3):223–30.
Bowen L, Wallace CJ, Glynn SM, Nuechterlein KH, Lutzker JR, Kuehnel TG. Schizophrenic individuals' cognitive functioning and performance in interpersonal interactions and skills training procedures. J Psychiatr Res. 1994;28(3):289–301.
Corrigan PW, Wallace CJ, Schade ML, Green MFJBT. Learning medication self-management skills in schizophrenia: relationships with cognitive deficits and psychiatric symptoms. Behav Ther. 1994;25(1):5–15.
Gupta M, Bassett E, Iftene F, Bowie CR. Functional outcomes in schizophrenia: understanding the competence-performance discrepancy. J Psychiatr Res. 2012;46(2):205–11.
Kern RS, Glynn SM, Horan WP, Marder SR. Psychosocial treatments to promote functional recovery in schizophrenia. Schizophr Bull. 2009;35(2):347–61.
McGurk SR, Twamley EW, Sitzer DI, McHugo GJ, Mueser KT. A meta-analysis of cognitive remediation in schizophrenia. Am J Psychiatry. 2007;164(12):1791–802.
Greenberg S, Rosenblum KL, McInnis MG, Muzik M. The role of social relationships in bipolar disorder: a review. Psychiatry Res. 2014;219(2):248–54.
Kambeitz-Ilankovic L, Betz LT, Dominke C, Haas SS, Subramaniam K, Fisher M, et al. Multi-outcome meta-analysis (moma) of cognitive remediation in schizophrenia: revisiting the relevance of human coaching and elucidating interplay between multiple outcomes. Neurosci Biobehav Rev. 2019;107:828–45.
Lejeune JA, Northrop A, Kurtz MM. A meta-analysis of cognitive remediation for schizophrenia: efficacy and the role of participant and treatment factors. Schizophr Bull. 2021;47(4):997–1006.
Vita A, Barlati S, Ceraso A, Nibbio G, Ariu C, Deste G, et al. Effectiveness, core elements, and moderators of response of cognitive remediation for schizophrenia. JAMA Psychiatry. 2021;78(8):848.
Wykes T, Huddy V, Cellard C, McGurk SR, Czobor P. A meta-analysis of cognitive remediation for schizophrenia: methodology and effect sizes. Am J Psychiatry. 2011;168(5):472–85.
Roder V, Mueller DR, Mueser KT, Brenner HD. Integrated psychological therapy (IPT) for schizophrenia: is it effective? Schizophr Bull. 2006;32(suppl_1):S81–93.
Eack SM, Greenwald DP, Hogarty SS, et al. Cognitive enhancement therapy for early-course schizophrenia: effects of a two-year randomized controlled trial. Psychiatr Serv. 2009;60(11):1468–76.
Wojtalik JA, Mesholam-Gately RI, Hogarty SS, Greenwald DP, Litschge MY, Sandoval LR, et al. Confirmatory efficacy of cognitive enhancement therapy for early schizophrenia: results from a multisite randomized trial. Psychiatr Serv. 2021;73(5):501–9.
Bartels SJ, Pratt SI, Mueser KT, et al. Long-term outcomes of a randomized trial of integrated skills training and preventive healthcare for older adults with serious mental illness. Am J Geriatr Psychiatry. 2014;22(11):1251–61.
Mueser KT, Pratt SI, Bartels SJ, et al. Randomized trial of social rehabilitation and integrated health care for older people with severe mental illness. J Consult Clin Psychol. 2010;78(4):561–73.
Hogarty GE, Flesher S, Ulrich R, et al. Cognitive enhancement therapy for schizophrenia: effects of a 2-year randomized trial on cognition and behavior. Arch Gen Psychiatry. 2004;61(9):866–76.
Turner DT, McGlanaghy E, Cuijpers P, van der Gaag M, Karyotaki E, MacBeth AJ. A Meta-analysis of social skills training and related interventions for psychosis. Schizophr Bull. 2017;44(3):475–91.
Hogarty GE, Greenwald DP. In: Hogarty G, Greenwald P, editors. Cognitive enhancement therapy: the training manual; 2006.
Eack SM, Greenwald DP, Hogarty SS, Keshavan MS. One-year durability of the effects of cognitive enhancement therapy on functional outcome in early schizophrenia. Schizophr Res. 2010;120(1-3):210–6.
Liberman RP, Wallace CJ, Blackwell G, Kopelowicz A, Vaccaro JV, Mintz J. Skills training versus psychosocial occupational therapy for persons with persistent schizophrenia. Am J Psychiatry. 1998;155(8):1087–91.
Pratt SI, Bartels SJ, Mueser KT, Forester B. Helping older people experience success: an integrated model of psychosocial rehabilitation and health care management for older adults with serious mental illness. Am J Psychiatr Rehabil. 2008;11(1):41–60.
Chan JY, Hirai HW, Tsoi KK. Can computer-assisted cognitive remediation improve employment and productivity outcomes of patients with severe mental illness? A meta-analysis of prospective controlled trials. J Psychiatr Res. 2015;68:293–300.
Brenner HD, Roder V, Hodel B, Kienzle N, Reed D, Liberman RP. Integrated psychological therapy for schizophrenic patients (IPT). Valley Stream: Hogrefe & Huber Publishers; 1994.
Hogarty GE, Flesher S. Practice principles of cognitive enhancement therapy for schizophrenia. Schizophr Bull. 1999;25(4):693–708.
McGurk SR, Mueser KT, Xie H, et al. Cognitive enhancement treatment for people with mental illness who do not respond to supported employment: a randomized controlled trial. Am J Psychiatry. 2015;172(9):852–61.
McGurk SR, Mueser KT, Xie H, et al. Cognitive remediation for vocational rehabilitation nonresponders. Schizophr Res. 2016;175(1-3):48–56.
McGurk SR, Mueser KT, Pascaris A. Cognitive training and supported employment for persons with severe mental illness: one-year results from a randomized controlled trial. Schizophr Bull. 2005;31(4):898–909.
McGurk SR, Mueser KT, DeRosa TJ, Wolfe R. Work, recovery, and comorbidity in schizophrenia: a randomized controlled trial of cognitive remediation. Schizophr Bull. 2009;35(2):319–35.
DeTore NR, Mueser KT, McGurk SR. Cognitive functioning as a predictor of response to cognitive remediation. J Psychiartic Res. 2019;113:117–24.
Keshavan MS, Hogarty GE. Brain maturational processes and delayed onset in schizophrenia. Dev Psychopathol. 1999;11(3):525–43.
Bowie CR, Grossman M, Gupta M, Oyewumi LK, Harvey PD. Cognitive remediation in schizophrenia: efficacy and effectiveness in patients with early versus long-term course of illness. Early Interv Psychiatry. 2014;8(1):32–8.
Corbera S, Wexler BE, Poltorak A, Thime WR, Kurtz MM. Cognitive remediation for adults with schizophrenia: does age matter? Psychiatry Res. 2017;247:21–7.
McGurk SR, Mueser K. Response to cognitive rehabilitation in older versus younger persons with severe mental illness. Am J Psychiatr Rehabil. 2008;11(1):90–105.
Revell ER, Neill JC, Harte M, Khan Z, Drake RJ. A systematic review and meta-analysis of cognitive remediation in early schizophrenia. Schizophr Res. 2015;168(1-2):213–22.
Thomas KR, Puig O, Twamley E. Age as a moderator of change following compensatory cognitive training in individuals with severe mental illnesses. Psychiatr Rehabil J. 2017;40(1):70.
Bellack AS, Mueser KT, Gingerich S, Agresta J. Social skills training for schizophrenia: a step-by-step guide. NY: Guilford Publications; 2013.
Medalia A, Saperstein AM. Does cognitive remediation for schizophrenia improve functional outcomes? Curr Opin Psychiatry. 2013;26(2):151–7.
Pena J, Ibarretxe-Bilbao N, Sanchez P, Inarte MB, Elizagarate E, Garay MA, et al. Combining social cognitive treatment, cognitive remediation, and functional skills training in schizophrenia: a randomized controlled trial. NPJ Schizophr. 2016;9(2):16037.
Friedman-Yakoobian M, Parrish EM, Thomas A, Lesser R, Gnong-Granato A, Eack S, et al. An integrated neurocognitive and social-cognitive treatment for youth at clinical high risk for psychosis: cognition for learning and for understanding everyday social situations (CLUES). Schizophr Res. 2019;208:55–9.
Santesteban-Echarri O, Piskulic D, Nyman RK, Addington J. Telehealth interventions for schizophrenia-spectrum disorders and clinical high-risk for psychosis individuals: a scoping review. J Telemed Telecare. 2018;26(1-2):14–20.
Lynch DA, Medalia A, Saperstein A. The design, implementation, and acceptability of a telehealth comprehensive recovery service for people with complex psychosis living in NYC during the covid-19 crisis. Front Psychiatry. 2020;11:581149.
Gleeson JF, Lederman R, Wadley G, Bendall S, McGorry PD, Alvarez-Jimenez M. Safety and privacy outcomes from a moderated online social therapy for young people with first-episode psychosis. Psychiatr Serv. 2014;65(4):546–50.
Gleeson JF, Alvarez-Jimenez M, Lederman R. Moderated online social therapy for recovery from early psychosis. Psychiatr Serv. 2012;63(7):719.
Eack SM, Hogarty SS, Greenwald DP, Litschge MY, Porton SA, Mazefsky CA, et al. Cognitive enhancement therapy for adult autism spectrum disorder: results of an 18-month randomized clinical trial. Autism Res. 2017;11(3):519–30.
DeLuca JS, Andorko ND, Chibani D, Jay SY, Rakhshan Rouhakhtar PJ, Petti E, et al. Telepsychotherapy with youth at clinical high risk for psychosis: clinical issues and best practices during the COVID-19 pandemic. J Psychother Integr. 2020;30(2):304–31.
Bashshur RL, Shannon GW, Bashshur N, Yellowlees PM. The empirical evidence for telemedicine interventions in mental disorders. Telemed J E Health. 2016;22(2):87–113.
Alvarez-Jimenez M, Alcazar-Corcoles MA, González-Blanch C, Bendall S, McGorry PD, Gleeson JF. Online, social media and mobile technologies for psychosis treatment: a systematic review on novel user-led interventions. Schizophr Res. 2014;156(1):96–106.
Mendelson D, Thibaudeau É, Sauvé G, Lavigne KM, Bowie CR, Menon M, et al. Remote group therapies for cognitive health in schizophrenia-spectrum disorders: feasible, acceptable, engaging. Schizophr Res Cogn. 2022;28:100230.
Hubley S, Lynch SB, Schneck C, Thomas M, Shore J. Review of key telepsychiatry outcomes. World J Psychiatry. 2016;6(2):69–82.
Sheehan DV, Lecrubier Y, Sheehan KH, et al. The Mini-international neuropsychiatric interview (M.I.N.I.): the development and validation of a structured diagnostic psychiatric interview for DSM-IV and ICD-10. J Clin Psychiatry. 1998;59(Suppl 20):22–33 quiz 34-57.
Killip S, Mahfoud Z, Pearce K. What is an intracluster correlation coefficient? Crucial concepts for primary care researchers. Ann Fam Med. 2004;2(3):204–8.
Sedgwick P. Cluster randomised controlled trials: sample size calculations. BMJ. 2013;346:e4654–f2839.
Diggle P, Diggle PJ, Heagerty P, Heagerty PJ, Liang K-Y, Zeger S. Analysis of longitudinal data: Oxford University Press; 2002.
Lu K, Luo X, Chen PY. Sample size estimation for repeated measures analysis in randomized clinical trials with missing data. Int J Biostat. 2008;4(1):9.
Donohue MC, Gamst AC, Edland SD. Longpower: sample size calculations for longitudinal data. 2015;R package version 1.0-11.
Schooler N, Hogarty G, Weissman M. Social adjustment scale II (SAS-II). In: Hargreaves WA, Atkisson CC, Sorenson SE, editors. Resource materials for community mental health program evaluations. Rockville: National Institute of Mental Health; 1979. p. 290–303.
Heinrichs DW, Hanlon TE, Carpenter WT Jr. The quality of life scale: an instrument for rating the schizophrenic deficit syndrome. Schizophr Bull. 1984;10(3):388–98.
Mueser KT, Kim M, Addington J, McGurk SR, Pratt SI, Addington DE. Confirmatory factor analysis of the quality of life scale and new proposed factor structure for the quality of life scale-revised. Schizophr Res. 2017;181:117–23.
Lindenmayer JP. Are shorter versions of the positive and negative syndrome scale (PANSS) doable? A critical review. Innov Clin Neurosci. 2017;14(11-12):73–6.
Ostergaard SD, Foldager L, Mors O, Bech P, Correll CU. The validity and sensitivity of PANSS-6 in the clinical antipsychotic trials of intervention effectiveness (CATIE) study. Schizophr Bull. 2018;44(2):453–62.
Mayer JD, Salovey P, Caruso DR, Sigarenios G. Measuring emotional intelligence with the MSCEIT V2.0. Emotion. 2003;3:97–105.
Patterson TL, Moscona S, McKibbin CL, Davidson K, Jeste DV. Social skills performance assessment among older patients with schizophrenia. Schizophr Res. 2001;48(2-3):351–60.
Larsen DL, Attkisson CC, Hargreaves WA, Nguyen TD. Assessment of client/patient satisfaction: development of a general scale. Eval Program Plann. 1979;2(3):197–207.
Rosenheck R, Leslie D, Sint K, Lin H, Robinson DG, et al. Cost-effectiveness of comprehensive, integrated care for first episode psychosis in the NIMH Raise early treatment program. Schizophr Bull. 2016;42(2):896–906.
Kahan BC, Morris TP. Adjusting for multiple prognostic factors in the analysis of randomised trials. BMC Med Res Methodol. 2013;13:99.
Valeri L, VanderWeele TJ. Mediation analysis allowing for exposure–mediator interactions and causal interpretation: theoretical assumptions and implementation with SAS and SPSS macros. Psychol Methods. 2013;18(2):137–50.
VanderWeele T. Explanation in causal inference: methods for mediation and interaction. NY: Oxford University Press; 2015.
Patient-Centered Outcomes Research Institute (PCORI). PCORI methodology standards. n.d. https://www.pcori.org/research/about-our-research/research-methodology/pcori-methodology-standards
Lagakos SW. The challenge of subgroup analyses—reporting without distorting. N Engl J Med. 2006;354(16):1667–9.
Benjamini Y, Hochberg Y. Controlling the false discovery rate: a practical and powerful approach to multiple testing. J R Stat Soc Series B Stat Methodol. 1995;57(1):289–300.
Westfall PH, Tobias RD, Wolfinger RD. Multiple comparisons and multiple tests using SAS. Cary: SAS Institute; 2011.
Hedeker D, Gibbons RD. Longitudinal data analysis, vol. 451. Hoboken: John Wiley & Sons; 2006.
Molenberghs G, Kenward M. Missing data in clinical studies, vol. 61. Hoboken: John Wiley & Sons; 2007.
Chan A-W, Tetzlaff JM, Gøtzsche PC, Altman DG, Mann H, Berlin J, et al. SPIRIT 2013 explanation and elaboration: guidance for protocols of clinical trials. BMJ. 2013;346:e7586.
Walker ER, Zahn R, Druss BGJPS. Applying a model of stakeholder engagement to a pragmatic trial for people with mental disorders. Psychiatr Serv. 2018;69(11):1127–30.
Schutt RK, Seidman LJ, Keshavan MS. Social neuroscience. Cambridge: Harvard University Press; 2015.
van Duin D, deWinter L, Kroon H, Veling W, van Weeghel J. Effects of IPS plus cognitive remediation in early psychosis: 18-month functioning outcomes of a randomized controlled trial. Schizophr Res. 2021;236:115–22.
All statements in this report, including its findings and conclusions, are solely those of the authors and do not necessarily represent the views of the Patient-Centered Outcomes Research Institute (PCORI), its Board of Governors or Methodology Committee.
Patient-Centered Outcomes Research Institute® (PCORI®) Award (CER-2018C3-14701). PCORI approved for funding the study protocol submitted by the researchers and negotiated some modifications in the study design prior to release of funding. PCORI continues to monitor study progress and provide feedback on analysis of data, but does not exercise authority over interpretation of data, contents of the final report, or decisions over submission for publication.
Ethics approval and consent to participate
The Committee on Clinical Investigations, Beth Israel Deaconess Medical Center, approved the study and its Informed Consent Forms on February 24, 2020. IRB Protocol #: 2020P000094. All procedures will be conducted in accordance with this approved protocol. Written informed consent for participation in the study will be obtained from all participants by a site-level trained research assistant prior to their involvement.
Consent for publication
Not applicable. Individual clinical data will not be used in this study.
The authors declare that they have no competing interests. No professional writers will be used to write any manuscripts that arise from this study.
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
The original version of this article was revised: the name has been updated.
About this article
Cite this article
Schutt, R.K., Xie, H., Mueser, K.T. et al. Cognitive Enhancement Therapy vs social skills training in schizophrenia: a cluster randomized comparative effectiveness evaluation. BMC Psychiatry 22, 583 (2022). https://doi.org/10.1186/s12888-022-04149-x
- Social skills training
- Cognitive enhancement therapy
- Cognitive remediation
- Community functioning
- Cluster randomized controlled trial
- Psychosocial rehabilitation
- Social cognition