Skip to main content
  • Research article
  • Open access
  • Published:

Comparison and predictors of treatment adherence and remission among patients with schizophrenia treated with paliperidone palmitate or atypical oral antipsychotics in community behavioral health organizations



Nonadherence to antipsychotic treatment increases the likelihood of relapse and progressive symptomatology in patients with schizophrenia. Atypical long-acting injectables, including paliperidone palmitate (PP), may increase adherence and improve symptoms. This study compared and assessed predictors of treatment patterns and symptom remission among schizophrenia patients treated with PP versus atypical oral antipsychotic therapy (OAT) in community behavioral health organizations (CBHOs).


This retrospective cohort analysis evaluated 763 patients with schizophrenia and new (PP-N; N = 174) or continuing (PP-C; N = 308) users of PP, or new users of OAT (N = 281) at enrollment in the REACH-OUT study (2010–2013). Treatment outcomes assessed at 1 year were discontinuation, and adherence, measured by proportion of days covered (PDC) or medication possession ratio (MPR). Remission status was assessed using the Structured Clinical Interview for Symptoms of Remission (SCI-SR). A machine learning platform, Reverse Engineering and Forward Simulation (REFS™), was used to identify predictors of study outcomes. Multivariate Cox and generalized linear regressions estimated the adjusted hazard ratios (HRs) or odds ratios (ORs) with 95% confidence intervals.


Among PP-N users, 27% discontinued their initial treatment regimen versus 51% (p < 0.001) of OAT users. PP-N (vs OAT; HR = 0.49 [0.31–0.76]) users and males (HR = 0.65 [0.46–0.92]) had significantly lower rates of discontinuation. Relative to OAT, PP-N had a 36% [31%–42%] higher MPR and a 10-fold increased achievement of PDC ≥80% (OR = 10.46 [5.72–19.76]). PP users were significantly more likely to achieve remission in follow-up (PP-N vs OAT: OR = 2.65 [1.39–5.05]; PP-C vs OAT: OR = 1.83 [1.03–3.25]).


Relative to OAT, PP was associated with improved adherence, less frequent treatment discontinuation, and improved symptom remission in this CBHO study population.

Peer Review reports


Various atypical antipsychotic medications to treat schizophrenia have been introduced to effectively manage positive symptoms, e.g., hallucinations and delusions [1]. However, nonadherence to treatment has been a common underlying issue in schizophrenia, leading to increased severity of symptoms and increased likelihood of relapse [2,3,4]. In addition, the increased risk of treatment nonadherence and poor health outcomes are further compounded by co-occurring tobacco, alcohol, or other substance abuse among schizophrenia patients [5]. Patients experience schizophrenia as a chronic disorder; the disease usually onsets in early adulthood and lasts throughout the patient’s lifetime [1]. Newly treated patients have an average of nine relapse episodes over 5.5 years, with the first relapse occurring at a median of 34 weeks after diagnosis [6]. As a result, schizophrenia-related costs to the healthcare system are high; a recent study has estimated that the mean cost per patient per month for patients with schizophrenia was $1387 higher ($1806 vs $419) than that for age- and gender-matched people without schizophrenia in the United States [7]. Moreover, schizophrenia patients who are nonadherent to their treatment regimen have even higher costs associated with their healthcare utilization (i.e., increased risk and duration of hospitalization) and require more emergency services [4, 8].

Atypical antipsychotic medications have been shown to be effective in managing the symptoms of schizophrenia and come in two major forms: oral formulations and long-acting injectable (LAI) formulations. LAIs work by providing stable levels of active drug within a patient that can be sustained over many weeks [9]. Since injections are administered by healthcare professionals, missed injections can easily be captured, giving the medical team the opportunity to intervene earlier [10]. LAIs have demonstrated superiority to atypical oral antipsychotic therapies (OATs) in clinical trials and observational studies [11,12,13,14]; 74% of patients in the Clinical Antipsychotic Trials of Intervention Effectiveness (CATIE) study discontinued their oral antipsychotic treatment within 18 months for various reasons [15]. Given the propensity for nonadherence in patients with schizophrenia and the consequences thereof, LAIs are a viable treatment option for schizophrenia in that they provide consistent coverage without the need for daily administration.

Paliperidone palmitate (PP) does not require oral supplementation during treatment initiation, as its pharmacokinetic profile allows both a rapid achievement of therapeutic plasma levels of paliperidone as well as a gradual and continuous release of the drug over the dosing interval, and it provides both acute symptom control and maintenance of effect [16,17,18,19,20,21]. Relative to OAT, PP has shown superiority with delayed time to relapse and improved treatment adherence [22, 23]. PP is generally well-tolerated, with the most common adverse events, occurring in less than a tenth of patients, being insomnia, worsening of schizophrenia, nasopharyngitis, headache, weight gain, and extrapyramidal symptoms [24, 25]. In a study population of previously incarcerated patients, the Paliperidone Palmitate Research in Demonstrating Effectiveness (PRIDE) trial reported a significant delay in treatment failure among those randomly assigned to PP, relative to those assigned to OAT [23]. Additionally, in a study conducted from the US healthcare payer perspective, PP was shown to be more cost-effective relative to OAT; specifically, PP patients had fewer mean annual days of relapse (8.7 days vs 17.8 days) and lower mean annual costs ($20,995 vs $22,481) [26].

Community behavioral health organizations (CBHOs) serve as the primary point of contact within the US healthcare system for patients, providing sustained outpatient care for patients who are publicly insured or uninsured and who often have chronic, severe psychiatric illnesses. These organizations offer high-quality integrated care using patient-centered approaches that focus on treatment plans to reduce and manage symptoms, as well as provide social support and services that prevent potential relapses [27]. Providing evidence of PP effectiveness in the CBHO setting is crucial given that most schizophrenia patients are treated in this setting. Additionally, much can be learned about the impact of treatment on quality of life, medication satisfaction, social functioning, symptom remission, and other patient-reported outcomes. The objectives of our study were to (1) compare the impact of PP vs OAT on medication adherence and symptom remission and (2) identify other patient factors predictive of adherence and remission among patients with schizophrenia treated with PP or OAT in CBHOs.


Study population

This was a retrospective cohort analysis of the prospective, observational Research and Evaluation of Antipsychotic Treatment in Community Behavioral Health Organizations, Outcomes (REACH-OUT) study [28]. This Janssen Pharmaceuticals–sponsored study of usual care of patients undergoing treatment for schizophrenia or bipolar I disorder at CBHOs was conducted between August 2010 and November 2013, approved by participating ethics committees and institutional review boards, and conducted in accordance with the ethical principles of the Declaration of Helsinki. Patients with schizophrenia or bipolar I diagnoses were enrolled in the REACH-OUT study, if they initiated treatment with risperidone: 1) 8 weeks prior to enrollment, or 2) at least 24 weeks prior to enrollment and did not have gaps between injections greater than 30 days. Patients that initiated risperidone treatment between 8 and 24 weeks prior to enrollment were excluded. Schizophrenia patients, who initiated treatment with PP or had been treated with PP any time prior to enrollment were included. Schizophrenia or bipolar I patients, who initiated treatment with an antipsychotic other than risperidone or PP within 8 weeks prior to enrollment were included. Written informed consent was obtained from all subjects prior to study enrollment. Data collection in REACH-OUT was done via patient interviews, clinician measures, and chart reviews at enrollment, and at follow-up study visits at 6 and 12 months for 1164 patients receiving care at 44 CBHOs across the United States between 2010 and 2013. For this analysis, we excluded patients found to be ineligible for REACH-OUT (N = 99), patients with bipolar I disorder at enrollment (N = 121), and patients on injectable risperidone, other LAIs, or other non-OAT antipsychotics (N = 181). Thus, our study population was restricted to the 763 eligible schizophrenia patients who had either initiated PP (N = 174) or an OAT regimen (N = 281) within 8 weeks of enrollment or continued previously initiated PP therapy at enrollment (N = 308) (Fig. 1).

Fig. 1
figure 1

Selection of the study population. Abbreviations: LAI, long-acting injectable; OAT, oral antipsychotic therapy; PP, paliperidone palmitate; REACH-OUT, Research and Evaluation of Antipsychotic Treatment in Community Behavioral Health Organizations, Outcomes study. Notes: Reasons for REACH-OUT ineligibility (N = 99) included unknown or ineligible age (N = 34, 34%), most recent antipsychotic unknown or not study-eligible (N = 22, 22%), did not meet diagnostic criteria for schizophrenia or bipolar I disorder (N = 18, 18%), patient unwilling to complete scheduled study interviews (N = 17, 17%), patient participating in concurrent clinical study (N = 4, 4%), and patient did not consent (N = 4, 4%)

The treatment status for analyses reflects the treatment regimen initiated or continued at enrollment. According to the REACH-OUT protocol, initiators of OAT were eligible if initiating an atypical oral antipsychotic medication or combination within the 8 weeks at or prior to enrollment. Similarly, new users of PP (PP-N) initiated within 8 weeks of enrollment. Continuing users (PP-C) consisted of patients initiating PP more than 8 weeks prior to enrollment. PP-C users may have initiated during or prior to the 6-month pre-enrollment study period.


Treatment pattern outcomes included discontinuation, and adherence. Patients were considered to have discontinued their enrollment regimen if there was any change in treatment, including medication substitutions or stoppage. Adherence was measured using medication possession ratio (MPR), calculated for the duration of PP or OAT treatment (initiation to discontinuation), and using 1-year proportion of days covered (PDC) calculated for the duration of follow-up (1 year from enrollment). These measures were calculated using the following formulae and reported as proportions or percentages.

$$ \mathrm{MPR}=\frac{\mathrm{Days}\ \mathrm{covered}\ \mathrm{by}\ \mathrm{injection}\ \mathrm{or}\ \mathrm{prescription}\ \mathrm{from}\ \mathrm{initiation}\ \mathrm{to}\ \mathrm{discontinuation},\mathrm{n}}{\mathrm{Days}\ \mathrm{from}\ \mathrm{initiation}\ \mathrm{to}\ \mathrm{discontinuation},\mathrm{n}} $$
$$ \mathrm{PDC}=\frac{\mathrm{Days}\ \mathrm{covered}\ \mathrm{by}\ \mathrm{injection}\ \mathrm{or}\ \mathrm{prescription}\ \mathrm{from}\ \mathrm{enrollment}\ \mathrm{to}\ 12\ \mathrm{mos},\mathrm{n}}{365\ \mathrm{days}} $$

The time period for evaluation of MPR was considered to be date of initiation until discontinuation, or loss to follow-up. For PP-C users who initiated prior to REACH-OUT, MPR was calculated from first injection on study. PDC was calculated for the 365-day period from REACH-OUT enrollment to 12 months for all study patients. For patients on OAT, days covered was calculated using the days’ supply from prescription date. Maximum coverage for each PP injection was assumed to be 35 days.

Symptom severity was ascertained using a 30-item questionnaire, the Structured Clinical Interview for Symptoms of Remission (SCI-SR), administered at each study visit. The SCI-SR is a standardized definition of remission that requires a clinician’s assessment of eight Positive and Negative Syndrome Scale (PANSS) items through assigning a score between 1 (absent) and 7 (extreme) to each area: PANSS domains used in the SCI-SR definition include delusions, unusual thought content, hallucinatory behavior, conceptual disorganization, mannerism/posturing, blunted affect, passive social withdrawal, and lack of spontaneity. Remission was defined as a score of ≤3 (mild) in all eight SCI-SR symptom areas [29].


Several additional measures collected via patient interview and/or chart review from the REACH-OUT database were evaluated as potential covariates, including:

  • Demographic information: age, gender, race, ethnicity, insurance coverage, education, and marital status

  • Clinical characteristics: markers of schizophrenia disease status (duration of illness (i.e., number of years since first schizophrenia diagnosis) and disease severity), vital signs (height, weight, systolic/diastolic blood pressure), comorbid conditions (hypertension, hyperlipidemia, heart disease, diabetes, asthma, lung condition, behavioral health disorders [International Classification of Diseases codes 290–319]), and miscellaneous physical comorbidities

  • Behavioral/lifestyle factors: alcohol abuse, substance use, physical activity, arrests, and smoking

  • Healthcare resource utilization: hospitalization, emergency department visits, outpatient visits, and assertive community treatment (ACT)

  • Patient-reported outcomes: Lehman Quality of Life (QOL) – Brief Version [30, 31]; Drug Attitude Inventory, Short Version (DAI-10) [32]; Medication Satisfaction Questionnaire [33]; Personal and Social Performance (PSP) scale [34]; and Scale to Assess Therapeutic Relationships – Patient Version (STAR-P) and Clinician Version (STAR-C) [35]

Subjective QOL domains included General Life Satisfaction, Satisfaction with Daily Activities, Satisfaction with Family Contact, Satisfaction with Social Relations, Job Satisfaction, Satisfaction with Safety, and Satisfaction with Health. These domains were scored on a scale of 1 (Terrible) to 7 (Delighted). Two objective QOL domains evaluated frequency of Family Contact and Social Contact, and were scored on a scale from 1 (Never) to 5 (≥1/day). Two additional objective QOL domains (Financial Adequacy and Victimization) evaluated the degree of financial adequacy and general victimization, ranging from 0 (No) to 1 (Yes). The DAI-10 was used to summarize patients’ attitude toward their medication regimen and was scored on a scale of −10 (poor relationship with antipsychotic medications) to +10 (positive attitude toward prescribed antipsychotics). The Medication Satisfaction Questionnaire consisted of a single evaluation of patient satisfaction with current antipsychotic medications, on a scale from 1 (Very Dissatisfied) to 7 (Very Satisfied). The PSP scale was used to measure patients’ success in psychosocial functioning, and was scored on a scale of 1 (extreme lack of autonomy in basic functioning) to 100 (excellent functioning). STAR-P and STAR-C were used to evaluate the therapeutic alliance, from the patient perspective (STAR-P) and the clinician perspective (STAR-C), respectively. These measures were scored on a scale of 0 to 48, with higher scores representing a positive patient/caregiver relationship.

Statistical analysis

Distributions of study outcomes, by treatment status, were tabulated and evaluated using Fisher’s exact test for categorical variables or the Wilcoxon-Mann-Whitney test for continuous variables. To identify meaningful predictors of treatment outcomes, we used a machine learning analytic platform, Reverse Engineering and Forward Simulation (REFS™) [36,37,38]. Briefly, REFS predictive modeling generates an ensemble comprising a specified number of individual models (N = 50) for the outcome of interest, which is learned empirically from the data using a hypothesis-free approach based on Bayesian scoring algorithms. From the ensemble for each study outcome, predictors were ranked and evaluated by their relative selection frequency (i.e., proportion of models in the ensemble in which the variable was selected) and distributions of effect estimates, in addition to the overall predictive accuracy of the ensemble in the form of area under the (receiver operating characteristic) curve (AUC) statistics.

Subsequently, we constructed multivariable Cox (time to discontinuation) or generalized linear (all other outcomes) regression models to further evaluate effects of treatment status on study outcomes. Covariates considered for inclusion in the adjusted models were drawn based on evidence from REFS prediction model ensembles and previous literature [39]. The final adjusted models were selected using a stepwise selection algorithm based on the Akaike information criterion for each proposed model. Effect estimates are reported as beta estimates (β; linear regression), odds ratios (ORs; logistic regression), or hazard ratios (HRs; Cox regression), with 95% confidence intervals (CIs). Any reported p values testing statistical significance are two-sided. All statistical analyses were performed using REFS or R statistical programming software (version 3.1.1).


Demographic characteristics and other selected patient factors, by PP or OAT status at enrollment, are summarized in Table 1. The mean age for PP and OAT users was 41.1 (±12.6) years and 42.1 (±13.4) years, respectively. On average, PP users were more likely than OAT initiators to be male, single, on Medicare and/or Medicaid, smokers, diagnosed with a lung condition, and living with chronic schizophrenia for a longer duration. PP users were less likely to be Hispanic or live in a private residence (Table 1).

Table 1 Selected Baseline Characteristics by Treatment Status at Enrollment, REACH-OUT (2010–2013)

At 12 months, 308 (64%) PP users and 180 (64%) OAT users had sufficient available data to evaluate treatment discontinuation. Of these patients, 20% (N = 62) of PP users discontinued treatment versus 51% (N = 92; p < 0.001) of OAT users (Table 2). Additionally, PP users were significantly more adherent than OAT users to their respective treatments, by both the MPR and 1-year PDC metrics (Table 2). Specifically, mean MPR for PP users was 0.84 versus 0.52 for OAT users (p < 0.001); mean PDC for PP users was 0.57 versus 0.31 for OAT users (p < 0.001) (Table 2). When evaluating the standard threshold of ≥80% treatment coverage, the proportion adherent was significantly higher among PP users for MPR (74% vs 25%, p < 0.001) and PDC (44% vs 9%, p < 0.001) (Table 2).

Table 2 Treatment-Related Outcomes at 12 Months by Treatment Status at Enrollment, REACH-OUT (2010–2013)

A higher proportion of PP users were in schizophrenia disease remission (all SCI-SR symptomatic domains mild or less) relative to OAT users at all time points (Fig. 2). Among those with available SCI-SR data at 12 months (PP-N, 49%; PP-C, 60%; OAT, 45%), 45% of PP-N users were in remission versus 39% of PP-C users and 23% of OAT users, respectively (p = 0.001). Furthermore, PP-N users in remission at 12 months represented a 25% increase from enrollment, relative to a 14% increase from enrollment among OAT initiators (Fig. 2).

Fig. 2
figure 2

Proportion of patients in remission by study visit and treatment status, REACH-OUT (2010–2013). Abbreviations: ENR, enrollment; OAT, oral antipsychotic therapy; PP-C, continuous user of paliperidone palmitate; PP-N, new user of paliperidone palmitate

Prediction model ensembles were generated for study outcomes from the REACH-OUT database using REFS. To summarize, an ensemble consisting of 50 individual models for treatment discontinuation provided moderately strong predictive accuracy (mean AUC = 0.74). Patient factors consistently selected in this ensemble as meaningful predictors of treatment discontinuation included various comorbidities, substance and alcohol abuse, hospitalizations, and poor adherence. Predictive ensembles for treatment adherence (PDC ≥80%; mean AUC = 0.76) and remission (mean AUC = 0.75) also performed well; consistent predictors included PP use and the absence of psychiatric comorbidities. Consistent, strong predictors of study outcomes identified using REFS were retained and evaluated as covariates for final generalized linear models.

In a multivariable Cox regression model, PP use was associated with a significantly lower rate of discontinuation (PP-N vs OAT: HR = 0.49, 95% CI: 0.31–0.76; PP-C vs OAT: HR = 0.28, 95% CI: 0.18–0.43). The only other statistically significant predictor of time to discontinuation in the adjusted model was gender (male vs female, HR = 0.65, 95% CI: 0.46–0.92).

PP use was significantly associated with higher MPR in an adjusted linear regression model (PP-N vs OAT, β = 0.36, 95% CI: 0.31–0.42; PP-C vs OAT, β = 0.39, 95% CI: 0.34–0.44). Additional correlates of higher MPR included shorter duration of schizophrenia (10-year increase in time from first schizophrenia diagnosis, β = −0.03, 95% CI: –0.05 to −0.01), older age (10-year increase, β = 0.03, 95% CI: 0.01–0.05), and history of arrests (β = 0.13, 95% CI: 0.01–0.25). In a multivariable logistic regression model for PDC ≥80%, initiation of PP resulted in a 10-fold increased likelihood of ≥80% treatment coverage for 1 year from enrollment, relative to OAT initiation (PP-N vs OAT, OR = 10.27, 95% CI: 5.55–19.00; PP-C vs OAT, OR = 16.47, 95% CI: 9.12–29.75). Additional patient factors associated with PDC ≥80% included ACT (OR = 3.00, 95% CI: 1.37–6.59), older age (per 10-year increase, OR = 1.29, 95% CI: 1.03–1.62), history of arrests (OR = 4.17, 95% CI: 1.14–15.32), private residence (OR = 1.89, 95% CI: 1.15–3.13), white race (vs black/African American, OR = 2.50, 95% CI: 1.49–4.17), heart disease (OR = 3.38, 95% CI: 1.36–8.72), and no history of substance abuse (OR = 2.27, 95% CI: 1.43–3.57) (Table 3).

Table 3 Predictors of Treatment Adherence (MPR and 1-year PDC ≥80%), REACH-OUT (2010–2013)

Initiators of PP at enrollment were nearly 2.5 times as likely to achieve disease remission, relative to OAT initiators, in the adjusted model (PP-N vs OAT, OR = 2.40, 95% CI: 1.26–4.57; PP-C vs OAT, OR = 1.97, 95% CI: 1.12–3.47) (Fig. 3). Positive predictors of disease remission in the adjusted model included female gender (OR = 1.75, 95% CI: 1.05–2.94), education level (high school graduate vs less, OR = 1.73, 95% CI: 1.07–2.78), favorable attitude toward medications (DAI-10 ≥ 8 vs <8, OR = 1.69, 95% CI: 1.06–2.69), and satisfaction with social relations (QOL domain, ≥4 [mixed/satisfied] vs <4 [dissatisfied], OR = 2.43, 95% CI: 1.59–3.73). A history of asthma or respiratory disease was negatively associated with remission in this model (OR = 0.55, 95% CI: 0.31–0.94) (Fig. 3).

Fig. 3
figure 3

Predictors of disease remission,a REACH-OUT (2010–2013). Abbreviations: DAI, Drug Attitude Inventory; PP-C, continuous user of paliperidone palmitate; PP-N, new user of paliperidone palmitate; REACH-OUT, Research and Evaluation of Antipsychotic Treatment in Community Behavioral Health Organizations, Outcomes study. aMultivariable logistic mixed effects regression model for disease remission (all SCI-SR domains mild or less), adjusted for all variables shown, plus proportion of days covered (by treatment, continuous), Personal and Social Performance scale score (≥70 vs <70), general life satisfaction (QOL domain, 4–7 [mixed/satisfied] vs 1–3 [dissatisfied]), and number of outpatient visits (continuous). Odds ratios and the 95% confidence intervals included


In this analysis of schizophrenia patients receiving care in CBHOs, we have compared PP and OAT use with respect to adherence and disease remission outcomes. Our results suggest that PP use is associated with increased treatment adherence, a decreased likelihood of discontinuation, and better symptom management, relative to OAT. According to our findings, predictors for discontinuation were nonadherence, OAT treatment status, healthcare resource use indicators (number of hospitalizations and ACT), substance abuse, and comorbid diagnoses. The strongest predictor of treatment adherence was PP use, which was also highly associated with remission. Other predictors of remission included female gender, education status, satisfaction with social relations, favorable attitude toward medications, and lack of lung conditions.

Poor treatment adherence needs to be addressed in order for schizophrenia patients to manage their disease and lead more satisfying lives. Our study corroborates findings from prior research demonstrating that LAIs, specifically PP, improve treatment stability among schizophrenia patients. Specifically, research in patients in Medicaid populations treated with LAIs versus OATs has shown greater antipsychotic adherence, lower rates of discontinuation (60-day continuous gap), and reduced risk of rehospitalization among those treated with LAIs versus OATs in the 6 months following a schizophrenia-related hospitalization [40, 41]. Our findings extend beyond the Medicaid population and confirm improved treatment pattern outcomes associated with PP for a heterogeneous population.

Relapse among schizophrenia patients impacts overall prognosis and healthcare utilization. Specifically, successive relapses can reduce the degree and duration of subsequent remission, worsen disability, and increase refractoriness to future treatment [42]. Furthermore, relapses are associated with high medical and nonmedical costs as well as productivity loss [43]. Therefore, managing relapses and delaying time to relapse is an important goal in schizophrenia treatment. Several studies have shown that relative to OATs, PP is associated with a significant delay in time to relapse. In a multicenter randomized trial, the Prevention of Relapse with Oral Antipsychotics versus Injectable Paliperidone Palmitate (PROSIPAL) study, 85% of PP patients were relapse-free at 469 days versus 249 days for the 85th percentile of OAT patients (p = 0.019) [22]. Similarly, the PRIDE study demonstrated PP superiority compared to OATs in delaying time to treatment failure during a 15-month period (PP: 39.8%; OAT: 53.7%) [23]. In addition, a higher proportion of PP users (95.2%) in PRIDE were adherent (MPR ≥80%) relative to OAT users (77.2%) [23]. Our findings confirm improved treatment adherence outcomes and extend beyond the PRIDE population of previously incarcerated patients.

Finally, in a claims analysis by Marcus et al., PP was significantly protective against nonadherence (PDC <80%, OR = 0.34, 95% CI: 0.22–0.54), discontinuation (≥60-day gap, OR = 0.39, 95% CI: 0.23–0.66), and rehospitalization (OR = 0.53, 95% CI: 0.30–0.94), as compared to OATs [44]. Our research complements these existing findings in suggesting the superiority of PP over OATs with respect to remission and treatment outcomes in a CBHO population.

Our use of REFS was advantageous in that it identified relevant predictors of study outcomes with more resolution and rigor than other available variable selection methods. REFS generates an ensemble (series) of prediction models, based on Bayesian mathematics, as opposed to a single maximum likelihood model, which can be susceptible to misspecification or chance (particularly in smaller data sets). From the distribution of effect estimates in the resulting ensembles, the degree of uncertainty of an effect estimate could be more closely assessed. Given the unique nature of the data collected in REACH-OUT, we believe this empirical, data-driven approach suitably complemented previous literature in specifying the final models presented here, with the goal of optimizing predictive value.

There are some limitations that should be considered when interpreting the results of our study. First, statistical power may have been suboptimal due to a relatively small sample size (particularly for PP initiators) and a limited follow-up period. However, we were able to detect several statistically significant differences in outcomes between the study cohorts. Second, the cohort of PP users was a combination of PP-N and PP-C users. To address this discrepancy, we adjusted for new and continuing PP status in primary models. OAT users were exclusively initiators at enrollment. Due to the design of the chart review form, these patients could not be further stratified by type of OAT, which may highlight additional treatment differences associated with the study outcomes. Third, calculations of adherence were inherently inconsistent due to the nature of LAI vs oral antipsychotic medications. Specifically, we have assumed PP users were covered for 35 days, where treatment coverage for OAT users was determined by days of supply. However, days’ supply may overestimate adherence among OAT users, since their actual fulfillment of the treatment was not captured. Finally, PDC was calculated for all eligible participants at enrollment, regardless of actual follow-up status, which may or may not have been differential by treatment status. To validate PDC calculations, we have also estimated MPR, which was applied only while patients were specifically under observation.

In this study of schizophrenia patients receiving care at CBHOs, our results suggest that PP users benefit from treatment stability as well as improved clinical management of symptoms. These findings were generally confirmed in our comparisons of initiators of PP versus initiators of OAT, suggesting potential clinical utility for PP beyond salvage therapy for patients who fail on OAT regimens. All analyses are exploratory and to be interpreted with caution. Further studies of the impact of LAI use on clinical outcomes of schizophrenia in alternative study populations are warranted.


Long-acting injectable PP was associated with improved adherence and lower treatment discontinuation. Additionally, PP users demonstrated significant association with achievement of disease remission relative to OAT users in this study of adults with schizophrenia receiving care at CBHOs.



assertive community treatment


area under the curve


Clinical Antipsychotic Trials of Intervention Effectiveness study


Community Behavioral Health Organization


confidence interval


Drug Attitude Inventory


hazard ratio


long-acting injectable


medication possession ratio


oral antipsychotic therapy


odds ratio


Positive and Negative Syndrome Scale


proportion of days covered


paliperidone palmitate


continuous user of paliperidone palmitate


new user of paliperidone palmitate


Paliperidone Palmitate Research in Demonstrating Effectiveness study


Prevention of Relapse with Oral Antipsychotics versus Injectable Paliperidone Palmitate


Personal and Social Performance scale


Research and Evaluation of Antipsychotic Treatment in Community Behavioral Health Organizations

Outcomes study


Reverse Engineering and Forward Simulation


Structured Clinical Interview for Symptoms of Remission


Scale to Assess Therapeutic Relationships – Clinician Version


Scale to Assess Therapeutic Relationships – Patient Version


  1. National Institute of Mental Health. Schizophrenia. Last revised November 2016. Accessed 9 Nov 2016.

  2. DiBonaventura M, Gabriel S, Dupclay L, Gupta S, Kim EA. Patient perspective of the impact of medication side effects on adherence: results of a cross-sectional nationwide survey of patients with schizophrenia. BMC Psychiatry. 2012;12:20. doi:10.1186/1471-244X-12-20.

    Article  PubMed  PubMed Central  Google Scholar 

  3. Emsley R, Chiliza B, Asmal L, Harvey BH. The nature of relapse in schizophrenia. BMC Psychiatry. 2013;13:50. doi:10.1186/1471-244X-13-50.

    Article  PubMed  PubMed Central  Google Scholar 

  4. Higashi K, Medic G, Littlewood KJ, Diez T, Granström O, De Hert M. Medication adherence in schizophrenia: factors influencing adherence and consequences of nonadherence, a systematic literature review. Ther Adv Psychopharmacol. 2013;3:200–18. doi:10.1177/2045125312474019.

    Article  PubMed  PubMed Central  Google Scholar 

  5. Ziedonis DM, Smelson D, Rosenthal RN, Batki SL, Green AI, Henry RJ, et al. Improving the care of individuals with schizophrenia and substance use disorders: consensus recommendations. J Psychiatr Pract. 2005;11:315–39.

    Article  PubMed  PubMed Central  Google Scholar 

  6. Lafeuille M-H, Gravel J, Lefebvre P, Fastenau J, Muser E, Doshi D, Duh MS. Patterns of relapse and associated cost burden in schizophrenia patients receiving atypical antipsychotics. J Med Econ. 2013;16:1290–9. doi:10.3111/13696998.2013.841705.

    Article  PubMed  Google Scholar 

  7. Fitch K, Iwasaki K, Villa KF. Resource utilization and cost in a commercially insured population with schizophrenia. Am Health Drug Benefits. 2014;7:18–26.

    PubMed  PubMed Central  Google Scholar 

  8. Offord S, Wong B, Mirski D, Baker RA, Lin J. Healthcare resource usage of schizophrenia patients initiating long-acting injectable antipsychotics vs oral. J Med Econ. 2013;16:231–9. doi:10.3111/13696998.2012.751025.

    Article  PubMed  Google Scholar 

  9. Kane JM, Kishimoto T, Correll CU. Assessing the comparative effectiveness of long-acting injectable vs oral antipsychotic medications in the prevention of relapse provides a case study in comparative effectiveness research in psychiatry. J Clin Epidemiol. 2013;66:S37–41. doi:10.1016/j.jclinepi.2013.01.012.

    Article  PubMed  PubMed Central  Google Scholar 

  10. Nasrallah HA, Lasser R. Improving patient outcomes in schizophrenia: achieving remission. J Psychopharmacol. 2006;20:57–61. doi:10.1177/1359786806071248.

    Article  PubMed  Google Scholar 

  11. Haddad PM, Taylor M, Niaz OS. First-generation antipsychotic long-acting injections v. Oral antipsychotics in schizophrenia: systematic review of randomised controlled trials and observational studies. Br J Psychiatry Suppl. 2009;52:S20–8. doi:10.1192/bjp.195.52.s20.

    Article  PubMed  Google Scholar 

  12. Lafeuille M-H, Laliberté-Auger F, Lefebvre P, Frois C, Fastenau J, Duh MS. Impact of atypical long-acting injectable versus oral antipsychotics on rehospitalization rates and emergency room visits among relapsed schizophrenia patients: a retrospective database analysis. BMC Psychiatry. 2013;13:221. doi:10.1186/1471-244X-13-221.

    Article  PubMed  PubMed Central  Google Scholar 

  13. Tiihonen J, Haukka J, Taylor M, Haddad PM, Patel MX, Korhonen PA. Nationwide cohort study of oral and depot antipsychotics after first hospitalization for schizophrenia. Am J Psychiatry. 2011;168:603–9. doi:10.1176/appi.ajp.2011.10081224.

    Article  PubMed  Google Scholar 

  14. Tiihonen J, Wahlbeck K, Lönnqvist J, Klaukka T, Ioannidis JP, Volavka J, Haukka J. Effectiveness of antipsychotic treatments in a nationwide cohort of patients in community care after first hospitalisation due to schizophrenia and schizoaffective disorder: observational follow-up study. BMJ. 2006;333:224. doi:10.1136/bmj.38881.382755.2F.

    Article  PubMed  PubMed Central  Google Scholar 

  15. Lieberman JA, Stroup TS, McEvoy JP, Swartz MS, Rosenheck RA, Perkins DO, et al. Effectiveness of antipsychotic drugs in patients with chronic schizophrenia. N Engl J Med. 2005;363:1092–3. doi:10.1056/NEJMx100054.

    Google Scholar 

  16. Alphs L, Bossie CA, Sliwa JK, DJ F, Ma YW, Hulihan J. Paliperidone palmitate and risperidone long-acting injectable in subjects with schizophrenia recently treated with oral risperidone or other oral antipsychotics. Neuropsychiatr Dis Treat. 2013;9:341–50. doi:10.2147/NDT.S36438.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  17. Hough D, Gopal S, Vijapurkar U, Lim P, Morozova M, Eerdekens M. Paliperidone palmitate maintenance treatment in delaying the time-to-relapse in patients with schizophrenia: a randomized, double-blind, placebo-controlled study. Schizophr Res. 2010;116:107–17. doi:10.1016/j.schres.2009.10.026.

    Article  PubMed  Google Scholar 

  18. Kim S, Solari H, Weiden PJ, Bishop JR. Paliperidone palmitate injection for the acute and maintenance treatment of schizophrenia in adults. Patient Prefer Adherence. 2012;6:533–45. doi:10.2147/PPA.S20657.

    PubMed  PubMed Central  Google Scholar 

  19. Kramer M, Litman R, Hough D, Lane R, Lim P, Liu Y, Eerdekens M. Paliperidone palmitate, a potential long-acting treatment for patients with schizophrenia. Results of a randomized, double-blind, placebo-controlled efficacy and safety study. Int J Neuropsychopharmacol. 2009;13:635–47. doi:10.1017/S1461145709990988.

    Article  PubMed  Google Scholar 

  20. Li H, Rui Q, Ning X, Xu H, Gu NA. Comparative study of paliperidone palmitate and risperidone long-acting injectable therapy in schizophrenia. Prog Neuro-Psychopharmacol Biol Psychiatry. 2011;35:1002–8. doi:10.1016/j.pnpbp.2011.02.001.

    Article  CAS  Google Scholar 

  21. Pandina G, Lane R, Gopal S, Gassmann-Mayer C, Hough D, Remmerie B, Simpson GA. Double-blind study of paliperidone palmitate and risperidone long-acting injectable in adults with schizophrenia. Prog Neuro-Psychopharmacol Biol Psychiatry. 2011;35:218–26. doi:10.1016/j.pnpbp.2010.11.008.

    Article  CAS  Google Scholar 

  22. Schreiner A, Aadamsoo K, Altamura AC, Franco M, Gorwood P, Neznanov NG, et al. Paliperidone palmitate versus oral antipsychotics in recently diagnosed schizophrenia. Schizophr Res. 2015;169:393–9. doi:10.1016/j.schres.2015.08.015.

    Article  PubMed  Google Scholar 

  23. Alphs L, Benson C, Cheshire-Kinney K, Lindenmayer JP, Mao L, Rodriguez SC, Starr HL. Real-world outcomes of paliperidone palmitate compared to daily oral antipsychotic therapy in schizophrenia: a randomized, open-label, review board-blinded 15-month study. J Clin Psychiatry. 2015;76:554–61. doi:10.4088/JCP.14m09584.

    Article  PubMed  Google Scholar 

  24. Gopal S, Vijapurkar U, Lim P, et al. A 52-week open-label study of the safety and tolerability of paliperidone palmitate in patients with schizophrenia. J Psychopharmacol. 2011;25:685–97. doi:10.1177/0269881110372817.

    Article  CAS  PubMed  Google Scholar 

  25. Owen RT. Paliperidone palmitate injection: its efficacy, safety and tolerability in schizophrenia. Drugs Today. 2010;46:463–71. doi:10.1358/dot.2010.46.7.1514647.

    Article  CAS  PubMed  Google Scholar 

  26. Edwards NC, Muser E, Doshi D, Fastenau J. The threshold rate of oral atypical anti-psychotic adherence at which paliperidone palmitate is cost saving. J Med Econ. 2012;15:623–34. doi:10.3111/13696998.2012.667465.

    Article  PubMed  Google Scholar 

  27. US Government Publishing Office. Mental Retardation Facilities and Community Mental Health Centers Construction Act of 1963. Public Law 88–164. Published October 31, 1963. Accessed 27 Apr 2016.

  28. Joshi K, Mao L, Biondi DM, Millet R. The research and evaluation of antipsychotic treatment in community behavioral health organizations, outcomes (REACH-OUT) study: real-world clinical practice in schizophrenia. Innov Clin Neurosci. 2016. In press.

  29. Andreasen NC, Carpenter WT Jr, Kane JM, Lasser RA, Marder SR, Weinberger DR. Remission in schizophrenia: proposed criteria and rationale for consensus. Am J Psychiatry. 2005;162:441–9. doi:10.1176/appi.ajp.162.3.441.

    Article  PubMed  Google Scholar 

  30. Anderson AL, McNeil DC, Reddon JR. Evaluation of Lehman's brief quality of life interview in assessing outcome in psychiatric rehabilitation in people with severe and persistent mental disorder. Soc Work Ment Health. 2008;1:43–59. doi:10.1300/J200v01n02_04.

    Article  Google Scholar 

  31. Dan A, Kumar S, Avasthi A, Grover SA. Comparative study on quality of life of patients of schizophrenia with and without depression. Psychiatry Res. 2011;189:185–9. doi:10.1016/j.psychres.2011.02.017.

    Article  PubMed  Google Scholar 

  32. Nielsen RE, Lindström E, Nielsen J, Levander S. DAI-10 is as good as DAI-30 in schizophrenia. Eur Neuropsychopharmacol. 2012;22:747–50. doi:10.1016/j.euroneuro.2012.02.008.

    Article  CAS  PubMed  Google Scholar 

  33. Vernon MK, Revicki DA, Awad AG, Dirani R, Panish J, Canuso CM, et al. Psychometric evaluation of the medication satisfaction questionnaire (MSQ) to assess satisfaction with antipsychotic medication among schizophrenia patients. Schizophr Res. 2010;118:271–8. doi:10.1016/j.schres.2010.01.021.

    Article  PubMed  Google Scholar 

  34. Nicholl D, Nasrallah H, Nuamah I, Akhras K, Gagnon DD, Gopal S. Personal and social functioning in schizophrenia: defining a clinically meaningful measure of maintenance in relapse prevention. Curr Med Res Opin. 2010;26:1471–84. doi:10.1185/03007991003798927.

    Article  CAS  PubMed  Google Scholar 

  35. McGuire-Snieckus R, McCabe R, Catty J, Hansson L, Priebe SA. New scale to assess the therapeutic relationship in community mental health care: STAR. Psychol Med. 2007;37:85–95. doi:10.1017/S0033291706009299.

    Article  PubMed  Google Scholar 

  36. Anderson JP, Parikh JR, Shenfeld DK, Ivanov V, Marks C, Church BW, et al. Reverse engineering and evaluation of prediction models for progression to type 2 diabetes: an application of machine learning using electronic health records. J Diabetes Sci Technol. 2015;10:6–18. doi:10.1177/1932296815620200.

    Article  PubMed  PubMed Central  Google Scholar 

  37. Steinberg GB, Church BW, McCall CJ, Scott AB, Kalis BP. Novel predictive models for metabolic syndrome risk: a “big data” analytic approach. Am J Manag Care. 2014;20:e221–8.

    PubMed  Google Scholar 

  38. Xing H, McDonagh PD, Bienkowska J, Cashorali T, Runge K, Miller RE, et al. Causal modeling using network ensemble simulations of genetic and gene expression data predicts genes involved in rheumatoid arthritis. PLoS Comput Biol. 2011;7:e1001105. doi:10.1371/journal.pcbi.1001105.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  39. Carbon M, Correll CU. Clinical predictors of therapeutic response to antipsychotics in schizophrenia. Dialogues Clin Neurosci. 2014;16:505–24.

    PubMed  PubMed Central  Google Scholar 

  40. Gilmer TP, Dolder CR, Lacro JP, Folsom DP, Lindamer L, Garcia P, Jeste DV. Adherence to treatment with antipsychotic medication and health care costs among Medicaid beneficiaries with schizophrenia. Am J Psychiatry. 2004;161:692–9. doi:10.1176/appi.ajp.161.4.692.

    Article  PubMed  Google Scholar 

  41. Weiden PJ, Kozma C, Grogg A, Locklear J. Partial compliance and risk of rehospitalization among California Medicaid patients with schizophrenia. Psychiatr Serv. 2004;55:886–91. doi:10.1176/

    Article  PubMed  Google Scholar 

  42. Wyatt RJ. Neuroleptics and the natural course of schizophrenia. Schizophr Bull. 1991;17:325–51.

    Article  CAS  PubMed  Google Scholar 

  43. Hui C. Relapse in schizophrenia. Hong Kong Med Diary. 2011;16:9–10.

    Google Scholar 

  44. Marcus SC, Zummo J, Pettit AR, Stoddard J, Doshi JA. Antipsychotic adherence and rehospitalization in schizophrenia patients receiving oral versus long-acting injectable antipsychotics following hospital discharge. J Manag Care Spec Pharm. 2015;21:754–68.

    Article  PubMed  Google Scholar 

Download references


The authors thank Neeta Tandon for her senior HECOR review and Matthew Grzywacz, PhD, of ApotheCom (Yardley, PA, USA) for his editorial assistance.


All funding for this study and for the editorial support used in the creation of this manuscript was provided by Janssen Scientific Affairs (Titusville, NJ) and GNS Healthcare (Cambridge, MA).

Availability of data and materials

Janssen is committed to data transparency and the responsible sharing of clinical trial data. If the journal is requesting the data tables underlying the results of our manuscript, they are included in the submission. If the journal is asking for the actual database (patient level data), Janssen makes their trial data available through the Yale Open Data Access (YODA) project. Requests can be made at:

Author information

Authors and Affiliations



JPA: Conception, design and/or planning of the study, analysis of the data interpretation of the results and drafting of the manuscript. ZI: Analysis of the data, interpretation of the results, and drafting of the manuscript. VA: Conception, design and/or planning of the study, analysis of the data and interpretation of the results and drafting of the manuscript. CB: Conception, design and/or planning of the study, analysis of the data, interpretation of the results and drafting of manuscript. KJ: Conception, design and/or planning of the study, analysis of the data and interpretation of the results, and drafting of the manuscript. All authors were involved in critically revising the manuscript for important intellectual content; have given final approval of the version to be published. They have participated sufficiently in the work to take public responsibility for appropriate portions of the content; and have agreed to be accountable for all aspects of the work in ensuring that questions related to the accuracy or integrity of any part of the work are appropriately investigated and resolved. All authors read and approved the final manuscript.

Corresponding author

Correspondence to Veronica Alas.

Ethics declarations

Ethics approval and consent to participate

The study was approved by the New England Institutional Review Board, and conducted in accordance with the ethical principles of the Declaration of Helsinki. Written informed consent was obtained from all subjects prior to study enrollment.

Consent for publication

Not applicable.

Competing interests

All authors were employed by GNS Healthcare or Janssen Scientific Affairs, LLC, at the time this research was conducted. KJ and CB are Johnson & Johnson stockholders. REFS™ is proprietary to GNS Healthcare.

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0 International License (, which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The Creative Commons Public Domain Dedication waiver ( applies to the data made available in this article, unless otherwise stated.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Anderson, J.P., Icten, Z., Alas, V. et al. Comparison and predictors of treatment adherence and remission among patients with schizophrenia treated with paliperidone palmitate or atypical oral antipsychotics in community behavioral health organizations. BMC Psychiatry 17, 346 (2017).

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: