Randomised controlled trial of a digitally assisted low intensity intervention to promote personal recovery in persisting psychosis: SMART-Therapy study protocol
- Neil Thomas1, 2Email authorView ORCID ID profile,
- John Farhall3, 4,
- Fiona Foley1,
- Susan L. Rossell1, 2, 5,
- David Castle5, 6,
- Emma Ladd7,
- Denny Meyer1,
- Cathrine Mihalopoulos8,
- Nuwan Leitan1,
- Cassy Nunan7,
- Rosalie Frankish7,
- Tara Smark7,
- Sue Farnan7,
- Bronte McLeod1,
- Leon Sterling9, 10,
- Greg Murray1,
- Ellie Fossey11, 12,
- Lisa Brophy13, 14 and
- Michael Kyrios1, 15
© The Author(s). 2016
Received: 29 August 2016
Accepted: 30 August 2016
Published: 7 September 2016
Psychosocial interventions have an important role in promoting recovery in people with persisting psychotic disorders such as schizophrenia. Readily available, digital technology provides a means of developing therapeutic resources for use together by practitioners and mental health service users. As part of the Self-Management and Recovery Technology (SMART) research program, we have developed an online resource providing materials on illness self-management and personal recovery based on the Connectedness-Hope-Identity-Meaning-Empowerment (CHIME) framework. Content is communicated using videos featuring persons with lived experience of psychosis discussing how they have navigated issues in their own recovery. This was developed to be suitable for use on a tablet computer during sessions with a mental health worker to promote discussion about recovery.
This is a rater-blinded randomised controlled trial comparing a low intensity recovery intervention of eight one-to-one face-to-face sessions with a mental health worker using the SMART website alongside routine care, versus an eight-session comparison condition, befriending. The recruitment target is 148 participants with a schizophrenia-related disorder or mood disorder with a history of psychosis, recruited from mental health services in Victoria, Australia. Following baseline assessment, participants are randomised to intervention, and complete follow up assessments at 3, 6 and 9 months post-baseline. The primary outcome is personal recovery measured using the Process of Recovery Questionnaire (QPR). Secondary outcomes include positive and negative symptoms assessed with the Positive and Negative Syndrome Scale, subjective experiences of psychosis, emotional symptoms, quality of life and resource use. Mechanisms of change via effects on self-stigma and self-efficacy will be examined.
This protocol describes a novel intervention which tests new therapeutic methods including in-session tablet computer use and video-based peer modelling. It also informs a possible low intensity intervention model potentially viable for delivery across the mental health workforce.
NCT02474524, 24 May 2015, retrospectively registered during the recruitment phase.
KeywordsRandomised controlled trial (RCT) Psychosis Schizophrenia Digital health e-therapy Tablet computers Peer support Illness self-management Personal recovery
In spite of early intervention and the routine use of antipsychotic medication, many people with psychotic disorders live with persisting symptoms and disability . For this group, alongside good clinical care and treatment, interventions are needed to help people to self-manage and adjust to persisting symptoms, and to maximise quality of life. Psychotherapeutic interventions such as cognitive behavioural therapy for psychosis have been heavily recommended in clinical practice guidelines [2–4]. However, psychotherapies, which require lengthy therapist training, have proven hard to deliver on a widespread routine basis within mental health services [5–8], and are provided only to a small minority of consumers . In addressing need, there may be value in developing interventions that make use of structured therapeutic resources that support practitioners in their delivery, and which can be delivered with a combination of in-session and self-guided use. These interventions, often referred to as low intensity interventions , are amenable to delivery without extensive specific training or prerequisite skill, and have the potential to be delivered by a broad mental health workforce, allowing for greater integration into routine service provision.
Group/classroom-format illness self-management courses accompanied by workbooks have already shown promise [11–15]. In parallel, online digital tools are beginning to be developed for psychosis [16–18]. Integrating digital tools into face-to-face mental health practice has significant implementation potential. The blending of face-to-face and online service delivery allows for practitioner support in using a digital tool, promotes more in-depth and individualised reflection upon standardised content, and helps to sustain engagement with materials. Strengths of the digital format include being easily disseminated, being directly accessible by consumers, offering more interactive therapeutic materials than workbooks, allowing for mobile device use to support self-management throughout the day, and potentially allowing peer-to-peer interaction to promote the development of supportive peer communities [16, 18].
Although initially having much poorer access to the Internet than the broader population , the majority of consumers with persisting psychosis are now online, particularly via internet-enabled smartphones [20, 21], and consumers have expressed positive attitudes towards use of digital technology in mental health care . Initial trials of digital interventions have shown self-guided use to be feasible . Similarly, workers within mental health services welcome the in-session use of digital technology as part of their work with this population . To test the integration of digital technology into routine mental health interactions, we sought to develop and trial a manualised therapeutic intervention that blended face-to-face interaction with a digital tool, and that could be used by mental health workers without an advanced level of training.
In developing such an intervention, a key therapeutic target is personal recovery, a domain which is now prioritised in mental health services worldwide [24–30]. Personal recovery is often contrasted with the traditional treatment targets of clinical recovery (minimisation of symptoms), and functional recovery (improving social functioning), as a process of living a full life irrespective of ongoing symptoms or disability [30–32]. Anthony  defines personal recovery as “a deeply personal, unique process of changing one's attitudes, values, feelings, goals, skills and/or roles. It is a way of living a satisfying, hopeful, and contributing life even with limitations caused by the illness” (p. 527). This conceptualisation of recovery has been strongly influenced by consumer accounts of their own recovery [30, 32]. Although recovery is typically characterised as an individual journey, synthesis of mental health service consumer accounts has led to consensus around particular processes as important in promoting personal recovery: becoming empowered and taking responsibility for self-managing mental health; developing hope and optimism; establishing a self-identity accepting of but not defined by illness; developing new meaning and life goals; and connecting with others [34–36].
How mental health services can most effectively support these recovery processes is little researched as yet. Although staff training in recovery-oriented practice is a common model, a major trial of training in recovery-oriented practice observed limited impact on a measure of these recovery processes . On the other hand, positive outcomes for measures of recovery have been found for group-format self-management interventions that incorporate material on recovery [11–13]. In the most widely used interventions, lived experience of mental health problems plays a key role in the intervention, with the intervention commonly facilitated by a peer who is in recovery, and with the group format being used to elicit discussion among peers . This accords with a recovery literature highlighting peer contact particularly important in promoting recovery from psychosis . Specifically, qualitative studies suggest peers providing positive role models, peers inspiring hope and motivation, and peer-to-peer interaction promoting social connection are core to the benefits of peer involvement .
Use of a digital medium provides opportunities to bring peer lived experience into the context of face-to-face mental health interactions with non-peer mental health practitioners. Audiovisual media can be utilised to feature peers as the main communicators of content, consistent with the origins of the concept of personal recovery in consumer narratives. Doing so potentially supports the benefits of peer positive role modelling and hope , and portraying peers discussing their recovery, in a humanised and dignified way, may be empowering and challenging of negative stereotypes about mental illness . In addition, the Internet can be used to support people using the tool to communicate with each other, potentially allowing for contribution of personal content, and the formation of supportive peer communities .
This study (acronym SMART-Therapy) is part of a broader research program (Self-Management and Recovery Technology) developing and trialling digital resources for specialist mental health services. It is a randomised controlled trial of a discrete psychosocial intervention to promote personal recovery in people with persisting psychosis. The intervention adopts a low intensity delivery model blending a course of individual face-to-face sessions with use of a website featuring lived experience content on recovery. The primary objective is to determine if this intervention will be superior to a time-equivalent comparison intervention that incorporates exploration of non-recovery-related digital resources on a measure of consumer-defined personal recovery. We will also examine a number of secondary outcomes, interview people about their experience of the intervention, use data as part of a broader economic evaluation, and examine mechanisms of change.
In examining the mechanisms of the intervention, we hypothesise effects on personal recovery via two measurable processes: (a) mental illness self-stigma and (b) self-efficacy for mental health recovery. Self-stigma, also referred to as internalised stigma, refers to the extent to which an individual adopts negative community stereotypes about mental illness and the extent to which these exert an influence on experience and behaviour . For persons with persisting psychosis, these may include stereotypes of being fundamentally different from others, of inherent dangerousness, of pessimistic long-term outcomes, and of being insufficiently capable of fulfilling social roles such as parenting and work . Internalised stigma includes components such as negative stereotype endorsement, alienation and social withdrawal . In studies of public stigma, contact with persons with mental illness that undermines negative stereotypes and humanises persons with a mental illness appears effective in reducing negative stereotypes . We hypothesise that positive portrayal of peers may have a parallel effect on internalised stigma .
A concept derived from social learning theory, self-efficacy refers to the expectancies that an individual has that they will be able to successfully engage in a target behaviour . Self-efficacy is a robust predictor of health behaviour change . In line with social learning theory we predict that observing peers discussing engagement in recovery oriented behaviours will facilitate behaviour change in our participants via increased expectancies of being able to engage in similar behaviours.
The SMART-Therapy trial is designed as a randomised, controlled, assessor-blinded superiority trial, with two parallel groups, using a 1:1 allocation ratio. The two groups both receive eight sessions with a mental health worker in addition to their routine care during a three month window, with one group being delivered the recovery intervention, and the other being delivered a befriending intervention as a comparison condition. Outcome measures are completed at baseline (prior to randomisation) and are repeated at 3, 6 and 9 months following baseline.
The study is being conducted across multiple public community mental health services in the state of Victoria, Australia, including clinical mental health services, and not-for-profit organisation-run community mental health support services. Potential participants are identified via referral from practitioners in collaborating services, supplemented by review of case-lists, plus advertising within services, and in print and online social media.
Inclusion criteria are: (a) age between 18 and 65 years inclusive; (b) diagnosis of a nonorganic psychotic disorder (schizophrenia-related disorder or bipolar disorder or major depressive disorder with psychotic features present within the past 2 years), confirmed using the Structured Clinical Interview for DSM-IV-TR Axis I Disorders (SCID) ; (c) sufficient conversational English for meaningful participation; (d) overall intellectual functioning within normal limits (having an IQ greater than 70, as estimated by the Wechsler Test of Adult Reading (WTAR) ) to ensure they have the cognitive capacity to provide consent, in addition to being able to adequately engage with the intervention. Exclusion criteria are: (e) initiation of a new antipsychotic medication, or commencement or completion of a formal psychological treatment, within the previous 8 weeks; (f) inpatient admission within the previous 8 weeks.
The recruitment target is 148 participants, providing 80 % power to detect medium effect sizes (d = 0.5) at α = .05 with 15 % attrition. This was based on an average effect size of d = 0.53 observed in a meta-analysis of online interventions .
Both conditions involve delivery of eight face-to-face one-to-one sessions of 50 min duration. Interventions are delivered by mental health support workers who are seconded to the project from roles in the not-for-profit community mental health support services sector. Workers in this sector were chosen to ensure that the intervention is suitable for delivery by the whole mental health workforce, rather than being restricted to postgraduate trained health professionals. These mental health workers are required to have a certificate level of training in mental health and at least 2 years of experience. They facilitate sessions for both conditions. Sessions are held alongside the participant’s usual routine treatment and care, which continues to be provided by their existing treating team without restriction from participation in the trial. Routine care would typically include antipsychotic medication, crisis support, symptom monitoring and case management, and may include receipt of formal psychological therapies.
SMART sessions are structured around use of the SMART website on a tablet computer, which is used by the facilitator and participant together during sessions. The SMART website is Drupal-based, optimised for tablet computer or smartphone use, and accessible via any internet browser including home computer.
Content domains are primarily based on the Connectedness–Hope–Identity–Meaning–Empowerment (CHIME) framework, which has been derived as a synthesis of consumer accounts of key processes involved in recovery . This is supplemented by basic symptom monitoring, coping enhancement, and behaviour change material from cognitive behavioural therapies. This creates seven self-management and recovery topics: (a) recovery, (b) managing stress, (c) physical health, (d) me, (e) empowerment, (f) relationships, and (g) life. Material for each topic is organized primarily around a series of videos, including a video introduction to the topic by a mental health consumer leader, followed by a combination of videos, text, and reflective exercises. The main videos are 2–3 min in length, each featuring a selection of persons with lived experience of psychosis reflecting on their experiences in relation to a particular aspect of recovery, and how they have navigated their mental health recovery in relation to this. Topics also include expert videos (a combination of peer leaders and non-peer academics) discussing conceptual points in more detail. Participants can share comments on each video with other users of the website in a comments feed. Due to the highly individual nature of recovery, and feedback during the consultation phase, resources were developed to include a pool of material that would be utilised in a flexible manner according to the participant’s own goals and recovery priorities. Participants’ entries for key exercises in each topic populate a “roadmap” page acting as a personalised summary of key points. Each topic also concludes by encouraging the person to identify key changes they intend to make, used to populate a task list. The site also contains a charting tool for self-monitoring stress, mood, sleep, physical health and self-esteem, and a peer-moderated member forum. Further details on the site development are detailed elsewhere .
Sessions involve use of the website to stimulate discussion about self-management and recovery. The worker has the following roles: (a) using website material to encourage the participant to reflect upon their own recovery, (b) facilitating the participant forming intentions to make changes on the basis of what has been discussed, (c) promoting familiarity with the site to independently use its features, and (d) encouraging the participant to use the site between sessions. The initial session includes orienting the participant to both the website and the concept of personal recovery. Participants are provided with their own login to the SMART website, and are introduced to the navigation, supported by a video on the site. This is followed by videos on the concept of personal recovery, and discussion about different material included in the website. This is used to identify priorities in the material for the therapist and participant to look at in subsequent sessions. In subsequent sessions, the worker and participant identify the area they want to focus on, with the core of the session involving a process of viewing website material together, using this as a prompt for discussion about the participant’s own recovery, and an invitation to complete reflective exercises on the site and/or make public posts about the content if willing. These sessions begin with a review of reflections since the last meeting, and any content which the person has used, and conclude by considering any changes the person would like to enact, and planning ongoing use of the site during the week.
Participants in the comparison condition receive sessions based upon a neutral social contact, involving shared activity and discussion about non-mental health topics such as current affairs, recent events and activities in the participant’s life, and the participant’s interests. This is a manualised intervention, the structure of which is based upon the befriending protocol of Bendall et al. , a credible and acceptable comparison condition , which has been used as a control condition to match for therapist time in a number of psychosocial intervention trials [54–56]. In this trial the standard protocol is augmented by use of the tablet computer to browse Internet sites related to participants’ interests, and as a potential means of facilitating discussion. This comparison condition was selected to provide a control for both therapist time, and for engagement in the use of technology.
In both groups, interventions are continued until, (a) all eight sessions are completed, (b) the end of the three month therapy delivery window is reached, or (c) the participant chooses to withdraw. If participants experience significant distress or deterioration in their mental state, researchers remind participants of their right to withdraw, and encourage them to make an informed choice about their ongoing participation. If participants wish to discontinue intervention sessions, they are still followed up for assessment if willing to do so.
Participant adherence to protocols will be monitored through records of session attendance. In secondary analyses, attendance of at least four sessions will be considered as the threshold for receiving a minimum dose of the intervention. Workers attend weekly supervision led by NT to ensure protocol fidelity. Worker fidelity will be further assessed by a researcher blind to participant allocation listening to a random sample of audio recordings of sessions, and being asked to allocate sessions to condition.
Allocation and blinding
The trial uses minimisation randomisation, conducted using the software Minimpy (http://sourceforge.net/projects/minimpy). This generates random allocations based on the stratification parameters for each participant enrolled in the study. The randomisation procedure and setup of the Minimpy parameters was prepared by the study statistician to stratify by type of mental health service the person receives (clinical vs community sector vs both), symptom severity (Positive and Negative Syndrome Scale (PANSS) total ≥ vs < 67), and frequency of Internet use (daily vs less than daily). Following commencement of the trial, the software is operated by a person independent of the research team, who receives participant codes from the research team as each baseline assessment is completed, runs the allocation, and advises of allocations via return email.
Research staff conducting assessments are blind to participant allocation throughout the study. Participants are unavoidably aware of the conditions they have been randomised to. To reduce expectancy effects for superiority of the intervention over the control condition, the trial was framed as a comparison between two ways of integrating technology into mental health service delivery, with a health condition using technology to discuss mental health recovery, and a social condition using technology to support discussion about the person’s interests. To maintain assessor blinding, participants are regularly reminded not to divulge their allocated condition or any details of their intervention to assessment staff, as are their mental health workers. Breaches in blindness are recorded and addressed by changing the rater whenever possible. Blindness is also assessed at each assessment following the intervention by asking raters to nominate a guessed treatment condition for the participant and to indicate their level of confidence. The principal investigator (NT) and project manager (FF) are not blinded, and respond to any clinical and research issues during the trial that require knowledge of a participant’s’ condition.
Schizophrenia Hope Scale
UCLA Loneliness Scale
Serious Adverse Events
Internet Use and Access
Recovery Style Questionnaire
Peer Identification Scale
Psychosis Attachment Measure
Medication and psychological treatment
Primary outcome measure
The primary outcome will be consumer-defined personal recovery, measured with the Process of Recovery Questionnaire (QPR ). This is a 22-item psychometrically validated self-report measure, developed collaboratively with mental health service consumers to measure recovery in people who experience psychosis . Items are scored using a 5-point Likert scale. This measure was selected due to good alignment with the CHIME model on which the intervention is based . The total score will be used as the main outcome measure, and the two subscales (for intrapersonal versus interpersonal recovery processes) will be reported.
Secondary outcome measures
Personal recovery dimensions
Two additional measures will be included for more focused measurement of specific personal recovery dimensions targeted by the intervention. Hope and optimism will be measured by the Schizophrenia Hope Scale, a validated 9-item self-report scale on which items are scored disagree, agree or strongly agree . Social connectedness will be indexed by version 3 of the UCLA Loneliness Scale , the most widely used measure of loneliness versus connection with others, which comprises 20 items which are self-rated on a 4-point response scale from never to always. The total scores on each of the measures will be used in analyses.
Psychotic symptom severity will be measured by the PANSS . The PANSS is a standardised interview-based measure of schizophrenia symptomatology widely used in treatment trials. It comprises seven items assessing positive symptoms, seven assessing negative symptoms and 16 items assessing general symptoms. The PANSS total score is used as part of the randomisation stratification, and the total and subscale scores will be analysed as secondary outcomes. Inter-rater reliability on the PANSS is checked every 3–6 months by research assistants co-rating face-to-face assessments, with recalibration of ratings occurring if necessary.
Although the PANSS has become a standard in psychosis treatment research, it has been criticised for providing a relatively indirect and insensitive index of the impact of psychosocial interventions on psychosis symptoms, where the impact of psychosis on distress and functioning is the key target, rather than symptom severity per se [8, 62]. Hence, the impact of psychotic symptoms will be measured by the Subjective Experience of Psychosis Scale (SEPS ). This is a psychometrically validated  questionnaire on which participants rate the current impact of psychotic symptoms on various aspects of distress, day-to-day functioning and subjective experience identified as important by mental health service users. The negative impact of psychosis is measured by the total of 29 5-point (not at all to very much negative) items. The SEPS is completed only by participants who have experienced psychotic symptoms in the week prior to the baseline assessment.
Emotional symptoms will be measured using the 21-item version of the Depression Anxiety Stress Scale (DASS-21, ), which consists of 3 subscales (Depression, Anxiety, Stress) comprising items rated on a 4-point scale of the extent to which the experience occurred over the past week from 0 (did not apply to me at all) to 3 (applied to me very much or most of the time). The DASS-21 is validated and psychometrically robust .
Quality of life
Quality of life will be measured using the Assessment of Quality of Life (AQoL)-8D  consisting of 35 items across eight dimensions comprisingphysical (independent living, pain, senses) and psychosocial (mental health, happiness, coping, relationships, self-worth) dimensions of quality of life. The AQoL-8D is comprehensively validated and demonstrates high test-retest reliability . The AQoL-8D total and physical and psychosocial scores will be used as a secondary outcome, and the utility scoring algorithm will be used to derive quality adjusted life years (QALYs) for an economic analysis which will integrate these data with those from other studies in the research program and be reported upon separately.
Collaboration with mental health services
We will also examine whether there are indirect effects of the intervention on the working relationship with the person’s usual services. This will be assessed using the Scale to Assess Therapeutic Relationships in Community Mental Health Care (STAR) Patient Version , a psychometrically validated self-report scale with 12 items on positive collaboration, positive clinician input, and non-supportive clinician input each rated from 0 (never) to 4 (always).
Subjective experience of intervention
At the three month assessment point, all participants complete questions asking “Overall, did the website make you feel better, or worse, or no different?” and “Do you feel that using the website made the impact of your mental health problems better, or worse, or no different?”, both answered on a 5-point scale ranging from much worse to much better. As well as providing an index of the subjective perceived benefit of the interventions, this measure will be used to assess for any perceived adverse effects. Although not a formally validated measure, analogous questions have proved sensitive to treatment condition in two previous trials of psychosis [55, 56]. In addition, a sample of SMART participants will be invited to take part in more detailed qualitative interviews about their experiences of the intervention, which will be reported upon separately.
Serious adverse events
In line with the Australian National Statement on Ethical Conduct in Human Research , in the context of this trial serious adverse events include events that lead to participant death, or that are life-threatening, require inpatient hospitalisation, or result in persistent or significant disability/incapacity. During the trial, potential serious adverse events are recorded, and are reviewed to determine whether they are likely to be related to interventions used in the trial. Analysis of outcomes will include a comparison of the number of participants in which an adverse event has occurred, and report of any adverse events attributed to either of the conditions.
Although differences in service use are not formally hypothesised as an outcome of the intervention, a resource use questionnaire (RUQ) is administered in order to assess for any differences in mental health service utilisation between the groups before, during, and following intervention. This will be used to derive costings for the economic analysis. The RUQ was modified for the local context of the study, assessing use of mental health services in Victoria including hospital admissions (number and days) and consultations with mental health service providers (number and length). This will be supplemented by Australian Government Medicare data: these administrative data are complete, but do not include the full spectrum of services which participants in this study are likely to use.
Self-stigma will be measured by the Internalised Stigma of Mental Illness Scale (ISMI), a psychometrically validated self-report measure with 29 4-point strongly agree to strongly disagree items which provide a total, and separate scores for Alienation, Stereotype Endorsement, Discrimination Experience, Social Withdrawal and Stigma Resistance .
Self-efficacy for personal recovery is measured by the item “How confident are you that in the future you will be able to live a satisfying life alongside any mental health problems you may have” rated from 0 (not confident I can do this at all) to 100 (highly confident I can do this). Also included, using the same response scale, are a self-efficacy for illness self-management item (“How confident are you that you can do things to manage any future mental health difficulties”), and a series of 12 items relating to content of the website. A 0–100 rating of confidence in achieving full clinical recovery is also administered to assess discrimination from this construct. To assist with validation of this new measure, the widely used Generalised Self-Efficacy Scale (GSES ) is also completed.
Covariates and moderating variables
Demographics collected include age, gender, employment, education, country of birth and ethnicity. Clinical variables include SCID diagnosis, age at first specialist mental health service contact, past psychiatric admissions, and past receipt of psychiatric treatment on an involuntary basis. Ratings are also obtained of level of independent use of the Internet and email (each rated as use without assistance, use with occasional assistance, use only with assistance); frequency of use of the Internet, email, and social media; and membership of mental health related forums or social networks. Additionally the following variables are collected:
Recovery style refers to individual differences in adaptive coping during recovery from psychosis, which range from an integrative style in which the person approaches their psychotic experiences with curiosity and interest, to a sealing over style in which they are encapsulate their psychosis and see it as separate . Past studies have suggested that persons with a sealing over style benefit less from interventions that involve discussion of mental health  making this a likely moderator of outcome. This will be measured using the Recovery Style Questionnaire, a self-report measure of 39 true/false items with good psychometric properties .
Mental health peer contact and identification
Because the intervention is primarily based upon peer delivery, its effects may be influenced by the extensiveness and quality of prior peer contact, and the degree to which the person identifies with mental health consumer peers. Positive peer contact is assessed using a brief description of experiencing a positive recovery from psychosis, to which the person indicates how many people they have encountered fitting this description on a 4-point ordinal scale. Peer group identification is assessed on a scale adapted from that of Watson et al. . Items, rated on a 9-point scale from not at all to very much, ask the extent to which the person identifies with, feels strong ties with, and sees themselves as part of the group of people that might be referred to as mental health consumers/patients/service users; how often they think about themselves as part of this group; and how close they feel to other members of the group. Higher scores indicate greater group identification. The original adaptation of this scale has good internal consistency .
Because of the explicitly interpersonal nature of the intervention, both in the form of the face-to-face sessions, and the use of peer material, attachment style is included as a potential moderator. This is measured by the Psychosis Attachment Measure , a brief 16-item self-report questionnaire designed to assess avoidant and anxious attachment styles in persons with psychotic disorders.
To determine whether cognitive function limits ability to benefit from resources, the WTAR will be used to control for premorbid IQ, in combination with tests of current cognitive functioning. The WTAR is administered as a test of premorbid IQ where participants are asked to read a list of 50 words that have atypical grapheme to phoneme translations (e.g., “liaison”, “ paradigm”) and are scored according to accuracy of pronunciation (raw score range 0–50). WTAR raw scores are standardised based on age then converted to the predicted WAIS-III IQ. While predicted IQ scores will be reported descriptively, these are associated with very large confidence intervals; standardised scores will therefore be used for analyses due to their greater precision. A small battery of cognitive functioning tests will additionally be administered. These include Digit Symbol Coding , Animal Fluency, Trails A and B , Digits Forward and Digits Backward. These tests assess the cognitive domains of executive function, verbal working memory, cognitive processing speed, task-switching and visual attention. Each assessment involves completion of paper and pencil cognitive assessments which are among the most frequently used in research and clinical practice and are well standardised with accepted norms. The battery takes a total of 8–10 min to complete. Each task will be briefly described below:
Digit Symbol Coding : Participants are asked to use a key to write the numbers in a grid that correspond to nonsense symbols over a 90 s period. The total number of correct responses in used as the performance measure.
Animal Fluency: Participants are asked to give the names of as many animals as they can think of within a one-minute period. The number of unique animals produced is used as the primary performance measure, with repetitive errors or perseverations also noted.
Trails A and B : For part A the participant is required to connect a series of 25 numbers in a circle as quickly as possible. They are asked to connect them in sequential order as quickly as possible without removing their pencil from the page. In part B the participant alternates between numbers and letters (i.e. 1, A, 2, B etc.). If the participant makes an error the experimenter corrects them before they move to the next circle. The time taken to complete each part is used as the performance metric.
Digits Forward and Digits Backward: For the forward condition participants are presented with lists of digits and must immediately repeat them. The length of the list increases on each trial. Participants are given two versions of each list length and score 1 point for each. After failing both versions of a list length the task is terminated. The total score is then calculated. For the backward condition, the participants need to reverse the order of the numbers.
Each of the primary variables for each task will be standardised based on age: Using these standardised scores a global mean current cognitive performance score will be calculated and used as a covariate in the analyses.
Number of sessions attended and amount of time spent on each session will be recorded. In the SMART condition, access of the online materials using the participant’s username will be recorded automatically and usage within sessions and outside of sessions recorded.
Receipt of other treatments
Antipsychotic medication doses and receipt of psychological treatment will be tracked at each assessment point. These will be compared between groups, and to exclude these as accounting for outcomes, their relationship with outcome will be examined (medication doses converted into chlorpromazine equivalents).
Data are initially recorded on paper forms which are securely stored and extracted into password protected electronic files. Data extraction is conducted by research staff blind to participant allocation. Procedures to identify potential errors include double entry from selected paper files, range checks, and examination of outliers.
The primary focus of the analysis is differential changes in the SMART group versus Befriending over the full post intervention period. Analyses will be undertaken using mixed-model repeated measures (MMRM) allowing for autoregressive dependence. MMRM is well-suited to ITT analyses because this approach uses all available information from subjects. Participants with incomplete data are not discarded and missing data are not replaced with estimated values or observations carried forward. Instead maximum-likelihood estimation is applied with the available data. Where there are significant group x time interactions, planned contrasts will compare changes from baseline under each intervention.
Where outcome measures exhibit non-normal distributions across time points, appropriate transformations will be applied. If any of the outcome measures show significant baseline differences between SMART and Befriending, and between completers and noncompleters, corrections for sample imbalance and attrition bias will be applied using propensity scoring. This will involve firstly constructing a logistic regression model with treatment condition as the outcome, and baseline clinical variables and demographics as predictors, plus a “missing-at-post-therapy” variable to index differential effects of attrition. Secondly the effects of any imbalance or attrition will be addressed by including the estimated probabilities from this model (propensity scores) as a covariate in analyses.
The differential effects on outcomes of the process variables (self-efficacy and self-stigma) and medication changes will be tested across conditions with a maximum likelihood Hierarchical Linear Modelling analysis. In these models time will be treated as a linear covariate. Baseline measures for cognitive function, recovery style, intervention engagement, diagnosis, demographics, baseline internet use and expectancy will be used as covariates, while testing interaction effects for condition and the baseline social engagement moderators (loneliness, attachment, peer group identification and peer contact). Structural equation modelling will then be used to demonstrate the mediation mechanism of outcome improvement for each condition separately, with bootstrap analyses used to obtain 95 % confidence intervals for all effect sizes.
Data will also be used as part of a broader economic evaluation of the use of the website involving cost-consequences analyses whereby incremental costs of interventions using the site are compared with the full spectrum of outcomes measured vs relevant comparators. For this study, QALYs will be calculated from AQoL8D data, and costs will be derived from estimates of the costs involved in running the website (website development and maintenance costs, forum moderation, and therapist activity data), adjusted for any differences in resource use between conditions. Standardised economic evaluation techniques will be used including incremental analysis of mean differences, with confidence intervals derived using bootstrapping.
This trial investigates the efficacy of a novel intervention based on putative personal recovery processes (CHIME) and common self-management materials. It will contribute a rigorously evaluated and disseminable intervention to the small but growing literature on mental health service interventions designed to support personal recovery from psychotic disorders. The trial has innovative elements including the integration of digital technology into mental health appointments using tablet computers, and the use of lived experience material to complement non-peer delivered mental health work. Examining this as a focused intervention in this study provides a potential model of low intensity provision in practice. As well as providing an intervention model for personal recovery in persisting psychosis, the trialling of using lived experience content as part of a non-peer intervention can also inform analogous interventions for different populations. Furthermore, examination of mechanisms of change associated with the lived experience content will be useful to inform both the use of this intervention, as well as informing the use of peer support more broadly.
Assessment of quality of life - 8 dimension
Depression anxiety stress scale 21
Diagnostic and statistical manual of mental disorders IV text revision
Generalised self-efficacy scale
Human research ethics committee
Internalised stigma of mental illness scale
Mixed-model repeated measures
Positive and negative syndrome scale
Quality adjusted life years
Process of recovery questionnaire
Randomised Controlled Trial
Resource use questionnaire
Structured clinical interview for DSM-IV-TR Axis I Disorders
Subjective experience of psychosis scale
Self-management and recovery technology
Scale to assess therapeutic relationships in community mental health care
University of California, Los Angeles
Wechsler adult intelligence scale III
Wechsler test of adult reading
This trial is funded by the State Government of Victoria Department of Health Mental Illness Research Fund (MIRF33). From 2012 to 16, CM was funded by a NHMRC ECR grant (APP1035887). The funders had no role in the design of the study or reporting of results.
Availability of data and materials
All authors participated in the design of the intervention and trial. NT, JF, MK, SR, DC, DM, CM, EF, FF, NL and BM developed the trial design, measures and analysis approach. NT, FF, NL, JF, EL, CN, SF, LS and GM designed the website and developed its content. NT, JF, FF, RF and TS developed the therapist intervention protocols. All authors read and approved the final manuscript.
The authors declare that they have no competing interests.
Consent for publication
Ethics approval and consent to participate
The study is being conducted in accordance with the Declaration of Helsinki and has been approved by the following governing ethics committees: Alfred Hospital Ethics Committee (139–14); Melbourne Health Mental Health Research and Ethics Committee (2014.087); St Vincent’s Hospital Human Research Ethics Committee (HREC-A 041.14); Austin Health Human Research Ethics Committee (HREC/15/Austin/308; ND 15/308); Peninsula Health Human Research Ethics Committee (HREC/15/Austin/308; SSA/15/PH/49); Eastern Health Human Research Ethics Committee (HREC/15/Austin/308; SERP09-2016); Swinburne University Human Research Ethics Committee (2014/119); La Trobe University Human Ethics Committee (UHEC acceptance of The Alfred HREC approved project – 139/14) and Deakin University Human Ethics Committee (2014–285). Research staff obtain full informed consent from all participants prior to completing the baseline assessment. There are no restrictions on reporting findings of this trial, which will be published in full in the peer-reviewed literature.
Open AccessThis article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated.
- Morgan VA, Waterreus A, Jablensky A, Mackinnon A, McGrath JJ, Carr V, et al. People living with psychotic illness in 2010: The second Australian national survey of psychosis. Aust N Z J Psychiatry. 2012;46:735–52.View ArticlePubMedGoogle Scholar
- National Institute for Health and Clinical Excellence. Psychosis and Schizophrenia in Adults: Treatment and Management. 2014. https://www.nice.org.uk/Guidance/cg178 Accessed 23 Aug 2016.
- Kreyenbuhl J, Buchanan RW, Dickerson FB, Dixon LB. The Schizophrenia Patient Outcomes Research Team (PORT): Updated treatment recommendations 2009. Schizophr Bull. 2010;36:94–103.View ArticlePubMedGoogle Scholar
- Galletly C, Castle D, Dark F, Humberstone V, Jablensky A, Killackey E. Royal Australian and New Zealand College of Psychiatrists clinical practice guidelines for the management of schizophrenia and related disorders. Aust N Z J Psychiatry. 2016;50:410–72.View ArticlePubMedGoogle Scholar
- Berry K, Haddock G. The implementation of the NICE guidelines for schizophrenia: barriers to the implementation of psychological interventions and recommendations for the future. Psychol Psychother. 2008;81:419–36.View ArticlePubMedGoogle Scholar
- Couldwell A, Stickley T. The Thorn course: rhetoric and reality. J Psychiatr Ment Health Nurs. 2007;14:625–34. doi:10.1111/j.1365-2850.2007.01143.x.View ArticlePubMedGoogle Scholar
- Farhall J, Thomas N. Cognitive and behavioural therapies for psychosis. Aust N Z J Psychiatry. 2013;47:508–11.View ArticlePubMedGoogle Scholar
- Thomas N. What’s really wrong with cognitive behavioural therapy for psychosis? Front Psychol. 2015;6, e323. doi:10.3389/fpsyg.2015.00323.Google Scholar
- Haddock G, Eisner E, Boone C, Davies G, Coogan C, Barrowclough C. An investigation of the implementation of NICE-recommended CBT interventions for people with schizophrenia. J Ment Health. 2014;23:162–5.View ArticlePubMedGoogle Scholar
- Bennett-Levy J, Richards D, Farrand P, Christensen H, Griffiths KM, Kavanagh DJ, et al. Oxford guide to low intensity CBT interventions. Oxford: Oxford University Press; 2010.View ArticleGoogle Scholar
- Barbic S, Krupa T, Armstrong I. A randomized controlled trial of the effectiveness of a modified Recovery Workbook program: Preliminary findings. Psychiatr Serv. 2009;60:491–7.View ArticlePubMedGoogle Scholar
- Cook JA, Jonikas JA, Hamilton MM, Goldrick V, Steigman PJ, Grey DD, et al. Impact of Wellness Recovery Action Planning on service utilisation and need in a randomised controlled trial. Psychiatr Rehab J. 2013;36:250–7.View ArticleGoogle Scholar
- Cook JA, Steigman P, Pickett S, Diehl S, Fox A, Shipley P, et al. Randomized controlled trial of peer-led recovery education using Building Recovery of Individual Dreams and Goals through Education and Support (BRIDGES). Schizophr Res. 2012;136:36–42.View ArticlePubMedGoogle Scholar
- Gilbert MM, Chamberlain JA, White CR, Mayers PW, Pawsey B, Liew D, et al. Controlled clinical trial of a self-management program for people with mental illness in an adult mental health service: The Optimal Health Program (OHP). Aust Health Rev. 2012;36:1–7.View ArticlePubMedGoogle Scholar
- Castle D, Crosse C, Morgain D, Rossell S, Thomas N, Phillipou A, et al. Helping people with a mental illness obtain work: The HOPE program. Australasian Psychiatry. 2016. Advance online publication. doi:10.1177/1039856216654400.
- Ben-Zeev D, Drake RE, Corrigan PW, Rotondi AJ, Nilsen W, Depp C. Using contemporary technologies in the assessment and treatment of serious mental illness. Am J Psychiatr Rehab. 2012;15:357–76.View ArticleGoogle Scholar
- Firth J, Torous J. Smartphone apps for schizophrenia: A systematic review. JMIR mHealth and uHealth. 2015;3, e102.View ArticlePubMedPubMed CentralGoogle Scholar
- Alvarez-Jimenez M, Alcazar-Corcoles MA, González-Blanch C, Bendall S, McGorry PD, Gleeson JF. Online, social media and mobile technologies for psychosis treatment: A systematic review on novel user-led interventions. Schizophr Res. 2014;156:96–106.View ArticlePubMedGoogle Scholar
- Borzekowski DLG, Leith J, Medoff DR, Potts W, Dixon LB, Balis R, Hackman AL, Himelhoch S. Use of the Internet and other media for health information among clinic outpatients with serious mental illness. Psychiatr Serv. 2009;60:1265–8.View ArticlePubMedGoogle Scholar
- Miller BJ, Stewart A, Schrimsher J, Peeples D, Buckley PF. How connected are people with schizophrenia? Cell phone, computer, email, and social media use. Psychiatry Res. 2015;225:458–63.View ArticlePubMedGoogle Scholar
- Record EJ, Medoff DR, Dixon LB, Klingaman EA, Park SG, Hack S, et al. Access to and use of the Internet by veterans with serious mental illness. Community Ment Health J. 2016;52:136–41.View ArticlePubMedGoogle Scholar
- Firth J, Cotter J, Touros J, Bucci S, Firth JA, Young AR. Mobile phone ownership and endorsement of “mHealth” among people with psychosis: A meta-analysis of cross-sectional studies. Schizophr Bull. 2016;42:448–55.View ArticlePubMedGoogle Scholar
- Williams A, Fossey E, Farhall J, Foley F, Thomas N. Going online: The potential for mental health workers to integrate recovery oriented e-resources into their practice. Am J Psychiatr Rehab. (In press).
- American Psychiatric Association. Position statement on the use of the concept of recovery. Washington, DC: American Psychiatric Association; 2005.Google Scholar
- Australian Government Department of Health and Ageing. Fourth National Mental Health Plan: An agenda for collaborative government action in mental health 2009–2014. Canberra: Commonwealth of Australia; 2009.Google Scholar
- Mental Health Commission of Canada. Changing directions, changing lives: The mental health strategy for Canada. Calgary: Mental Health Commission of Canada; 2012.Google Scholar
- National Institute for Mental Health in England. NIMHE Guiding Statement on Recovery. London: Department of Health; 2005.Google Scholar
- New Freedom Commission on Mental Health. Achieving the Promise: Transforming Mental Health Care in America. Rockville: U.S. Department of Health and Human Services; 2003.Google Scholar
- Tse S, Cheung E, Kan A, Ng R, Yau S. Recovery in Hong Kong: service user participation in mental health services. Int Rev Psychiatry. 2012;24:40–7.View ArticlePubMedGoogle Scholar
- Davidson L, Schmutte T, Dinzeo T, Andres-Hyman R. Remission and recovery in schizophrenia: Practitioner and patient perspectives. Schizophr Bull. 2008;34:5–8.View ArticlePubMedGoogle Scholar
- Whitley R, Drake RE. Recovery: a dimensional approach. Psychiatr Serv. 2010;61:1248–50.View ArticlePubMedGoogle Scholar
- Bellack AS. Scientific and consumer models of recovery in schizophrenia: Concordance, contrasts, and implications. Schizophr Bull. 2006;32:432–42.View ArticlePubMedPubMed CentralGoogle Scholar
- Anthony WA. Recovery from mental illness: the guiding vision of the mental health system in the 1990s. J Psychosoc Rehabil. 1993;16:11–23.View ArticleGoogle Scholar
- Andresen R, Oades L, Caputi P. The experience of recovery from schizophrenia: towards an empirically-validated stage model. Aust N Z J Psychiatry. 2003;37:586–94.View ArticlePubMedGoogle Scholar
- Leamy M, Bird V, Le Boutillier C, Williams J, Slade M. A conceptual framework for personal recovery in mental health: systematic review and narrative synthesis. Br J Psychiatry. 2011;199:445–52.View ArticlePubMedGoogle Scholar
- Law H, Morrison AP. Recovery in psychosis: A Delphi study with experts by experience. Schizophr Bull. 2014;40:1347–55.View ArticlePubMedPubMed CentralGoogle Scholar
- Slade M, Bird V, Clarke E, Le Boutillier C, McCrone P, Macpherson R, et al. Supporting recovery in patients with psychosis through care by community-based adult mental health teams (REFOCUS): A multisite, cluster, randomised, controlled trial. Lancet Psychiatry. 2015;2:503–14.View ArticlePubMedGoogle Scholar
- Petros R, Solomon P. Reviewing illness self-management programs: A selection guide for consumers, practitioners, and administrators. Psychiatr Serv. 2015;66:1180–93.View ArticlePubMedGoogle Scholar
- Walker G, Bryant W. Peer support in adult mental health services: A metasynthesis of qualitative findings. Psychiatr Rehab J. 2013;36:28–34.View ArticleGoogle Scholar
- Thomas N, Farhall J, Foley F, Farnan S, Ladd E, Leitan N, et al. Using the Internet to integrate peer stories and self-management resources into mental health services. In: Kellehear K, Lane A, Cassaniti M, Tooth B, Chapman C, Robertson S, editors. Proceedings of the 24th TheMHS Mental Health Services Conference, Perth 2014. Balmain: TheMHS; 2015.Google Scholar
- Thomas N, McLeod B, Jones N, Abbott J. Developing internet interventions to target the individual impact of stigma in health conditions. Internet Interventions. 2015;2:351–8.View ArticleGoogle Scholar
- Naslund JA, Aschbrenner KA, Marsch LA, Bartels J. The future of mental health care: Peer-to-peer support and social media. Epidem Psychiatr Serv. 2016;25:113–22.View ArticleGoogle Scholar
- Corrigan PW, Watson AC, Barr L. The self-stigma of mental illness: implications for self-esteem and self-efficacy. J Soc Clin Psychol. 2006;25(8):875–84.View ArticleGoogle Scholar
- Hinshaw SP, Stier A. Stigma as related to mental disorders. Annu Rev Clin Psychol. 2008;4(1):367–93.View ArticlePubMedGoogle Scholar
- Boyd Ritsher J, Otilingam PG, Grajales M. Internalized stigma of mental illness: psychometric properties of a new measure. Psychiatry Res. 2003;121(1):31–49.View ArticleGoogle Scholar
- Corrigan PW, Morris SB, Michaels PJ, Rafacz JD, Rüsch N. Challenging the public stigma of mental illness: a meta-analysis of outcome studies. Psychiatr Serv. 2012;63(10):963–73.View ArticlePubMedGoogle Scholar
- Bandura A. Self-efficacy: toward a unifying theory of behavioral change. Psycho Rev. 1977;84(2):191.View ArticleGoogle Scholar
- Schwarzer R. Social-cognitive factors in changing health-related behaviors. Current Directions Psychol Sci. 2001;10:47–51.View ArticleGoogle Scholar
- First MB, Spitzer RL, Gibbon M, Williams JBW. Structured Clinical Interview for DSM-IV-TR Axis I Disorders. New York: Biometrics Research Department, New York State Psychiatric Institute; 2007.Google Scholar
- Wechsler D. Wechsler Test of Adult Reading. New York: Psychological Corporation; 2001.Google Scholar
- Barak A, Hen L, Boniel-Nissam M, Shapira N. A comprehensive review and a meta-analysis of the effectiveness of Internet-based psychotherapeutic interventions. J Tech Hum Serv. 2008;26:109–60.View ArticleGoogle Scholar
- Bendall S, Killackey E, Jackson HJ, Gleeson J. Befriending Manual. Melbourne: ORYGEN Research Centre, University of Melbourne; 2003.Google Scholar
- Bendall S, Jackson HJ, Killackey E, Allot K, Johnson T, Harrigan S, Gleeson J, McGorry PD. The credibility and acceptability of Befriending as a control therapy in a randomized controlled trial of cognitive behaviour therapy for acute first episode psychosis. Behav Cog Psychoth. 2006;34:277–91.View ArticleGoogle Scholar
- Jackson HJ, McGorry PD, Killackey E, Bendall S, Allot K, Dudgeon P, et al. The ACE Project: a randomised controlled trial of CBT versus befriending for first episode psychosis: Acute phase and one-year follow-up results. Psychol Med. 2007;38:725–35.PubMedGoogle Scholar
- Shawyer F, Farhall J, Mackinnon A, Trauer T, Sims E, Ratcliff K, et al. A randomised controlled trial of acceptance-based cognitive behavioural therapy for command hallucinations in psychotic disorders. Behav Res Ther. 2012;50:110–21.View ArticlePubMedGoogle Scholar
- Shawyer F, Farhall J, Thomas N, Hayes SC, Gallop B, Copolov D, Castle D. A randomised controlled trial of acceptance and commitment therapy for psychosis. Brit J Psychiatry. (In press).
- Neil ST, Kilbride M, Pitt L, Nothard S, Welford M, Sellwood W, Morrison AP. The questionnaire about the process of recovery (QPR): A measurement tool developed in collaboration with service users. Psychosis. 2009;2:145–55.View ArticleGoogle Scholar
- Williams J, Leamy M, Pesola F, Bird V, Le Boutillier C, Slade M. Psychometric evaluation of the Questionnaire about the Process of Recovery (QPR). Br J Psychiatry. 2015;207(6):551–5.View ArticlePubMedGoogle Scholar
- Choe K. Development and preliminary testing of the Schizophrenia Hope Scale, a brief scale to measure hope in people with schizophrenia. Int J Nurs Stud. 2014;51(6):927–33.View ArticlePubMedGoogle Scholar
- Russell DW. UCLA Loneliness Scale (Version 3): Reliability, validity, and factor structure. J Pers Assess. 1996;66(1):20–40.View ArticlePubMedGoogle Scholar
- Kay SR. Positive and Negative Syndromes in Schizophrenia. New York: Brunner/Mazel; 1991.Google Scholar
- Birchwood M, Trower P. The future of cognitive–behavioural therapy for psychosis: not a quasi-neuroleptic. B J Psychiatry. 2006;188(2):107–8.View ArticleGoogle Scholar
- Haddock G, Wood L, Watts R, Dunn G, Morrison AP, Price J. The Subjective Experiences of Psychosis Scale (SEPS): Psychometric evaluation of a scale to assess outcome in psychosis. Schizophr Res. 2011;133(1–3):244–9.View ArticlePubMedGoogle Scholar
- Lovibond SH, Lovibond PF. Manual for the Depression Anxiety Stress Scales. Sydney: Psychology Foundation of Australia; 1995.Google Scholar
- Richardson J, Khan M, Iezzi A, Sinha K, Mihalopoulos C, Herrman H, Schweitzer I. The AQoL-8D (PsyQoL) MAU Instrument: Overview. Melbourne: Monash University Centre for Health Economics; 2009.Google Scholar
- Richardson JR, Iezzi A, Khan MA, Maxwell A. Validity and reliability of the Assessment of Quality of Life (AQoL)-8D multi-attribute utility instrument. Patient. 2014;7:85–96.View ArticlePubMedGoogle Scholar
- McGuire-Snieckus R, McCabe R, Catty J, Hansson L, Priebe S. A new scale to assess the therapeutic relationship in community mental health care: STAR. Psychol Med. 2007;37(1):85–95.View ArticlePubMedGoogle Scholar
- National Health and Medical Research Council. National Statement on Ethical Conduct in Human Research 2007 (Updated May 2015). Canberra: National Health and Medical Research Council; 2015.
- Schwarzer R, Jerusalem M. Generalized Self-Efficacy scale. In: Weinman J, Wright S, Johnston M, editors. Measures in health psychology: A user’s portfolio. Causal and control beliefs. Windsor. UK: NFER-Nelson; 1995. p. 35–7.Google Scholar
- McGlashan TH. Recovery style from mental illness and long-term outcome. J Nerv Ment Dis. 1987;175:681–5.View ArticlePubMedGoogle Scholar
- Tait L, Birchwood M, Trower P. Adapting to the challenge of psychosis: personal resilience and the use of sealing-over (avoidant) coping strategies. Br J Psychiatry. 2004;185(5):410–5.View ArticlePubMedGoogle Scholar
- Drayton M, Birchwood M, Trower P. Early attachment experience and recovery from psychosis. Br J Clin Psychol. 1998;37:269–84.View ArticlePubMedGoogle Scholar
- Watson AC, Corrigan P, Larson JE, Sells M. Self-Stigma in People With Mental Illness. Schizophr Bull. 2007;33(6):1312–8.View ArticlePubMedPubMed CentralGoogle Scholar
- Berry K, Wearden A, Barrowclough C. Liversidge. Attachment styles, interpersonal relationships and psychotic phenomena in a non-clinical student sample. Pers Indiv Diff. 2006;41(4):707–18.View ArticleGoogle Scholar
- Keefe RS, Goldberg TE, Harvey PD, Gold JM, Poe MP, Coughenour L. The Brief Assessment of Cognition in Schizophrenia: reliability, sensitivity, and comparison with a standard neurocognitive battery. Schizophr Res. 2004;68(2–3):283–97.View ArticlePubMedGoogle Scholar
- Reitan R. Halstead-Reitan Neuropsychological Test Battery: Theory and Clinical Interpretation. Tucson: Reitan Neuropsychology; 1985.Google Scholar