Participants

Eligible participants were residents of the catchment area of the New South Wales (Australia) Southern Area Health Service who were over 17 years of age, who volunteered for training in response to publicity, who were available over the period of the trial, and who were willing to receive interviews assessing trial outcomes. Participants had to volunteer as individuals rather than as a group (e.g. a whole workplace). Publicity took the form of talks to community groups, newspaper ads, a press release and radio interviews.

Eligible clusters were the 16 Local Government Areas (cities or shires) in the catchment area of the Southern Area Health Service in 2003. This catchment is located in south-east New South Wales, runs approximately 370 km from north to south and approximately 160 km from east to west, and had a population of 194,435 in 2001. The Local Government Areas varied from popular coastal areas to farming communities to rural towns and ranged in population size from less than 5000 to over 50,000.

Intervention

Participants received a nine-hour Mental Health First Aid course, in three weekly sessions of three hours each. Training was administered in the local area in groups of up to 25 participants, with a minimum of 10 participants per course. As documentation of the intervention, there is a lesson plan for each session and a participants' manual containing material that was given to take away [2]. All instructors were given training and a teaching kit of lesson plans, videos, books, master copies of handouts and a set of transparencies. Educators received a one-week training program in how to conduct Mental Health First Aid courses and subsequent supervision in running a course. They were trained by Betty Kitchener who devised the Mental Health First Aid course. The course teaches how to help people in the crisis situations of being suicidal, having a panic attack, being exposed to a traumatic event, or in an acute psychotic state. The symptoms, risk factors and evidenced-based treatments (medical, psychological, alternative and self-help) for the mental disorders of anxiety, depressive and substance use and psychotic disorders are also taught. Figure 1 shows the five steps of providing mental health first aid taught in the course. Participants received training either immediately (experimental Local Government Areas) or after 6 months on a wait-list (control Local Government Areas).

Training was administered by educators who were recruited from the staff of the Southern Area Health Service. Expressions of interest to become Mental Health First Aid instructors were sought from staff of the Area Health Service and associated community organisations. Five Mental Health first Aid instructors were recruited from a pool of 10 applicants for these positions. All the instructors had experience in mental health work and also a background in training, working with communities or health promotion work. A project coordinator with experience in mental health and health promotion (Ms Karen Peterson), who was employed to work on the project half time, also trained as an instructor. The same instructors taught courses in each paired Local Government Area, so that this factor did not differ between the immediate and wait-list Local Government Areas. The coordinator monitored a sample of courses taught during the trial to assess fidelity to the lesson plans. A fidelity checklist of topics that had to be covered was developed for each session. Four of the instructors had all three course sessions checked, while one of the instructors only had two sessions checked. The percentage of topics covered correctly was 100% for four of the instructors and 81% for one of the instructors.

Objectives

The hypotheses were that individuals trained in Mental Health First Aid, when compared to wait-list controls, would have increased knowledge of mental disorders and their treatments, decreased social distance, increased confidence in providing help, and that they would provide greater help to people experiencing mental health problems.

Outcomes

Outcomes were measured in January–February of 2003 (the pre-test assessment), the courses were run for the intervention group in March–April of 2003, and outcomes were measured again in July–August 2003 (the follow-up assessment). The wait-list control group received courses in September–October 2003, after the follow-up assessment was completed.

All outcomes were measured at the individual level by telephone interview. The interview content was based on the questionnaire used in the uncontrolled trial of Mental Health First Aid [3]. The pre-test interview covered the following: whether the participant had ever experienced a mental health problem (yes/no), whether a family member had ever experienced a mental health problem (yes/no), the participant's confidence in helping someone (five-point scale from 1. not at all to 5. extremely), contact in the last six months with anyone with a mental health problem (yes/no), how many people, whether any help offered (yes/no), what type of help (open-ended question), recognition of the problem in a case vignette (randomly assigned to be a case of depression or one of schizophrenia), what participant would do to help if they knew the person in the vignette (this "mental health first aid intention" involved the presence or absence of 8 elements, arrived at by a qualitative analysis of a sample of the responses, and added up to give a scorefrom 0–8), ratings of the likely helpfulness of a range of interventions for the person in the vignette (scored to give a scale of percentage agreement with mental health professionals about treatment [3]), a social distance scale relating to the person in the vignette [8], whether the participant had had a problem like the one in the vignette, whether a family member had had a problem like the one in the vignette, participant's reason for doing the course, and sociodemographic characteristics of the participant (age, gender, education, non-English speaking background, aboriginality). The follow-up questionnaire was the same as the pre-test questionnaire except that it omitted the sociodemographic questions.

All outcomes were measured by a scripted telephone interview administered by professional interviewers. In order to reduce the length of the interview, participants were individually randomly assigned to receive either a depression vignette or a schizophrenia vignette, with the same questions asked in respect to each vignette. The interviewers were provided with an ID, name and phone number of each participant and knew whether they were giving the first or second interview to the participant. While they were not told whether the participant was in the experimental or control group, information about which group they were assigned to was given at the end of the interview script. As far as was practical given the very different sizes of the Local Government Area pairs, the same interviewers interviewed participants in each pair.

Sample size determination

For power calculations and sample size determination, a conservative assumption was made that the waitlist control group would show improvements, possibly due to increased awareness of mental health issues, of about 50% of that of the experimental group. This corresponds to effect sizes in the range 0.28–0.31 for changes on scales and in the range 0.02–0.04 for changes in identifying the correct diagnosis. Sample size estimates using nQuery Advisor software [9] indicated that a sample size of 200 participants in each of the two groups would be sufficient to detect differences with power of at least 80% in 2-sided tests at the 0.05 level. Clustering effects of individuals in 16 Local Government Areas involved design effects of unknown magnitude in the analysis. It was assumed that these would be of the order of 20%, so that a total achieved sample sizes of 250 in each group would be sufficient to detect differences with 80% power.

Randomization: Sequence generation

Randomization to immediate participation or wait-list was at the level of Local Government Area. The Local Government Areas were matched in pairs to have similar population and social characteristics. The variables used for matching were population size, interior vs coastal location, and an index of population education/occupation. The first listed LGA of each pair was assigned to the immediate or wait-list group at random, using the Random Integers option of Random.org [10] to generate a 1 or a 2 for each pair. For LGA pairs receiving a 1, the first member of the pair received immediate training, while for those receiving a 2 it was the second member of the pair.

Each individual participant was randomly assigned a variable (values of 1 or 2) to determine which case vignette they received during their interviews. This was done using the Random Integers option of Random.org [10]. Those assigned a 1 received the interview based on a vignette of a person who is depressed and those assigned 2 received a vignette of a person with schizophrenia.

Randomization: Allocation concealment

Allocation was on the basis of cluster. In other words, the participant's Local Government Area determined whether they received immediate or wait-list training. Participants were not informed about their allocation to immediate or wait-list training until the end of their baseline interview.

Randomization: implementation

Local Government Areas were matched in pairs and Anthony Jorm assigned these randomly to immediate training or wait-list. Participants were not able to attend a class from outside their own Local Government Area. There was a recruitment period for all Local Government Areas which was organized by the coordinator Karen Peterson. The coordinator and the participants who were recruited were blind to the allocation of the Local Government Area during the recruitment period. Anthony Jorm revealed the allocation to Karen Peterson after the recruitment period ended. Karen Peterson then organized class times either immediately or after a waiting period, depending on the allocation of each Local Government Area in the pair.

Randomization: Blinding (masking)

At the time of the baseline interview, the participants did not know whether they were in an immediate or wait-list Local Government Area. However, interviewers had information at the end of the interview script telling whether the participant was assigned an immediate class or had to wait. Blinding of participants was not possible at subsequent interviews. Participants knew whether or not they had received training. While interviewers were not told the allocation of the participants in subsequent interviews, this might have become obvious during the interview if participants mentioned whether or not they had done the course. Interviewers were given a scripted interview to minimize any bias in the assessment due to knowledge of allocation.

Ethics

Ethical approval for the study was given by the Australian National University Human Research Ethics Committee and by the ethics committee of the South Western Sydney Area Health Service.

Statistical methods

For outcomes measured on a numeric scale, the change from pre-test to follow-up was analysed using linear regression. For binary outcomes, individuals scoring the same at pre-test and at follow-up were not used, and for those who changed, the direction of change was analysed as a binary outcome using logistic regression. Standard errors and p-values were adjusted for the cluster design using the Huber-White "sandwich" variance estimator, treating the 16 LGAs as the clusters. Analyses were corrected for differences between the LGA pairs by including this as an 8-level fixed-effect factor in the regression models. Missing data were imputed using best-subsets regression. All analysis was done using Stata version 8.2 [11].

Recruitment and Participant flow

Recruitment of participants took place in October and November of 2002. Figure 2 shows the number of participants and clusters at each stage of the trial.

Baseline data

Numbers analyzed

The data were analyzed by an intention-to-treat approach, with single imputation used for missing data. As shown in Figure 2, the number of participants analyzed was the same as the number randomly allocated.

Outcomes and estimation

The intraclass correlations for the continuous outcomes were: for agreement with health professionals about treatment, 0.15 (95% confidence interval 0.01, 0.29); for number of people in contact with that had a mental health problem, 0.02 (0, 0.06); for confidence in providing help, 0.03 (0, 0.07); for mental health first aid intention, 0.002 (0, 0.02); and for social distance, 0.04 (0, 0.08). Thus for all but one outcome, the correlation was small, justifying our assumption of a modest design effect.

Adverse events

Given that an educational intervention was evaluated with a non-clinical sample, there was no justification for a systematic inquiry into adverse events. Informally, no adverse events were reported.