Validation of brief screening tools for depressive and alcohol use disorders among TB and HIV patients in primary care in Zambia

Study design and setting

This was a cross-sectional study under the auspices of the Zambia AIDS-Related TB (ZAMBART) Project. The study was conducted in PHC centres that provide both TB and HIV diagnoses and treatment. The TB/ART clinics within the PHC centres are serviced by clinical officers and nurses, many of whom have limited training in screening for MDDs and AUDs. These centres cover both urban and rural settings and are the first point of entry in the referral process for the majority of TB and HIV patients in the early stages of disease. Sixteen PHC centres were selected for the study on the basis of high TB and HIV prevalence [30] and these are distributed in seven districts, of which Lusaka (the capital city) contains four centres and the other six districts have two centres each. The size of the catchment area for these PHC centres varied from 25,000 inhabitants in rural communities to 147,000 inhabitants in urban communities.

Training of the Field staff

Sixteen lay research assistants and ten mental health clinical assistants (mental health workers with a diploma in mental health), recruited from the study communities, were trained separately over two days. On the first day, the lay research assistants were trained on how to screen for common mental disorders using the CES-D and AUDIT tools. On the second day the mental health clinical assistants were trained on how to diagnose common mental disorders using the MINI. The training for both groups also covered information on the study protocol, informed consent procedures, ethical considerations and data quality issues. The training was led by two psychiatrists, two TB/HIV specialists and a data manager. Inter-rater reliability assessments formed part of this training. The intra-class correlation value for interrater reliability of the lay research assistants was 0.98 while that of the mental health clinical assistants was 0.99. These results showed a high degree of inter-rater reliability in each group of field staff.

Participants

Eligible participants were aged 16 years or older, attending the TB or ART clinics of one of the 16 PHCs, and had started TB treatment or ART during the month before the study. Patients with whom it was not possible to complete the informed consent procedures owing to serious medical illness were not eligible. Patients with dual TB and HIV infection were included, and data pertaining to both TB therapy and ART were documented.

Procedures

A consecutive series of eligible patients were recruited from the PHC centres from December 2009 to January 2010. Participants were informed about the study and asked to provide informed consent after which data collection commenced in two separate interviews which were carried out on the same day of their routine clinic visits. The first interview was a screening interview conducted by the trained lay research assistants. Participants were screened for depression with the CES-D and excessive alcohol use with the AUDIT. Socio-demographic data and HIV/TB clinical data were also obtained in this interview. DSM IV criteria were assessed using the MINI in a second interview conducted by the trained mental health clinical assistants. Confirmatory clinical data on TB treatment and ART status were also obtained in this second interview from the participants' medical records. The trained mental health clinical assistants conducting the second interview with the MINI were blinded to all data collected by the trained lay research assistants conducting the first interview with the CES-D and AUDIT. We chose the CES-D and AUDIT because they are brief, easy to administer and have been widely used in cross-cultural studies, including African settings [31].

Reference standard

The MINI was used as the 'gold-standard' in generating psychiatric diagnoses. The MINI is a short, structured diagnostic interview that was developed in 1990 by psychiatrists and clinicians in the United States and Europe for DSM-IV psychiatric disorders [29]. The MINI is divided into modules, each corresponding to a diagnostic category. For the purposes of this study, only modules covering current MDD and AUDs were selected. DSM-IV criteria have been used in previous studies in Zambia [22, 23].

Screening tools

The CES-D is a self-report scale with 20 items designed to measure depressive symptoms in general population samples. Each item is assigned a value 0-4. There are four items that are positive-worded that have to be reverse scored, before computing the total score by adding each of the 20 items. The minimum score is 0 and the maximum score is 60. The CES-D measures common symptoms of major depression, including depressive mood, feelings of guilt and worthlessness, psychomotor retardation, loss of appetite, and sleep disturbance within the week prior to the interview. A score of 16 and above in the general population suggests symptoms of depression [27]. Reliability, validity, and factor structure have been found to be similar across a wide variety of demographic characteristics in general population samples that have been tested [32, 33]. In Uganda, the CES-D has been used to assess the prevalence of depression in HIV infected individuals, although it was not validated in this population [8].

The AUDIT was developed by the World Health Organisation (WHO) as a simple method of screening for excessive alcohol consumption in the past 12 months [28, 34]. It consists of 10 questions on recent alcohol use (items 1-3), alcohol dependency syndromes (items 4-6) and alcohol-related problems (items 7-10). Each of the 10 questions is rated on a four-point scale. The total score ranges from 0 to 40. A total score of 8 or more is recommended as an indicator of hazardous drinking behaviour [34].The AUDIT was developed and validated in multinational samples involving Kenya [28] and has been validated in South Africa [31].

Translation of the MINI and screening tools

The MINI, CES-D and AUDIT were translated to the dominant languages (Bemba, Nyanja, Tonga and Lozi) in the study communities. The translated questionnaires were forward-translated by professional translators working for the Zambia National Broadcasting Corporation (ZNBC). The forward-translated instruments were then back-translated into English by community representatives with experience in the translation of research questionnaires. Discrepancies in conceptual and semantic equivalence were resolved through an informal committee consensus approach with both forward and back-translators. Following on all this, all translated versions of the questionnaires were discussed by the research team; comprising members who were fluent in the native languages, until final versions of the questionnaires were agreed upon. These instruments were tested before use in the field.

Definition of cases

The sample was categorised into cases and non-cases of psychiatric disorders based on the MINI outputs of (i) current MDD and (ii) AUD.

Analysis

Data were analysed using Stata 11 (College Station, Texas, USA). Median total scores on the CES-D and AUDIT respectively were compared against MINI-defined diagnoses using the Wilcoxon rank-sum test. The internal consistency of these screening tools was assessed using Cronbach's alpha. We used confirmatory factor analysis to examine how a four-factor model of 'depressive symptoms', 'somatic symptoms', 'positive experiences' and 'interpersonal difficulties' for the CES-D [27], and a two-factor model of 'alcohol consumption' and 'alcohol related problems' for the AUDIT [35] fit the observed data. We used a combination of the chi-square to degrees of freedom ratio (χ²/df) of <2; Tucker-Lewis index (TLI) and Comparative fit index (CFI) of >0.95, and Standardized Root Mean-Square Residual (SRMR) of <0.08 as our rule of thumb for goodness of fit of the models [36].

Non-parametric area under the receiver operating characteristic curve (AUROC) analyses were performed to estimate the diagnostic accuracy of the screening tools [37]. Cut-off scores that simultaneously gave high sensitivity and high PPV were selected [38]. The data for the AUDIT was stratified by gender as previous research had shown that cut-off for the AUDIT was gender specific [39, 40].

Ethical Considerations

The study was approved by the University of Zambia Biomedical Research Ethics committee and endorsed by the Ministry of Health in Zambia. Written informed consent for participation and publication was obtained from the patients prior to the commencement of any study related procedures. Data were collected anonymously and all participants were identified by a unique study code. The questionnaires and electronic database were linked by these unique barcodes that were kept separately in a password protected database.

Characteristics of the participants

Seven hundred and forty four patients participated in the first interview with the CES-D and AUDIT. Of these, 649 patients (87.2%) completed the MINI diagnostic interview (Figure 1). There was little evidence of a difference in age (p = 0.45), gender (p = 0.12), median CES-D score (p = 0.47) or median AUDIT score (p = 0.49) between those who did, and did not, complete the MINI diagnostic interview. Of the 649 participants who completed the MINI diagnostic interview, the majority (77%) were recruited at TB clinics, and of these, 54% were also HIV positive (Table 1).

Internal consistency

Case detection properties

CES-D scores tended to be higher among MINI-defined current MDD cases than non-MDD cases (median 28 vs. 18; p < 0.001). Similarly, the AUDIT scores were higher among MINI-defined AUD cases than non-cases (median 22 vs.12; p < 0.001). The AUDIT scores were also higher among MINI-defined AUD cases than non-cases for women (median 24 vs.10; P < 0.001) and men (median 20 vs.13; p < 0.001) respectively.

The CES-D and AUDIT showed good discrimination in detecting current MDD and AUD cases from non-cases respectively (AUROC for CES-D = 0.78, and for AUDIT = 0.98 for women and 0.75 for men respectively), indicating that these were accurate screening tools. The difference in performance of the AUDIT was significantly better for women than for men (p < 0.0001) (Figure 2).

For each CES-D and AUDIT, cut-off points, sensitivity and PPV were obtained (Figure 3). The optimum cut-off score of the CES-D in screening for current MDD was 22. This achieved a sensitivity of 73% and PPV of 76% (Figure 3A). For the AUDIT, the optimum cut-off score for screening AUDs was 24 for women (sensitivity of 60% and PPV of 60%), and 20 for men (sensitivity of 55% and PPV of 50%) (Figure 3B).