The sample

POMH-UK invited all National Health Service (NHS) Trusts in the United Kingdom providing specialist mental health services to participate in a QIP focusing on the quality of monitoring of patients prescribed lithium. This was done in a number of ways which included; (1) e-mail communication with the POMH leads in eligible Trusts; (2) discussion with clinicians and clinical audit staff at POMH regional workshops, and: (3) letters of invitation to Trust chief executives, medical directors, chief pharmacists and clinical governance leads. Thirty-eight Trusts chose to participate. Very few UK services have a central register of patients prescribed lithium and Trusts used a variety of methods to identify their sample. These included a census of prescriptions, pharmacy records, pathology records and the caseloads of individual clinical teams. Services could enter data for as many patients as they wished.

Data collection

For each patient the following data were collected: age, gender, ethnicity, and primary psychiatric diagnosis.

For the subsample of patients who had started lithium treatment in the past year, the following data were collected: the presence of documented pre-treatment measures of renal and thyroid function and body weight (or BMI or waist circumference), and documented evidence that the patient had been advised of the side effects of lithium, the risk factors for lithium toxicity and the signs and symptoms of toxicity.

For the remaining patients, who had been prescribed lithium for longer than year, the data collected included the number of occasions on which a serum lithium level, renal and thyroid function tests and a measure of body weight had been measured over the past 12 months. Multiple tests conducted within the same month were counted as a single test as these were likely to have been conducted for a purpose other than routine monitoring.

For each patient, all the data were collected from their clinical records, and submitted to POMH using a secure web-based system called SNAP. Data collection fields relevant to the audit standards were mandatory in that it was not possible to submit data for cases where the mandatory fields had not been completed. The identity of each Trust submitting data was known to POMH, but the identities of the individual clinical teams and patients were not. Only the national level data are reported here.

Trust Questionnaire

Each Trust was sent a questionnaire relating to systems for managing patients who were prescribed lithium, both within the Trust and across the interface with primary care. With respect to systems within the Trust, the questionnaire covered whether: (1) there were locally adopted guidelines for managing patients prescribed lithium; (2) Trust clinicians had electronic access to pathology results; (3) care was wholly or partly delivered through lithium clinics and; (4) whether the Trust had access to an electronic database containing details of all patients prescribed lithium, and if so, whether this system generated automatic reminders that blood tests were due. Further questions were asked about systems for sharing care between the mental health Trust and primary care.

Statistical analysis

Each of the four outcomes of interest (measures of serum lithium level, renal function, thyroid function and body weight) was treated as a binary measure; whether or not the standard had been met. Logistic regression analyses were conducted to explore the contribution of several possible explanatory variables (age, gender, ethnicity, psychiatric diagnosis, and care provider) to these binary outcomes. The separate effect of each predictor variable upon each outcome was tested in a series of univariable analyses. Subsequently, the joint effect of the variables upon each outcome was examined in multivariable analyses, using a backwards selection procedure to retain only the statistically significant variables. Data were analysed using SPSS, version 17.

The sample

Four hundred and thirty six clinical teams from 38 mental health Trusts submitted data for 3,373 patients. 1,972 (59%) patients were female, 2,667 (79%) were white British, and the mean age of the sample was 55 years (sd 16, range 17-94 years). For 1,919 (57%) patients the primary clinical diagnosis was bipolar disorder, 857 (25%) unipolar depression, 370 (11%) a psychotic spectrum disorder (ICD10 F20-29), 161 (5%) another psychiatric diagnosis, and for 66 (2%) no psychiatric diagnosis was documented.

Performance against the standards in the sub-sample of patients who had been prescribed lithium for less than 1 year

397 patients had been prescribed lithium for less than 1 year. Of these, 334 (84%) had a documented baseline test of renal function including creatinine; the respective figures for thyroid function and body weight were 325 (82%) and 145 (37%).

With respect to documentation regarding the provision of relevant information to patients, this was present for the side effects of lithium in 244 (62%) cases, the risk factors for toxicity in 166 (42%), and the signs and symptoms of toxicity in 178 (45%) of cases. These proportions did not differ for the sub-groups of patients who were either younger than 65 years or older than 65 years.

Performance against the standards in the sub-sample of patients who had been prescribed lithium for more than a year

2,976 patients had been prescribed lithium for more than a year. With respect to lithium serum levels, 68% of cases had 2 or more documented tests in the previous year, thus meeting the QOF standard, while 30% had 4 or more tests in the last year, reaching the NICE standard. With respect to tests of renal function, which included creatinine, 81% of cases had one or more documented tests in the last year, thereby meeting the QOF standard, while 55% had two or more documented tests and therefore met the NICE standard. The respective figures for thyroid function were 82% and 50%. For 206 (7%) patients there was no documented evidence that any of the recommended monitoring tests/measures had been conducted in the previous year.

Factors predicting monitoring performance

The univariable analyses examined the effect of potentially relevant clinical or demographic factors (age, gender, ethnicity, ICD-10 psychiatric diagnosis and type of clinical service providing care, e.g. general adult psychiatry, learning disability, forensic service, etc.) on whether the four outcomes were met. At a significance level of p ≤ 0.001, age (being over 65 years) and service type (elderly mental health services) were associated with monitoring of serum lithium level and renal function, service type (again essentially elderly mental health services) was associated with monitoring of thyroid function, while diagnosis (schizophrenia spectrum disorder) and service type (forensic and learning disability services) were associated with measurement of body weight.

The multivariable analyses addressed biochemical monitoring, and revealed that only service type (elderly mental health services) was associated with meeting the standards for monitoring serum lithium (OR 1.34; 95% CI 1.13-1.58) and renal function (OR 1.45; 1.12-1.87), both at a significance of p ≤ 0.001.

Trust questionnaire

All 38 Trusts returned a completed questionnaire. Twenty-eight (74%) Trusts reported having fully adopted formal guidelines; most using the monitoring standards recommended in the NICE bipolar guidelines (n = 20) or British National Formulary (n = 11). Twenty-four (63%) Trusts reported having at least one lithium clinic, but only 8 (21%) had Trust-wide electronic access to results and 1 (3%) a local electronic database specifically for lithium that automatically produced prompts when tests were due. Fourteen (37%) Trusts had formally agreed, shared-care guidelines for patients managed concurrently with primary care, and 5 (13%) had electronic systems shared Trust-wide between primary and secondary care.

Why is recommended monitoring not carried out?

Possible explanations for suboptimal monitoring may implicate procedural, patient and/or practitioner variables.

The use of incentivised care in improving monitoring practice

Mental health Trusts within the UK are required to implement NICE guidelines and progress with this is monitored by the Care Quality Commission (CQC). In contrast, there are no sanctions for General Practitioners who fail to meet QOF targets, rather a positive benefit in the form of payment when these targets are met.

In our sample, the primary care QOF targets with respect to monitoring of serum lithium was met in over two-thirds of cases, and the target with respect to renal and thyroid function in just over four fifths. The NHS Information Centre (QOF statistics for England, 2008/9) lists these targets as having been met for 91% and 97.4% of patients respectively within primary care in 2008/9 [27]. As it is likely that the care of the majority of patients who are prescribed lithium is shared between primary and secondary care, less apparent monitoring in our secondary care sample may partly reflect communication issues between these sectors.

Proponents of a system like QOF argue that it can improve the implementation of evidence-based interventions [28] in primary care and constitutes an important quality improvement tool. However, critics have expressed concerns that QOF targets are too low with poor discriminatory value [29], and that incentivised care will never be an adequate substitute for professional judgment [30]. Our finding that the proportion of patients monitored in line with QOF targets was higher in primary than secondary care supports the view that the QOF system is a viable quality improvement tool. There is however, a need for more objective and transparent setting of QOF targets and increasing convergence between these and NICE standards.

Study strengths and limitations

A possible limitation of our study is a bias in the selection of patient samples for audit by each participating Trust. Such bias is unlikely to be unidirectional in that clinical teams that consider they are performing well in relation to meeting the relevant practice standards may choose to participate, whereas Trusts may choose to submit data for teams that they suspect are performing less well. The net result of competing sources of bias is unknown. Poor documentation standards or quality of case note review in this audit could also account for observed failures in monitoring practice.

A strength of the work is that our audit sample is larger than those of all previously published studies combined and is drawn from across the UK. Trusts that participated in the audit are representative of all NHS mental health Trusts [31] and so it is likely that our findings are generalisable to practice in other Trusts and representative of current clinical practice in the UK.