Studies considered by the committee during revisions to the age of onset criterion
We identified one document that we considered the key document describing the evidence to support the ADHD criteria revision process [13] (Additional file 1: Additional Document). The document refers to a systematic literature review published by the “workgroup age-of-onset subcommittee” [14] and one published study [15] as evidence for the change to the age of onset criterion. The review included 32 studies related to the age of onset criterion of varying designs and with different objectives. Based on these studies, the Committee commented on a) the magnitude of change (to the criterion), b) the reason/evidence for change, c) the potential negative consequences considered and d) additional objections and response. This key document had been available previously on the American Psychiatric Association (APA) website, but is no longer publicly available. On full text review of the 33 studies referred to by the key document, 17 studies addressing the checklist items were categorised and analysed (Fig. 1) [15,16,17,18,19,20,21,22,23,24,25,26,27,28,29,30,31].
Studies relevant to the age of onset criterion and the checklist items published between 1990 and 2013
Our searches of the available literature found 20 relevant studies (Fig. 1) [15,16,17,18,19,20,21,22,23,24,25,26,27,28,29,30,31,32,33,34]. We did not locate any studies of prevalence, precision, benefit or harm additional to those used by the Committee. However, we identified a further 3 studies related to prognosis [32,33,34].
Risk of bias and strength of evidence of the included studies
The published form of the literature review [14] referenced in the key document is a 2-page document. It provides a link to a supplementary online table that describes some features of the included studies (study objective, source of age of onset information, DSM version, study groups and results). We assessed the literature review as low quality according to the AMSTAR-2 quality assessment tool for systematic reviews [8] (Additional file 1: Table S4). The review did not meet 11 of the 13 applicable AMSTAR-2 items (of the 16 AMSTAR-2 items, 3 were not applicable as no meta-analysis was conducted): it did not specify a research question, outline apriori methods, explain how studies were selected and included, extract data in duplicate, list and justify excluded studies, describe included studies sufficiently, assess risk of bias of the included studies or account for risk of bias when interpreting the results or explain heterogeneity. In addition, funding sources for the included studies and the review authors’ potential conflict of interest were not reported (Item 10 and 16). The review partially met, or met, only two AMSTAR-2 criteria: the comprehensiveness of the search strategy and duplicate study selection. Flaws in the conduct or reporting of several critical domains of bias suggest that the review may not provide an accurate and comprehensive summary of the available studies.
Analysis of studies assessing prevalence
The key document included a primary study assessing the effect of the change in the age of onset criterion on prevalence using a prospective cohort study of 2322 British twins assessed at 7 and 12 years of age using information from mothers and teachers [15]. At age 12, 66 children (3.3%) met the study criteria for ADHD (mother or teacher report of six or more inattentive and/or six or more hyperactivity-impulsivity symptoms) with age of onset of symptoms before 7 years. An additional 2 children presented with symptoms meeting the ADHD diagnostic criteria between age 7 and 12. This study estimated that the increase in the prevalence of ADHD because of the change in the criterion would be 0.1% (Additional file 1: Table S5a).
The research evidence related to the effect on prevalence used to support the change to the age of onset criterion consists of a single study. We assessed the risk of bias in this study as low (Additional file 1: Table S5b). However, there is considerable uncertainty about the applicability of the evidence because of the type of participants (twins), failure to assess the effects on prevalence of the change to both the age and impairment requirements of the criterion, the prospective measurement of symptom onset and the low prevalence of ADHD in the population studied (3.3%).
Analysis of studies assessing prognosis
The key document considered the prognostic ability of the age of onset criterion using 11 studies [16,17,18,19,20,21,22,23,24,25,26] identified by the literature review [14] and we identified an additional 3 studies [32,33,34] in our searches (Fig. 1). These studies of cross-sectional and cohort design in child, adolescent or adult populations, compared a range of outcomes between variably defined groups with ‘early’ onset of ADHD symptoms or impairing symptoms (sometimes termed ‘Full ADHD’) and ‘late’ onset of ADHD symptoms or impairing symptoms (Additional file 1: Table S6a). Study results were mixed, with prognosis varying between ‘early’ and ‘late’ symptom onset groups for the same outcomes between studies, and for different outcomes within studies. Some of these studies were mentioned and referenced in the text of the literature review [14], while others appeared in the supplementary online table only.
The research evidence related to the prognostic ability of the age of onset criterion consists of multiple studies that we assessed to be at moderate or high risk of bias on 3 or more of the 6 domains of bias [11]. Risk of bias from confounding was present in most studies, and the potential for selective reporting or absence of an appropriate statistical model in all studies (Additional file 1: Table S6b). The possibility of bias arising from the retrospective recall of the age of symptom onset was judged to be considerable in the studies of cross-sectional design. In assessing the strength of the evidence, there was inconsistency in the results of studies of similar design, and the applicability of these studies is questionable as many were performed in clinical samples and none evaluated the prognostic ability of the criterion in individuals identified by the new but not old (DSM-IV-TR) age of onset criterion.
Analysis of studies assessing precision
The key document considered 3 studies [27,28,29] included in the literature review [14] related to the precision of methods of measuring age of onset. These studies evaluated the reporting of the date, or the age of onset of symptoms or behaviours by the same informant (using the same or different method of obtaining onset information) at different time intervals (1 week, 1 year and 5 years) (Additional file 1: Table S7a). These studies found poor to moderate agreement on: the date of symptom onset when data were collected at interviews 1 week apart; but reasonable stability of mothers reporting DSM-III symptoms over a 1-year period; and parent or self-reports of ‘later’ age of onset of impairing symptoms after a 5-year interval.
The research evidence related to the precision that the Committee considered in changing the age of onset criterion was not applicable to the checklist item. The item requires evaluation of the agreement between the same clinicians at different times (repeatability), and between different clinicians (reproducibility) on their judgment of whether an individual meets, or does not meet, the new age of onset criterion. None of the available studies provide this information. In addition, we assessed the studies used by the Committee to be at high or unclear risk of bias because the study population was not suited to evaluate the precision of the age of onset criterion, the interviewers were not those who would perform the test in everyday practice, and because of the availability of other clinical information which may influence assessors coding of the date or age of onset. (Additional file 1: Table S7b).
Analysis of studies assessing benefits and/or harms
The Committee considered 2 studies [30, 31] relating to the potential benefits and harms of treatment. The first of these single arm studies evaluated the response to methylphenidate among a population of adults with ‘late onset’ ADHD [30]. In the second, children, adolescents and adults were treated with methylphenidate and treatment response was compared between participants meeting DSM-IV ADHD criteria and participants meeting all criteria except symptom onset before age 7 [31] (Additional file 1: Table S8a). The adverse effects of the medication were reported in one study. There were no studies evaluating the potential harms arising from a diagnosis of ADHD according to the new age of onset criterion or arising from treatment of individuals classified by the new criterion.
To assess risk of bias in single-arm studies included in systematic reviews, we used a tool currently being trialed by the Cochrane Kidney Group (Beller, E. Personal communication. 2018. Feb 14). The research evidence related to the benefit or harm for individuals diagnosed by the new age of onset criterion consists of two single arm studies that are at high risk of bias due to lack of a control group, selection bias, lead time bias, bias due to adjunctive therapies, attrition bias and selective reporting of outcomes (Additional file 1: Table S8b). Further, the applicability of the studies to the checklist item is uncertain as the studies do not evaluate the benefits or harms arising from the treatment of those individuals identified by the new, but not old age of onset criterion (that is, the additional ‘milder’ and ‘older’ individuals identified by the new but not old criterion).