Research article | Open | Open Peer Review | Published:
Reliability and validity of the international dementia alliance schedule for the assessment and staging of care in China
BMC Psychiatryvolume 17, Article number: 371 (2017)
Clinical and social services both are important for dementia care. The International Dementia Alliance (IDEAL) Schedule for the Assessment and Staging of Care was developed to guide clinical and social care for dementia. Our study aimed to assess the validity and reliability of the IDEAL schedule in China.
Two hundred eighty-two dementia patients and their caregivers were recruited from 15 hospitals in China. Each patient-caregiver dyad was assessed with the IDEAL schedule by a rater and an observer simultaneously. The Clinical Dementia Rating (CDR), Mini-Mental Status Examination (MMSE), and Caregiver Burden Inventory (CBI) were assessed for criterion validity. IDEAL repeated assessment was conducted 7-10 days after the initial interview for 62 dyads.
Two hundred seventy-seven patient-caregiver dyads completed the IDEAL assessment. Inter-rater reliability for the total score of the IDEAL schedule was 0.93 (95%CI = 0.92-0.95). The inter-class coefficient for the total score of IDEAL was 0.95 for the interviewers and 0.93 for the silent raters. The IDEAL total score correlated with the global CDR score (ρ = 0.72, p < 0.001), the CDR-sum of box (CDR-SOB, ρ = 0.74, p < 0.001), the total score of MMSE (ρ = −0.65, p < 0.001) and CBI (ρ = 0.70, p < 0.001). All item scores of the IDEAL schedule were associated with the CDR-SOB (ρ = 0.17 ~ 0.79, all p < 0.05).
The IDEAL schedule is a valid and reliable tool for the staging of care for dementia in the Chinese population.
Dementia is one of the biggest global public health problems for the elderly. Among the 46.8 million people worldwide now living with dementia, most live in low- or middle-income countries .In China, the disease burden and care demands of dementia have exponentially increased with the rapid growth of the population. However, no disease-modifying treatment exists . Thus, dementia care is critical to maintaining a high quality of life during disease progression. Disease staging models that provide more precise information may be crucial to guide better person-centered care for persons with dementia in China.
Most of the current clinical staging scales, e.g., the Clinical Dementia Rating (CDR) scale , Global Deterioration Scale (GDS) , and Functional Assessment Staging (FAST) , focus on cognition or functional performance, without the inclusion of dementia care. Integrative assessment is one of the key components for psychosocial care of dementia . A staging model including dementia care needs can provide more evidence for health care services. Therefore, it is imperative to include dementia care in the severity assessment framework.
The International Dementia Alliance (IDEAL) schedule was developed by the IDEAL study group based on the consensus on diagnosis and care of dementia patients. The IDEAL schedule stages dementia based on multiple perspectives, including social support and professional or informal care . Use of the multidimensional schedule might improve the organization of regular or professional care for people with dementia. The IDEAL schedule is short and can be completed in 15 min. Health professionals, general practitioners, social workers and psychiatric nurses can use it. The IDEALschedule has been shown to have good content validity and high reliability in several countries [7,8,9]. However, the test-retest reliability has not been examined before. Besides, the Chinese version of IDEAL has not been validated. Testing the psychometric properties of Chinese IDEAL would provide more evidence for its application in dementia care in China.
Therefore, the present study first aimed to examine the psychometric property of the Chinese version of the IDEAL schedule, including internal consistency, test-retest reliability and inter-rater reliability. The secondary purpose was to evaluate the convergent validity of the schedule.
From June to December 2015, 282 persons with dementia and their primary caregivers were recruited from memory clinics and neurology and psychiatry specialist clinics of 15 hospitals in 11 cities in China, including Beijing, Taiyuan, Guangzhou, Wuxi, Yangzhou, Tianjin, Changsha, Qingdao, Baoding, Shenzhen, and Hangzhou (see study raters and participating hospitals in Additional file 1). Persons with dementia and their informants were consecutively sampled.
Patients with dementia due to neurodegenerative diseases were consecutively recruited. A clinical diagnosis was made according to the criteria for dementia cited in the International Classification of Diseases, 10th Revision (ICD-10) . To be included in the study, caregivers needed to visit the patients at least once a week. The exclusion criteria were as follows: The patient’s caregiver was not available, or the contact between caregiver and patient was insufficient (i.e., less than once a week); the patient was not able to speak Mandarin Chinese; the diagnosis remained unclear after diagnostic workup; the cognitive disorder was not due to neurodegenerative disease, e.g., brain tumour, delirium, etc.
The study was approved to be conducted in all participating hospitals by the institutional review board of Peking University Institute of Mental Health (Sixth Hospital). The ethics committee of Sir Run Run Shaw Hospital, Zhejiang University reviewed and additionally approved to conduct the study at Sir Run Run Shaw Hospital, Zhejiang University. Written informed consent was obtained from each patient-caregiver dyad. The patient and his/her legal guardian both provided written consent for the patient to participate in the study.
Translation of IDEAL schedule
Two bilingual geriatric psychiatrists translated the IDEAL schedule from English to Mandarin Chinese. One professional translator reviewed the translation, made further changes and agreed on the semi-final Chinese version. Another independent professional translator translated the semi-final Chinese scale back into English, and the study team compared the translated and back-translated versions. After that, further changes were made to formulate the final Chinese version.
To examine the convergent validity of individual items and the IDEAL schedule, we selected the following instruments as the reference:
Clinical dementia rating (CDR)
The CDR scale is a dementia staging instrument with good inter-rater reliability  and concurrent validity . It consists of 6 domains: memory, orientation, judgment and problem solving, community affairs, home and hobbies, and personal care. Each domain scores from 0 (normal) through 3 (severe dementia), representing the different severity of dementia. The sum of box (CDR-SOB, range from 0 to 18) and an overall score of CDR were both used in the present study.
Mini-mental state examination (MMSE)
The MMSE  is widely used as a test for the general assessment of cognitive function. The test covers several cognitive domains, including orientation, immediate and short-term memory, concentration, naming, reading, comprehension, writing, and visual-motor abilities. The total score of MMSE ranges from 0 (severe impairment) to 30 (normal cognition).
Caregiver burden inventory (CBI)
The CBI is a 24-item inventory to evaluate caregiver burden. It consists of five factors: time-dependence burden, developmental burden, physical burden, social burden, and emotional burden .
Rater training of IDEAL schedule
Twenty-six raters from15 sites received didactic training on how to administer the IDEAL schedule. The raters were asked to score for four cases examples after the training. During the training period, the intra-class correlation coefficients (ICC) of the total score of the IDEALschedule was 0.91.
Procedures of assessment
For each dyad, two raters attended the IDEAL schedule interview: one of them conducted the interview (interviewer), and the other silently observed the interview (silent rater). The interviewer and the silent rater did not communicate and discuss the scores. Patients and caregivers were interviewed separately. The raters for MMSE, CBI and CDR were blind to the score of IDEAL. Sixty-two dyads repeated the IDEAL assessments7-10 days after the first interview.
All analyses were performed with SPSS (version 16, SPSS Inc., Chicago, IL). Means and standard deviations were calculated for continuous data, and counts and prevalence rates for categorical data. Cronbach’s alpha coefficient and correlation coefficients between items were calculated to evaluate the internal consistency of the IDEAL schedule . ICCs were used to assess the inter-rater reliability and the test-retest reliability . For the concurrent validity, we used Spearman partial correlation analysis to examine the relationship between the individual items and the total score of the IDEAL schedule with the total score of the CDR-SOB, MMSE and CBI. The total score of IDEAL was also correlated with the global score of CDR.
A total of 277 patient-caregiver dyads (response rate of 98.23%) completed the assessment with the IDEAL schedule. As summarized in Table 1, 179 (64.6%) patients were female (mean age: 73.8 ± 8.7 years). Most patients were married (77.3%) and lived independently (96%). A total of 201 patients (71.5%) had received more than 9 years of schooling. Approximately 92.4% of caregivers were either spouses or children, and 148 (53.4%) caregivers were female. Majority of the patients were diagnosed with Alzheimer’s disease (n = 253, 91.3%). Other diagnoses included frontotemporal dementia (n = 9), Lewy body dementia (n = 6), Parkinson’s disease with dementia (n = 1), cerebral amyloid angiopathy (n = 2), and unspecified dementia (n = 6).
As illustrated in Fig. 1, there were significant differences in the total scores of MMSE, CBI and IDEAL by the severity of dementia determined by the global score of CDR (CDR = 0.5, n = 51; CDR = 1, n = 85; CDR = 2, n = 86; CDR = 3, n = 55). The total score of the IDEAL schedule increased with dementia advancing.
The internal consistency, Cronbach’s alpha coefficient was 0.85. Table 2 shows correlation coefficients between individual items and the total IDEAL score. The correlation coefficients were satisfying for most individual items.
Table 3 shows the ICC coefficients (interviewer vs. silent rater) for the total score of the IDEAL schedule was 0.93 (95%CI = 0.92-0.95), indicating good inter-rater reliability. About 70% of the ICCs were higher than 0.8 (range 0.59-0.88), except for domain of physical health (0.72, 95% CI = 0.65-0.77), behavioural and psychological symptoms (0.79, 95% CI = 0.75-0.83), and social support (0.59, 95%CI = 0.51-0.66).
The ICC coefficient for the total score of IDEAL was 0.95 for the interviewers and 0.93 for the silent raters. For individual items, the test-retest reliability ranged between 0.72 and 0.96 for interviewers and between 0.67 and 0.94 for silent raters (see in Additional file 2: Table S1). The results indicated the rating of the IDEAL schedule was duplicable within 7-10 days.
Table 4 summarizes the correlation coefficients of the IDEAL schedule against validated instruments. The total score of IDEAL correlated with the CDR-global (ρ = 0.72, p < 0.001), the CDR-SOB (ρ = 0.74, p < 0.001), the total score of MMSE (ρ = −0.65, p < 0.001) and CBI (ρ = 0.70, p < 0.001). All item scores were associated with the CDR-SOB (ρ = 0.17 ~0.79, all p < 0.05). The total scores of MMSE and CBI were significantly correlated with most item scores except for physical health (Table 4). More correlation coefficients between the IDEAL individual items and CBI factors are shown in Additional file 3: Table S2.
The present study demonstrated adequate validity and reliability of the IDEAL schedule in staging dementia among Chinese people. Our study is the first one, to our knowledge, which evaluates the test-retest reliability of the IDEAL schedule.
The Cronbach’s alpha indicates that the IDEAL schedule has good structure validity. The finding is consistent with the previous study by Semrau et al. . The IDEAL schedule intends to stage dementia based on clinical symptoms and care needs . The assessment of social support and care needs is as important as other aspects to reflect an accurate picture of dementia care in practice.
Similar to studies on original version and among Irish and Spanish, the present study demonstrates moderate to excellent inter-rater reliability in Chinese [7,8,9]. The only concern lies in the relative low inter-rater reliability of assessment of social support level. The Spanish study ruled out the item of social support with factor analysis. Nevertheless, we recommend to include the level of social support in dementia staging system due to two main reasons. First, several studies have provided evidence that a low level of social support is an important risk factor for cognitive impairment [17,18,19,20,21]. Bennett and colleagues even argue that social networks modify the relation of some measures of Alzheimer’s disease pathology and level of cognitive function . Second, social support is one of the key components of the psychosocial intervention of dementia care . Social support is a multidimensional construct having perceived and objective elements . Further studies need to provide supplemental definition and scoring anchors for a more reliable assessment of social support.
Compared with the Irish and Spanish studies, our finding adds test-retest reliability of the IDEAL schedule. Test-retest reliability is one of the important psychometric properties. Comparison of our results with previous studies is difficult because the test-retest reliability was not reported before [7,8,9]. The present study demonstrates adequate test-retest reliability for both interviewers and silent raters over an interval of 7-10 days. It supports the high stability of assessment with IDEAL over a short period.
Finally, the concurrent validity of the global IDEAL score when compared with the CDR-SOB and MMSE is similarly considered to be adequate. We observe a strong association between the total score of IDEAL and the CDR-SOB and MMSE. The finding is consistent with previous studies [7, 9]. Besides, the present study reveals a significant correlation between the total score of IDEAL, the carer distress item score and the total score CBI. In the Spanish study, Lopez-Anton et al. reported a correlation between care distress and the score of Zarit burden interview (ZBI) . Our findings are comparable to the Spanish study and confirm that the IDEAL schedule could assess caregiver burden effectively.
An important limitation of this study is that participants were recruited at 15 sites across 11 cities in China, and researchers from different cities in different local cultures may have a different understanding of each domain of the IDEAL schedule, which may lead to interviewer bias. However, all raters from the participating sites had attended extensive training and supervision and reached high inter-rater reliability before starting the study. The qualification procedure is acceptable to minimize rater bias for multi-center clinical studies.
This study indicates adequate validity and reliability of the Chinese version of the IDEAL schedule. The main psychometric properties of the IDEAL schedule confirm its application for clinical practice of dementia care in the Chinese population. To better assess the items of social support and care needs, further research is needed to provide precise definition and scoring anchors for individual items.
Alzheimer's Disease International. World Alzheimer Report. The Global Impact of Dementia: An analysis on prevalence, incidence, cost and trends. 2015. Accessible online on 2017/09/30. https://www.alz.co.uk/research/WorldAlzheimerReport2015.pdf.
Nowrangi MA, Rao V, Lyketsos CG. Epidemiology, assessment, and treatment of dementia. Psychiatr Clin North Am. 2011;34(2):275–94. vii
Hughes CP, Berg L, Danziger WL, Coben LA, Martin RL. A new clinical scale for the staging of dementia. Br J Psychiatry J Ment Sci. 1982;140:566–72.
Reisberg B, Ferris SH, de Leon MJ, Crook T. The global deterioration scale for assessment of primary degenerative dementia. Am J Psychiatry. 1982;139(9):1136–9.
Reisberg B. Functional assessment staging (FAST). Psychopharmacol Bull. 1988;24(4):653–9.
Vasse E, Moniz-Cook E, Rikkert MO, Cantegreil I, Charras K, Dorenlot P, Fumero G, Franco M, Woods B, Vernooij-Dassen M. The development of quality indicators to improve psychosocial care in dementia. Int Psychogeriatr. 2012;24(6):921–30.
Semrau M, Burns A, Djukic-Dejanovic S, Eraslan D, Han C, Lecic-Tosevski D, Lobo A, Mihai A, Morris J, Palumbo C, et al. Development of an international schedule for the assessment and staging of care for dementia. J Alzheimers Dis. 2015;44(1):139–51.
Schepens ME, Lutomski JE, Bruce I, Olde Rikkert MG, Lawlor BA. Reliability and validity of the international dementia alliance schedule for the assessment and staging of Care in Ireland. Am J Geriatr Psychiatry. 2016;24(4):297–300.
Lopez-Anton R, Barrada JR, Santabarbara J, Posadas-de Miguel M, Aguera L, Burillo C, Franco M, Lopez-Alvarez J, Mesa P, Petidier R, et al. Reliability and validity of the Spanish version of the IDEAL schedule for assessing care needs in dementia: cross-sectional, multicenter study. Int J Geriatr Psychiatry. 2017;
World Health Organization. International classification of diseases, 10th revision (ICD-10). Geneva: World Health Organization; 1999.
Burke WJ, Miller JP, Rubin EH, Morris JC, Coben LA, Duchek J, Wittels IG, Berg L. Reliability of the Washington University clinical dementia rating. Arch Neurol. 1988;45(1):31–2.
Welsh KA, Butters N, Hughes JP, Mohs RC, Heyman A. Detection and staging of dementia in Alzheimer's disease. Use of the neuropsychological measures developed for the consortium to establish a registry for Alzheimer’s disease. Arch Neurol. 1992;49(5):448–52.
Folstein MF, Folstein SE, McHugh PR. “Mini-mental state”. A practical method for grading the cognitive state of patients for the clinician. J Psychiatr Res. 1975;12(3):189–98.
Chou KR, Jiann-Chyun L, Chu H. The reliability and validity of the Chinese version of the caregiver burden inventory. Nurs Res. 2002;51(5):324–31.
Terwee CB, Bot SD, de Boer MR, van der Windt DA, Knol DL, Dekker J, Bouter LM, de Vet HC. Quality criteria were proposed for measurement properties of health status questionnaires. J Clin Epidemiol. 2007;60(1):34–42.
Ellis JL. Probability interpretations of intraclass reliabilities. Stat Med. 2013;32(26):4596–608.
Kawachi I, Berkman LF. Social ties and mental health. J Urban Health. 2001;78(3):458–67.
Seeman TE, Lusignolo TM, Albert M, Berkman L. Social relationships, social support, and patterns of cognitive aging in healthy, high-functioning older adults: MacArthur studies of successful aging. Health Psychol. 2001;20(4):243–55.
Zunzunegui MV, Alvarado BE, Del Ser T, Otero A. Social networks, social integration, and social engagement determine cognitive decline in community-dwelling Spanish older adults. J Gerontol Ser B Psychol Sci Soc Sci. 2003;58(2):S93–S100.
Green AF, Rebok G, Lyketsos CG. Influence of social network characteristics on cognition and functional status with aging. Int J Geriatr Psychiatry. 2008;23(9):972–8.
Melchiorre MG, Chiatti C, Lamura G, Torres-Gonzales F, Stankunas M, Lindert J, Ioannidi-Kapolou E, Barros H, Macassa G, Soares JF. Social support, socio-economic status, health and abuse among older people in seven European countries. PLoS One. 2013;8(1):e54856.
Bennett DA, Schneider JA, Tang Y, Arnold SE, Wilson RS. The effect of social networks on the relation between Alzheimer's disease pathology and level of cognitive function in old people: a longitudinal cohort study. Lancet Neurol. 2006;5(5):406–12.
Morano C, Morano B. Social assessment (chapter 11). In: Gallo J, Bogner H, Fulmer T, Paveza G, editors. Handbook of geriatric assessment. 4th ed. Boston: Jones and Bartlett Publishers; 2006.
The authors thank all research participants for the time and effort they dedicated to the study.
This work was supported by the Beijing Municipal Science and Technology Commission (Z151100004015236) and the National Natural Science Foundation of China (71490732, 81500918). The funding sources did not play any role in the design and conduct of the study; collection, analysis, and interpretation of data; and preparation of the manuscript; or in the decision to submit the paper for publication.
Availability of data and materials
The databases are not freely available to the public because lack of specific consent in the ethics committee approval, but will be available from the corresponding author on reasonable request.
Ethics approval and consent to participate
The institutional review board of Peking University Institute of Mental Health (Sixth Hospital) approved the study to be conducted in all participating hospitals. The ethics committee of Sir Run Run Shaw Hospital, Zhejiang University additionally approved to conduct the study at Sir Run Run Shaw Hospital, Zhejiang University. Written informed consent was obtained from each patient-caregiver dyad. The patient and his/her legal guardian both provided written consent for the patient to participate in the study.
Consent for publication
The authors declare that they have no competing interests.
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Study groups, raters and participating hospitals (in alphabetic order by province or administrative city). (DOCX 15 kb)
Intra-class correlation coefficients (ICC) for item scores and the total score of IDEAL (test-retest reliability). (DOCX 25 kb)
Correlation of item scores of IDEAL, Chinese version, against factor scores of CBI. (DOCX 16 kb)