Web-based tools can be used reliably to detect patients with major depressive disorder and subsyndromal depressive symptoms
© Lin et al; licensee BioMed Central Ltd. 2007
Received: 31 August 2006
Accepted: 10 April 2007
Published: 10 April 2007
Although depression has been regarded as a major public health problem, many individuals with depression still remain undetected or untreated. Despite the potential for Internet-based tools to greatly improve the success rate of screening for depression, their reliability and validity has not been well studied. Therefore the aim of this study was to evaluate the test-retest reliability and criterion validity of a Web-based system, the Internet-based Self-assessment Program for Depression (ISP-D).
The ISP-D to screen for major depressive disorder (MDD), minor depressive disorder (MinD), and subsyndromal depressive symptoms (SSD) was developed in traditional Chinese. Volunteers, 18 years and older, were recruited via the Internet and then assessed twice on the online ISP-D system to investigate the test-retest reliability of the test. They were subsequently prompted to schedule face-to-face interviews. The interviews were performed by the research psychiatrists using the Mini-International Neuropsychiatric Interview and the diagnoses made according to DSM-IV diagnostic criteria were used for the statistics of criterion validity. Kappa (κ) values were calculated to assess test-retest reliability.
A total of 579 volunteer subjects were administered the test. Most of the subjects were young (mean age: 26.2 ± 6.6 years), female (77.7%), single (81.6%), and well educated (61.9% college or higher). The distributions of MDD, MinD, SSD and no depression specified were 30.9%, 7.4%, 15.2%, and 46.5%, respectively. The mean time to complete the ISP-D was 8.89 ± 6.77 min. One hundred and eighty-four of the respondents completed the retest (response rate: 31.8%). Our analysis revealed that the 2-week test-retest reliability for ISP-D was excellent (weighted κ = 0.801). Fifty-five participants completed the face-to-face interview for the validity study. The sensitivity, specificity, positive, and negative predictive values for major depressive disorder were 81.8% and 72.7%, 66.7%, and 85.7% respectively. The overall accuracy was 76.4%.
The evidence indicates the ISP-D is a reliable and valid online tool for assessing depression. Further studies should test the ISP-D in clinical settings to increase its applications in clinical environments with different populations and in a larger sample size.
Clinical depression is a highly prevalent mental illness. The World Health Organization Global Burden of Disease study in 1997 predicted that clinical depression will be the second most burdensome illness in the world by the year 2020 . The prevalence of depressive symptoms is high, ranging from 20% to 41% of the total population [2, 3]. The prevalence of major depressive disorder (MDD) and subsymdormal depressive (SSD) symptoms in an Australian population were found to be 6.8% and 12.9%, respectively . Prevalence of minor depressive disorder (MinD) has been reported to be in the range of 2.6% to 4.5% . Depression is associated with considerably high mortality through suicide, medical illness, and increased risk of accidental death . The high prevalence of depression, together with its significant morbidity, potential treatability, and costs, has made improved detection and management a priority for policymakers and health care agencies . Nevertheless many individuals with depression remain undetected or go untreated . Screening is a frequently proposed strategy for increasing detection of depression; however the diagnostic rate for depression by primary care practitioners is low [9–12]. Many primary care providers fail to diagnose depression in their patients either because they do not know how to screen for depression, have insufficient time, prefer not to label a patient with a mental illness, or do not know how to proceed with a positive result . Furthermore, the current cost of screening for depression may be too high relative to the projected benefit in quality-adjusted life years (QALYs) for many countries to implement. For example, Valenstein et al. found that the cost of screening every 5 years or of one-time screening were $50,988/QALY and $32,053/QALY, respectively . These obstacles impede the progress of screening for depression, which may result in delayed assessment or treatment of the disease. Such delays in diagnosis and treatment can ultimately result in profound consequences for patients including great financial costs, and disruption of interpersonal relationships and employment, as well as diminished overall mental and physical health. For these reasons, it is important to consider programs that can assess risk of depression in a manner that is easier, less costly, and more confidential than the face-to-face interview.
Computer-administered depression assessment programs, such as the computer-administered version of the Hamilton Depression Rating Scales , the computerized version of the Center for Epidemiological Studies Depressed Mood Scale (CES-D) , the computer-adaptive test for depression (D-CAT) , and the electronic version of Mini-International Neuropsychiatric Interview (M.I.N.I.)  have been developed in recent years. They give standardized information about a patient's psychopathology , and could reduce both the time and the tremendous financial cost of processing and analyzing diagnostic data . Previous researches have indicated that patients preferred the computer-mediated to the clinician interviews for assessment in sensitive areas such as suicide, substance abuse, sexual behavior, and HIV-related symptoms. Patients felt less embarrassed, more relaxed, and were generally more honest when using computer assessments [20–23]. Despite the wealth of encouraging research, few computer aids are being used in regular mental health care in nonresearch settings. One of the reasons for this may be that computer-administered programs still require that people come to a clinical setting to take the test.
The Internet provides advantages that may greatly improve the utility of computer-administered diagnostic programs. Firstly, Web-based programs are an anonymous method of accessing information about socially "unacceptable" illnesses . Furthermore Internet programs can reach a large number of individuals across a broad geographic area at a low cost, and may also identify many persons who have never received treatment . Individuals may surf the Internet conveniently to find information about whether they have depression or whether they should seek treatment. There are several programs through which people may be assessed for depression through the Internet, including an Internet-based depression screening test adapted from the CES-D scale  and the Web-Based Depression and Anxiety Test (WB-DAT) .
Because they are anonymous, Internet studies are difficult to perform. As yet, only one study has made an attempt to evaluate the validity of a Web-based instrument for depression . In that study, the researchers reported that the WB-DAT had good diagnostic concordance with Structured Clinical Interview for DSM-IV Axis I Disorders (SCID-I/P) diagnosis for MDD. However, the participants in the sample were recruited from several clinical research projects, a subject population that is not representative of Internet users at large. No previous academic reports could be found on the reliability of such programs. In the present study, we describe the design of the Internet-based Self-assessment Program for Depression (ISP-D) and evaluate the test-retest reliability and criterion validity of the ISP-D.
System and program design
The ISP-D was developed by a committee consisting of 16 certified psychiatrists in Taiwan. Its aim is to provide Internet users a concise, interactive, and individualized tool to self-assess depressive symptoms and screen for depressive disorders. It included 3 depression diagnoses: MDD, MinD, and SSD. These three diagnoses were chosen because they include the same nine symptoms and they represent different levels of depression severity. MDD was defined in the DSM-IV criteria . MinD was defined in accordance with the DSM-IV research criteria, that is, an episode with depressed mood or decreased interest, fulfilling at least two (but fewer than five) A criteria for a major depressive disorder [26, 27]. SSD was used to evaluate subthreshold depression, which was defined as having two or more symptoms of depression criteria excluding the A criteria [27–29]. The presence of each symptom was determined using one to three questions, which were adapted primarily from the M.I.N.I. , and Taiwanese Depression Questionnaire (TDQ) . Some revisions were made to improve understanding and readability on the Internet browser, e.g. more explanations or examples were included and different font colors or styles were incorporated to highlight important messages. Because the program was designed with branching, different users may get different numbers of questions (9 to 24) during a complete assessment.
The interactive ISP-D system was hosted on a Microsoft Windows 2000 server of PsychPark  in Taiwan and developed with Active Server Page 3.0 and HTML 3.0. The database system was constructed with a Microsoft SQL server 2000. Visual Basic (VB) script was used for interactive program and real-time data analysis to confirm that all questions had been completed. The Web site and Simple Mail Transfer Protocol (SMTP) service were hosted on Internet Information Services (IIS) 5.0. The transmission of account/password was protected by Secure Sockets Layer on the server. PsychPark, one of the largest and most popular mental health websites among the Chinese population, was established in 1995 as a virtual organization. PsychPark is free and membership is open to the general population.
Participants were volunteers recruited via the Internet. They found the study advertisement on PsychPark, which was listed on the Web resources of major portal websites, such as Yahoo and MSN. During the recruitment period, weekly electronic papers were sent to the subscribers of PsychPark e-paper. Individuals who wanted to know whether they had depressive disorders or wanted to recheck their existing depressive disorders were encouraged to join the study . The only inclusion criterion was that participants must be at least 18 years of age because the program was not designed to assess adolescent depression. Participants were excluded if they could not complete the study protocol. The study was approved by the Institute Review Board of the Yuli Veterans Hospital, Taiwan and was performed in compliance with the Helsinki Declaration.
Recruitment for the ISP-D test-retest reliability study was performed from September 2001 to January 2002. The entire process of the reliability study was completed via the Internet. When a participant entered the study website, the program began with an introduction about the aims and the procedure of the study. Each participant had to set up a PsychPark user account and password online to participate in the study, and was asked to visit a real clinic as soon as possible if he/she faced a severe mental condition. After reading a complete description of the study on the browser, the participant signed an informed consent form. Each participant was then asked questions on his or her sociodemographic data. There were brief instructions on how to answer subsequent questions about depressive symptoms and related conditions. We encouraged the participants to answer the questions honestly so that their assessment would achieve more accurate results.
After completing the self-administered assessments, the participants were asked to make an appointment with a psychiatrist of our research team near his/her location for the validity study. Our research team included 16 certified psychiatrists who were located in most of the geographic areas of Taiwan. The psychiatrists had been trained to use the M.I.N.I. , a clinical diagnostic interview based on the DSM-IV. They performed face-to-face semi-structured interviews with the M.I.N.I. and made psychiatric diagnoses according to the DSM-IV. Recruitment for the validity study was performed from September 2001 to May 2002 .
Subjects were classified as having MDD, MinD, SSD, or as no depression specified (ND). The system's test-retest reliability was analyzed by computing weighted kappa (κ) statistics for various time intervals. In order to examine the test-retest reliability for each diagnosis, κ values were recalculated for the groupings of MDD vs. non-MDD, MinD vs. non-MinD, and SSD vs. non-SSD. The test-retest reliability for each depressive symptom was calculated using the same method. To analyze the representation of our retest sample, sociodemographic data were compared between the retest responders and non-responders. To analyze the variables affecting test-retest agreement, we compared the sociodemographic data between participants with test-retest agreement and those without agreement.
The psychiatrist's diagnosis was regarded as the criterion for the validity statistics. Validity statistics included sensitivity, specificity, positive predictive value (PPV), and negative predictive value (NPV). To analyze the representation of our validity sample, sociodemographic data were compared between participants entering the validity study (validity sample) and those not entering the validity study (non-validity sample). To analyze the variables affecting the agreement between the diagnoses of ISP-D and psychiatrists, we compared the sociodemographic data between participants with agreement and those without agreement.
A Chi-square test was used to compare categorical variables and t-tests were used to compare continuous variables between group pairs. Standard deviations were calculated for all mean values. A p value of <0.05 (two-tailed) was considered significant for all statistical tests. The statistical software used was SPSS 10.0 for Windows. Weighted kappa was analyzed with the SAS System 9.0 for Windows.
Test-retest reliability study
Demographic data for the ISP-D test, retest, and validity samples
Test sample (n = 579)
Retest sample (n = 184)*
Validity sample (n = 55)†
P < 0.001
Mean ± SD
26.5 ± 6.6
26.2 ± 6.6
29.7 ± 7.5
Mean ± SD
14.9 ± 2.1
15.2 ± 1.9
15.3 ± 2.0
P = 0.027
ISP-D test-retest diagnosis reliabilities by test-retest time interval
(A)Within 2 weeks of test-retest time interval
(B) Between 2 and 4 weeks of test-retest time interval
(C) Greater than 4 weeks of test-retest time interval
ISP-D test-retest reliabilities for depressive symptoms by test-retest time interval
κ (agreement) by time interval
<2 weeks (N = 59)
≥2 to ≤4 weeks (N = 86)
>4 weeks (N = 39)
Diminished interest or pleasure
Insomnia or hypersomnia
Fatigue or loss of energy
Worthlessness or excessive guilt
Diminished ability to think or concentrate
Thoughts of death
Comparison of ISP-D diagnoses and psychiatrists' diagnoses for major depressive disorder
In our analysis, the ISP-D was found to be a reliable and valid tool for Internet users. The reliability and validity of data collected through the Internet have been questioned because the data were based on information provided directly by the patient's experience and data gathering were administered remotely. In the study, the overall test-retest reliability for ISP-D was excellent within a 2-week interval. Regarding the test-retest reliability for each diagnosis within a 2-week interval, MDD and SSD were found to be excellent. Only the reliability of MinD was low, which may be related to the relatively fewer number of cases of MinD. It is possible that MinD was not a stable diagnosis or that it may have been a stage of another depressive disorder. The high reliability of SSD implied that its diagnosis is worthy of further studies of validity and psychopathology. As the test-retest interval increased to greater than 2 weeks, the test-retest reliabilities of MDD and SSD decreased. Because MinD has similar symptomatic criteria to MDD and SSD, the good reliabilities of MDD and SSD lessened the possibility of problems with the questionnaire design or the media bias of the Internet.
The analysis of test-retest reliability for each depressive symptom showed that the reliabilities for "diminished interest or pleasure", "fatigue or loss of energy", and "thoughts of death" were excellent and that those for other symptoms were fair within a 2-week interval. The reliabilities were poor only for "depressed mood" and "thoughts of death" with a time interval between 2 and 4 weeks, and were poor only for "decreased appetite", "activity disturbance", and "fatigue or loss of energy" with a time interval greater than 4 weeks. The test-retest reliability with a time interval of greater than 2 weeks may be affected by the fluctuating course of depressive disorders themselves. According to Judd et al. ., the symptomatic course of MDD is dynamic and changeable, and MDD, MinD, and SSD symptom levels commonly alternate over time in the same patients as a symptomatic continuum of illness activity of a single clinical disease. Cuijpers and Smit found that the incidence of MDD in subjects with SSD is larger than in subjects without SSD . Thus in spite of a possible fluctuating course, the test-retest reliability of each depressive symptom in the ISP-D was fair to excellent within a 2-week interval.
For the validity study, the M.I.N.I. was applied in the clinical interview. The concordance between the M.I.N.I. and SCID-P diagnoses was demonstrated to be very good (sensitivity 96%, specificity 88%, PPV 87%, and NPV 97%) . κ values of inter-rater reliability for the M.I.N.I. and for test-retest reliability of MDD were previously reported to be 1.00 and 0.87, respectively , indicating that the M.I.N.I. is a good criterion for validity. As of yet, only one study has previously attempted to evaluate the validity of a Web-based instrument for depression diagnosis . In that study Farvolden et al. showed that the WB-DAT had sensitivity, specificity, PPV, and NPV of 79%, 89%, 75%, and 93%, respectively, relative to SCID-I/P diagnosis for MDD . Our result is similar to that of Farvolden et al. and confirms that the Internet may be a valid tool for the assessment of depression. Farvolden et al.'s study differed from the present study in that the participants in their study were recruited from several clinical research projects and the test was performed in a clinical environment. By contrast, our study participants were recruited remotely, directly from the Internet, and the ISP-D was used to evaluate depressive disorders with differing severities including MDD, MinD, and SSD. All the other validity studies for depression have been conducted in writing (pen and paper tests). One such study by Haringsma et al.  used the same assessment tool as that used in our study (the M.I.N.I.) to assess the criterion validity of the CES-D in a sample of self-referred seniors. They found that with the optimal cut-off score of 25 for MDD, sensitivity was 85%, specificity 64%, and PPN 63%. In Bagby's review article , the mean sensitivity, specificity, PPV, and NPV of the Hamilton Depression Rating Scale from 7 studies were found to be 76%, 91%, 77%, and 92%, respectively. In Nyklicek's study  of the Edinburgh Depression Scale (EDS) with 951 randomly selected women of peri-menopausal age, test sensitivity, specificity, PPV, and NPV were 58.8%, 95.0%, 49.1%, and 91.7%, respectively, with a cut-off score of 12. The results of our study demonstrated that screening for depression via the Internet may have similar validity as screening tests conducted in writing, most of which have been reported to be highly sensitive and specific .
Despite the satisfying results of the present examination of the reliability and validity of the ISP-D, the study has several limitations. The first limitation of our study is the potential for a self-selected effect. The high prevalence of MDD, MinD, and SSD in the study may be due to a self-selected effect of the participants. That is, people who had depressive symptoms may have been more motivated to participate in the study than non-depressed persons in the general population. On the other hand, many severely depressed patients may not have access to the Internet. The self-selected effect was apparent in the demographic characteristics of our study sample, with the majority of participants being young, single, well educated women. This sampling bias limits our ability to generalize our findings to the general population. The second limitation is the relatively low response rate in the test-retest reliability study, a characteristic inherent to Internet studies [41, 42]. Importantly, our statistical analysis showed that the sociodemographic characteristics did not affect the agreement of the test-retest reliability or the validity of the ISP-D, despite these limitations. The third limitation is that the kappa statistics for each diagnosis was recoded and recalculated. The interpretation should be cautious because of possible recoding bias.
Because the mean Internet interview time for the ISP-D was relatively short, and participants did not need to go to a clinic, the ISP-D may be useful as an auxiliary tool for screening or follow-up of depression as an alternative to the face-to-face interview. There is little additional cost to online interviews beyond maintenance of the system on the server. Computer-aided interviewing gives standardized information about a patient's psychopathology and diagnosis. It allows patients to work at their own pace and is available whenever a computer terminal is available. Furthermore, results can be scored and presented to the patients and/or clinicians immediately. Indeed the standardized manner of administration and scoring of computer-administered rating scales may actually improve reliability and insure greater completeness of the information gathered .
The ISP-D can generate personalized reports, which summarize each individual's responses and possible diagnostic categories. The automatically generated final report was designed to be printed and shared with a health care professional. In addition, links to related Internet articles and information about resources such as clinics and hospitals, virtual clinics, and online groups can be provided when participants have positive findings.
To our knowledge, this study was the first to investigate both the reliability and validity of a Web-based depression program. We found that the test-retest reliability of the ISP-D was excellent within a 2-week interval and its criterion validity was comparable to that observed with written tests. The current study offers Internet users an alternative way to assess depression by oneself in a short amount of time. The ISP-D provides a continuously available, inexpensive, and easily maintained depression screening method that is accessible to a large number of individuals across a broad geographic area. In order to broaden the application of the ISP-D, further studies conducted with various populations (outpatients, inpatients etc.) in larger sample sizes (especially for validity) should be conducted.
This study was supported by the National Science Council (NSC 95-2314-B-480 -003 and NSC 93-2314-B-480 -003) and Yuli Veterans Hospital, Taiwan. We would like to thank Steven E. Locke, M.D., Cheng-Hsiu Lin, M.D., Lih-Jong Shen, M.D. and Chih-Chien Lin, M.D. for their contributions to this study and discussion of ISP-D. We thank Susan S.F. Gau, M.D., Ph.D. for the consultation of statistics.
- Murray CJ, Lopez AD: Alternative projections of mortality and disability by cause 1990-2020: Global Burden of Disease Study. Lancet. 1997, 349: 1498-1504. 10.1016/S0140-6736(96)07492-2.View ArticlePubMedGoogle Scholar
- Zung WW, Broadhead WE, Roth ME: Prevalence of depressive symptoms in primary care. J Fam Pract. 1993, 37: 337-344.PubMedGoogle Scholar
- Crum RM, Cooper-Patrick L, Ford DE: Depressive symptoms among general medical patients: prevalence and one- year outcome. Psychosom Med. 1994, 56: 109-117.View ArticlePubMedGoogle Scholar
- Goldney RD, Fisher LJ, Dal GE, Taylor AW: Subsyndromal depression: prevalence, use of health services and quality of life in an Australian population. Soc Psychiatry Psychiatr Epidemiol. 2004, 39: 293-298. 10.1007/s00127-004-0745-5.View ArticlePubMedGoogle Scholar
- Hermens ML, van Hout HP, Terluin B, van der Windt DA, Beekman AT, van DR, de HM: The prognosis of minor depression in the general population: a systematic review. Gen Hosp Psychiatry. 2004, 26: 453-462. 10.1016/j.genhosppsych.2004.08.006.View ArticlePubMedGoogle Scholar
- Fawcett J: The morbidity and mortality of clinical depression. Int Clin Psychopharmacol. 1993, 8: 217-220. 10.1097/00004850-199300840-00002.View ArticlePubMedGoogle Scholar
- Valenstein M, Vijan S, Zeber JE, Boehm K, Buttar A: The cost-utility of screening for depression in primary care. Ann Intern Med. 2001, 134: 345-360.View ArticlePubMedGoogle Scholar
- Davidson JR, Meltzer-Brody SE: The underrecognition and undertreatment of depression: what is the breadth and depth of the problem?. J Clin Psychiatry. 1999, 60 Suppl 7: 4-9.PubMedGoogle Scholar
- Joukamaa M, Lehtinen V, Karlsson H: The ability of general practitioners to detect mental disorders in primary health care. Acta Psychiatr Scand. 1995, 91: 52-56.View ArticlePubMedGoogle Scholar
- Rost K, Zhang M, Fortney J, Smith J, Coyne J, Smith GR: Persistently poor outcomes of undetected major depression in primary care. Gen Hosp Psychiatry. 1998, 20: 12-20. 10.1016/S0163-8343(97)00095-9.View ArticlePubMedGoogle Scholar
- Cohen-Cole SA, Boker J, Bird J, Freeman AM: Psychiatric education for primary care: a pilot study of needs of residents. J Med Educ. 1982, 57: 931-936.PubMedGoogle Scholar
- Schulberg HC, Burns BJ: Mental disorders in primary care: epidemiologic, diagnostic, and treatment research directions. Gen Hosp Psychiatry. 1988, 10: 79-87. 10.1016/0163-8343(88)90092-8.View ArticlePubMedGoogle Scholar
- Klinkman MS: The role of algorithms in the detection and treatment of depression in primary care. J Clin Psychiatry. 2003, 64 Suppl 2: 19-23.PubMedGoogle Scholar
- Kobak KA, Reynolds WM, Griest JH: Computerized and clinician assessment of depression and anxiety: respondent evaluation and satisfaction. J Pers Assess. 1994, 63: 173-180. 10.1207/s15327752jpa6301_14.View ArticlePubMedGoogle Scholar
- Ogles BM, France CR, Lunnen KM, Bell MT, Goldfarb M: Computerized depression screening and awareness. Community Ment Health J. 1998, 34: 27-38. 10.1023/A:1018760128239.View ArticlePubMedGoogle Scholar
- Fliege H, Becker J, Walter OB, Bjorner JB, Klapp BF, Rose M: Development of a computer-adaptive test for depression (D-CAT). Qual Life Res. 2005, 14: 2277-2291. 10.1007/s11136-005-6651-9.View ArticlePubMedGoogle Scholar
- Sheehan DV, Lecrubier Y, Sheehan KH, Amorim P, Janavs J, Weiller E, Hergueta T, Baker R, Dunbar GC: The Mini-International Neuropsychiatric Interview (M.I.N.I.): the development and validation of a structured diagnostic psychiatric interview for DSM-IV and ICD-10. J Clin Psychiatry. 1998, 59 Suppl 20: 22-33.PubMedGoogle Scholar
- Cawthorpe D: An evaluation of a computer-based psychiatric assessment: evidence for expanded use. Cyberpsychol Behav. 2001, 4: 503-510. 10.1089/109493101750527060.View ArticlePubMedGoogle Scholar
- Lissman TL, Boehnlein JK: A critical review of internet information about depression. Psychiatr Serv. 2001, 52: 1046-1050. 10.1176/appi.ps.52.8.1046.View ArticlePubMedGoogle Scholar
- Black DW, Belsare G, Schlosser S: Clinical features, psychiatric comorbidity, and health-related quality of life in persons reporting compulsive computer use behavior. J Clin Psychiatry. 1999, 60: 839-844.View ArticlePubMedGoogle Scholar
- Locke SE, Kowaloff HB, Hoff RG, Safran C, Popovsky MA, Cotton DJ, Finkelstein DM, Page PL, Slack WV: Computer interview for screening blood donors for risk of HIV transmission. MD Comput. 1994, 11: 26-32.PubMedGoogle Scholar
- Erdman HP, Klein MH, Greist JH, Skare SS, Husted JJ, Robins LN, Helzer JE, Goldring E, Hamburger M, Miller JP: A comparison of two computer-administered versions of the NIMH Diagnostic Interview Schedule. J Psychiatr Res. 1992, 26: 85-95. 10.1016/0022-3956(92)90019-K.View ArticlePubMedGoogle Scholar
- Slack WV, Hicks GP, Reed CE, Van Cura LJ: A computer-based medical-history system. N Engl J Med. 1966, 274: 194-198.View ArticlePubMedGoogle Scholar
- Houston TK, Cooper LA, Vu HT, Kahn J, Toser J, Ford DE: Screening the public for depression through the Internet. Psychiatr Serv. 2001, 52: 362-367. 10.1176/appi.ps.52.3.362.View ArticlePubMedGoogle Scholar
- Farvolden P, McBride C, Bagby RM, Ravitz P: A Web-based screening instrument for depression and anxiety disorders in primary care. J Med Internet Res. 2003, 5: e23-10.2196/jmir.5.3.e23.View ArticlePubMedPubMed CentralGoogle Scholar
- American Psychiatric Association: Diagnostic and Statistical Manual of Mental Disorders. 4th ed ,. 1994, Washington (DC):Google Scholar
- Rapaport MH, Judd LL: Minor depressive disorder and subsyndromal depressive symptoms: functional impairment and response to treatment. J Affect Disord. 1998, 48: 227-232. 10.1016/S0165-0327(97)00196-1.View ArticlePubMedGoogle Scholar
- Judd LL, Akiskal HS, Paulus MP: The role and clinical significance of subsyndromal depressive symptoms (SSD) in unipolar major depressive disorder. J Affect Disord. 1997, 45: 5-17. 10.1016/S0165-0327(97)00055-4.View ArticlePubMedGoogle Scholar
- Sadek N, Bona J: Subsyndromal symptomatic depression: a new concept. Depress Anxiety. 2000, 12: 30-39. 10.1002/1520-6394(2000)12:1<30::AID-DA4>3.0.CO;2-P.View ArticlePubMedGoogle Scholar
- Lee Y, Yang MJ, Lai TJ, Chiu NM, Chau TT: Development of the Taiwanese Depression Questionnaire. Chang Gung Med J. 2000, 23: 688-704.PubMedGoogle Scholar
- PsychPark . 2007, [http://www.psychpark.org/]
- Internet-based Self-assessment Program for Depression (ISP-D, in traditional Chinese). 2007, [http://www.psychpark.org/guideline/depression/test/]
- Lin C, Li YC, Bai YM, Tsai SJ, Hsiao MC, Wu CH, Liu CY, Chen JY: The Validity of an Internet-based Self-assessment Program for Depression. Proc AMIA Symp. 2003, 911-Google Scholar
- Cicchetti DV, Sparrow SA: Developing criteria for establishing interrater reliability of specific items: applications to assessment of adaptive behavior. Am J Ment Defic. 1981, 86: 127-137.PubMedGoogle Scholar
- Judd LL, Akiskal HS, Maser JD, Zeller PJ, Endicott J, Coryell W, Paulus MP, Kunovac JL, Leon AC, Mueller TI, Rice JA, Keller MB: A prospective 12-year study of subsyndromal and syndromal depressive symptoms in unipolar major depressive disorders. Arch Gen Psychiatry. 1998, 55: 694-700. 10.1001/archpsyc.55.8.694.View ArticlePubMedGoogle Scholar
- Cuijpers P, Smit F: Subthreshold depression as a risk indicator for major depressive disorder: a systematic review of prospective studies. Acta Psychiatr Scand. 2004, 109: 325-331. 10.1111/j.1600-0447.2004.00301.x.View ArticlePubMedGoogle Scholar
- Haringsma R, Engels GI, Beekman AT, Spinhoven P: The criterion validity of the Center for Epidemiological Studies Depression Scale (CES-D) in a sample of self-referred elders with depressive symptomatology. Int J Geriatr Psychiatry. 2004, 19: 558-563. 10.1002/gps.1130.View ArticlePubMedGoogle Scholar
- Bagby RM, Ryder AG, Schuller DR, Marshall MB: The hamilton depression rating scale: has the gold standard become a lead weight?. Am J Psychiatry. 2004, 161: 2163-2177. 10.1176/appi.ajp.161.12.2163.View ArticlePubMedGoogle Scholar
- Nyklicek I, Scherders MJ, Pop VJ: Multiple assessments of depressive symptoms as an index of depression in population-based samples. Psychiatry Res. 2004, 128: 111-116. 10.1016/j.psychres.2004.05.017.View ArticlePubMedGoogle Scholar
- McAlpine DD, Wilson AR: Screening for depression in primary care: what do we still need to know?. Depress Anxiety. 2004, 19: 137-145. 10.1002/da.20000.View ArticlePubMedGoogle Scholar
- Lin CC, Bai Y, Chen JY: Reliability of information provided by patients of a virtual psychiatric clinic. Psychiatr Serv. 2003, 54: 1167-1168. 10.1176/appi.ps.54.8.1167.View ArticlePubMedGoogle Scholar
- Umefjord G, Petersson G, Hamberg K: Reasons for consulting a doctor on the Internet: Web survey of users of an Ask the Doctor service. J Med Internet Res. 2003, 5: e26-10.2196/jmir.5.4.e26.View ArticlePubMedPubMed CentralGoogle Scholar
- The pre-publication history for this paper can be accessed here:http://www.biomedcentral.com/1471-244X/7/12/prepub