Skip to main content

Archived Comments for: The Four-Dimensional Symptom Questionnaire (4DSQ): a validation study of a multidimensional self-report questionnaire to assess distress, depression, anxiety and somatization

Back to article

  1. Does the 4DSQ provide any added value to general practitioners’ diagnoses?

    Kaj Sparle Christensen, Research Unit for General Practice, University of Aarhus

    1 November 2006

    We have read the recently published article about the Four-Dimensional Symptom Questionnaire (4DSQ) with great interest. The development of self-evaluation questionnaires for the assessment of mental disorders in primary care is important both for clinical work and research. However, we have some serious concerns about the study by B Terluin et al:

    1. The 4DSQ has been author-copyrighted although it includes 50 items that resemble items from the Symptom Checklist very much.

    2. Authors have assessed criterion validity of distress and somatisation using GP diagnoses as the gold standard. As a consequence the 4DSQ may bring no added value when used for assessment of distress and somatisation in general practice (Christensen et al. 2005). Furthermore the GPs’ diagnoses are subject to large variations (Rosendal et al. 2003) and do not necessarily agree with other concepts such as the diagnosis of somatoform disorder (Schilte et al. 2000; Fink et al. 2005; Dew et al. 2005)

    3. Criterion validity of Depression was assessed using 55 patients selected by GPs for SDI interview twice by their GP with an interval of 1-2 days. Patients were asked to fill in the 4DSQ at both occasions. In order to increase the power of analysis, the measurements of both times were combined (102 complete ‘cases’) as if the retest were independent observations. Obviously this is not the case.

    4. Criterion validity of Anxiety was assessed using the SCID interview but on a sample of patients selected on the basis of a positive result of a screening test for anxiety (circularism).

    5. Using imprecise and different patient selection criteria makes no sense when one wishes to assess criterion validity of the 4DSQ scales in primary care. We found no information on whether the 4DSQ was answered before or after diagnostic assessment. The authors have put a lot of work into the evaluation of construct validity of the 4DSQ, but when it comes to criterion validity we think that much work needs to be done yet.

    We appreciate the effort put into the development and validation of patient questionnaires in relation to mental disorders by Terluin and others. GPs and patients are increasingly familiar with the use of self-evaluation questionnaires and good questionnaires used for case-finding/high-risk screening may improve the recognition of mental disorders in primary care (Christensen et al. 2003). General screening, however, seems of little effect (Gilbody et al. 2001) and we therefore find it important that new questionnaires provide added value to the GPs’ own diagnoses in addition to rigorous validation.

    Kaj Sparle Christensen, senior researcher, PhD, general practitioner

    Marianne Rosendal, senior researcher, PhD, general practitioner

    The Research Unit for General Practice

    Aarhus University

    Vennelyst Boulevard 6

    DK-8000 Aarhus C

    Denmark

    e-mail: kasc@alm.au.dk

    Reference List

    Christensen, K. S., Toft, T., Frostholm, L., Oernboel, E., Fink, P. & Olesen, F. 2003. The FIP Study: A Randomised Controlled Trial of Screening and Registration of Psychiatric Disorders. British Journal of General practice 53: 758-763.

    Christensen, K. S., Toft, T., Frostholm, L., Ornbol, E., Fink, P. & Olesen, F. 2005. Screening for common mental disorders: who will benefit? Results from a randomised clinical trial. Fam. Pract. 22: 428-434.

    Dew, K., Dowell, A., McLeod, D., Collings, S. & Bushnell, J. 2005. "This glorious twilight zone of uncertainty": mental health consultations in general practice in New Zealand. Soc. Sci. Med. 61: 1189-1200.

    Fink, P., Rosendal, M. & Olesen, F. 2005. The classification of somatisation and functional somatic symptoms in primary care. Aust. N. Z. J Psychiatry 39: 772-781.

    Gilbody, S. M., House, A. O. & Sheldon, T. A. 2001. Routinely administered questionnaires for depression and anxiety: systematic review. BMJ 322: 406-409.

    Rosendal, M., Bro, F., Fink, P., Christensen, K. S. & Olesen, F. 2003. Diagnosis of somatisation: effect of an educational intervention in a cluster randomised controlled trial. Br. J. Gen. Pract. 53: 917-922.

    Schilte, A. F., Portegijs, P. J., Blankenstein, A. H. & Knottnerus, J. A. 2000. Somatisation in primary care: clinical judgement and standardised measurement compared. Soc. Psychiatry Psychiatr. Epidemiol. 35: 276-282.

    Competing interests

    None

  2. Does the 4DSQ provide any added value to general practitioners’ diagnoses? Authors’ reply

    Berend Terluin, Department of General Practice, VU University Medical Center, Amsterdam

    20 February 2007

    We appreciate the valuable comments made by Christensen and Rosendal, who apparently have scrutinised our paper (Terluin et al., 2006), which we consider a great honour.

    1. Christensen and Rosendal observed some resemblance between the 4DSQ and the Symptom Checklist. Some resemblance is, of course, inevitable since the possibilities for the self-description of distress, depression, anxiety and somatization are limited. However, we would like to stress that none of the 4DSQ items have been copied from the Symptom Checklist or any other existing questionnaire. Moreover, it should be noticed that the phrasing of the questions and the response categories of the 4DSQ are quite different from the Symptom Checklist. The 4DSQ has been author-copyrighted to prevent commercial use of the questionnaire, but the 4DSQ is freely available for non-commercial use in health care and research (see 4DSQ).

    2. Christensen and Rosendal stated that, since the 4DSQ Distress and Somatization scales have been validated against the GPs’ diagnoses, the 4DSQ may not add any diagnostic value to the GP’s own diagnosis. This is, in itself, a valid conclusion. However, we did not aim to demonstrate any added value of the 4DSQ. We set out to demonstrate validity, which can be considered to be a prerequisite of any added value to be demonstrated in the future. We did acknowledge the limitations of our “criterion”, which can hardly be considered a gold standard. However, the (moderate) associations of the Distress and Somatization scores with the GPs’ diagnoses do not exclude any added value of the 4DSQ. There is a real possibility that the 4DSQ is, in fact, a better measure of distress and somatization than the GP’s diagnosis. However, correlations are not suitable to demonstrate this. We will return to the added-value issue at the end of this reply.

    3. Christensen and Rosendal rightly criticised our analysis of the criterion validity of the Depression score because we ignored the clustering of measurements within patients. Therefore, we present more appropriate analyses below.

    Table 1 shows separate ROC analyses for T1 and T2. The results are in line with those presented in Figure 2 in the article under discussion (Terluin et al., 2006). The confidence intervals were a bit larger, as expected, because of smaller numbers in the subgroups. We used multilevel analysis (software program MLwiN 1.1) to estimate the differences in mean 4DSQ scores between patients with and without a major depression diagnosis while controlling for the clustering of measurements within patients. Levels were measurements and patients. A comparison between Table 2 (below) and the corresponding Table 6 of the article under discussion shows that the multilevel analysis could no longer demonstrate significant differences in the Anxiety and Somatization scores, whereas the difference in Depression scores became relatively more pronounced. The logistic regression analysis, to predict major depression by the 4DSQ scales, was also repeated using multilevel analysis. Table 3 shows that the results were comparable with those presented in Table 7 in the article under discussion.

    4. Christensen and Rosendal also criticised the selection of the sample in which we assessed the criterion validity of the 4DSQ Anxiety scale. In this case circularity is not the problem. The pre-selection on the basis of anxiety symptoms did not make it easier to distinguish patients with an anxiety disorder from those without. Rather, it made it more difficult because the patients without diagnosable (SCID) anxiety disorders were by no means typical of all patients without anxiety disorders in general practice. Probably, most of those patients without a diagnosable anxiety disorder (24% of the sample) were (very) close to the diagnostic threshold of an anxiety disorder making it difficult for any assessment instrument to distinguish between patients with and without an anxiety disorder diagnosis.

    5. Christensen and Rosendal think that much work needs to be done with respect to the criterion validity of the 4DSQ. We agree with them. Certainly, since no true gold standards are available for distress, depression, anxiety and somatization, the final word on the criterion validity of the 4DSQ is not to be expected in the near future. Nevertheless, we hope to soon present more data on validity issues.

    Finally, Christensen and Rosendal expressed their view that new questionnaires should provide added value to the GPs’ own diagnosis. We could not agree with them more, and we would like to add that new questionnaires should specifically be helpful to our patients. GPs and other health care providers would not be inclined to start using the 4DSQ and to continue to do so if this would not bring any benefit to their patients and themselves. We have not yet examined this extensively in a systematic manner, but nevertheless we have made some interesting observations.

    From our own experience and from what we have heard from colleagues who use the 4DSQ in general practice, we have learned that GPs find using the 4DSQ a very efficient method to get a first impression of the patients’ psychological problems along the four most relevant dimensions. The Distress dimension is judged to be especially relevant for primary care as it permits to identify psychological problems that are often stress-related and not classifiable according to formal psychiatric criteria.

    In addition, perhaps the most useful and gratifying application of the 4DSQ is to help patients acknowledge their distress and discuss it with their doctor. In patients presenting with physical symptoms, GPs often find it difficult to explore psychosocial issues. Many of such patients are worried about their symptoms and seek reassurance. Although they may have some latent awareness of their psychosocial problems, their preoccupation with possible disease prevents them from fully acknowledging these problems. However, most patients are willing to accept a work-up along two tracks simultaneously, a somatic and a psychosocial one. While the doctor performs a somatic diagnostic work-up (physical examination, laboratory tests, etc.), the patient can be persuaded to list any psychological symptoms by filling in the 4DSQ. When the patient returns to discuss the results of the tests, the 4DSQ scores can be discussed. In our experience, the process of filling in the 4DSQ increases the patients’ awareness of their distress and its causes, as well as their readiness to discuss this with the doctor. We have rarely encountered patients denying any distress when they had an elevated Distress score. The added value of the 4DSQ appears to go beyond diagnosis alone: it facilitates the doctor-patient communication.

    Berend Terluin, MD, PhD, general practitioner,

    on behalf of the co-authors,

    Department of General Practice,

    VU University Medical Center

    Van der Boechorststraat 7

    1081 BT Amsterdam

    The Netherlands

    e-mail: b.terluin@vumc.nl

    Reference

    Terluin B, Van Marwijk HWJ, Adèr HJ, De Vet HCW, Penninx BWJH, Hermens MLM, Van Boeijen CA, Van Balkom AJLM, Van der Klink JJL, Stalman WAB: The Four-Dimensional Symptom Questionnaire (4DSQ): a validation study of a multidimensional self-report questionnaire to assess distress, depression, anxiety and somatization. BMC Psychiatry 2006; 6:34.

    Table 1: Area’s under the ROC curve (AUC) for the 4DSQ scores with respect to detecting a DSM-IV major depression

    4DSQ scales- - - - T1 (N = 54) - - - - 95% CI - - - - - - T2 (N = 48) - - - - 95% CI

    Distress- - - - - - - - - - 0.81 - - - - - 0.70-0.93 - - - - - - - - 0.81 - - - - - 0.68-0.93

    Depression - - - - - - - 0.81 - - - - - 0.70-0.93 - - - - - - - - 0.85 - - - - - 0.75-0.96

    Anxiety - - - - - - - - - - 0.62 - - - - - 0.47-0.77 - - - - - - - - 0.64 - - - - - 0.49-0.80

    Somatization - - - - - - 0.69 - - - - - 0.55-0.84 - - - - - - - - 0.67 - - - - - 0.52-0.83

    Table 2: Differences in mean 4DSQ scores between patients with and without a DSM-IV major depression diagnosis

    4DSQ scales - - - scale range - - - mean difference - - - - - - se - - - - - - - - - - - - p

    Distress - - - - - - - - - 0-32 - - - - - - - - - 2.18 - - - - - - - - - 0.92 - - - - - - - - - 0.017

    Depression- - - - - - - 0-12 - - - - - - - - - 3.54 - - - - - - - - - 0.80 - - - - - - - - < 0.0001

    Anxiety- - - - - - - - - - 0-24 - - - - - - - - - 0.07 - - - - - - - - - 0.81 - - - - - - - - - 0.936

    Somatization- - - - - - 0-32 - - - - - - - - - 0.84 - - - - - - - - - 0.96 - - - - - - - - - 0.379

    Table 3: Logistic regression analysis with major depression diagnosis as dependent variable and the 4DSQ scores as independent variables

    - - - - - - - - - - - - - - - - - - - - Odds ratioa - - - - - - - 95% CI - - - - - - - - - - - p

    Initial model

    4DSQ Distress- - - - - - - - - - - 1.11 - - - - - - - - - 0.90, 1.37 - - - - - - - - - 0.342

    4DSQ Depression - - - - - - - - 1.49 - - - - - - - - - 1.07, 2.07 - - - - - - - - - 0.018

    4DSQ Anxiety - - - - - - - - - - - 0.98 - - - - - - - - - 0.85, 1.18 - - - - - - - - - 0.849

    4DSQ Somatization - - - - - - - 1.11 - - - - - - - - - 0.99, 1.26 - - - - - - - - - 0.077

    Final model

    4DSQ Depression - - - - - - - - 1.62 - - - - - - - - - 1.29, 2.06 - - - - - - - - - 0.0002

    4DSQ Somatization - - - - - - - 1.13 - - - - - - - - - 1.01, 1.25 - - - - - - - - - 0.0257

    a the odds ratio is associated with one unit of the scale

    Competing interests

    BT is the copyright owner of the 4DSQ and receives copyright fees from companies that use the 4DSQ on a commercial basis. BT received fees from various institutions for workshops on the application of the 4DSQ in primary care settings.

Advertisement