Skip to main content

Only one cluster per condition is an invalid design for a cluster randomized trial and should be re-labeled a quasi-experimental study. Response to: “Effect of behavioral activation on time and frequency domain heart rate variability in older adults with subthreshold depression: a cluster randomized controlled trial in Thailand”

Abstract

Ayudhaya et al. examined the effect of Behavioral Activation on daily step count and heart rate variability among older adults with depression in a study labeled a cluster randomized controlled trial (cRCT). However, only one cluster was assigned to either of the study conditions. Such a design would have zero degrees of freedom for inferential testing, because the variation due to cluster membership cannot be estimated apart from the variation due to treatment assignment. Thus, the intervention effect is completely confounded with the cluster effect. The study should be labeled a quasi-experimental study, not a cRCT. Accordingly, the numerical results should be interpreted as associations but not evidence for causal relationships.

Peer Review reports

Ayudhaya et al. [1] conducted a study to examine the effect of Behavioral Activation (BA) on daily step count (as a physical activity indicator) and heart rate variability (as a depression-related autonomic function indicator) among community-dwelling older adults with mild to moderate depression. The article is labeled as a cluster randomized controlled trial (cRCT). However, there is only one cluster (i.e., hospitals) assigned to either of the study conditions (BA plus usual care or usual care only), which makes the study design invalid for estimating the causal effect of treatment assignment. Forty-one participants were recruited in each hospital after they were assigned to study conditions.

In cRCTs, there can be and typically are correlated data within each cluster (i.e., unit of randomization) so “errors” (loosely speaking, model residuals) cannot be assumed to be independent across study participants within clusters. Moreover, when clusters (rather than individuals) are assigned to study conditions, the study has a hierarchical structure (individuals nested in clusters) [2]. Thus, the degrees of freedom for testing the intervention effect depends both on the number of individual participants and on the number of clusters per condition [3]. A purported cRCT with two conditions and only one cluster assigned to each condition, such as Ayudhaya et al.’s study [1], would have zero degrees of freedom (n1 + n2–2) for inferential testing. That is, the variation due to cluster membership cannot be estimated apart from the variation due to treatment assignment, and a valid analysis of the intervention effect could not exist [4].

We commend Ayudhaya et al. for addressing an important question about depression among the older population, and for their collegiality in providing clarification about their study design. The authors’ clarification confirmed our understanding from reading the published methods.

Because in Ayudhaya et al.’s study [1] the intervention effect is completely confounded with the cluster effect, it should be labeled a quasi-experimental study, not a cRCT. Accordingly, the numerical results should be interpreted as associations between BA and the outcomes but not evidence for causal relationships [5].

Availability of data and materials

Not applicable.

Abbreviations

BA:

Behavioral Activation

cRCT:

Cluster Randomized Controlled Trial

References

  1. Ayudhaya WSN, Pityaratstian N, Eungpinichpong W, et al. Effect of behavioral activation on time and frequency domain heart rate variability in older adults with subthreshold depression: a cluster randomized controlled trial in Thailand. BMC Psychiatry. 2022;22:319.

    Article  Google Scholar 

  2. Murray DM, Varnell SP, Blitstein JL. Design and analysis of group-randomized trials: a review of recent methodological developments. Am J Public Health. 2004;94:423–32.

    Article  Google Scholar 

  3. Jamshidi-Naeini Y, Brown AW, Mehta T, et al. A practical decision tree to support editorial adjudication of submitted parallel cluster randomized controlled trials. Obesity. 2022;30:565–70.

    Article  Google Scholar 

  4. Varnell SP, Murray DM, Baker WL. An evaluation of analysis options for the one-group-per-condition design: can any of the alternatives overcome the problems inherent in this design? Eval Rev. 2001;25:440–53.

    Article  CAS  Google Scholar 

  5. Shadish WR, Cook TD, Campbell DT. Experimental and quasi-experimental designs for generalized causal inference. Boston: Houghton Mifflin. 2002.

Download references

Acknowledgements

Not applicable.

Funding

Authors supported in part by NIH grants R25DK099080, R25HL124208, U24AG056053, P30AG050886, and the Gordon and Betty Moore Foundation. The opinions expressed are those of the authors and do not necessarily represent those of the NIH or any other organization.

Author information

Authors and Affiliations

Authors

Contributions

Conceptualization: DBA, YJ-N, LG-A, ABS, CJV; Investigation: YJ-N; Writing original draft: YJ-N; Review and editing: DBA, LG-A, ABS, CJV. All authors approved the final draft.

Corresponding author

Correspondence to David B. Allison.

Ethics declarations

Ethics approval and consent to participate

Not applicable.

Consent for publication

Not applicable.

Competing interests

Dr. Allison and his institution (Indiana University) have received payments for consultation, grants, contracts, in-kind donations, and contributions from multiple for-profit and not-for-profit entities interested in statistical design and analysis of experiments, but not directly related to the research questions addressed in the paper question.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Jamshidi-Naeini, Y., Golzarri-Arroyo, L., Siddique, A.B. et al. Only one cluster per condition is an invalid design for a cluster randomized trial and should be re-labeled a quasi-experimental study. Response to: “Effect of behavioral activation on time and frequency domain heart rate variability in older adults with subthreshold depression: a cluster randomized controlled trial in Thailand”. BMC Psychiatry 23, 35 (2023). https://doi.org/10.1186/s12888-022-04491-0

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1186/s12888-022-04491-0

Keywords

  • Cluster randomized trial
  • Degrees of freedom
  • Inferential testing
  • Nesting