- Research article
- Open Access
The Therapeutic Engagement Questionnaire (TEQ): a service user-focused mental health nursing outcome metric
BMC Psychiatry volume 19, Article number: 384 (2019)
Therapeutic engagement (TE) has been described as the crux of mental health nursing but despite its perceived importance, to date, there is no measurement tool that captures it. As a result, there is no way of determining the contribution of mental health nursing interaction to service user recovery, in acute inpatient mental health settings or the wider care quality agenda.
To develop and validate a TE measurement tool in partnership with Service Users (SUs) and Registered Mental Health Nurses (RMHNs). The TEQ was developed in 3 stages: 1) item generation (and pre-testing), 2) item reduction using Principal Component Analysis (PCA), and 3) validation across Mental Health Trusts in England.
The final questionnaire has two versions, (SU and RMHN version), each scored within two contexts (1–1 SU-RMHN interactions and overall environment and atmosphere of the ward) and includes 20 items with two sub-scales (care interactions and care delivery). Psychometric evaluation of the TEQ shows high inter-scale correlations (0.66–0.95 SU; 0.57–0.90 RMHN), sound sub-scale internal consistency (> 0.95), concurrent validity (> 0.60) and adequate score variability for both versions of the TEQ. In summary, the TEQ behaves well as a measurement tool.
The TEQ can determine the collaborative and empathic nature of RMHN-SU interactions, capture if SUs are treated with dignity and respect and recognise that the principles of the recovery approach are being respected. The TEQ can also provide robust monitoring of nursing activity, offer opportunity for transparency of activity, feed into healthcare organizations’ key performance indicators and provide reassurance about the nature and quality of nurses’ work.
Since the inception of Peplau’s (1952)  seminal work which emphasised the primacy of the nurse–patient relationship, therapeutic engagement (TE) has been considered the crux of psychiatric nursing . Current evidence suggests that TE is beneficial, and is of significant clinical importance [3,4,5]. Indeed, service users (SUs) value positive attitudes, being listened to, and being able to trust those who provide care.
Given the perceived importance of therapeutic engagement in mental health nursing it is necessary to evaluate how such engagement contributes to SU recovery and the overall quality agenda of healthcare organisations [6, 7]. Such a metric has not yet been developed, consequently there is no way to measure the nature of ‘face-time’ and TE as part of the SU experience as perceived by SUs and registered mental health nurses (RMHNs).
An array of rating scales exist to measure therapeutic engagement with a variety of titles, for example the Working Alliance Inventory (WAI) , and the Helping Alliance Scale (HAS) . Their tendency is either to measure TE within research , or measure the qualitative nature, of TE, making it difficult to quantify and assess the quality. The Scale To Assess the Therapeutic Relationship (STAR) , was developed to assess the relationship between multidisciplinary clinicians and SUs who experience mental illness; despite its merits it was not designed in partnership with SUs in acute inpatient mental healthcare settings [10, 11]. Whilst the co-production of tools in partnership with SUs is in its infancy, such involvement gives greater credibility to the final product.
From the tools identified, none of them assess the 1-1 SU-RMHN interactions in acute inpatient mental healthcare settings nor the overall environment and atmosphere of the ward.
The aim of this study was to develop such a tool using psychometric methodology, resulting in a short and simple tool that can quantify and recognise nursing engagement activity in the monitoring and enhancement of SU care and recovery.
Stage 1 of 3 – item generation
A previous article by Chambers et al. (2016)  provides an explicit description of the questionnaire, the two versions and the two contexts. In this article, stage 1 of the development and validation of TEQ entitled the item generation stage, which took place in co-production with SUs and RMHNs, is described in detail. In the item generation stage, the original 25-item (statement) questionnaire encompassing five themes (with a 4 point Likert scale response format) was generated based on the literature, feedback from a therapeutic engagement workshop (n=70) involving service users, clinical nurses and nurse academics, findings from the ‘Lived Experience of Detained Patients’ project (in-depth interviews with 19 detained service users)  and review of the tool (pre-testing) by both parties. Both parties liked the use of ‘me’ or ‘I’ in the statements and that the positively worded statements appeared to be sensitive to care interactions and delivery. Clarity of instructions, statement-stems and the 4-point Likert response options were also discussed and agreed. Clinical appropriateness of each statement was ensured by the RMHNs.
The TEQ has a SU and RMHN version and both are scored in relation to two contexts - SU-RMHN 1-1 interactions in acute inpatient mental healthcare settings and the environment and atmosphere of the ward12.
This article describes the next two stages of its development, item reduction and psychometric evaluation, of both versions of the tool.
Stage 2 of 3 – item reduction
In this stage, the questionnaire was administered to 86 SUs and 68 RMHNS from 4 Mental Health NHS Trusts across different regions of England. Standard item reduction techniques were then used to develop the sub-scales - care delivery and care interactions – in both versions of the TEQ.
Procedure and participants
The newly revised TEQ which encompasses 20-items instead of the original 25 was administered face-to-face to 154 participants. Purposive sampling was adopted. Ward managers identified eligible SUs and RMHNs for the study who were then invited by the research team to participate in the study. Adult service users with the following eligibility criteria were invited to complete the SU version of the questionnaire within their care environment (with support from a person of their choice if needed who was not their named nurse): residing for more than one week within an adult acute inpatient mental healthcare setting, mental capacity to consent (as determined by the ward nursing staff and treating Psychiatrist using the four-point British Medical Association mental capacity test) and good command of the English language as the TEQ has been initially developed in English. Registered mental health nurses working in an acute inpatient mental health setting attached to the Mental Health NHS Trust participating in the study with a permanent work contract were invited to complete the revised nurse version of the questionnaire within their work environment. Data were collected within a 3-month period. The response rate was not calculated as the project was not able to access data on service users who had not consented to participate.
The data collected from the questionnaires underwent a Principal Components Analysis (PCA) with oblimin rotation. Principal Component Analysis (PCA) is adequate for the development of a measurement tool and is the most commonly used in exploratory factor analysis determining the underlying domains (factors and structural validity) of measurement tools . Oblimin rotation is the standard method and allows the factor to be correlated which often provides sound factor structures. Factors were retained with eigenvalues over one, and items were chosen if the items loaded on the factor > 0.40 as per ‘rule of thumb’ .
Two factors were evident and were named care interactions and care delivery by our ‘expert’ group during the item generation stage. For the environment and atmosphere context of the SU version, 8 items fell under the sub-scale care delivery and 12 items in the care interactions sub-scale. For the 1-1 interactions context of the SU version of the TEQ, 9 items also fell under the care delivery factor and 11 under the care interactions factor.
For 1-1 interactions, two factors were retained - the first factor was very strong with 66% of variance. For the environment and atmosphere context of the ward, only one factor was retained (75% of variance). The loadings were not as clearly separated as we had previously observed . Based on the statistical analysis at this stage of the development of the TEQ and SU feedback about all the items we saw no compelling reason to drop items prior to the validation stage.
The internal consistency for the sub-scales was examined using Cronbach’s α  estimates exceeded 0.80 which indicated good consistency . The αs were very high (α = 0.98 for 1-1 interactions; 0.97 for environment and atmosphere context of the ward), which remained the same when each item was deleted one at a time from each sub-scale.
For both versions of the TEQ, missing items were low. Participants who had any missing data were deleted as per rationale for multivariate analysis. Seventy-two per cent, 79%, 81% and 92% respectively of respondents endorsed all 20 items (100% complete data). The number of respondents who failed to complete 3 or more items was very low (0.01%, 0.00%, 0.03%, 0.00% respectively). Therefore, scale scores could be computed for most of the respondents. Distribution for item response scales was symmetrical and not skewed and items within each scale had similar mean scores and standard deviations. All correlations between items and the total score were high (0.77-0.91). Scales scores spanned the entire scale range and were not notably skewed, mean scores were near the scale mid-point however floor and ceiling effects were moderately high (maximum 30.3%).
For the environment and atmosphere context of the RMHN version, 9 items fell under care delivery and 11 items in the care interactions sub-scales; these items were the same as the SU version. For the 1-1 interactions context of the RMHN version of the TEQ, 4 items also fell under the care delivery factor and 16 under the care interactions factor.
For 1-1 interactions, two factors were retained with eigenvalues over 1, and they explained 61% of the variance; the first factor was very strong with 66% of variance. For the environment and atmosphere context of the ward, two factors were retained (62% of variance). After rotation, both sets of responses were loaded similarly to the pattern observed for the service users.
The internal consistencies for the sub-scales were very high (α = 0.95 for 1-1 interactions; 0.96 for environment and atmosphere of the ward), which remained the same when each item was deleted one at a time from each sub-scale.
After item reduction, the 4-point Likert scale response format remained. It should also be known that at this stage both groups of participants were given a platform to state any problems with the wording and content of the statements, to produce any missing statements and/or topic areas to be included in the tool. There were no examples of this therefore the scale was not revised prior to analysis.
Stage 3 of 3 – validation
In this stage, the psychometric properties of the TEQ i.e. data quality, scaling assumptions, acceptability, reliability and validity, were evaluated in a national sample (with wide geographical spread) of 628 SUs and 543 RMHNs. Tables 1 and 2 show the response data for each sub-scale of the service user version and nurse version of the TEQ.
Procedure and participants
To assess the psychometric properties of the TEQ, the questionnaire was authenticated against two validated questionnaires that measure therapeutic alliance in community settings and research respectively, The Scale To Assess the Therapeutic Relationship (STAR)  and The Helping Alliance Scale (HAS) . The STAR assesses therapeutic relationships in community psychiatry. The clinician version of the STAR has three subscales: Positive Collaboration, Emotional Difficulties, Positive Clinician Input. The patient version of the STAR also has three sub-scales: Positive Collaboration, Positive Clinician Input, Non-Supportive Clinician Input, and the HAS measures the strength of the patient-therapist therapeutic alliance.
The aim of the validation was to examine the evidence that the TEQ was a measure of therapeutic engagement for the populations mentioned. This authentication occurred with service users and RMHNs in 26 England Mental Health Trusts with wide geographical spread. Table 3 shows the participant characteristics which appeared to cover the diversity of service users and nurses. Eligible services users and RMHNs (as per the item reduction stage) from all 26 Trusts were invited to participate in this stage of the study by completing the three questionnaires either in their care or work environment.
In total, 628 SUs and 543 RMHNs completed the appropriate version of the TEQ across the participating Trusts. The number of SUs and RMHNs who completed the whole HAS was 392 and 401 respectively; the number of SUs and RMHNs who completed the whole STAR was 445 and 453 respectively.
Data analysis and Results
To help determine the TEQ’s adequacy, convergent validity was determined by examining the sub-scale correlations between the TEQ sub-scales in each version of the TEQ using Pearson’s product-moment correlation coefficient. We predicted that the sub-scales in each version of the TEQ and type of questioning of the TEQ would correlate highly (>0.70). Correlations ranged from 0.66-0.95. Strong correlations were also found in the RMHN version of the TEQ with the exception of the care delivery sub-scale in each of the contexts of the TEQ i.e. 1:1 and environment and atmosphere of the ward (0.57). Tables 4 and 5 show these correlations.
It should be known that limited numbers of SUs and RMHNs missed responding to 3 items or more.
Concurrent validity was determined by examining the sub-scale correlations between the TEQ sub-scales with the STAR and HAS. The analyses showed significant, moderate correlations (>0.60) in the 2 versions of the TEQ in both contexts. Given that the HAS and STAR were designed with other populations in mind, the direction, magnitude and pattern of the correlations are generally consistent.
The aim of this study was to develop a TE measurement tool that combines the service user perspective with a rigorous scientific approach. The TEQ includes 20 items with two sub-scales - care interactions and care delivery. The questionnaire is easy to administer, has versions for SUs and RMHNs and has satisfactory psychometric properties. The inter-scale correlations are high (0.66-0.95 SU; 0.57-0.90 RMHN) and the TEQ exhibits sound sub-scale internal consistency (>0.95). The authentication shows acceptable concurrent validity and is supported by significant, moderate correlations with the other measures used for authentication. The majority of the relationships between the sub-scales and authentication measures were expected however there are apparent weaker correlations between the care delivery sub-scales in both the contexts of the RMHN version of the TEQ. Reasoning could be that the nurses make a distinction between types of interaction within this sub-scale.
The questionnaire’s psychometric properties show that in general the TEQ behaves well as an assessment scale. Indeed, the TEQ has the capacity and necessary psychometric properties to measure and quantify TE in adult acute in-patient psychiatric settings from the perspectives of both SUs and RMHNs.
Scoring and interpretation of the TEQ
For clinical purposes the TEQ should be completed at SU discharge. The SU and their primary/named nurse (with whom they should have had recovery-focused interactions however brief) will independently complete their respective version of the TEQ so that responses can be matched when reported. There is no scoring system attached to the TEQ given the number of items in each sub-scale and context for each version. Individual items and/or the groups of items in the sub-scales should be reviewed. The higher the score, the better the engagement. A holistic viewpoint is the most informative way to understand SU and RMHN scores. Although simplistic, this method ensures that service users' thoughts, feelings, beliefs, and attitudes are supported by nursing staff.
The TEQ was developed and validated in accordance with psychometric theory  and developed within the NHS in partnership with SUs, RMHNs and clinical nurse academics, therefore it is highly relevant and useful to clinical practice. This questionnaire measures and quantifies the nature of RMHN-SU interaction and is therefore of national interest having the potential to make more explicit and visible the skills of RMHNs, something which Brown and Fowler  identified as lacking as far back as 1979. The TEQ has the benefit of consisting of items and domains that are specific to a particular SU group and are therefore relevant and important to patients and clinicians . Measurement tools such as the TEQ are necessary to monitor the quality of the clinical environment and help secure delivery of the best possible care to SUs.
Several practical limitations of the study should be noted. The study relied upon participants’ self-report. A substantial literature exists concerning the numerous problems of self-report data [22, 23]. Of particular concern, is the issue of social desirability which may have affected the magnitude of the results. Imminent studies of the TEQ may benefit from inclusion of other data sources like feedback via interviews with ward managers, SUs discharged and named nurses .
Due to the transient nature of SUs and nursing staff in these settings it was not possible to include the entire SU and nurse population eligible across all the participating Trusts. The results are only based on the ‘lived’ experience, thoughts, feelings and/or opinions of the participants recruited. Nevertheless, the study population is deemed representative of SUs and RMHNs across England so may be generalised to a large extent. More investigation is needed into the perception of nurse care delivery as they appear to view their delivery in 1-1 interactions with SUs differently to their delivery in general within the psychiatric setting as a whole.
Establishing the validity of any measurement tool is an ongoing process and future work is planned to internationally validate and implement the TEQ.
The availability of a reliable and valid TE measurement tool to assess RMHN-SU interactions is central to an improved understanding of the role and contribution of RMHNs to service user recovery. We anticipate that information gathered by the TEQ will help to advise mental health nursing staff at all levels of seniority about the nature of TE experienced by SUs. We hope that the questionnaire will inform the mental health nursing profession about SU involvement in the decision-making/control over their care plan and monitoring of their treatment and/or care to ensure it is offered with dignity and compassion. The TEQ is able to determine the collaborative and empathic nature of RMHN-SU interactions, capture if SUs are treated with dignity and respect and recognise that the principles of the recovery approach are being respected.
The TEQ could help provide robust monitoring of nursing activity, offer opportunity for transparency of activity and feed into healthcare organisations' key performance indicators and other outcome measures. It will provide reassurance for Directors of Nursing about the nature and quality of nurses’ work and the degree to which they are aspiring to working in partnership with SUs as a means to enabling ‘recovery’.
Registered Mental Health Nurse
Peplau HE. Interpersonal relations: a theoretical framework for application in nursing practice. Nurs Sci Q. 1952;5:13–8.
McAndrew S, Chambers M, Nolan F, Thomas B, Watts P. Measuring the evidence: reviewing the literature of the measurement of therapeutic engagement in acute mental health inpatient wards. Int J MentHealth Nurs. 2014;23:212–20.
Nesset MB, Rossberg JI, Almvik R, Friis S. Can a focused staff training programme improve the ward atmosphere and patient satisfaction in a forensic psychiatric hospital? A pilot study. Scand J Caring Sci. 2009;23(1):117–24.
Gamble C, Dodd G, Grellier J, et al. Zoning: focused support: a Trust wide implementation project. J Psychiatr Ment Health Nurs. 2010;17:79–86.
Health Education England. Raising the Bar. In: Shape of Caring: A Review of the Future Education and Training of Registered Nurses and Care Assistants: Health Education England in partnership with the Nursing and Midwifery Council: Health Education England; 2015.
NHS England. 2015. http://www.england.nhs.uk/about/imp-our-mission/. Accessed 23 Jan 2018.
Spilsbury K, Meyer J. Defining the nursing contribution to patient outcome: lessons from a review of the literature examining nursing outcomes, skill mix, and changing roles. J Clin Nurs. 2001;10:3–14.
Horvath AO, Greenberg LS. Development and validation of the Working Alliance Inventory. J Couns Psychol. 1989;36(2):223–33.
Priebe S, Gruyters T. The role of the helping alliance in psychiatric community care. A prospective study. J Nerv Ment Dis. 1993;181(9):552–7.
McGuire-Snieckus R, McCabe R, Catty J, Hansson L, Priebe S. A new scale to assess the therapeutic relationship in community mental health care: STAR. Psychol Med. 2007;37:85–95.
McCabe R, Priebe S. The therapeutic relationship in the treatment of severe mental illness: a review of methods and findings. Int J Soc Psychiatry. 2004;50:115–28.
Chambers M, McAndrew S, Nolan F, Thomas B, Watts P, Kantaris X. Service user involvement in the co-production of a mental health nursing metric. Health Expect. 2016;20:1–7.
Chambers M, Gallagher A, Borschmann R, Gillard S, Turner K, Kantaris X. The experiences of detained mental health service users: issues of dignity in care. BMC Med Ethics. 2014;15:50 10, 1186, 1472–6939-15-50.
De Ve HC, Ader HJ, Terwee CB, Pouwer F. Are factor analytical techniques used appropriately in the validation of health status questionnaires? A systematic review on the quality of factor analysis of the SF-36. Qual Life Res. 2005;14:1203–18.
Stevens A, Abrams K, Brazier J, Fitzpatrick R, Lilford R. The advanced handbook of methods in evidence-based healthcare. London: Sage; 2001.
Tabachnick BG, Fidell LS. Using Multivariate Statistics. In: Pearson New International Edition Paperback. 6th ed. England: Pearson Education Limited; 2014.
Cronbach LJ. Coefficient alpha and the internal structure of tests. Psychometrika. 1951;16:297–334.
Luborsky L, Barber JP, Siqueland L, Johnson S, Najavits LM, Frank A, Daley D. The Revised Helping Alliance Questionnaire (HAq-II): Psychometric Properties. J Psychother Pract Res. 1996;5(3):260–71.
Nunnally JC, Bernstein IH. Psychometric Theory. 3rd ed. New York: McGraw-Hill; 1994.
Guyatt GH, Osoba D, Wu AW, Wyrwich KW, Norman GR. Methods for explaining the clinical significance of health status measures. Clin Ther. 2002;24(Suppl):5.
Brown MM, Fowler GR. Psychodynamic Nursing: A Biosocial Orientation V. B. Saunders Co: Philadelphia; 1971.
Brown Kirk W, Moskowitz DS. Dynamic stability of behavior: the rhythms of our interpersonal lives. J Pers. 1998;66:105–34.
Turk J. Impact of cystic fibrosis on family functioning. Pediatrics. 1994;34:67–71.
Edwards R, Telfair J, Cecil H, Lenoci J. Self-efficacy as a predictor of adult adjustment to sickle cell disease. Psychosom Med. 2000;63:850–8.
The authors would like to thank the nursing staff and service users who participated in the project who generously gave their time, the nurse researchers who collected the data at each of the project sites and the nursing management at the project sites who helped with the organisation of the project. The authors would also like to thank the National Institute for Health Research, Collaboration for Leadership in Applied Health Research and Care (CLAHRC) South London for supporting this project.
BT and MC obtained the funding for the study. MC is principal investigator. MC and XK co- designed the study. SMc, FN and PW were local co-investigators. XK was responsible for running the study. RG and XK analysed the data and interpreted the results. MC and XK wrote the paper; SMc, FN and PW co-wrote the paper. All authors have read and approved the manuscript.
The study was funded by a grant from the Department of Health and Kingston University Enterprise team. The views and opinions expressed do not necessarily reflect those of the funding bodies.
Ethics approval and consent to participate
Ethical approval was obtained from the NHS National Research Ethics Service (NRES) prior to the project commencing (13/EM/0469 East Midlands, Leicester). The project was registered with the Research and Development Committee for each of the Mental Health Trusts involved in the research. The nature and objectives of the project were explained to all potential participants and written informed consent was obtained from each participant prior to data collection. All participants were assured of their confidentiality and their right to withdraw from the project without penalty.
Consent for publication
The authors declare that they have no competing interests.
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
About this article
Cite this article
Chambers, M., McAndrew, S., Nolan, F. et al. The Therapeutic Engagement Questionnaire (TEQ): a service user-focused mental health nursing outcome metric. BMC Psychiatry 19, 384 (2019). https://doi.org/10.1186/s12888-019-2326-x
- Therapeutic Engagement
- Service Users
- Registered Mental Health Nurses
- Mental Health Nursing