Skip to main content
Download PDF

Cardiovascular risk assessment methods yield unequal risk predictions: a large cross-sectional study in psychiatric secondary care outpatients

BMC Psychiatry volume 23, Article number: 536 (2023) Cite this article

Abstract

Background

Patients with a mental illness are more likely to develop, and die from, cardiovascular diseases (CVD), necessitating optimal CVD-risk (CVR)-assessment to enable early detection and treatment. Whereas psychiatrists use the metabolic syndrome (MetS)-concept to estimate CVR, GPs use absolute risk-models. Additionally, two PRIMROSE-models have been specifically designed for patients with severe mental illness. We aimed to assess the agreement in risk-outcomes between these CVR-methods.

Methods

To compare risk-outcomes across the various CVR-methods, we used somatic information of psychiatric outpatients from the PHAMOUS-, and MOPHAR-database, aged 40–70 years, free of past or current CVD and diabetes. We investigated: (1) the degree-of-agreement between categorical assessments (i.e. MetS-status vs. binary risk-categories); (2) non-parametric correlations between the number of MetS-criteria and absolute risks; and (3) strength-of-agreement between absolute risks.

Results

Seven thousand twenty-nine measurements of 3509 PHAMOUS-patients, and 748 measurements of 748 MOPHAR-patients, were included. There was systematic disagreement between the categorical CVR-assessments (all p < 0.036). Only MetS-status versus binary Framingham-assessment had a fair strength-of-agreement (κ = 0.23–0.28). The number of MetS-criteria and Framingham-scores, as well as MetS-criteria and PRIMROSE lipid-scores, showed a moderate-strong correlation (τ = 0.25–0.34). Finally, only the continuous PRIMROSE desk and lipid-outcomes showed moderate strength-of-agreement (ρ = 0.91).

Conclusions

The varying methods for CVR-assessment yield unequal risk predictions, and, consequently, carry the risk of significant disparities regarding treatment initiation in psychiatric patients. Considering the significantly increased health-risks in psychiatric patients, CVR-models should be recalibrated to the psychiatric population from adolescence onwards, and uniformly implemented by health care providers.

Trial registration

The MOPHAR research has been prospectively registered with the Netherlands Trial Register on 19th of November 2014 (NL4779).

Peer Review reports

Background

Across a wide range of psychiatric diagnoses, patients have a life expectancy that is reduced by 10–20 years compared to the general population [1]. Various explanations have been proposed for this health disparity [2]. First, compared to their healthy peers, patients with psychotic disorders, major depression, bipolar disorder and anxiety disorders may have differential and mostly increased exposure to risk factors, such as an unhealthy lifestyle (e.g., smoking, substance use disorders including alcohol, unhealthy diets and less physical activity [3,4,5,6,7,8]). Psychiatric patients are more likely to develop cardiovascular diseases (CVD), and their CVD-related mortality risk is doubled to tripled [9, 10]. Second, increased mortality may be due to iatrogenic effects of psychopharmacological treatment, including weight gain [11]. Third, patients may experience inequity in and access to health care services [2]. Other risk-increasing factors are sympathetic overactivity, genetic polymorphisms interacting with cardiovascular risk factors, inflammation, and platelet dysfunction [12]. Finally, mere presence of psychiatric disease has been demonstrated to independently contribute to CVD-risk [13]. All of these established risk factors are known to interact synergistically and increase the likelihood to develop CVD, especially when multiple factors co-occur [14, 15].

Given the high mortality risk, it is widely acknowledged that somatic health screening should routinely be offered to patients with psychiatric disorders [16]. However, the physical health status of psychiatric patients is not always routinely monitored [17], and when monitored, deficits are not systematically treated [18]. Furthermore, in the Netherlands, as in the vast majority of other countries, there is a lack of clarity and consensus about which professional should be responsible for detecting and managing physical problems in these patients: the treating psychiatrist or the general practitioner? Fragmented medical and mental health care systems, and lack of integrated services, could be additional factors that contribute to the increasing health gap between psychiatric and non-psychiatric patients [17, 19].

Generally, in psychiatric care, different approaches are used compared to general practice to assess cardiovascular risks, which can lead to divergent predictions of CVD-morbidity and -mortality, depending on the service the patient attends. Subsequently, patients with a psychiatric diagnosis may receive, or not, varying (prophylactic) somatic treatments depending on whether their psychiatrist takes care of the primary prevention of CVD, as opposed to their GP.

Psychiatrists, on the one hand, generally use the presence or absence of the metabolic syndrome (MetS) to assess cardiovascular risk [20]. MetS entails a cluster of factors identifying patients at increased cardiometabolic risk [21]. Individuals with MetS are at increased risk for developing CVD, and are three times more likely to die hereof [22]. Psychiatric patients are especially vulnerable, as illustrated by the MetS prevalence being 58% higher in this group compared to the general population [12, 22]. The presence of MetS predicts CVD events with an overall accuracy of 63% (95% CI area under the receiver-operating characteristic curve (AUC): 56%–69%; [23]). An AUC of 0% means that the model wrongly predicts the outcome in every instance, whereas an AUC of 100% indicates perfect prediction. Nonetheless, some experts dispute the diagnostic, prognostic, and therapeutic value of the MetS concept. A first criticism entails the dichotomous nature (i.e. presence/absence) of the concept, and consequently its relative insensitivity to subtle early warning signs of CVD [24, 25]. Overall, patients are monitored on its presence or absence, thus having the risk of being treated only when MetS is present, leaving people who display one or two risk factors sometimes untreated. Another point of criticism is that important additional cardiovascular determinants, such as age, sex, and smoking status, are not taken into account in the MetS-system.

GPs, on the other hand, generally use continuous -and thus more flexible- scoring systems, such as the Framingham [26], and ‘Systematic Coronary Risk Evaluation’ (SCORE)-model [27], the latter being used in the Dutch GP-setting. These models potentially mitigate the aforementioned shortcomings of MetS, because they quantify the absolute CVD-risk, and include additional relevant cardiovascular determinants, such as age, sex and smoking status. Unfortunately, these models are not adapted to the increased CVD-risk psychiatric patients experience.

Framingham predicts the 10-year morbidity risk to develop the first cardiovascular event, expressed as a percentage. It has a median AUC of 77% (95% CI: 58%–84%), based on 28 studies [28]. Rigal and colleagues demonstrated in their meta-analysis that individuals with a severe mental illness (SMI) had Framingham coronary scores that were 1.6 fold higher than in control subjects from the general population [29], confirming the elevated risk of these patients. However, there are indications that Framingham overestimates the risk of coronary heart disease when the formula is applied to European populations [30]. In addition, not all studies agree that Framingham is superior to MetS with regard to CVD-risk prediction [31,32,33].

The second continuous scoring model, SCORE, predicts the 10-year CVD-mortality risk expressed as a percentage [27]. The median AUC of SCORE for CVD-prediction was 75% (95% CI: 62%–91%), based on 28 studies [28]. Its implementation is recommended by the Dutch College of General Practitioners in their cardiovascular risk management (CVRM) guideline, as well as by the European Association for Cardiovascular Prevention & Rehabilitation [34]. Absolute SCORE-risks have been studied in the SMI-population, yielding mean risks between 0.9% and 1.8% [35]. On the other hand, the European Psychiatric Association (EPA) recommends in their Position Statement the use of relative SCORE-risks, especially in the younger SMI-population. According to the smoking habits, systolic blood pressure, and total cholesterol of patients with SMI, the risk of fatal cardiovascular disease is compared to the general population (that is, people without SMI of the same age with normal risk factors) [36]. However, the Statement does not mention cut-off values for which relative risks prophylactic treatment should be initiated. In addition, some studies argue that SCORE underestimates the true CVD-risk [37, 38].

The large-scale studies performed to establish 10-year CVD-mortality and morbidity-risks for SCORE and Framingham, were carried out in the general population, which may limit the generalizability of the calculated risks to the psychiatric population. Risk-calculators calibrated or adjusted to the SMI-population can potentially improve the risk-estimation and may better address underestimations [39, 40]. The PRIMROSE desk and lipid-models [41] are among the few models that take the CVD risk-increasing effects of SMI-diagnosis, psychosocial and -pharmacological treatment into account [2, 11] by adding these risk factors to the algorithm, besides the traditional, well-established cardiovascular predictors [41, 42]. The PRIMROSE models were validated in the UK in a SMI-population aged 30 to 90 years, and estimate the 10-year risk percentage to incur the first cardiovascular event. The authors described AUCs around 80% (range 95% CI: 74%–83%) for CVD prediction [41]. Perry et al., however, demonstrated that PRIMROSE systematically underpredicted the CVR in adolescents with, or at risk for, psychosis [43].

To summarize, there are at least four different CVD-risk assessment-models in use by clinicians for psychiatric patients with a range of psychiatric problems, each with advantages and drawbacks. Currently, it is unclear (1) whether the use of any of the four CVR-assessment models under study, when applied to the same patient, leads to a similar CVD-risk and subsequent management, and (2) whether there is any evidence suggesting the superiority of one CVD-risk assessment method over another with respect to sensitivity and specificity in predicting CVD events, notably for psychiatric patients in secondary mental healthcare [44]. In light of the theoretical situation that the use of a suboptimal CVR-evaluator may contribute to an increase in excess life years lost due to heart diseases in mental health patients because of underestimated risk and as a consequence undertreatment [19, 45], this is not a trivial question. It is currently unclear whether the assessment methods used in general healthcare (mostly Framingham, SCORE), when used in mental health care, are equally predictive as PRIMROSE which has been specifically designed for psychiatric populations. Moreover, these models need comparison with the criteria of presence/absence of MetS, which to date seems to be the most commonly used CVR-estimator in mental healthcare. Hence, the aims of this study are to cross-sectionally compare current CVR-monitoring systems in a large and heterogenous group of mental health care outpatients. We aimed to investigate whether there is a systematic difference in the agreement between the outcomes of MetS, Framingham, SCORE, modified SCORE, PRIMROSE lipid, and PRIMROSE desk, within the same person. Second, we aimed to determine the strength of the agreements between these assessment methods. Finally, we evaluated the strength of the correlations between the total number of MetS criteria that are met, and the various CVD-risk predictions.

Methods

Study design and population

We used two databases: PHAMOUS (‘PHArmacotherapy Monitoring and Outcome Survey’) and MOPHAR (‘Monitoring Outcomes of psychiatric PHARmacotherapy’) to answer our research questions. Both databases were used separately, instead of a combined one, in order to verify the transdiagnostic consistency and generalizability of the findings. The PHAMOUS and MOPHAR study protocols have been described in detail elsewhere [46, 47]. In short, PHAMOUS (N = 13,215 patients with a combined total of 42,776 measurements as of November 2021) has been including patients aged 18 years and older since 2006, originally focusing on schizophrenia, schizoaffective disorder, or other psychotic disorders, at four mental health facilities in the Northern Netherlands. For the last couple of years, PHAMOUS has been including SMI diagnosed by DSM-criteria in a broader diagnostic spectrum. MOPHAR (N = 2098 patients with a combined total of 2,986 measurements) started a decade later, and includes adults with any psychiatric diagnosis from outpatient departments of mental health facilities in the Northern Netherlands.

Patients were eligible for our study if they were aged between 40–70 years, because (i) all of our CVD risk models have been validated for this age range; (ii) CVR is substantially age-related, it is generally acknowledged that CVR-monitoring is most useful from age 40 on in the general population. Patients were included if they completed at least one of the yearly somatic assessments. Exclusion criteria were (1) a history of CVD, because we selected CVD-risk models that target primary prevention, (2) the presence of diabetes mellitus, because diabetic patients are automatically classified as patients with a high to very high CVD risk, and should be assessed with other risk engines, such as ADVANCE [48].

Patients are screened yearly by the MOPHAR or PHAMOUS-program, on somatic, and psychiatric (co)morbidities, disease severity, psychosocial functioning, current medication usage, registration of side effects. Further, a basic physical examination (e.g. measurement of waist circumference and blood pressure) is carried out by trained nurses, and laboratory measurements are collected, including blood lipids (i.e. low and high-density lipoprotein (LDL and HDL), total cholesterol, triglycerides) and fasting glucose. PHAMOUS and MOPHAR are highly similar with respect to the somatic parameters assessed, and differ on assessment of symptom parameters, as a result of differences between their populations (MOPHAR targeting mostly patients with affective and anxiety disorders, PHAMOUS mostly psychotic patients). If the same patient had multiple risk score and MetS assessments, we included every complete set of data points. We used statistical tests and measurements that reliably account for the paired nature of the risk comparisons [49]. Using multiple data points has the advantage that it accommodates the detection of intraindividual changes between assessments (regarding treatment, or risk factor exposition, which could induce alterations in MetS-status and/or CVD-risks).

Measurements

A detailed overview of the various models, and addressed CVD-predictors, can be found in Fig. 1 and Appendix 1.

Fig. 1
figure 1

Summary of the outcomes, features of the cohorts used for CVR-derivation, applied thresholds, CVR-variables, and accuracy of the various CVR-models (i.e., MetS, Framingham, SCORE, modified SCORE, PRIMROSE desk, and PRIMROSE lipid)

Full size image

Metabolic syndrome

Patients are considered to have MetS if they fulfil three or more of the following criteria [50]: 1) waist circumference ≥ 88/102 cm (female/male); 2) systolic blood pressure ≥ 130 mmHg or diastolic blood pressure ≥ 85 mmHg or receiving antihypertensive drug treatment; 3) HDL-cholesterol < 1.30/1.03 mmol/L (female/male) or receiving lipid-lowering drugs; 4) triglycerides ≥ 1.7 mmol/L or receiving lipid-lowering drugs; and 5) fasting glucose ≥ 6.1 mmol/L or receiving antidiabetic medication(s). There are 16 unique combinations of MetS, to which we collectively refer as ‘MetS-profiles’.

Framingham

The Framingham risk-score estimates the 10-year CVD-morbidity risk, and is derived from large scaled population-based North-American samples without CVD, aged 30–74 years. Because our total cholesterol and HDL-cholesterol results were expressed in SI-units (i.e. mmol/L), we multiplied our laboratory results with 38.6698 to convert these to mg/dL [51] to be entered in the risk model. Patients with a Framingham risk-score of at least 10% were considered to have a substantially increased cardiovascular risk [52].

SCORE

The SCORE-formula is derived from individuals of the general European population aged 40–70 years, without known CVD. Patients with a SCORE between 5%-10% are considered to have a moderate to high risk, ≥ 10% a very high risk [53]. To compare the various morbidity risks, we applied multipliers specified by the Dutch CVRM-guideline to convert the SCORE-mortality risk-score to estimated morbidity-scores (‘modified SCORE’). ‘Modified SCORE’ risks of at least 10% were considered to be substantial.

PRIMROSE

The PRIMROSE desk and lipid-models entail 10-year CVD-morbidity risk-estimations from a study conducted in the UK in a SMI-population aged 30 to 90 years. Because we did not have sufficient information on social deprivation, we instead averaged the ‘best’ and ‘worst case risk scenario’ by assuming the lowest, respectively, highest Townsend quintiles in the PRIMROSE-formulas. A cut off value of ≥ 10% was used for the PRIMROSE-risk to be considered substantial.

Statistical analyses

We used R version 4.1.2 [54] to determine the CVR-assessments by using ‘available case’-analyses. A description of the prediction-formulas can be found in Appendix 1.

Comparisons between MetS-status and dichotomized CVD-risk categories

First, we compared the agreement between these methods as present versus absent. Because the binary MetS is a nominal variable (i.e. the absence of MetS is generally associated with a ‘low CVD risk’, the presence of MetS with an ‘intermediate to (very) high CVD risk’), we dichotomized the continuous outcomes of Framingham, SCORE, modified SCORE, and PRIMROSE models, to the same CVD-risk outcome categories as MetS. ‘Modified SCORE’, Framingham, and PRIMROSE risks of ≥ 10% were considered to be an ‘intermediate to (very) high CVD risk’. For SCORE we used a cut-off of 5%, in line with the Dutch CVRM guideline.

We determined whether there was a systematic difference between the agreement of the presence or absence of MetS and each of the dichotomized risk-categories, by performing McNemar’s tests with the mcnemar.test-function of R’s stats-package [49, 55]. Subsequently, we assessed the strength of agreement between these comparisons by measuring Cohen’s kappa κ with 95% confidence intervals by using the Kappa.test-function of R’s fmsb-library [49]. Kappa can be interpreted as follows: below 0 is poor agreement, 0–0.20 slight, 0.21–0.40 fair, 0.41–0.60 moderate, 0.61–0.80 substantial, and > 0.80 almost perfect agreement [56].

Mean CVD-risks stratified by number of MetS-criteria met, and their correlation

Furthermore, we stratified the mean CVD-risks by the number of MetS-criteria that were met, and by ‘MetS-profile’. The correlation between the number of MetS-criteria and absolute risk-scores, was calculated by Kendall’s tau with 95% confidence intervals (by using R’s KendallTauB-function in the DescTools-package). We used Botsch’s guidelines for the interpretation of the magnitude of τ, that is: |τ|< 0.10 = very weak, |τ| = 0.10–0.19 = weak, |τ| = 0.20–0.29 = moderate, and |τ|≥ 0.30 = strong strength of association between the number of MetS-criteria and CVD-risk score [57].

Pairwise comparisons between continuous CVD-risks

The assessment of the degree of agreement between the continuous risk-outcomes of Framingham, modified SCORE, and PRIMROSE lipid and desk models, took place in a pairwise fashion and was quantified by Lin’s concordance correlation-coefficients ρ (by the CCC-function in R’s DescTools-library). McBride’s cut-off values were used to assess the strength-of-agreement [58], i.e. almost perfect agreement if ρ > 0.99, substantial agreement if ρ is 0.95–0.99, moderate if ρ is 0.90–0.95, and poor strength of agreement if ρ < 0.90.

Results

Of the 42,776 combined measurements of the PHAMOUS-database, 36% had sufficient information to be eligible for inclusion; which was 66% for the MOPHAR-database. Most of the measurements were excluded based on age (< 40; > 70 years), followed by presence of a history of CVD, and/or DM. Almost half of the eligible PHAMOUS-measurements, from 3509 unique patients, met our inclusion criteria, compared to more than one third of MOPHAR’s (see Fig. 2).

Fig. 2
figure 2

Flowcharts with reasons for exclusion, total amount of included measurements (as well as number of unique patients), and distribution of frequency of yearly assessments, from the PHAMOUS-, and MOPHAR-database

Full size image

For a more detailed comparison between the in- and excluded groups, as well as missing values, see Supplementary material 1. Characteristics of the included samples can be found in Table 1.

Table 1 Descriptive table with characteristics of measurements satisfying our inclusion criteria (i.e. from patients without CVD or diabetes, and aged between 40 and 70 years) in the PHAMOUS (n = 7029 measurements from 3509 unique patients), and MOPHAR-database (n = 748 measurements from 748 unique patients)
Full size table

Almost one third of the PHAMOUS-measurements met the MetS-criteria (30.2%; 95% CI: 29.1%–31.3%), compared to more than one in five in MOPHAR (21.4%; 95% CI: 18.5%–24.5%). The majority of the measurements that fulfilled MetS, satisfied three criteria (1062/2084 = 51% for PHAMOUS, vs. 95/160 = 59% for MOPHAR, see Table 1).

Cross-tables with frequencies and percentages of (dis)agreement between the outcomes of the CVR-assessment methods (based on binary risk classifications from six methods) are depicted in Table 2. The largest range between these percentages occurred in the measurements fulfilling MetS-criteria, both in the PHAMOUS (i.e. range of percentages of (dis)agreement between CVR-assessment methods with MetS belonging to the highest CVD-risk category was 7.2%-62.1%, vs. lowest CVD-risk-category: 37.9%-92.8%), and in the MOPHAR-database (i.e. range highest CVD-risk category: 3.7%-60.4%, vs. lowest CVD-risk-category: 39.6%-96.3%).

Table 2 Cross tables with concordant (light grey), and discordant (dark grey) frequencies of binary CVD-risk classifications from six assessment methods (i.e., dichotomous MetS-status (= presence vs. absence of MetS) vs. CVD-risk categories (= ‘intermediate-(very) high risk’ vs. ‘low risk’) based on thresholds of continuous Framingham, SCORE, Modified SCORE, PRIMROSE desk, and PRIMROSE lipid scores)
Full size table

MetS-status versus dichotomized CVD-risk categories in PHAMOUS and MOPHAR

We evaluated whether there was a systematic difference between MetS-status and the dichotomized CVD-risk categories in the PHAMOUS-, and MOPHAR-database separately. Indeed, the McNemar’s chi-squared p-values indicated significant differences between the binary risk-classifications of the CVR-assessment methods (all p < 0.036; Table 2).

The extent of agreement was ‘fair’ between the categorical risk-classifications by MetS and Framingham (range κ = 0.23–0.28 (95% CI = 0.18–0.38)), all the other comparisons agreed ‘poorly’ to ‘slightly’ (κ’s ≤ 0.14, range 95% CI: -0.16–0.26), as shown in Table 2.

Mean CVD-risks by number of MetS-criteria, and their correlation

Next, we examined the mean CVD-risks by the number of MetS-criteria that were fulfilled, as well as their correlations.

PHAMOUS

The lowest mean 10-year CVD-morbidity risks of the PHAMOUS-database occurred when none of the MetS-criteria were fulfilled, ranging from 2.38% (95% CI: 2.21%–2.55%; PRIMROSE lipid), to 7.55% (95% CI: 7.00%–8.10%; Framingham; see Fig. 3A).

Fig. 3
figure 3

Mean 10-year CVD-risks stratified by the number of MetS criteria, and risk-model a PHAMOUS, b MOPHAR

Full size image

The highest mean risks occurred when all of the MetS-criteria were fulfilled, with a minimum of 5.61% (95% CI: 5.10%–6.14%; PRIMROSE desk), and a maximum of 18.28% (95% CI: 16.72%–19.83%; Framingham). We refer to Supplementary material 2 for subgroup CVD-risk analyses per number of MetS-criteria fulfilled and MetS-profile.

There was a moderate positive correlation between the number of MetS-criteria and Framingham risk-scores (Kendall’s τ = 0.25, 95% CI: 0.23–0.27), as well as between the number of MetS-criteria and PRIMROSE lipid-scores (τ = 0.26, 95% CI: 0.24–0.28). The other correlations were all (very) weak, with τ’s < 0.13.

MOPHAR

The range of the lowest mean 10-year CVD-morbidity risks of the MOPHAR-database, was 1.51% (for the PRIMROSE lipid-group; 95% CI: 1.24%-1.78%) to 5.77% in the Framingham-group that had zero MetS-criteria (95% CI: 4.53%–7.00%).

The highest mean risks occurred when all of the MetS-criteria were fulfilled, with a maximum of 18.57% in the Framingham-group (95% CI: 9.49%–27.65%; see Fig. 3B).

There was a strong positive correlation between the number of MetS-criteria and Framingham risk-scores (τ = 0.34, 95% CI: 0.28–0.40), as well as between the number of MetS-criteria and PRIMROSE lipid-scores (τ = 0.32, 95% CI: 0.24–0.40). Other correlations were weak with τ’s < 0.19.

Pairwise comparisons between continuous CVD-risk outcomes in the PHAMOUS and MOPHAR databases

We found a moderate strength-of-agreement between the continuous risk outcomes of the PRIMROSE lipid and desk-model in the MOPHAR-database (ρ = 0.91 (95% CI: 0.89–0.93), Fig. 3B in Suppl. 1). The pairwise comparisons of continuous CVD-risk outcomes of PHAMOUS, and the other pairwise comparisons of MOPHAR, all agreed poorly with ρ’s between 0.20 and 0.85.

Discussion

This study directly compared cardiovascular risk-assessment methods as used in psychiatric practice (MetS, PRIMROSE-models), as well as in GP-practices (SCORE, and Framingham-model), across two large transdiagnostic and heterogeneous psychiatric patient samples receiving secondary care. CVD-risk was on average 2.52%-4.27% for psychiatric practice methods, and 1.38%-12.5% for GP-practice methods in these two samples. We found evidence for systematic disagreement between the categorical methods of CVD-risk assessment. Furthermore, the strength-of-agreement between these binary CVR-assessment methods was generally poor to slight. Only MetS and Framingham corresponded slightly, as evidenced by fair agreement in both study groups.

Our findings provide supporting evidence that CVD-risk is elevated in psychiatric populations. Indeed, the mean Framingham and SCORE-risks in our samples were comparable to the estimated risks in studies performed in bipolar [59,60,61], depressive [62], and schizophrenic populations [63, 64]. Additionally, we found support for higher CVD-risks in more severely ill patients, as demonstrated by higher mean risks in the PHAMOUS-patients compared to the MOPHAR-group. This can be explained by a higher prevalence of SMI in the PHAMOUS-cohort, their longer illness duration, less favourable lower mean HDL, higher triglycerides, higher smoking and excessive drinking frequencies, and higher use of antipsychotics. Accurate assessment of CVD-risk is therefore of high importance in this group.

However, we found that CVD-risk assessments lead to uneven risk predictions, especially for more severely ill patients. The predominantly poor degree of agreement between the continuous CVR-comparisons is in accordance with the findings of Berry and colleagues, who reported similar ρ’s between 0.54 and 0.77 for Framingham, PRIMROSE lipid, and PRIMROSE desk [44]. Indeed, the largest individual disparity between continuous CVR-outcomes occurred in the PHAMOUS-database, where the ‘modified SCORE’ predicted a CVD-risk of 100%, versus only 5.94% by the PRIMROSE lipid-model, for the same 63-year old smoking male with an unspecified SMI and MetS (Fig. 4B, Suppl. 1). This can be explained by the greater emphasis by the PRIMROSE lipid-model on the CVD-protective effect of his high HDL (i.e. 2.5 mmol/L), whereas the modified SCORE-model more strongly emphasizes his high systolic blood pressure (i.e. 185 mmHg), and age (i.e. 63 years old). This indicates that various CVD-risk assessment methods, used in different clinical settings, yield uneven risk predictions.

Only in a few instances, we found some agreement between CVD-risk assessment methods, but only between those that are used in the same clinical setting (i.e., in psychiatric practice). The PRIMROSE lipid and PRIMROSE desk-model had at most a moderate strength-of-agreement when continuous CVD-outcomes were compared, in both samples. Moderate strength-of-agreement can be expected between the PRIMROSE-models, because they share most parameters, and have been derived from, and validated in, the same SMI-sample, and should therefore produce comparable risk-predictions. Secondly, moderate to strong correlations were found between the number of MetS-criteria and Framingham risk-scores, as well as between the number of MetS-criteria and PRIMROSE lipid-scores. These results are in line with previous findings of Wilson et al., and Wannamethee and colleagues, who demonstrated positive associations between the number of MetS-criteria fulfilled and increased CVD-risks measured by Framingham [33, 65].

Considering the inconsistencies in reported CVR-criteria and subsequent risk-estimations between the various models, we can conclude that persons with a psychiatric disorder have a substantial chance to attain a varying risk profile depending on the context (general practice versus psychiatric care) and related CVR-risk management-model in which they are monitored and treated. As a consequence, psychiatric patients might not receive necessary treatments due to significant underpredictions of the CVD-risk (e.g., by SCORE or PRIMROSE), or receive unnecessary treatments due to overpredicted CVD-risks (e.g., by Framingham). The differences in assessed risks highlight the necessity to provide recommendations on the preferred model. Such recommendations depend on certain quality criteria.

Most importantly, a CVR-assessment model should include relevant predictors to optimize sensitivity and specificity to reliably predict CVD-morbidity. With this cross-sectional study, we demonstrated that various CVR-assessment models yield unequal risk predictions; the next step should involve the investigation of the relative contribution of each of the predictors in each model, and the comparison of the individual predicted risks with the actual occurrence of CVD-events within 10 years. Besides the ‘traditional’ cardiovascular risk factors such as demographic, inflammatory, metabolic, physiologic, and lifestyle factors [66], one should also include relevant environmental (including psychosocial and psychological) factors in the model that are known to increase the cardiovascular risk, especially in the psychiatric population (such as insomnia, vital exhaustion, depression, anxiety, anger/hostility, social isolation/loneliness, optimism, psychological distress, adverse childhood experiences; see Supplementary Table 2 of ref. [40]). Future research should therefore focus on combining longitudinal risk assessments (over ≥ 10 years) with the occurrence of CVD-events in large psychiatric populations, to enable well-founded recommendations on which (recalibrated) CVD-risk model should be employed in the psychiatric population.

In the meantime, as described by Perry et al., we advise clinicians to use MetS because it is not restricted to older individuals (with SCORE: ≥ 40 years), and is therefore more suitable for cardiometabolic risk assessment in the younger psychiatric population [43], whereas SCORE [67] and PRIMROSE [43] underpredict. Reversely, Framingham overpredicts this CV risk (as shown by Danish, German, and Dutch studies [68,69,70]). The underprediction is mainly caused by age being more strongly weighted than other risk factors; ‘older’ CVR-estimators (such as Framingham) overpredict the risk due to a decrease in coronary heart disease incidence and mortality rates in the last four decades [30]. However, earlier initiation of somatic treatment nowadays (e.g., with statins, and anti-hypertensive medication) could be partly responsible for this overprediction as well [71]. Furthermore, MetS has the advantage that it is relatively easy and fast to use, and it does not require rather complicated prediction formulas which can be cumbersome in the daily clinical practice.

Strengths and limitations

Strengths of our study include the relatively large naturalistic samples of psychiatric patients, which included patients with diverse diagnoses and severity levels. Furthermore, the use of two databases allowed us to substantiate the generalizability of the results across populations.

Limitations include the applied age-restriction, excluding patients that are younger than 40 (or older than 70). The younger population is not routinely screened for CVD, even though psychiatric disorders most often originate in early adulthood, and presence of a psychiatric disorder should be considered as an independent cardiovascular risk factor, also at earlier age [13, 43, 72,73,74,75].

Second, we employed ‘available case’-analyses even though not all missing data were expected to be missing completely at random, because the missingness is probably influenced by some patient characteristics, such as disease severity. This is, however, largely compensated by the considerable sample size [76].

Third, we cannot exclude some biases. Patients who are experiencing a severe exacerbation of their psychiatric illness are, for instance, less likely to be assessed, which can induce response-bias. Detection-bias cannot be ruled out either, because practitioners may encourage medicated, or more severely ill patients, to participate in the monitoring programs more strongly than unmedicated, or less severely ill patients.

Conclusions

To conclude, the commonly used CVD-risk assessments generally show strong disagreement. Given that GPs and psychiatrists rely on different methods, this means that CVD-assessments lead to disparate results. As a consequence, psychiatric patients might not receive necessary treatments due to significant underpredictions of the CVD-risk (e.g., by SCORE or PRIMROSE), or receive unnecessary treatments due to overpredicted CVD-risks (e.g., by Framingham). Future research should compare the quality of the predictive performances of the established CVD-risk models, preferably calibrated to psychiatric patients aged 18 and above. We favour the implementation of a single CVD-risk model for psychiatric patients by the healthcare practitioners, on the condition that the model performs equally well in the various settings. In this way we can, hopefully, start to close the cardiovascular health gap between the psychiatric and non-psychiatric population.

Availability of data and materials

The datasets used and/or analysed during the current study are available from the corresponding author on reasonable request.

Abbreviations

CVD:

Cardiovascular diseases

CVR:

Cardiovascular risk

MetS:

Metabolic syndrome

GP:

General practitioner

PRIMROSE:

PRedIction and Management of cardiovascular Risk in peOple with SEvere mental illnesses

PHAMOUS:

Pharmacotherapy Monitoring and Outcome Survey

MOPHAR:

Monitoring Outcomes of Psychiatric Pharmacotherapy

SCORE:

Systematic Coronary Risk Evaluation

CVRM:

Cardiovascular risk management

AUC:

Area under the receiver-operating characteristic curve

NHG:

'Nederlands Huisartsen Genootschap' (The Dutch College of General Practitioners)

SMI:

Severe mental illness

DSM:

Diagnostic and Statistical Manual of Mental Disorders

LDL:

Low-density lipoprotein

HDL:

High-density lipoprotein

References

  1. Chesney E, Goodwin GM, Fazel S. Risks of all-cause and suicide mortality in mental disorders: a meta-review. World Psychiatry. 2014;13(2):153–60.

    Article  PubMed  PubMed Central  Google Scholar 

  2. WHO Guidelines. Guidelines for the management of physical health conditions in adults with severe mental disorders. 2018. Available from: https://apps.who.int/iris/bitstream/handle/10665/275718/9789241550383-eng.pdf.

  3. de Leon J, Diaz FJ. A meta-analysis of worldwide studies demonstrates an association between schizophrenia and tobacco smoking behaviors. Schizophr Res. 2005;76(2–3):135–57.

    Article  PubMed  Google Scholar 

  4. Dipasquale S, Pariante CM, Dazzan P, Aguglia E, McGuire P, Mondelli V. The dietary pattern of patients with schizophrenia: a systematic review. J Psychiatr Res. 2013;47(2):197–207.

    Article  PubMed  Google Scholar 

  5. Jackson JG, Diaz FJ, Lopez L, de Leon J. A combined analysis of worldwide studies demonstrates an association between bipolar disorder and tobacco smoking behaviors in adults. Bipolar Disord. 2015;17(6):575–97.

    Article  PubMed  Google Scholar 

  6. Jakobsen AS, Speyer H, Nørgaard HCB, Karlsen M, Hjorthøj C, Krogh J, et al. Dietary patterns and physical activity in people with schizophrenia and increased waist circumference. Schizophr Res. 2018;199:109–15.

    Article  PubMed  Google Scholar 

  7. Jørgensen KB, Nordentoft M, Hjorthøj C. Association between alcohol and substance use disorders and psychiatric service use in patients with severe mental illness: a nationwide Danish register-based cohort study. Psychol Med. 2018;48(15):2592–600.

    Article  PubMed  Google Scholar 

  8. Vancampfort D, Firth J, Schuch FB, Rosenbaum S, Mugisha J, Hallgren M, et al. Sedentary behavior and physical activity levels in people with schizophrenia, bipolar disorder and major depressive disorder: a global systematic review and meta-analysis. World Psychiatry. 2017;16(3):308–15.

    Article  PubMed  PubMed Central  Google Scholar 

  9. Correll CU, Solmi M, Veronese N, Bortolato B, Rosson S, Santonastaso P, et al. Prevalence, incidence and mortality from cardiovascular disease in patients with pooled and specific severe mental illness: a large-scale meta-analysis of 3,211,768 patients and 113,383,368 controls. World Psychiatry. 2017;16(2):163–80.

    Article  PubMed  PubMed Central  Google Scholar 

  10. Liu NH, Daumit GL, Dua T, Aquila R, Charlson F, Cuijpers P, et al. Excess mortality in persons with severe mental disorders: a multilevel intervention framework and priorities for clinical practice, policy and research agendas. World Psychiatry. 2017;16(1):30–40.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  11. Piña IL, Di Palo KE, Ventura HO. Psychopharmacology and Cardiovascular Disease. J Am Coll Cardiol. 2018;71(20):2346–59.

    Article  PubMed  Google Scholar 

  12. Penninx B, Lange SMM. Metabolic syndrome in psychiatric patients: overview, mechanisms, and implications. Dialogues Clin Neurosci. 2018;20(1):63–73.

    Article  PubMed  PubMed Central  Google Scholar 

  13. Quadackers DMC, Cath DC, Liemburg EJ, Houtman IEM, Oud MJT, Berger MY, et al. Stemmings- en angststoornissen als risicofactor voor cardiovasculaire ziekten. [Anxiety and mood disorders are independent risk factors for cardiovascular diseases]. Nederlands Tijdschrift voor Geneeskunde. 2021;165(44):26. 8.

  14. Jackson R, Lawes CM, Bennett DA, Milne RJ, Rodgers A. Treatment with drugs to lower blood pressure and blood cholesterol based on an individual’s absolute cardiovascular risk. Lancet. 2005;365(9457):434–41.

    Article  CAS  PubMed  Google Scholar 

  15. Wilson P. Overview of established risk factors for cardiovascular disease. UpToDate. Available from: https://www.uptodate.com/contents/overview-of-established-risk-factors-for-cardiovascular-disease.

  16. Firth J, Siddiqi N, Koyanagi A, Siskind D, Rosenbaum S, Galletly C, et al. The Lancet Psychiatry Commission: a blueprint for protecting physical health in people with mental illness. The Lancet Psychiatry. 2019;6(8):675–712.

    Article  PubMed  Google Scholar 

  17. De Hert M, Cohen D, Bobes J, Cetkovich-Bakmas M, Leucht S, Ndetei DM, et al. Physical illness in patients with severe mental disorders. II. Barriers to care, monitoring and treatment guidelines, plus recommendations at the system and individual level. World Psychiatry. 2011;10(2):138–51.

    Article  PubMed  Google Scholar 

  18. Bruins J, Pijnenborg GHM, van den Heuvel ER, Visser E, Corpeleijn E, Bartels-Velthuis AA, et al. Persistent low rates of treatment of metabolic risk factors in people with psychotic disorders: a PHAMOUS Study. J Clin Psychiatry. 2017;78(8):1117–25.

    Article  PubMed  Google Scholar 

  19. Erlangsen A, Andersen PK, Toender A, Laursen TM, Nordentoft M, Canudas-Romo V. Cause-specific life-years lost in people with mental disorders: a nationwide, register-based cohort study. Lancet Psychiatry. 2017;4(12):937–45.

    Article  PubMed  Google Scholar 

  20. Jakovljević M, Crncević Z, Ljubicić D, Babić D, Topić R, Sarić M. Mental disorders and metabolic syndrome: a fatamorgana or warning reality? Psychiatr Danub. 2007;19(1–2):76–86.

    PubMed  Google Scholar 

  21. Alberti KG, Zimmet PZ. Should we dump the metabolic syndrome? No. BMJ. 2008;336(7645):641.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  22. Cornier M-A, Dabelea D, Hernandez TL, Lindstrom RC, Steig AJ, Stob NR, et al. The metabolic syndrome. Endocr Rev. 2008;29(7):777–822.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  23. Kang GD, Guo L, Guo ZR, Hu XS, Wu M, Yang HT. Continuous metabolic syndrome risk score for predicting cardiovascular disease in the Chinese population. Asia Pac J Clin Nutr. 2012;21(1):88–96.

    CAS  PubMed  Google Scholar 

  24. Gale EA. Should we dump the metabolic syndrome? Yes. BMJ. 2008;336(7645):640.

    Article  PubMed  PubMed Central  Google Scholar 

  25. Kahn R, Buse J, Ferrannini E, Stern M. The metabolic syndrome: time for a critical appraisal: joint statement from the American Diabetes Association and the European Association for the Study of Diabetes. Diabetes Care. 2005;28(9):2289–304.

    Article  PubMed  Google Scholar 

  26. D’Agostino RB Sr, Vasan RS, Pencina MJ, Wolf PA, Cobain M, Massaro JM, et al. General cardiovascular risk profile for use in primary care: the Framingham Heart Study. Circulation. 2008;117(6):743–53.

    Article  PubMed  Google Scholar 

  27. Conroy RM, Pyörälä K, Fitzgerald AP, Sans S, Menotti A, De Backer G, et al. Estimation of ten-year risk of fatal cardiovascular disease in Europe: the SCORE project. Eur Heart J. 2003;24(11):987–1003.

    Article  CAS  PubMed  Google Scholar 

  28. Damen JA, Hooft L, Schuit E, Debray TP, Collins GS, Tzoulaki I, et al. Prediction models for cardiovascular disease risk in the general population: systematic review. BMJ. 2016;353:i2416.

    Article  PubMed  PubMed Central  Google Scholar 

  29. Rigal A, Colle R, El Asmar K, Elie-Lefebvre C, Falissard B, Becquemont L, et al. Framingham coronary score in individuals with symptoms or diagnoses of mental disorders: a review and meta-analysis. Psychiatry Res. 2018;263:41–7.

    Article  PubMed  Google Scholar 

  30. Empana JP, Ducimetière P, Arveiler D, Ferrières J, Evans A, Ruidavets JB, et al. Are the Framingham and PROCAM coronary heart disease risk functions applicable to different European populations? The PRIME Study. Eur Heart J. 2003;24(21):1903–11.

    Article  CAS  PubMed  Google Scholar 

  31. Girman CJ, Rhodes T, Mercuri M, Pyörälä K, Kjekshus J, Pedersen TR, et al. The metabolic syndrome and risk of major coronary events in the Scandinavian Simvastatin Survival Study (4S) and the Air Force/Texas Coronary Atherosclerosis Prevention Study (AFCAPS/TexCAPS). Am J Cardiol. 2004;93(2):136–41.

    Article  CAS  PubMed  Google Scholar 

  32. McNeill AM, Rosamond WD, Girman CJ, Golden SH, Schmidt MI, East HE, et al. The metabolic syndrome and 11-year risk of incident cardiovascular disease in the atherosclerosis risk in communities study. Diabetes Care. 2005;28(2):385–90.

    Article  PubMed  Google Scholar 

  33. Wannamethee SG, Shaper AG, Lennon L, Morris RW. Metabolic syndrome vs Framingham Risk Score for prediction of coronary heart disease, stroke, and type 2 diabetes mellitus. Arch Intern Med. 2005;165(22):2644–50.

    Article  PubMed  Google Scholar 

  34. Piepoli MF, Hoes AW, Agewall S, Albus C, Brotons C, Catapano AL, et al. 2016 European Guidelines on cardiovascular disease prevention in clinical practice: The Sixth Joint Task Force of the European Society of Cardiology and Other Societies on Cardiovascular Disease Prevention in Clinical Practice (constituted by representatives of 10 societies and by invited experts) Developed with the special contribution of the European Association for Cardiovascular Prevention & Rehabilitation (EACPR). Atherosclerosis. 2016;252:207–74.

    Article  CAS  PubMed  Google Scholar 

  35. Foguet-Boreu Q, Fernandez San Martin MI, Flores Mateo G, Zabaleta Del Olmo E, Ayerbe García-Morzon L, Perez-Piñar López M, et al. Cardiovascular risk assessment in patients with a severe mental illness: a systematic review and meta-analysis. BMC Psychiatry. 2016;16:141.

    Article  PubMed  PubMed Central  Google Scholar 

  36. De Hert M, Dekker JM, Wood D, Kahl KG, Holt RI, Möller HJ. Cardiovascular disease and diabetes in people with severe mental illness position statement from the European Psychiatric Association (EPA), supported by the European Association for the Study of Diabetes (EASD) and the European Society of Cardiology (ESC). Eur Psychiatry. 2009;24(6):412–24.

    Article  PubMed  Google Scholar 

  37. NICE. Cardiovascular disease: risk assessment and reduction, including lipid modification: London: National Institute for Health and Care Excellence (NICE); 2016. Available from: https://www.ncbi.nlm.nih.gov/books/NBK554923/.

  38. Perk J, De Backer G, Gohlke H, Graham I, Reiner Z, Verschuren M, et al. European Guidelines on cardiovascular disease prevention in clinical practice (version 2012). The Fifth Joint Task Force of the European Society of Cardiology and Other Societies on Cardiovascular Disease Prevention in Clinical Practice (constituted by representatives of nine societies and by invited experts). Eur Heart J. 2012;33(13):1635–701.

    Article  CAS  PubMed  Google Scholar 

  39. Allan GM, Nouri F, Korownyk C, Kolber MR, Vandermeer B, McCormack J. Agreement among cardiovascular disease risk calculators. Circulation. 2013;127(19):1948–56.

    Article  PubMed  Google Scholar 

  40. Visseren FLJ, Mach F, Smulders YM, Carballo D, Koskinas KC, Bäck M, et al. 2021 ESC Guidelines on cardiovascular disease prevention in clinical practice. Eur Heart J. 2021;42(34):3227–337.

    Article  PubMed  Google Scholar 

  41. Osborn DPJ, Hardoon S, Omar RZ, Holt RIG, King M, Larsen J, et al. Cardiovascular risk prediction models for people with severe mental illness: results from the prediction and management of cardiovascular risk in people with severe mental illnesses (PRIMROSE) research program. JAMA Psychiat. 2015;72(2):143–51.

    Article  Google Scholar 

  42. Hippisley-Cox J, Coupland C, Brindle P. Development and validation of QRISK3 risk prediction algorithms to estimate future risk of cardiovascular disease: prospective cohort study. BMJ. 2017;357:j2099.

    Article  PubMed  PubMed Central  Google Scholar 

  43. Perry BI, Upthegrove R, Crawford O, Jang S, Lau E, McGill I, et al. Cardiometabolic risk prediction algorithms for young people with psychosis: a systematic review and exploratory analysis. Acta Psychiatr Scand. 2020;142(3):215–32.

    Article  CAS  PubMed  Google Scholar 

  44. Berry A, Drake RJ, Webb RT, Ashcroft DM, Carr MJ, Yung AR. Investigating the Agreement Between Cardiovascular Disease Risk Calculators Among People Diagnosed With Schizophrenia. Front Psychiatry. 2018;9:685.

    Article  PubMed  PubMed Central  Google Scholar 

  45. Cohen D. Een ernstige psychiatrische aandoening verkort de levensverwachting. Huisarts en wetenschap. 2015;58:16–8.

    Article  Google Scholar 

  46. Bartels-Velthuis AA, Visser E, Arends J, Pijnenborg GHM, Wunderink L, Jörg F, et al. Towards a comprehensive routine outcome monitoring program for people with psychotic disorders: The Pharmacotherapy Monitoring and Outcome Survey (PHAMOUS). Schizophr Res. 2018;197:281–7.

    Article  PubMed  Google Scholar 

  47. Simoons M, Ruhé HG, van Roon EN, Schoevers RA, Bruggeman R, Cath DC, et al. Design and methods of the “monitoring outcomes of psychiatric pharmacotherapy” (MOPHAR) monitoring program - a study protocol. BMC Health Serv Res. 2019;19(1):125.

    Article  PubMed  PubMed Central  Google Scholar 

  48. Woodward M, Hirakawa Y, Kengne AP, Matthews DR, Zoungas S, Patel A, et al. Prediction of 10-year vascular risk in patients with diabetes: the AD-ON risk score. Diabetes Obes Metab. 2016;18(3):289–94.

    Article  CAS  PubMed  Google Scholar 

  49. Watson PF, Petrie A. Method agreement analysis: a review of correct methodology. Theriogenology. 2010;73(9):1167–79.

    Article  CAS  PubMed  Google Scholar 

  50. Grundy SM, Cleeman JI, Daniels SR, Donato KA, Eckel RH, Franklin BA, et al. Diagnosis and management of the metabolic syndrome: an American Heart Association/National Heart, Lung, and Blood Institute Scientific Statement. Circulation. 2005;112(17):2735–52.

    Article  PubMed  Google Scholar 

  51. Young DS. Implementation of SI units for clinical laboratory data. Style specifications and conversion tables. Ann Intern Med. 1987;106(1):114–29.

    Article  CAS  PubMed  Google Scholar 

  52. Anderson TJ, Grégoire J, Hegele RA, Couture P, Mancini GB, McPherson R, et al. 2012 update of the Canadian Cardiovascular Society guidelines for the diagnosis and treatment of dyslipidemia for the prevention of cardiovascular disease in the adult. Can J Cardiol. 2013;29(2):151–67.

    Article  PubMed  Google Scholar 

  53. NHG. NHG-Standaard Cardiovasculair risicomanagement (M84). Available from: https://richtlijnen.nhg.org/files/pdf/93_Cardiovasculair%20risicomanagement_juni-2019.pdf.

  54. R Core Team. R: A language and environment for statistical computing. Vienna, Austria: R Foundation for Statistical Computing. https://www.R-project.org/.

  55. Fagerland MW, Lydersen S, Laake P. The McNemar test for binary matched-pairs data: mid-p and asymptotic are better than exact conditional. BMC Med Res Methodol. 2013;13:91.

    Article  PubMed  PubMed Central  Google Scholar 

  56. Landis JR, Koch GG. The measurement of observer agreement for categorical data. Biometrics. 1977;33(1):159–74.

    Article  CAS  PubMed  Google Scholar 

  57. Botsch RE. Chapter 12. Significance and Measures of Association. [Available from: http://polisci.usca.edu/apls301/Text/Chapter%2012.%20Significance%20and%20Measures%20of%20Association.htm.

  58. McBride GB. A proposal for strength-of-agreement criteria for Lin's concordance correlation coefficient. Available from: https://www.medcalc.org/download/pdf/McBride2005.pdf.

  59. Garcia-Portilla MP, Saiz PA, Bascaran MT, Martínez AS, Benabarre A, Sierra P, et al. Cardiovascular risk in patients with bipolar disorder. J Affect Disord. 2009;115(3):302–8.

    Article  PubMed  Google Scholar 

  60. Grover S, Nebhinani N, Chakrabarti S, Avasthi A, Basu D, Kulhara P, et al. Cardiovascular risk factors among bipolar disorder patients admitted to an inpatient unit of a tertiary care hospital in India. Asian J Psychiatr. 2014;10:51–5.

    Article  PubMed  Google Scholar 

  61. Slomka JM, Piette JD, Post EP, Krein SL, Lai Z, Goodrich DE, et al. Mood disorder symptoms and elevated cardiovascular disease risk in patients with bipolar disorder. J Affect Disord. 2012;138(3):405–8.

    Article  PubMed  PubMed Central  Google Scholar 

  62. Hoffman BM, Sherwood A, Smith PJ, Babyak MA, Doraiswamy PM, Hinderliter A, et al. Cardiovascular disease risk, vascular health and erectile dysfunction among middle-aged, clinically depressed men. Int J Impot Res. 2010;22(1):30–5.

    Article  CAS  PubMed  Google Scholar 

  63. Ratliff JC, Palmese LB, Reutenauer EL, Srihari VH, Tek C. Obese schizophrenia spectrum patients have significantly higher 10-year general cardiovascular risk and vascular ages than obese individuals without severe mental illness. Psychosomatics. 2013;54(1):67–73.

    Article  PubMed  Google Scholar 

  64. Sicras-Mainar A, Rejas-Gutiérrez J, Navarro-Artieda R, Blanca-Tamayo M. C-reactive protein as a marker of cardiovascular disease in patients with a schizophrenia spectrum disorder treated in routine medical practice. Eur Psychiatry. 2013;28(3):161–7.

    Article  CAS  PubMed  Google Scholar 

  65. Wilson PW, D’Agostino RB, Parise H, Sullivan L, Meigs JB. Metabolic syndrome as a precursor of cardiovascular disease and type 2 diabetes mellitus. Circulation. 2005;112(20):3066–72.

    Article  CAS  PubMed  Google Scholar 

  66. Brotman DJ, Walker E, Lauer MS, O’Brien RG. In search of fewer independent risk factors. Arch Intern Med. 2005;165(2):138–45.

    Article  PubMed  Google Scholar 

  67. Schoofs MC, Akkermans RP, de Grauw WJ, Schalk BW, van Dis I, Tjin ATJ, et al. Performance of the SCORE and Globorisk cardiovascular risk prediction models: a prospective cohort study in Dutch general practice. Br J Gen Pract. 2023;73(726):e24–33.

    Article  PubMed  Google Scholar 

  68. Hense HW, Schulte H, Löwel H, Assmann G, Keil U. Framingham risk function overestimates risk of coronary heart disease in men and women from Germany–results from the MONICA Augsburg and the PROCAM cohorts. Eur Heart J. 2003;24(10):937–45.

    Article  PubMed  Google Scholar 

  69. Thomsen TF, McGee D, Davidsen M, Jørgensen T. A cross-validation of risk-scores for coronary heart disease mortality based on data from the Glostrup Population Studies and Framingham Heart Study. Int J Epidemiol. 2002;31(4):817–22.

    Article  PubMed  Google Scholar 

  70. van der Heijden AA, Ortegon MM, Niessen LW, Nijpels G, Dekker JM. Prediction of coronary heart disease risk in a general, pre-diabetic, and diabetic population during 10 years of follow-up: accuracy of the Framingham, SCORE, and UKPDS risk functions: The Hoorn Study. Diabetes Care. 2009;32(11):2094–8.

    Article  PubMed  PubMed Central  Google Scholar 

  71. Damen JA, Pajouheshnia R, Heus P, Moons KGM, Reitsma JB, Scholten R, et al. Performance of the Framingham risk models and pooled cohort equations for predicting 10-year risk of cardiovascular disease: a systematic review and meta-analysis. BMC Med. 2019;17(1):109.

    Article  PubMed  PubMed Central  Google Scholar 

  72. de Graaf R, Bijl RV, Spijker J, Beekman AT, Vollebergh WA. Temporal sequencing of lifetime mood disorders in relation to comorbid anxiety and substance use disorders–findings from the Netherlands Mental Health Survey and Incidence Study. Soc Psychiatry Psychiatr Epidemiol. 2003;38(1):1–11.

    Article  PubMed  Google Scholar 

  73. Goldstein BI, Carnethon MR, Matthews KA, McIntyre RS, Miller GE, Raghuveer G, et al. Major depressive disorder and bipolar disorder predispose youth to accelerated atherosclerosis and early cardiovascular disease: a scientific statement from the American Heart Association. Circulation. 2015;132(10):965–86.

    Article  PubMed  Google Scholar 

  74. Merikangas KR, Akiskal HS, Angst J, Greenberg PE, Hirschfeld RM, Petukhova M, et al. Lifetime and 12-month prevalence of bipolar spectrum disorder in the National Comorbidity Survey replication. Arch Gen Psychiatry. 2007;64(5):543–52.

    Article  PubMed  PubMed Central  Google Scholar 

  75. van der Werf M, Hanssen M, Köhler S, Verkaaik M, Verhey FR, van Winkel R, et al. Systematic review and collaborative recalculation of 133,693 incident cases of schizophrenia. Psychol Med. 2014;44(1):9–16.

    Article  PubMed  Google Scholar 

  76. Vandenbroucke JP, von Elm E, Altman DG, Gøtzsche PC, Mulrow CD, Pocock SJ, et al. Strengthening the Reporting of Observational Studies in Epidemiology (STROBE): explanation and elaboration. PLoS Med. 2007;4(10):e297.

    Article  PubMed  PubMed Central  Google Scholar 

Download references

Acknowledgements

K. Wardenaar, PhD at the department of psychiatry, UMCG, for his critical appraisal of our R-syntax.

Collaborating author names of the PHAMOUS investigators-group: Agna Bartels-Velthuis, PhD, Richard Bruggeman, PhD, Stynke Castelein, PhD, Frederike Jörg, PhD, Henderikus Knegtering, PhD, Marieke Pijnenborg, PhD.

Funding

The work for this paper was funded by Espria (number GR18-130b), and a societal PhD grant from the University Medical Center Groningen (RGOC 671677). Espria was not involved in the development, execution or writing process of this study.

Author information

Authors and Affiliations

  1. Mental Health Services Drenthe, P.O. box 30007, 9400 RA, Assen, The Netherlands

    Davy Quadackers & Danielle Cath

  2. Department of Psychiatry, University of Groningen, University Medical Center Groningen, Rob Giel Research Center, P.O. box 30.001, 9700 RB, Groningen, The Netherlands

    Davy Quadackers, Edith Liemburg, Fionneke Bos, Ellen Visser & Danielle Cath

  3. Department of Clinical Psychology & Experimental Psychopathology, University of Groningen, Faculty of Behavioural and Social Sciences, Grote Kruisstraat 2/1, 9712 TS, Groningen, The Netherlands

    Fionneke Bos

  4. Research Department, Lentis Psychiatric Institute, Hereweg 80, 9725 AG, Groningen, The Netherlands

    Bennard Doornbos

  5. Department of Clinical Pharmacy, Wilhelmina Hospital, Assen, The Netherlands

    Arne Risselada

  6. Department of General Practice & Elderly Care Medicine, University of Groningen, University Medical Center Groningen, Groningen, the Netherlands

    Marjolein Berger

Authors
  1. Davy Quadackers
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Edith Liemburg
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Fionneke Bos
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Bennard Doornbos
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Arne Risselada
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Marjolein Berger
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Ellen Visser
    View author publications

    You can also search for this author in PubMed Google Scholar

  8. Danielle Cath
    View author publications

    You can also search for this author in PubMed Google Scholar

Consortia

PHAMOUS investigators

  • Agna Bartels-Velthuis
  • , Richard Bruggeman
  • , Stynke Castelein
  • , Frederike Jörg
  • , Henderikus Knegtering
  •  & Marieke Pijnenborg

Contributions

MB conceived the study together with DC. DC supervised the study. EL contributed to the data and sample collection of the MOPHAR cohort. EV contributed to the data and sample collection of the PHAMOUS-cohort. DQ carried out the statistical analysis, interpreted the data together with DC, EL, and FB, and wrote the draft of the manuscript. All authors critically reviewed, commented, and approved the final manuscript.

Corresponding author

Correspondence to Davy Quadackers.

Ethics declarations

Ethics approval and consent to participate

The research aspects of MOPHAR were approved by the independent medical ethics committee (RTPO 928, rTPO Leeuwarden, The Netherlands). The PHAMOUS assessments were carried out in accordance with the Declaration of Helsinki (October 2013), and local and international ethical standards, as confirmed by the ethical committee of the University Medical Center of Groningen in the Netherlands.

Consent for publication

Not applicable.

Competing interests

The authors declare that they have no competing interests.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary Information

Additional file 1:

Supplementary document 1. [Quadackers_suppl1.docx; contains descriptive tables by ‘inclusion’ and missing status, pairwise comparisons of continuous outcomes of CVD-risk models, and appendix 1 (specifies which MetS criteria and riskfunctions we employed].

Additional file 2:

Supplementary document 2. [Quadackers_suppl2.docx; contains the mean CVD-risks stratified per MetS-criterium and MetS-profile].

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Quadackers, D., Liemburg, E., Bos, F. et al. Cardiovascular risk assessment methods yield unequal risk predictions: a large cross-sectional study in psychiatric secondary care outpatients. BMC Psychiatry 23, 536 (2023). https://doi.org/10.1186/s12888-023-05022-1

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1186/s12888-023-05022-1

Keywords

BMC Psychiatry

ISSN: 1471-244X

Contact us