Skip to main content
Download PDF

The assessment of capacity limitations in psychiatric work disability evaluations by the social functioning scale Mini-ICF-APP

BMC Psychiatry volume 21, Article number: 480 (2021) Cite this article

Abstract

Objective

Insurers frequently commission medical experts to estimate the degree of the remaining work capacity (RWC) in claimants for disability benefits. The social functioning scale Mini-ICF-APP allows for a rating of activity and participation limitations in thirteen capacity domains, considered as particularly relevant for work ability. The current study sought to evaluate the role of the Mini-ICF-APP ratings in psychiatric work disability evaluations, by examining how the capacity limitation ratings varied with the claimants’ primary psychiatric diagnoses and how the ratings were related to RWC estimates.

Methods

Medical experts estimated the RWC of 946 claimants with mental disorders and rated their activity and participation limitations using the Mini-ICF-APP, with higher ratings reflecting more severe limitations. The ratings were compared between claimants with different psychiatric diagnoses by analyses of variance. The mean Mini-ICF-APP rating across all capacity domains as well as all capacity-specific ratings were entered in simple or multiple regression models to predict the RWC in an alternative job.

Results

The Mini-ICF-APP capacity limitation ratings in all domains but mobility were higher for claimants with personality and behavior disorders as compared to those with mood disorders or with neurotic, stress-related and somatoform disorders. The largest differences were observed in social capacities (e.g. group integration: F 2, 847 = 78.300, P < 0.001). In claimants with depression, all ratings increased with the severity of the diagnosis (all Fs 2, 203 > 16.393, all Ps < 0.001). In the overall sample, the mean Mini-ICF-APP rating showed a strong negative correlation with the estimated RWC (r = −.720, P < 0.001). Adding the capacity-specific ratings to the prediction model improved this prediction only marginally.

Discussion

The Mini-ICF-APP allows for documenting claimants’ activity and participation limitations, which is likely to increase the transparency of medical experts’ RWC estimates and enables them to check the plausibility of such estimates. However, our study showed that despite the strong association between RWC and Mini-ICF-APP ratings, half of the RWC variance was unrelated to the capacity limitations documented in the Mini-ICF-APP.

Peer Review reports

Introduction

Statutory disability insurances provide disability benefits for individuals with strong restrictions in their work capacity due to disease- or injury-related impairments, in case the work capacity cannot be restored or improved by vocational integration services. In 2019, the Swiss disability insurance paid disability benefits of 5.4 billion CHF (5.7 billion US $) to a total of 247′000 claimants [1]. The latter number corresponds to 4% of all insured Swiss citizens. In Switzerland, the rate of individuals receiving disability benefits is thus higher than the rate of the unemployed [2].

Work disability assessments by medical experts are internationally the most common procedure to determine claimants’ entitlement for benefits [3]. The definition of the expert’s role in these assessments depends on the national social security system. In Switzerland, medical experts assess the claimants’ remaining work capacity (RWC) in percent, as a basis for determining claimants’ eligibility for disability benefits and the amount of benefits they are entitled to. There are currently four levels of disability benefits, varying by percentage of invalidity (“Invalidität”). In order to qualify for such benefits, claimants must suffer from at least 40% invalidity, defined as a 40% income loss due to permanent work incapacity in relation to the income they could achieve without a disability. An invalidity of 40% qualifies for 25% of full disability benefits (“Viertelsrente”). Beyond 40% invalidity, there is a 25% increase of full disability benefits with a 10% increase of invalidity. This means that claimants with 50, 60, and 70% invalidity receive 50, 75, and 100% of full disability benefits, respectively [4].

These numbers suggest that small variance in the estimated RWC can have considerable impact on the granting of disability benefits. Ideally, the estimation of RWC should primarily depend on claimants’ health-related impairments. Unfortunately, the interrater agreement in RWC estimates is commonly limited even for the very same claimants [5,6,7], which might ultimately result in unjust financial compensation. Pizala [8] criticized that, in particular, psychiatric work disability evaluations often lack information on what considerations RWC estimates are based on, implying an insufficient transparency and a lack of objective data in these evaluations.

One way to tackle such a shortcoming is the implementation of standardized instruments in work disability evaluations that a) document activity and participation limitations of claimants, and b) form an empirical basis for the RWC estimated by the medical expert. Instruments for assessing work disability in a quantitative way are still sparse, albeit the production of core sets derived from the International Classification of Functioning, Disability and Health, ICF [9] have created a basis for changing the landscape of instruments in work disability assessments, but also provided a taxonomic framework for job placement and vocational rehabilitation [10,11,12,13].

The social functioning scale Mini-ICF-APP represents a rating instrument for activity and participation limitations of individuals with mental disorders [14,15,16] and is increasingly used in work disability evaluations. The name Mini-ICF-APP reflects the fact that the to-be-rated activity and participation domains were derived from the ICF [9], whereby APP stands for activity and participation limitations in mental/psychiatric disorders. The Mini-ICF-APP encompasses ratings of thirteen different capacity domains, which are considered as particularly relevant for work participation, namely (I) adherence to regulations, (II) planning and structuring of tasks, (III) flexibility, (IV) applying expertise, (V) competence to judge and decide, (VI) endurance, (VII) assertiveness, (VIII) contact with others, (IX) group integration, (X) intimate relationships, (XI) non-work activities, (XII) self-care, and (XIII) mobility. The ratings allow the differentiation of five levels of limitations, numerically coded from “0” to “4”. These levels are a) no disability (0 to 4% incapacity), b) mild disability (5 to 24% incapacity), c) moderate disability (25 to 49% incapacity), d) severe disability (50 to 95% incapacity), e) total disability (96 to 100% incapacity, [14]). The Mini-ICF-APP is available in several languages, including German, English, French, Italian, and Polish [14,15,16,17,18,19].

The role of the Mini-ICF-APP in work disability evaluation is so far little investigated. The current study focused on two important aspects, namely the question whether the Mini-ICF-APP ratings can capture diagnosis-specific capacity limitations and the question how the ratings relate to RWC estimates. With regard to the first aspect, previous research suggested that the nature of the mental disorder has some impact on the extent of activity and participation limitations. An Italian community-based study revealed larger Mini-ICF-APP sum scores in patients with schizophrenia than in patients with major depression and larger sum scores in patients with major depression than in patients with anxiety disorders [19]. Such findings presumably reflect in parts the differential severity of mental disorders, with increasing severity of mental disorders associated with more capacity limitations and higher Mini-ICF-APP total scores [19, 20]. However, such findings may partly also reflect disorder-specific capacity limitations, which has been investigated only to minimal extent so far (for different limitations in different work anxieties, see [21]). As one study aim, we sought to reveal such disorder-specific capacity limitations. To this end, we contrasted the ratings in each capacity domain and across all capacity domains between claimants for disability benefits with a) mood disorders, b) neurotic, stress-related and somatoform disorders, and c) disorders of adult personality and behavior. Moreover, for mood disorders, we compared the Mini-ICF-APP ratings of patients with mild, moderate and severe depressive episodes in order to test whether activity and participation limitations would increase with the severity of the disorder.

The second and major aim of the current study was to investigate the association between RWC estimates and Mini-ICF-APP ratings, which was previously addressed in two studies, including an own one. In both previous studies, the same psychiatrist rated their activity and participation limitations and estimated the claimants’ RWC. In a sample of 447 claimants for disability benefits, we showed that the RWC estimates and Mini-ICF-APP ratings agreed, as to-be-expected, on a group level: claimants with high RWC showed low levels of capacity limitations and claimants with low RWC showed high levels of limitations [22]. Assessing this kind of agreement in a sample of 121 claimants in more detail, Habermeyer and colleagues [23] revealed a significant correlation of r = 0.663 between work disability (as complement to RWC) and the Mini-ICF-APP capacity limitation sum score. However, up to now, it is unknown whether primarily the global functional disability (as reflected in the Mini-ICF-APP sum score or its respective mean score) contributes to the RWC estimate, or whether some domains of activity and participation are of higher relevance for medical experts when estimating the RWC. To answer this question, we calculated several linear regression models with the Mini-ICF-APP mean score, ratings in individual domains, or both as predictors for the RWC and compared how well these models explained the RWC variance. Moreover, in extension to our previous study [22], we contrasted the Mini-ICF-APP ratings between claimants with high, moderate, and low RWC.

Methods

Participants and data collection

From February 2010 to October 2016, 946 claimants for disability benefits undergoing a multidisciplinary work disability evaluation at MEDAS Zentralschweiz (Lucerne, Switzerland) were rated with the Mini-ICF-APP as part of their psychiatric assessment. This included 447 claimants of our previous study [22]. The mean age of the claimants (532 female, 414 male) was 48.9 years (SD 8.5 years). Details on the distribution of gender and age across in the major psychiatric diagnoses can be found in Supplementary Table S1. Only claimants who underwent a psychiatric evaluation and received an ICD-10 diagnosis from the chapter “F – Mental and behavioural disorders” were included. The same psychiatrists conducted the Mini-ICF-APP ratings as part of their evaluation and estimated the RWC resulting from mental disorders. The pool of raters consisted of seven psychiatrists, three of whom conducted 84% of all ratings. For six individuals, the experts refrained from estimating the RWC. Consequently, the regression analyses for predicting the RWC were based on 940 cases only.

Statistical analyses

Due to data protection regulations, we only analysed pooled data and did not differentiate between the psychiatrists (due to their limited number, the anonymity of the medical experts would not have been guaranteed). In addition to the ratings in each of the 13 domains, we calculated the average rating across domains (MICFmean) as a measure for global capacity limitation. Linden et al. [15] named this average rating ‘global value’. Descriptive statistics on the summed MICF rating (MICFtotal) were also calculated. In analogy to the International Classification of Functioning [9], the ratings in each domain range from “0” to “4”, with higher ratings corresponding to more severe activity and participation limitations. Mini-ICF-APP data are often not normally distributed, but tend to show a right skew. The analysis of variance (ANOVA), used for analysing group differences, provides relatively robust results for data that are not normally distributed [24].

First, we compared all Mini-ICF-APP ratings in a univariate ANOVA between the three major groups of mental disorders as psychiatric diagnosis. Based on the ICD-10 (https://icd.who.int/browse10/2019/en#/V), these groups were (a) mood disorders (F30-F39), (b) neurotic, stress-related and somatoform disorders (‘neurotic disorders’, F40-F48), and (c) disorders of adult personality and behavior (‘personality disorders’, F60-F69). Moreover, for mood disorders, we compared the ratings in another univariate ANOVA between patients with mild (F32.0), moderate (F32.1), and severe (F32.2) depressive episodes. Inclusion of age and sex as co-variables had no relevant impact on the ANOVA results. For the sake of brevity, age and sex were therefore not considered as co-variables. The p values as reported in Tables 1 to 3 were not corrected for multiple testing. All significant F values were followed by post-hoc least significant difference tests, with an α criterion of P = 0.01. The F statistics provided identical results for MICFmean and MICFtotal, as one value can be linearly derived from the other (MICFtotal = k * MICFmean; with k as the number of domains = 13).

Table 1 Mini-ICF-APP ratings of claimants with different psychiatric diagnoses
Full size table
Table 2 Mini-ICF-APP ratings of claimants with depressive episodes
Full size table
Table 3 Mini-ICF-APP ratings in claimants with high, moderate, and low RWC
Full size table

The psychiatric experts estimated the RWC in the last and in an alternative job on the general labor market. An alternative job considers the claimant’s disabilities and accounts for adjustments to compensate them. This adjustment lowers the impact of disabilities on the work capacity and is used by the insurer to determine invalidity. We sought to reveal how the expert’s estimate of the RWC in an alternative job related to the Mini-ICF-APP ratings. For this purpose, we first compared the Mini-ICF-APP ratings in a univariate ANOVA between claimants with low, moderate, and high RWC (poor: 30% or less RWC; moderate: RWC between > 30 and < 70%; high: RWC of 70% or above). This approach extends the one by Jeger et al. [22] who exclusively contrasted the low and high RWC groups. Second, we used the Mini-ICF-APP ratings for predicting the RWC estimates by the medical expert. To this end, we ran four different kinds of linear regressions. A) In a simple linear regression analysis, we used the average Mini-ICF-APP rating (MICFmean) as predictor. B) In a multiple stepwise regression analysis, we used the domain-specific Mini-ICF-APP ratings as predictors, without considering MICFmean. C) In a hierarchical multiple regression analysis, we used MICFmean in the first step and, in a second step, we determined which domain-specific Mini-ICF-APP ratings would further improve the prediction of the RWC. D) In a final univariate regression analysis, we swapped the dependent and independent variable of the first simple linear regression analysis and used the RWC for predicting the average Mini-ICF-APP ratings. The latter analysis was conducted in order to overcome limitations of the other regression models due to the (‘bounded’) data distribution of RWC, as detailed in the results section.

Results

Mini-ICF-APP ratings: effects of the psychiatric diagnosis

848 of the 946 participants had either a mood disorder (F30-F39), a neurotic disorder (F40-F48), or a personality disorder (F60-F69) as primary psychiatric diagnosis. The distribution of the participants across these diagnoses, including information about their age and sex, is provided in Table 1. Due to their insufficient sample sizes, participants with other primary diagnoses were not considered in this analysis. Descriptive Mini-ICF-APP data on the total sample can be found in Supplementary Tables S2 and S3.

The univariate ANOVA with diagnosis as between-subject factor revealed significant differences in all domains but mobility between the three groups of psychiatric disorders (F30-F39 vs. F40-F48 vs. F60-F69). Claimants with personality disorders were on average more limited in activity and participation as compared to claimants with one of the other two disorders; claimants with mood disorders were on average more limited than claimants with neurotic disorders (Table 1, MICFmean). The percentage of RWC estimates ≤30% was also highest among claimants with personality disorders (Table 1, bottom row). Across all three sub-samples, limitations in the capacity domains flexibility and endurance were most pronounced, as compared to other capacity domains. Claimants with personality disorder showed additionally pronounced limitations in their social functioning (contact with others, group integration, intimate relationships, adherence to regulations), but were relatively well functioning in some other capacity domains (e.g. competence to judge and decide, Table 1 and Supplementary Table S4).

Mini-ICF-APP ratings of claimants with depressive episodes

There were 204 participants with either mild, moderate, or severe depressive episodes (Table 2). The MICFmean across these claimants was 1.46 (SD 0.59). The univariate ANOVA revealed that all Mini-ICF-APP ratings varied between the three diagnoses, with increased severity of the diagnosis being associated with higher Mini-ICF-APP ratings, as verified in post-hoc tests (Table 2 and Supplementary Table S5). The percentage of claimants with an RWC ≤ 30% was considerably larger in claimants with severe depressive episodes, as compared to claimants with mild and moderate depressive episodes (Table 2, bottom row).

Relationship between Mini-ICF-APP ratings and RWC estimates

The Mini-ICF-APP ratings were compared between the claimants of the three subsamples, as defined by the estimated RWC in an alternative job. Across all participants with RWC estimates (n = 940), the RWC was on average 50.6% (SD 29.4%). The initial univariate ANOVA revealed significant differences in all Mini-ICF-APP domains, with (as expected) higher Mini-ICF-APP ratings being associated with lower RWC (Table 3). The rating profiles (reflecting the differential Mini-ICF-APP item difficulty) within each of the three RWC groups were very similar: All three subsamples exhibited the highest ratings for endurance and flexibility and the lowest ratings for self-care and mobility (Table 3), similar to previous reports [15]. Of note, relatively strong limitations in endurance were observed even in claimants with a high RWC: Almost 70% of the claimants with high RWC were rated at least as moderately limited in this domain (ratings ≥2, Table S6).

Regression analyses

In the initial simple linear regression analysis, we observed a strong association between the RWC and mean Mini-ICF-APP ratings (MICFmean, R2 = 0.518, Fig. 1, Table 4 top row). The stepwise multiple regression revealed a slightly higher R2 when four (or more) domain-specific ratings were entered as predictors, whereby the improvement of R2 was below 0.01 after entering a fifth predictor. Such model with five predictors encompassed the ratings for flexibility, endurance, intimate relationships, planning and structuring a task, and group integration (Table 4, middle row, R2 = 0.550). In a combined hierarchical multiple regression model, we first entered MICFmean as predictor and subsequently entered the five domain-specific capacity ratings (as identified in the stepwise multiple regression) as additional predictors. In this model, the improvement of R2 was below 0.01 after entering endurance and intimate relationships as additional predictors (Table 4, bottom row, R2 = 0.548).

Fig. 1
figure 1

Simple linear regression model with MICFmean as predictor. Scatterplot of the data, with the MICFmean ratings on the x-axis and the RWC, as estimated by the medical expert, on the y axis. The black line represents the regression curve with MICFmean as predictor for the RWC. This curve fit is compromised by heteroscedasticity, all data points with RWC = 100% are for example on the right side of the curve

Full size image
Table 4 Summary of the regression models
Full size table

Unfortunately, all models are flawed by heteroscedasticity, meaning the residual variance is not equally distributed and it shows a systematic linear trend with the dependent variable, as shown for the simple linear regression model with MICFmean as predictor (Fig. 2). Thus, the regression models as described in Table 4 inform about the magnitude of the association between Mini-ICF-APP ratings and RWC estimate, but the estimation of the regression curves is flawed by the bounded data distribution. A more reliable regression curve can be obtained by reversing the dependent and independent variables, meaning to predict MICFmean based on the RWC estimate (Fig. 3). (One could equally well argue that high Mini-ICF-APP ratings should be associated with low RWC estimates and that a low RWC should result in high Mini-ICF-APP ratings.) The data displayed in Fig. 3 is the very same data as displayed in Fig. 1, with just x and y axes swapped. The regression curve in Fig. 3 is less steep than one would expect from Fig. 1, but it now runs at each RWC level through the data points (MICFmean predicted = − 0.015 * RWC + 2.136, R2 = 0.518).

Fig. 2
figure 2

Systematic prediction error of the simple linear regression model with MICFmean as predictor. Scatterplot of the prediction error for RWC in dependence of the RWC, reported by the medical expert to the insurer. The plot indicates that high RWCs were underestimated and low RWCs were overestimated, when predicted based on the MICFmean score, as compared to the reported RWC.

Full size image
Fig. 3
figure 3

Simple linear regression model with RWC as predictor. Scatterplot of the data, with the RWC, as estimated by the medical expert, on the x axis and MICFmean ratings on the y axis. The data are thus identical to those displayed in Fig. 1, with just the axes swapped. The black line represents the regression curve, with RWC values used for predicting MICFmean

Full size image

In a subsequent explorative analysis, we compared the MICFmean values between 10% levels of RWC in a univariate ANOVA, followed by post-hoc pairwise comparisons, in order to reveal how well these RWC levels could be differentiated based on the MICFmean. To this end, the values of RWC were rounded to multiples of 10%. The MICFmean difference between 100 and 0% RWC was 1.49 (SE 0.06), meaning for each RWC decrease by 10% there was on average a MICFmean increase by 0.15, which corresponds to approximately 2 points for MICFtotal. MICFmean varied highly significantly between the eleven rounded levels of RWC (F 10, 929 = 100.155, P < 0.001). However, the MICFmean differences between two neighboring RWC levels varied between as little as −.02 (SD 0.08, 30 and 40% RWC) and as much as 0.29 (SD 0.06, 40 and 50%, Table 5). The MICFmean differences between 20, 30, and 40% RWC, 60 and 70% RWC, as well as between 90 and 100% RWC did not reach statistical significance in the post-hoc testing.

Table 5 MICFmean and MICFtotal for each 10% level of RWC
Full size table

Discussion

The study sought to analyze the difference in Mini-ICF-APP capacity limitation ratings between claimants with different psychiatric diagnoses and to determine how well the expert’s RWC estimate could be inferred from his corresponding Mini-ICF-APP ratings. The major findings are discussed below.

Mini-ICF-APP ratings and psychiatric diagnoses

The comparison of the Mini-ICF-APP ratings between claimants with personality, mood, and neurotic disorders showed that the activity and participation limitations were most severe in claimants with personality disorders. Study participants with personality disorders particularly showed strong social limitations (contact with others, group integration, intimate relationships, adherence to regulations). This finding is in line with views that impairments in social, occupational, or other major areas of life represent an integral aspect of personality disorders [25, 26]. Yet, also in other domains (such as flexibility and endurance), the psychiatrists rated them as more limited, with flexibility generally considered as fundamental aspect of health, but also of social functioning [27]. Social interactions have been regarded as constituting element of work [28] and it is hard to imagine any kind of work which lacks this element. Given the degree and the kind of their capacity limitations, it was little surprising that experts assigned low RWCs to claimants with personality disorders more than twice as frequently than to claimants with mood or neurotic disorders (Table 1).

Nevertheless, the severity of capacity limitations in claimants with personality disorder was unexpected, also given the high prevalence of personality disorders in the general population [29]. Claimants with personality disorder and less severe capacity limitations appear to be underrepresented in the study sample. It is not fully clear why this could had been the case. It would be worrying if in order to qualify as ‘candidate’ for disability benefits the threshold of capacity limitations would be higher for individuals with personality disorders than for other disorders, and if less severe capacity limitations of such individuals would be downplayed and attributed to their difficult personality (rather than to their disorder). Future studies should pay special attention to capacity limitations in individuals with personality disorders to clarify this issue.

Among claimants with depressive episodes, the deficits increased across all domains with the severity of the diagnosis. This was expected as the severity of depressive episodes is largely defined by a) the symptom load (with more and more distressing symptoms present with increasing severity), and b) the ability to continue activities. The symptom load can be expected to affect activities and participation, as some symptoms of depressive episodes describe likewise limitations in function and activity/participation, as e.g. the reduction of energy. The finding shows that for claimants with depressive episodes the extent of limitations in activity and participation largely agreed with their psychiatric diagnosis. In contrast to the differences between claimants with personality disorders and claimants with mood and neurotic disorders, rating profiles of the three sub-samples with depressive episodes were quite similar. This finding is in line with previous studies, showing an association between MICFtotal and the severity of the psychiatric symptoms [19, 20].

When placing the here investigated Mini-ICF-APP capacity limitations into context with previous research, it is important to consider the inclusion criteria of our study. The Mini-ICF-APP sum score of our participants with depressive episodes was on average 3 to 4 points higher than the scores for community-based patient samples with such diagnosis [16, 19]. Likewise, the sum score of the total sample was more than 7 points higher than the sum score of the sample reported in the Mini-ICF-APP manual, consisting of 213 psychosomatic rehabilitation patients [15]. The ratings in our sample corresponded to those observed for psychiatric inpatients after admission [20]. The on average high ratings in our study sample presumably reflect the fact that all study participants filed disability benefits. This step is usually proceeded only after return-to-work programs failed, implying persisting capacity deficits [30].

To sum up, aside from increased ratings with increased severity of the mental disorder, the current study revealed for the first time disorder-specific alterations of the Mini-ICF-APP capacity limitation ratings, with claimants with personality disorders showing pronounced limitations particularly in their social functioning.

Mini-ICF-APP ratings and work-capacity ratings

In line with previous reports [15, 22, 23, 31], Mini-ICF-APP ratings showed significant differences between claimants with high, moderate, and low RWC (Table 4), which documents a consistency of expert estimates in RWC with observed limitations in activity and participation on a group level. Across all capacity domains, claimants with low RWC had the highest and those with high RWC the lowest Mini-ICF-APP ratings. The regression analyses showed, as expected, that the RWC linearly decreased with increasing Mini-ICF-APP ratings. This finding is well in line with another Swiss study that revealed a strong linear relationship between the Mini-ICF-APP sum score and work incapacity (as complement to work capacity [23]). However, in both studies, the estimation of the exact linear relationship is evidently hampered by heteroscedasticity (an unequal distribution of the residual variance).

In our study, inspection of the residual variance showed that high RWCs were underestimated and low RWCs were overestimated when using MICFmean as predictor in the linear regression analysis (Fig. 2). For the curve fit, this leads to the impression of a mirrored z-form, with the regression curve as diagonal slash (Fig. 1). Such a z-form is similarly present in Fig. 2 of Habermeyer et al. [23]. Heteroscedasticity cannot be easily dissolved as it is related to the data distribution: As evident from Fig. 1, the RWC variance is considerably lower for MICFmean ratings < 1 and > 2 than for MICFmean ratings between 1 and 2. Related and likewise importantly, the assigned RWC values are bounded as the RWC can neither be worse than 0% nor better than 100%.

To tackle the problem of heteroscedasticity, we swapped the dependent and independent variable. The swapping resulted in a linear curve fit with a similar level of residual MICFmean variance for each level of RWC (Fig. 3). To some extent, this regression curve can be used for estimating the plausibility of an assigned RWC based on MICFmean. However, Figs. 1 and 3 illustrate that empirically MICFmean ratings between 1 and 2 were associated with any RWC (0 to 100%). This indicates that, even though the statistical correlation between MICFmean and RWC was relatively high, it was not high enough to derive an individual’s RWC from his/her MICFmean rating with sufficient confidence. About half of the RWC variance was unrelated to the capacity limitations documented in the Mini-ICF-APP although the very same medical expert provided the Mini-ICF-APP ratings and RWC estimate. One could argue that, when estimating the RWC, medical experts considered not just the amount of limitations, but also their kind, and they weighted limitations in some domains higher than in others. However, the multiple regression analysis did not reveal evidence that the consideration of domain-specific ratings resulted in a noticeably better RWC prediction than just considering MICFmean as predictor.

The poor predictive power of the capacity-specific Mini-ICF-APP data might be to some extent due to the limited number of response options, which range from “0” (no disability) to “4” (total disability), with the latter response option being hardly ever applied in our sample (Supplement Table S3). The limited number of response options implies that the variance of ratings in each domain is limited, which constrains the explanatory power of these ratings as predictors for RWC as more finely graded outcome variable. Moreover, the usability of the Mini-ICF-APP ratings as predictors is restricted by how the Mini-ICF-APP ratings refer to the quantity of limitations: A rating of “0” refers to a range of 4% limitations (0 to 4%), whereas a rating of “3” refers to a range of 45% limitation (50 to 95%, [15, 16]). For rehabilitation and therapy, a coarse categorization of limitations might be considered as sufficient, because all limitations would anyway require a qualitative (rather than a quantitative) specification.

However, for the purpose of work disability evaluations, the Mini-ICF-APP should allow for a more finely graded rating. This raises the question whether it is possible to rate capacities like assertiveness based on clinical interviews and medical and other records with sufficiently high precision. Moreover, what is a sufficiently high precision? For RWC ratings, most stakeholders in Switzerland expect a maximum acceptable difference between two raters between 10 and 20% [32]. In a naturalistic study setting, Kunz et al. [7] showed that the RWC disagreements between two experts were > 20% in approximately one third of the evaluations, even though the medical experts received additional training beforehand. Thus, for Mini-ICF-APP ratings, a precision between 10 and 20% would be an optimistic expectation. Ideally, the grading of the Mini-ICF-APP ratings should correspond to the commonly used 10% RWC levels, when these ratings are supposed to serve the purpose to document capacity limitations contributing to a diminished RWC. However, to what degree psychiatric experts are able to reliably discriminate 10% differences in their capacity ratings of individual Mini-ICF-APP items when using clinical judgment needs to be tested empirically.

An essential aspect for determining the role of the Mini-ICF-APP in work disability evaluations is how the instrument is actually used in this context. The manual suggests that just 10 min are required to provide the ratings and to analyze the results. This does not correspond to the perceptions in insurance medicine [33]. These authors stress that for work disability evaluations first the relevant information needs to be gathered from different sources and subsequently checked for consistency. They recommend that the ratings should be accompanied by a “narrative explanation”, informing about based on which information the expert provided his rating. Such narrative explanations would increase the transparency and (to some extent) the plausibility of the evaluation, as the information relevant for the evaluation is documented, like it is suggested for injury-related disability evaluations [34]. For an example, see Kunze [35]. We propose that ideally these narrative explanations refer to the job demands in the last job as well as in an alternative job, and are based on functional interviews [34, 36].

To sum up, RWC estimates cannot be derived from Mini-ICF-APP ratings with sufficient precision, even when considering capacity-specific ratings. About half of the RWC variance was unrelated to activity and participation limitations documented in the Mini-ICF-APP ratings. This large variability only enables a plausibility check of RWC estimates and the identification of gross outliers (values in far distance to the regression curve, as depicted in Fig. 3).

Study limitations

The reported data were obtained in a large sample of claimants for disability benefits in a real-life setting. As one major study limitation, the number of medical experts as participants was quite limited. Given that three of them conducted more than 80% of the ratings, the individual rating behavior could have had some impact on the results, even though the profiles of the Mini-ICF-APP ratings presented here are generally well in accordance with previous reports [15, 22, 23, 31]. For example, there could have been some variance between experts how they conceptualized the individual domains of the Mini-ICF-APP and limitations herein. Moreover, even though all participating experts worked in the same assessment center and likely shared some standards, there might still have been systematic differences between them. These differences may have included the preference for diagnosing certain psychiatric disorders over others, a focus on specific limitations during exploration, or how finely graded their RWC estimation was. Future studies should include a larger number of medical experts in order to minimize the impact of individual rater characteristics on the results, even though this would not necessarily improve the RWC prediction based on the Mini-ICF-APP ratings. However, a detailed analysis of an expert’s rating profile might provide some valuable feedback for the expert himself in order to identify in which aspects his rating behavior varies from the rating behavior of others (e.g. by being too conservative or too liberal in the ratings).

The basic concept of the current study is that activity and participation limitations can be translated into an RWC estimate. However, the specification and quantification of such limitations in the work context are not trivial, even though anchor points for the rating are clearly defined in the Mini-ICF-APP manual. The current study just considers the endpoints: namely, on the one hand the activity and participation limitations as rated in the Mini-ICF-APP, and on the other hand the RWC, as estimated by the expert. The process of the formation of these values was not considered in this study. Given this, the study does not provide insights about from which sources the RWC variance stems that is unrelated to the Mini-ICF-APP ratings. It might stem from fuzzy ratings, maybe related to variable, inconsistent limitations of the claimant, to insufficient knowledge of medical experts about work and work requirements, or related to the lack of reliable information at hand, e.g. when a claimant was out of work for an already considerable period. However, it might also stem from imprecise RWC estimates, as previous studies showed considerable variance between experts for the very same case [5,6,7].

Conclusions

The Mini-ICF-APP allows the documentation of activity and participation limitations of claimants with psychiatric disorders. If accompanied by narrative explanations, these ratings can help stakeholders (including claimants, case managers, treating physicians, lawyers, judges etc.) to understand in condensed form based on what observations the medical expert estimated the RWC. The Mini-ICF-APP ratings thus provide a bridge between the psychiatric diagnosis and RWC, with various mental activity and participation domains systematically considered. The consistency between the Mini-ICF-APP capacity limitation ratings and RWC is likely to increase if the claimants’ work-related limitations are assessed in the light of the demands of particular jobs and occupations. Given this, medical experts should always report MICFmean, capacity-specific ratings, and provide narrative explanations. Our study showed that it is possible, in principle, to assess the plausibility of RWC estimates based on the mean Mini-ICF-APP rating as well as to check the plausibility of a claimant’s Mini-ICF-APP profile with regard to his or her psychiatric diagnosis. However, the considerable variance in Mini-ICF-APP ratings at each RWC level as well as within psychiatric diagnoses implies that only gross inconsistencies would stand out as evidently implausible data.

Availability of data and materials

Data are available from the senior author (JJ) upon reasonable request.

Abbreviations

RWC:

remaining work capacity

ICD:

International Classification of Diseases

ANOVA:

analysis of variance

ICF:

International Classification of Functioning, Disability and Health

APP:

activity and participation limitations in psychiatric diseases [‘Aktivitäts- und Partizipationsbeeinträchtigungen bei psychischen Erkrankungen’]

MICF:

Mini-ICF-APP

MICFmean :

mean rating in the Mini-ICF-APP

MICFtotal :

summed rating score in the Mini-ICF-APP

References

  1. State Secretariat for Economics Affairs SECO. Die Lage auf dem Arbeitsmarkt im Mai 2020. [The situation on the job market in May 2020]. 2020. Retrieved April, 13, 2021 from: https://www.seco.admin.ch/seco/de/home/Publikationen_Dienstleistungen/Publikationen_und_Formulare/Arbeit/Arbeitslosenversicherung/Die_Lage_auf_dem_Arbeitsmarkt/lage_arbeitsmarkt_2020/arbeitsmarkt_mai_2020.html.

  2. Federal Social Insurance Office Switzerland BSV. IV-Statistik 2019 [Disability insurance statistics 2019]. 2020. Retrieved April, 13, 2021 from: https://www.bsv.admin.ch/bsv/de/home/sozialversicherungen/iv/statistik.html

  3. Geiger BB, Garthwaite K, Warren J, Bambra C. Assessing work disability for social security benefits: international models for the direct assessment of work capacity. Disabil Rehabil. 2018;40(24):2962–70. https://doi.org/10.1080/09638288.2017.1366556.

    Article  PubMed  Google Scholar 

  4. Information Centre OASI/DI. Invalidenrenten der IV [Disability benefits of disability insurance]. 2021. Retrieved April, 13, 2021 from: https://www.ahv-iv.ch/p/4.04.d.

  5. Dickmann JRM, Broocks A. Das psychiatrische Gutachten im Rentenverfahren - wie reliabel? [the psychiatric assessment in the disability benefits process - how reliable?]. Fortschr Neurol Psychiatr. 2007;75(07):397–401. https://doi.org/10.1055/s-2006-944303.

    Article  CAS  PubMed  Google Scholar 

  6. Barth J, de Boer WEL, Busse JW, Hoving JL, Kedzia S, Couban R, et al. Inter-rater agreement in evaluation of disability: systematic review of reproducibility studies. BMJ. 2017;356:j14. https://doi.org/10.1136/bmj.j14.

    Article  PubMed  PubMed Central  Google Scholar 

  7. Kunz R, von Allmen DY, Marelli R, Hoffmann-Richter U, Jeger J, Mager R, et al. The reproducibility of psychiatric evaluations of work disability: two reliability and agreement studies. BMC Psychiatry. 2019;19(1):205. https://doi.org/10.1186/s12888-019-2171-y.

    Article  PubMed  PubMed Central  Google Scholar 

  8. Pizala H. Qualitative evaluation von psychiatrischen Gutachten unter besonderer Berücksichtigung des methodischen Vorgehens der Gutachter [qualitative evaluation of psychiatric assessments in consideration of the methodological rater's approach]. Inaugural dissertation. Basel: Faculty of Medicine, University of Basel 2010.

  9. World Health Organization WHO. International Classification of Functioning, Disability and Health. 2001. Retrieved April, 13, 2021 from: https://apps.who.int/iris/bitstream/handle/10665/42407/9241545429.pdf.

  10. Homa DB. Using the international classification of functioning, disability and health (ICF) in job placement. Work. 2007;29(4):277–86.

    PubMed  Google Scholar 

  11. Brage S, Donceel P, Falez F. Development of ICF core set for disability evaluation in social security. Disabil Rehabil. 2008;30(18):1392–6. https://doi.org/10.1080/09638280701642950.

    Article  PubMed  Google Scholar 

  12. Finger ME, Escorpizo R, Glässel A, Gmünder HP, Lückenkemper M, Chan C, et al. ICF Core set for vocational rehabilitation: results of an international consensus conference. Disabil Rehabil. 2012;34(5):429–38. https://doi.org/10.3109/09638288.2011.608145.

    Article  PubMed  Google Scholar 

  13. Sengers JH, Abma FI, Wilming L, Roelofs PDDM, Heerkens YF, Brouwer S. Content Validation of a Practice-Based Work Capacity Assessment Instrument Using ICF Core Sets. Journal of Occupational Rehabilitation. 2020.

  14. Linden M, Baron S, Muschalla B, Molodynski A. Mini-ICF-APP social functioning scale. Oxford: Hogrefe; 2014.

    Google Scholar 

  15. Linden M, Baron S, Muschalla B. Mini-ICF-APP : Mini-ICF-Rating für Aktivitäts- und Partizipationsstörungen bei psychischen Erkrankungen. Manual [Mini-ICF-APP: Mini-ICF-Rating for activity and participation in mental disorders. Manual.] Bern: Huber; 2009.

  16. Molodynski A, Linden M, Juckel G, Yeeles K, Anderson C, Vazquez-Montes M, et al. The reliability, validity, and applicability of an English language version of the Mini-ICF-APP. Soc Psychiatry Psychiatr Epidemiol. 2013;48(8):1347–54. https://doi.org/10.1007/s00127-012-0604-8.

    Article  PubMed  Google Scholar 

  17. Wciórka J, Anczewska M, Jahołkowski P, Świtaj P. Psychometric evaluation of the polish version of the MINI-ICF-APP – a concise measure of limitations on activity and restrictions on participation according to the international classification of functioning, disability and health (ICF) – in people with mental disorders. Postępy Psychiatrii i Neurologii. 2018;27(3):218–31. https://doi.org/10.5114/ppn.2018.78715.

    Article  Google Scholar 

  18. Fauchère W, Habicht M, Burri E, Jost E. La MINI-CIF-APP vous aide à évaluer l’impact des troubles psychiques sur la vie personnelle et professionnelle des personnes [MINI-ICF-APP: Assessing the impact of mental disorders on people's private and professional lives]: Hogrefe; 2020.

  19. Balestrieri M, Isola M, Bonn R, Tam T, Vio A, Linden M, et al. Validation of the Italian version of Mini-ICF-APP, a short instrument for rating activity and participation restrictions in psychiatric disorders. Epidemiol Psychiatr Sci. 2013;22(1):81–91. https://doi.org/10.1017/S2045796012000480.

    Article  CAS  PubMed  Google Scholar 

  20. Egger ST, Weniger G, Bobes J, Seifritz E, Vetter S. Exploring the factor structure of the mini-ICF-APP in an inpatient clinical sample, according to the psychiatric diagnosis. Rev Psiquiatr Salud Ment. 2020. https://doi.org/10.1016/j.rpsm.2020.05.008.

  21. Muschalla B. Different work capacity impairments in patients with different work-anxieties. Int Arch Occup Environ Health. 2016;89(4):609–19. https://doi.org/10.1007/s00420-015-1099-x.

    Article  PubMed  Google Scholar 

  22. Jeger J, Trezzini B, Schwegler U. Applying the ICF in disability evaluation: a report based on clinical experience. In: Escorpizo R, Brage S, Homa D, Stucki G, editors. Handbook of vocational rehabilitation and disability evaluation: application and implementation of the ICF. Cham: Springer International Publishing; 2015. p. 397–410. https://doi.org/10.1007/978-3-319-08825-9_18.

    Chapter  Google Scholar 

  23. Habermeyer B, Kaiser S, Kawohl W, Seifritz E. Rentenrelevante Arbeitsunfähigkeit und Mini-ICF-APP. [Assement of incapacity to work and the Mini-ICF-APP]. Neuropsychiatr. 2017;31(4):182–6. https://doi.org/10.1007/s40211-017-0246-x.

    Article  PubMed  Google Scholar 

  24. Blanca MJ, Alarcón R, Arnau J, Bono R, Bendayan R. Non-normal data: Is ANOVA still a valid option? Psicothema. 2017;29(4):552–7. https://doi.org/10.7334/psicothema2016.383.

    Article  PubMed  Google Scholar 

  25. Skodol AE, Gunderson JG, McGlashan TH, Dyck IR, Stout RL, Bender DS, et al. Functional impairment in patients with schizotypal, borderline, avoidant, or obsessive-compulsive personality disorder. Am J Psychiatr. 2002;159(2):276–83. https://doi.org/10.1176/appi.ajp.159.2.276.

    Article  PubMed  Google Scholar 

  26. Boland JK, Damnjanovic T, Anderson JL. Evaluating the role of functional impairment in personality psychopathology. Psychiatry Res. 2018;270:1017–26. https://doi.org/10.1016/j.psychres.2018.03.049.

    Article  PubMed  Google Scholar 

  27. Kashdan TB, Rottenberg J. Psychological flexibility as a fundamental aspect of health. Clin Psychol Rev. 2010;30(7):865–78. https://doi.org/10.1016/j.cpr.2010.03.001.

    Article  PubMed  PubMed Central  Google Scholar 

  28. Dunkel W, Weihrich M. Arbeit als Interaktion [work as interaction]. In: Böhle F, Voß GG, Wachtler G, editors. Handbuch Arbeitssoziologie: band 1: Arbeit, Strukturen und Prozesse [handbook of sociology of work: vol 1: work, structure and processes]. Wiesbaden: Springer Fachmedien Wiesbaden; 2018. p. 201–30. https://doi.org/10.1007/978-3-658-14458-6_6.

    Chapter  Google Scholar 

  29. Volkert J, Gablonski TC, Rabung S. Prevalence of personality disorders in the general adult population in Western countries: systematic review and meta-analysis. Br J Psychiatry. 2018;213(6):709–15. https://doi.org/10.1192/bjp.2018.202.

    Article  PubMed  Google Scholar 

  30. Bolliger C, Fritschi T, Zürcher P, Hümbelin O. Eingliederung veor Rente: evaluation der Früherfassung, der Frühintervention und der Integrationsmassnahmen in der Invalidenversicherung [return to work prior to disability benefits: evaluation of the early registration, early intervention and return to work programs in disability insurance]. Federal Social Insurance Office, Switzerland (BSV): Bern, 2012.

  31. Muschalla B, Poguntke KJ, Linden M. Assessment of capacity impairment in patients with mental disorders by routine clinical assessment and by structured assessment with the Mini-ICF-APP. Psychopathology. 2019;52(4):248–55. https://doi.org/10.1159/000502123.

    Article  PubMed  Google Scholar 

  32. Schandelmaier S, Leibold A, Fischer K, Mager R, Hoffmann-Richter U, Bachmann MS, et al. Attitudes towards evaluation of psychiatric disability claims: a survey of Swiss stakeholders. Swiss Med Wkly. 2015;145:w14160. https://doi.org/10.4414/smw.2015.14160.

    Article  PubMed  Google Scholar 

  33. Liebrenz M, Schleifer R, Canela C, Seifritz E, Ebner G, Jeger J. Abklärungstiefe im Fokus [clarification depth in focus]. InFo Neurologie Psychiatrie. 2014;12:1–6.

    Google Scholar 

  34. Schwegler U, Trezzini B, Schiffmann B. Current challenges in disability evaluation and the need for a goal-oriented approach based on the ICF: a qualitative stakeholder analysis in the context of the Swiss accident insurance. Disabil Rehabil. 2019:1–13.

  35. Kunze P. Begutachtungen mit dem Mini-ICF-APP in der Schweizer Invalidenversicherung (IV) – Zwei Fallbeispiele [Assessments with the Mini-ICF-APP in the Swiss Disability Insurance (IV) - two case studies]. Praxis Klinische Verhaltensmedizin Rehabil. 2019;107(3):232–6.

    Google Scholar 

  36. Bachmann M, de Boer W, Schandelmaier S, Leibold A, Marelli R, Jeger J, et al. Use of a structured functional evaluation process for independent medical evaluations of claimants presenting with disabling mental illness: rationale and design for a multi-center reliability study. BMC Psychiatry. 2016;16(1):271. https://doi.org/10.1186/s12888-016-0967-6.

    Article  PubMed  PubMed Central  Google Scholar 

  37. World Medical Association. Declaration of Helsinki – Ethical Principles for Medical Research Involving Human Subjects: World Medical Association 1964. Retrieved April, 13, 2021 from: https://www.wma.net/policies-post/wma-declaration-of-helsinki-ethical-principles-for-medical-research-involving-human-subjects/.

  38. Federal Council Switzerland. Humanforschungsgesetz [Swiss Human Research Act]. 2011. Retrieved April, 13, 2021 from: https://www.admin.ch/opc/de/classified-compilation/20061313/index.html.

Download references

Acknowledgements

The authors greatly appreciate the support of Brigitte Walter-Meyer in editing the manuscript.

Funding

The study received no funding.

Author information

Authors and Affiliations

  1. Department of Clinical Research, EbIM Research & Education, University of Basel Hospital, Basel, Switzerland

    Timm Rosburg & Regina Kunz

  2. Swiss Paraplegic Research, Nottwil, Switzerland

    Bruno Trezzini & Urban Schwegler

  3. Department of Health Sciences and Medicine, University of Lucerne, Lucerne, Switzerland

    Bruno Trezzini & Urban Schwegler

  4. MEDAS Zentralschweiz, Lucerne, Switzerland

    Jörg Jeger

Authors
  1. Timm Rosburg
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Regina Kunz
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Bruno Trezzini
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Urban Schwegler
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Jörg Jeger
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

TR drafted the manuscript, conducted all statistical analyses, and interpreted the data; JJ initiated the study, supervised the data collection, created and managed the data basis, contributed to the data interpretation and discussion; RK revised the manuscript and contributed to the data interpretation and discussion; BT co-initiated the study, contributed to the discussion, and provided critical feedback to the manuscript; US contributed to the discussion and provided critical feedback to the manuscript. All authors read and approved the final manuscript.

Corresponding author

Correspondence to Timm Rosburg.

Ethics declarations

Ethics approval and consent to participate

All reported data were collected for the purpose of work disability evaluations and retrospectively analyzed for study purposes. The Ethics Committee Northwest/Central Switzerland approved the study in July 2014, with an amendment October 2016 (“Utility of the Mini-ICF-APP for quality assurance in psychiatric disability evaluation”, EKNZ: 2014–122). The study was granted an exemption from requiring an informed consent of the claimants, as it was considered as impractical and disproportionate to obtain such consent from the claimants. Such an exemption is in accordance with §32 of the Declaration of Helsinki ([37] and national legal regulations (§34 Swiss Human Research Act, ‘Humanforschungsgesetz’, [38]).

Consent for publication

Not applicable.

Competing interests

RK declares a part-time employment with the Swiss National Accident Insurance Fund (Suva), a non-profit organisation. Suva did not have any role in the design, performance, data collection, and analysis of this work or interpretation of the data. All other authors declare that they have no competing interests.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary Information

Additional file 1 Table S1

Study sample. Table S2 Mean Mini-ICF-APP ratings across all participants. Table S3 Distribution of the Mini-ICF-APP ratings across the total sample. Table S4 Mini-ICF-APP ratings for the different psychiatric diagnoses. Table S5 Mini-ICF-APP ratings for claimants with depressive episodes. Table S6 Mini-ICF-APP ratings for the three remaining work capacity (RWC) levels.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Rosburg, T., Kunz, R., Trezzini, B. et al. The assessment of capacity limitations in psychiatric work disability evaluations by the social functioning scale Mini-ICF-APP. BMC Psychiatry 21, 480 (2021). https://doi.org/10.1186/s12888-021-03467-w

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1186/s12888-021-03467-w

Keywords

BMC Psychiatry

ISSN: 1471-244X

Contact us