Is the PANSS used correctly? a systematic review
BMC Psychiatry volume 11, Article number: 113 (2011)
The PANSS (Positive and Negative Syndrome Scale) is one of the most important rating instruments for patients with schizophrenia. Nevertheless, there is a long and ongoing debate in the psychiatric community regarding its mathematical properties.
All 30 items range from 1 to 7 leading to a minimum total score of 30, implying that the PANSS is an interval scale. For such interval scales straightforward calculation of relative changes is not appropriate. To calculate outcome criteria based on a percent change as, e.g., the widely accepted response criterion, the scale has to be transformed into a ratio scale beforehand. Recent publications have already pointed out the pitfall that ignoring the scale level (interval vs. ratio scale) leads to a set of mathematical problems, potentially resulting in erroneous results concerning the efficacy of the treatment.
A Pubmed search based on the PRISMA statement of the highest-ranked psychiatric journals (search terms "PANSS" and "response") was carried out. All articles containing percent changes were included and methods of percent change calculation were analysed.
This systematic literature research shows that the majority of authors (62%) actually appear to use incorrect calculations. In most instances the method of calculation was not described in the manuscript.
These alarming results underline the need for standardized procedures for PANSS calculations.
The PANSS is currently the most established scale in patients with schizophrenia. For example in the high impact journal "Schizophrenia Bulletin" Kay's publication on the Positive and Negative Syndrome Scale (PANSS) for Schizophrenia is the most frequently cited article with more than 4000 citations (pubmed 05/2011) . Despite its common use there still seems to be profound uncertainty within the psychiatric community regarding its mathematical properties. The pitfall relates to the calculation of proportions (including percent changes), which are used in common outcome criteria like response.
Dichotomized measures such as response can be understood more intuitively than mean values and are specifically endorsed by the European Medicines Agency http://www.ema.europa.eu/htms/human/ich/ichefficacy.htm.
As pointed out in a previous paper , the PANSS is a 30 item interval scale ranging from 1-7 which implies that computations of ratios (e.g. percent changes, like calculation of XX% PANSS reduction from baseline to final endpoint) are not appropriate. Ignoring this fact leads to severe mathematical problems, resulting in an underestimation of the actual response rate and potentially even to erroneous results. Comparing results with and without PANSS scale level transformation into a ratio scale revealed that up to 50% of test decisions may differ . In a comment on this article , Leucht et al. have cited such erroneous calculation methods as one reason for low response rates in studies on second generation antipsychotic drugs.
To avoid incorrect calculations the best solution would be to subtract the theoretical minimum (which is 30 for the total score), resulting in a score range starting from zero. Percent changes (PCs) have to be calculated using this corrected version of the PANSS, which converts the PANSS into a ratio scale. Although Leucht et al. [4, 5] have emphasized this necessity previously, the uncertainty in the psychiatric community remains.
In our previous report we already cited some articles performing the correction, as well as some others ignoring the pitfall. These examples also included approval studies of atypical antipsychotics, where a correct calculation would seem to be particularly important . However, the mentioned articles were neither representative, nor did they give any answer to the scope of the problem. So far, knowledge concerning the relative frequency of incorrectly calculated PANSS PCs has been limited. If papers with erroneous calculations turn out to be negligible in comparison to similar publications as a whole, then most researchers seem to be aware of this pitfall. If not, we need to open a wider debate on this issue, because results of studies using different methods for the calculation of PCs can, strictly speaking, not be compared.
Thus, the aim of this review article is to further investigate the scope of incorrect PANSS calculations based on a systematic review of all articles published in the top ten journals with the highest impact factors in psychiatry, with a focus on the question: Is the PANSS used correctly?
All articles in this review were found by a systematic literature search in the top-ranked psychiatric journals using Pubmed http://www.pubmed.com based on the PRISMA statement . The Impact Factor for psychiatric journals according to the 2008 Journal Citation Reports® Science Edition (Thomson Reuters, 2009) was used as ranking index. Journals focusing on topics not related to the PANSS and schizophrenia, such as Molecular Psychiatry or journals specialising in adolescent psychiatry, were excluded.
Based on these criteria, a predefined Pubmed search was carried out in the 10 highest-ranked journals entering the search terms "PANSS" and "response" with no restrictions regarding date of publication. The search term "response" was expected to be linked to the calculation of PCs in the PANSS.
Articles were included if they contained PCs in the PANSS in any form: Study inclusion criteria as well as outcome parameters were of interest, as well as continuous PCs and dichotomous response criteria. All articles containing PCs were included in this review and their methods of PC calculation were analyzed. The authors of articles with insufficient method descriptions were contacted (twice in case of no reply).
A classification was performed independently by two experienced researchers (MO and FS) into articles with PC as primary and those with PC as secondary outcome and into articles using PC as inclusion criteria. In case of disagreement a third researcher (SM) was consulted so that all articles could be satisfactorily classified.
Articles grouped according to their PC calculation method were sub-classified according to their year of publication, their outcome parameter and their particular citation number, using nonparametric, rank-based statistics and corresponding tests.
The ten highest-ranked psychiatric journals according to their impact factor 2008 included three journals, which did not fit our search criteria (MOL PSYCHIATR, J CHILD PSYCHOL PSYC and J AM ACAD CHILD PSY). These three journals were therefore replaced by the three subsequent journals on the impact list (PSYCHOL MED, J PSYCHIATR RES, J NEUROL NEUROSUR). The search in Pubmed in January 2011 resulted in 68 publications including both terms, "PANSS" and "response". Of all articles, 39 actually used PANSS PC values ([8–46]) and for 33 articles the method of calculation could finally be determined. Table 1 shows the main results in detail.
In summary, in at least 62% of all publications (24 out of 39) the PANSS PC was calculated without the necessary score correction. The PC calculation method was rarely specified within the text. It was possible only in seven articles, to deduce the calculation method without correspondence with the authors: In two articles with score correction an explanation of the method was included and in five articles without correction the calculation method could be identified through an examination of the presented results.
Most of the articles were from the past few years (median:2007, range:1995-2010), without any noticeable difference (p = 0.23) between articles with (median:2008, range:1995-2010) and without score correction (median:2006.5, range:1998-2010). The number of citations ranged from 0 to 447 with a median of 18. As with the year of publication, there was no significant difference (p = 0.94) regarding the number of citations in the two groups. There is a significant negative rank correlation of -0.70 between citation number and publication year (p < 0.001).
Regarding the outcome classification of the articles, 33 of the 39 articles could be classified concordantly by researchers MO and FS, and in six cases a third researcher (SM) was consulted for the final decision. In twelve of the 39 publications the primary outcome was based on PC; in five (42%) of these corrected score values were used, five (42%) used uncorrected scores, and in two (17%) the method remained unclear. The majority of the articles found presented PCs as secondary outcomes:4 (15%) with correction, 19 (70%) without, and 4 (15%) articles with unknown status. There was no significant difference between outcome classification and method (p = 0.09).
The influence of the PC calculation method on the results of double blind placebo controlled trials has already been described and quantified in detail in our previous article . There are two main issues, which need to be considered: (1) Results of studies without correction cannot be compared to studies with correction. A 50% response criterion, for example, denotes two different facts: With corrected scores it corresponds to a 50% reduction of the measured symptoms, whereas without correction it corresponds to a 50% reduction of the score value, which is something very different. (2) Results are not only incomparable, but could even lead to different conclusions: While one method might reveal a significant treatment effect, the other might lead to the opposite result . In articles with PC as primary outcome this is particularly problematic, since without correction even the main conclusion might be erroneous. A special issue in this context are approval studies, which are obliged to follow guidelines like the EMEA guidelines and therefore regularly include outcome measures with PCs. For one approval study  an erroneous calculation of the PANSS PC has already been shown .
In combination with the results of the present review it becomes even more apparent that there is a strong need for clarification in terms of the PANSS calculation: Although some authors use corrected scores, in the majority of cases the correction is not performed. Most importantly, the non-awareness of this problem is mirrored by the fact that only in two articles the score correction was described in the Methods section. This suggests that most researchers conducting schizophrenia trials are not even aware of this pitfall. Considering the fact that we probably did not identify all relevant articles in our literature search by focussing on the searching term of "response" one could assume that there are even more publications with incorrect PANSS calculations.
This is even more remarkable keeping in mind that the papers reviewed were published in high impact journals. So we can answer the question posed at the beginning of this article: Yes, the PANSS is used incorrectly!
What solutions can be made? First of all, it would be helpful to recalculate studies which have used the PANSS PC as primary outcome without correction.
For future work with the PANSS a consensus in the psychiatric research field is needed: Is it enough to correct the score every time PCs are used or should the PANSS be rescaled? Leucht et al., in their comment on our previous paper, prefer the radical solution: The PANSS items should be rescaled into a scale ranging from 0 to 6. This would be the most straightforward solution and could avoid future problems with PCs. Additionally, renaming the scale as e.g. "PANSS-0" or "PANSS (ratio version)", as suggested previously, could prevent new confusion, which might otherwise arise with different scale versions.
Again, we emphasize the necessity of further discussion and a broad consensus on future action in the psychiatric community. Until this is achieved we recommend that, for PANSS PC calculations, all researchers at least use the scale correction and include a short statement in the description of methods.
Kay SR, Fiszbein A, Opler LA: The positive and negative syndrome scale (PANSS) for schizophrenia. Schizophr Bull. 1987, 13: 261-276.
Obermeier M, Mayr A, Schennach-Wolff R, Seemuller F, Moller HJ, Riedel M: Should the PANSS Be Rescaled?. Schizophr Bull. 2010, 36: 455-460. 10.1093/schbul/sbp124.
Leucht S, Kissling W, Davis JM: The PANSS Should Be Rescaled. Schizophr Bull. 2010, 36: 461-462. 10.1093/schbul/sbq016.
Leucht S, Davis JM, Engel RR, Kane JM, Wagenpfeil S: Defining 'response' in antipsychotic drug trials: recommendations for the use of scale-derived cutoffs. Neuropsychopharmacology. 2007, 32: 1903-1910. 10.1038/sj.npp.1301325.
Leucht S, Davis JM, Engel RR, Kissling W, Kane JM: Definitions of response and remission in schizophrenia: recommendations for their use and their presentation. Acta Psychiatr Scand Suppl. 2009, 7-14.
FDA: Application No.:021253. FDA. FDA. 2004, Ref Type: Electronic Citation, [http://www.accessdata.fda.gov/drugsatfda_docs/nda/2004/21253_Zyprexa.TOC.cfm]
Moher D, Liberati A, Tetzlaff J, Altman DG: Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement. BMJ. 2009, 339: b2535-10.1136/bmj.b2535.
Heresco-Levy U, Javitt DC, Ermilov M, Mordel C, Silipo G, Lichtenstein M: Efficacy of high-dose glycine in the treatment of enduring negative symptoms of schizophrenia. Arch Gen Psychiatry. 1999, 56: 29-36. 10.1001/archpsyc.56.1.29.
Breier A, Meehan K, Birkett M, David S, Ferchland I, Sutton V, et al: A double-blind, placebo-controlled dose-response comparison of intramuscular olanzapine and haloperidol in the treatment of acute agitation in schizophrenia. Arch Gen Psychiatry. 2002, 59: 441-448. 10.1001/archpsyc.59.5.441.
Emsley R, Rabinowitz J, Medori R: Time course for antipsychotic treatment response in first-episode schizophrenia. Am J Psychiatry. 2006, 163: 743-745. 10.1176/appi.ajp.163.4.743.
Sikich L, Frazier JA, McClellan J, Findling RL, Vitiello B, Ritz L, et al: Double-blind comparison of first- and second-generation antipsychotics in early-onset schizophrenia and schizo-affective disorder: findings from the treatment of early-onset schizophrenia spectrum disorders (TEOSS) study. Am J Psychiatry. 2008, 165: 1420-1431. 10.1176/appi.ajp.2008.08050756.
Marder SR, Kramer M, Ford L, Eerdekens E, Lim P, Eerdekens M, et al: Efficacy and safety of paliperidone extended-release tablets: results of a 6-week, randomized, placebo-controlled study. Biol Psychiatry. 2007, 62: 1363-1370. 10.1016/j.biopsych.2007.01.017.
Campbell DB, Ebert PJ, Skelly T, Stroup TS, Lieberman J, Levitt P, et al: Ethnic stratification of the association of RGS4 variants with antipsychotic treatment response in schizophrenia. Biol Psychiatry. 2008, 63: 32-41. 10.1016/j.biopsych.2007.04.018.
Kegeles LS, Slifstein M, Frankle WG, Xu X, Hackett E, Bae SA, et al: Dose-occupancy study of striatal and extrastriatal dopamine D2 receptors by aripiprazole in schizophrenia with PET and [18F]fallypride. Neuropsychopharmacology. 2008, 33: 3111-3125. 10.1038/npp.2008.33.
Sechter D, Peuskens J, Fleurot O, Rein W, Lecrubier Y: Amisulpride vs. risperidone in chronic schizophrenia: results of a 6-month double-blind study. Neuropsychopharmacology. 2002, 27: 1071-1081. 10.1016/S0893-133X(02)00375-5.
Kinon BJ, Chen L, scher-Svanum H, Stauffer VL, Kollack-Walker S, Zhou W, et al: Early response to antipsychotic drug therapy as a clinical marker of subsequent response in the treatment of schizophrenia. Neuropsychopharmacology. 2010, 35: 581-590. 10.1038/npp.2009.164.
Levine SZ, Rabinowitz J: Trajectories and Antecedents of Treatment Response Over Time in Early-Episode Psychosis. Schizophr Bull. 2008
Haas M, Eerdekens M, Kushner S, Singer J, Augustyns I, Quiroz J, et al: Efficacy, safety and tolerability of two dosing regimens in adolescent schizophrenia: double-blind study. Br J Psychiatry. 2009, 194: 158-164. 10.1192/bjp.bp.107.046177.
Peuskens J: Risperidone in the treatment of patients with chronic schizophrenia: a multi-national, multi-centre, double-blind, parallel-group study versus haloperidol. Risperidone Study Group. Br J Psychiatry. 1995, 166: 712-726. 10.1192/bjp.166.6.712.
Schennach-Wolff R, Seemuller FH, Mayr A, Maier W, Klingberg S, Heuser I, et al: An early improvement threshold to predict response and remission in first-episode schizophrenia. Br J Psychiatry. 2010, 196: 460-466. 10.1192/bjp.bp.109.069328.
Fleischhacker WW, Eerdekens M, Karcher K, Remington G, Llorca PM, Chrzanowski W, et al: Treatment of schizophrenia with long-acting injectable risperidone: a 12-month open-label trial of the first long-acting second-generation antipsychotic. J Clin Psychiatry. 2003, 64: 1250-1257. 10.4088/JCP.v64n1017.
Volpi S, Potkin SG, Malhotra AK, Licamele L, Lavedan C: Applicability of a genetic signature for enhanced iloperidone efficacy in the treatment of schizophrenia. J Clin Psychiatry. 2009, 70: 801-809. 10.4088/JCP.08m04391.
Canuso CM, Lindenmayer JP, Kosik-Gonzalez C, Turkoz I, Carothers J, Bossie CA, et al: A randomized, double-blind, placebo-controlled study of 2 dose ranges of paliperidone extended-release in the treatment of subjects with schizoaffective disorder. J Clin Psychiatry. 2010, 71: 587-598. 10.4088/JCP.09m05564yel.
Meltzer HY, Bobo WV, Nuamah IF, Lane R, Hough D, Kramer M, et al: Efficacy and tolerability of oral paliperidone extended-release tablets in the treatment of acute schizophrenia: pooled data from three 6-week, placebo-controlled studies. J Clin Psychiatry. 2008, 69: 817-829. 10.4088/JCP.v69n0515.
Glick ID, Poyurovsky M, Ivanova O, Koran LM: Aripiprazole in schizophrenia patients with comorbid obsessive-compulsive symptoms: an open-label study of 15 patients. J Clin Psychiatry. 2008, 69: 1856-1859. 10.4088/JCP.v69n1202.
Kahn RS, Schulz SC, Palazov VD, Reyes EB, Brecher M, Svensson O, et al: Efficacy and tolerability of once-daily extended release quetiapine fumarate in acute schizophrenia: a randomized, double-blind, placebo-controlled study. J Clin Psychiatry. 2007, 68: 832-842. 10.4088/JCP.v68n0603.
Chan HY, Lin WW, Lin SK, Hwang TJ, Su TP, Chiang SC, et al: Efficacy and safety of aripiprazole in the acute treatment of schizophrenia in Chinese patients with risperidone as an active control: a randomized trial. J Clin Psychiatry. 2007, 68: 29-36. 10.4088/JCP.v68n0104.
Zhong KX, Sweitzer DE, Hamer RM, Lieberman JA: Comparison of quetiapine and risperidone in the treatment of schizophrenia: A randomized, double-blind, flexible-dose, 8-week study. J Clin Psychiatry. 2006, 67: 1093-1103. 10.4088/JCP.v67n0712.
nil Yagcioglu AE, Kivircik Akdede BB, Turgut TI, Tumuklu M, Yazici MK, Alptekin K, et al: A double-blind controlled study of adjunctive treatment with risperidone in schizophrenic patients partially responsive to clozapine: efficacy and safety. J Clin Psychiatry. 2005, 66: 63-72. 10.4088/JCP.v66n0109.
Lane HY, Chang YC, Cheng YC, Liu GC, Lin XR, Chang WH: Effects of patient demographics, risperidone dosage, and clinical outcome on body weight in acutely exacerbated schizophrenia. J Clin Psychiatry. 2003, 64: 316-320. 10.4088/JCP.v64n0314.
Lindenmayer JP, Czobor P, Volavka J, Lieberman JA, Citrome L, Sheitman B, et al: Olanzapine in refractory schizophrenia after failure of typical or atypical antipsychotic treatment: an open-label switch study. J Clin Psychiatry. 2002, 63: 931-935. 10.4088/JCP.v63n1011.
Emsley RA, Roberts MC, Rataemane S, Pretorius J, Oosthuizen PP, Turner J, et al: Ethnicity and treatment response in schizophrenia: a comparison of 3 ethnic groups. J Clin Psychiatry. 2002, 63: 9-14. 10.4088/JCP.v63n0103.
Dossenbach MR, Kratky P, Schneidman M, Grundy SL, Metcalfe S, Tollefson GD, et al: Evidence for the effectiveness of olanzapine among patients nonresponsive and/or intolerant to risperidone. J Clin Psychiatry. 2001, 62 (Suppl 2): 28-34.
Lindenmayer JP, Iskander A, Park M, Apergi FS, Czobor P, Smith R, et al: Clinical and neurocognitive effects of clozapine and risperidone in treatment-refractory schizophrenic patients: a prospective study. J Clin Psychiatry. 1998, 59: 521-527. 10.4088/JCP.v59n1005.
Koenigsberg HW, Reynolds D, Goodman M, New AS, Mitropoulou V, Trestman RL, et al: Risperidone in the treatment of schizotypal personality disorder. J Clin Psychiatry. 2003, 64: 628-634. 10.4088/JCP.v64n0602.
Case M, Stauffer VL, scher-Svanum H, Conley R, Kapur S, Kane JM, et al: The heterogeneity of antipsychotic response in the treatment of schizophrenia. Psychol Med. 2010, 1-10.
Mizrahi R, Korostil M, Starkstein SE, Zipursky RB, Kapur S: The effect of antipsychotic treatment on Theory of Mind. Psychol Med. 2007, 37: 595-601. 10.1017/S0033291706009342.
Copolov DL, Link CG, Kowalcyk B: A multicentre, double-blind, randomized comparison of quetiapine (ICI 204,636, 'Seroquel') and haloperidol in schizophrenia. Psychol Med. 2000, 30: 95-105. 10.1017/S0033291799001476.
Musil R, Spellmann I, Riedel M, Dehning S, Douhet A, Maino K, et al: SNAP-25 gene polymorphisms and weight gain in schizophrenic patients. J Psychiatr Res. 2008, 42: 963-970. 10.1016/j.jpsychires.2007.11.003.
Lipkovich I, Deberdt W, Csernansky JG, Buckley P, Peuskens J, Kollack-Walker S, et al: Predictors of risk for relapse in patients with schizophrenia or schizoaffective disorder during olanzapine drug therapy. J Psychiatr Res. 2007, 41: 305-310. 10.1016/j.jpsychires.2006.07.016.
Bozina N, Kuzman MR, Medved V, Jovanovic N, Sertic J, Hotujac L: Associations between MDR1 gene polymorphisms and schizophrenia and therapeutic response to olanzapine in female schizophrenic patients. J Psychiatr Res. 2008, 42: 89-97. 10.1016/j.jpsychires.2006.10.002.
Prikryl R, Ceskova E, Kasparek T, Kucerova H: Neurological soft signs, clinical symptoms and treatment reactivity in patients suffering from first episode schizophrenia. J Psychiatr Res. 2006, 40: 141-146. 10.1016/j.jpsychires.2005.03.008.
Peet M, Horrobin DF: A dose-ranging exploratory study of the effects of ethyl-eicosapentaenoate in patients with persistent schizophrenic symptoms. J Psychiatr Res. 2002, 36: 7-18. 10.1016/S0022-3956(01)00048-6.
Girgis RR, Merrill DB, Vorel SR, Edward K, Kimberly P, You M, et al: Aripiprazole versus haloperidol treatment in early-stage schizophrenia. J Psychiatr Res. 2010
Giegling I, Drago A, Schafer M, Moller HJ, Rujescu D, Serretti A: Interaction of haloperidol plasma level and antipsychotic effect in early phases of acute psychosis treatment. J Psychiatr Res. 2010, 44: 487-492. 10.1016/j.jpsychires.2009.11.004.
McEvoy JP, Daniel DG, Carson WH, McQuade RD, Marcus RN: A randomized, double-blind, placebo-controlled, study of the efficacy and safety of aripiprazole 10, 15 or 20 mg/day for the treatment of patients with acute exacerbations of schizophrenia. J Psychiatr Res. 2007, 41: 895-905. 10.1016/j.jpsychires.2007.05.002.
The pre-publication history for this paper can be accessed here:http://www.biomedcentral.com/1471-244X/11/113/prepub
We would like to thank T. Coutts for the linguistic revision of the manuscript.
The authors declare that they have no competing interests.
MO performed the analyses of the found articles and elaborated the conception of the manuscript, including a first draft. RS-W participated in the conception of the analysis and revised the manuscript critically. SM reviewed the included articles and assisted in the sequence alignment. H-JM, MR and DK revised the manuscript critically at each step of the analysis. FS reviewed the found articles and revised the manuscript critically. All authors read and approved the final manuscript.
Authors’ original submitted files for images
Below are the links to the authors’ original submitted files for images.
About this article
Cite this article
Obermeier, M., Schennach-Wolff, R., Meyer, S. et al. Is the PANSS used correctly? a systematic review. BMC Psychiatry 11, 113 (2011). https://doi.org/10.1186/1471-244X-11-113