This article has Open Peer Review reports available.
Is the PANSS used correctly? a systematic review
© Obermeier et al; licensee BioMed Central Ltd. 2011
Received: 18 March 2011
Accepted: 18 July 2011
Published: 18 July 2011
The PANSS (Positive and Negative Syndrome Scale) is one of the most important rating instruments for patients with schizophrenia. Nevertheless, there is a long and ongoing debate in the psychiatric community regarding its mathematical properties.
All 30 items range from 1 to 7 leading to a minimum total score of 30, implying that the PANSS is an interval scale. For such interval scales straightforward calculation of relative changes is not appropriate. To calculate outcome criteria based on a percent change as, e.g., the widely accepted response criterion, the scale has to be transformed into a ratio scale beforehand. Recent publications have already pointed out the pitfall that ignoring the scale level (interval vs. ratio scale) leads to a set of mathematical problems, potentially resulting in erroneous results concerning the efficacy of the treatment.
A Pubmed search based on the PRISMA statement of the highest-ranked psychiatric journals (search terms "PANSS" and "response") was carried out. All articles containing percent changes were included and methods of percent change calculation were analysed.
This systematic literature research shows that the majority of authors (62%) actually appear to use incorrect calculations. In most instances the method of calculation was not described in the manuscript.
These alarming results underline the need for standardized procedures for PANSS calculations.
The PANSS is currently the most established scale in patients with schizophrenia. For example in the high impact journal "Schizophrenia Bulletin" Kay's publication on the Positive and Negative Syndrome Scale (PANSS) for Schizophrenia is the most frequently cited article with more than 4000 citations (pubmed 05/2011) . Despite its common use there still seems to be profound uncertainty within the psychiatric community regarding its mathematical properties. The pitfall relates to the calculation of proportions (including percent changes), which are used in common outcome criteria like response.
Dichotomized measures such as response can be understood more intuitively than mean values and are specifically endorsed by the European Medicines Agency http://www.ema.europa.eu/htms/human/ich/ichefficacy.htm.
As pointed out in a previous paper , the PANSS is a 30 item interval scale ranging from 1-7 which implies that computations of ratios (e.g. percent changes, like calculation of XX% PANSS reduction from baseline to final endpoint) are not appropriate. Ignoring this fact leads to severe mathematical problems, resulting in an underestimation of the actual response rate and potentially even to erroneous results. Comparing results with and without PANSS scale level transformation into a ratio scale revealed that up to 50% of test decisions may differ . In a comment on this article , Leucht et al. have cited such erroneous calculation methods as one reason for low response rates in studies on second generation antipsychotic drugs.
To avoid incorrect calculations the best solution would be to subtract the theoretical minimum (which is 30 for the total score), resulting in a score range starting from zero. Percent changes (PCs) have to be calculated using this corrected version of the PANSS, which converts the PANSS into a ratio scale. Although Leucht et al. [4, 5] have emphasized this necessity previously, the uncertainty in the psychiatric community remains.
In our previous report we already cited some articles performing the correction, as well as some others ignoring the pitfall. These examples also included approval studies of atypical antipsychotics, where a correct calculation would seem to be particularly important . However, the mentioned articles were neither representative, nor did they give any answer to the scope of the problem. So far, knowledge concerning the relative frequency of incorrectly calculated PANSS PCs has been limited. If papers with erroneous calculations turn out to be negligible in comparison to similar publications as a whole, then most researchers seem to be aware of this pitfall. If not, we need to open a wider debate on this issue, because results of studies using different methods for the calculation of PCs can, strictly speaking, not be compared.
Thus, the aim of this review article is to further investigate the scope of incorrect PANSS calculations based on a systematic review of all articles published in the top ten journals with the highest impact factors in psychiatry, with a focus on the question: Is the PANSS used correctly?
All articles in this review were found by a systematic literature search in the top-ranked psychiatric journals using Pubmed http://www.pubmed.com based on the PRISMA statement . The Impact Factor for psychiatric journals according to the 2008 Journal Citation Reports® Science Edition (Thomson Reuters, 2009) was used as ranking index. Journals focusing on topics not related to the PANSS and schizophrenia, such as Molecular Psychiatry or journals specialising in adolescent psychiatry, were excluded.
Based on these criteria, a predefined Pubmed search was carried out in the 10 highest-ranked journals entering the search terms "PANSS" and "response" with no restrictions regarding date of publication. The search term "response" was expected to be linked to the calculation of PCs in the PANSS.
Articles were included if they contained PCs in the PANSS in any form: Study inclusion criteria as well as outcome parameters were of interest, as well as continuous PCs and dichotomous response criteria. All articles containing PCs were included in this review and their methods of PC calculation were analyzed. The authors of articles with insufficient method descriptions were contacted (twice in case of no reply).
A classification was performed independently by two experienced researchers (MO and FS) into articles with PC as primary and those with PC as secondary outcome and into articles using PC as inclusion criteria. In case of disagreement a third researcher (SM) was consulted so that all articles could be satisfactorily classified.
Articles grouped according to their PC calculation method were sub-classified according to their year of publication, their outcome parameter and their particular citation number, using nonparametric, rank-based statistics and corresponding tests.
Summary of calculation methods in single journals
Abbreviated Journal Title
(Impact Factor 2008)
No. of articles with correction
No. of articles
No. of articles, unknown method
ARCH GEN PSYCHIAT (14.273)
AM J PSYCHIAT (10.545)
BIOL PSYCHIAT (8.672)
SCHIZOPHRENIA BULL (6.592)
BRIT J PSYCHIAT (5.077)
J CLIN PSYCHIAT (5.053)
PSYCHOL MED (4.718)
J PSYCHIATR RES (4.679)
J NEUROL NEUROSUR PS (4.622)
In summary, in at least 62% of all publications (24 out of 39) the PANSS PC was calculated without the necessary score correction. The PC calculation method was rarely specified within the text. It was possible only in seven articles, to deduce the calculation method without correspondence with the authors: In two articles with score correction an explanation of the method was included and in five articles without correction the calculation method could be identified through an examination of the presented results.
Most of the articles were from the past few years (median:2007, range:1995-2010), without any noticeable difference (p = 0.23) between articles with (median:2008, range:1995-2010) and without score correction (median:2006.5, range:1998-2010). The number of citations ranged from 0 to 447 with a median of 18. As with the year of publication, there was no significant difference (p = 0.94) regarding the number of citations in the two groups. There is a significant negative rank correlation of -0.70 between citation number and publication year (p < 0.001).
Regarding the outcome classification of the articles, 33 of the 39 articles could be classified concordantly by researchers MO and FS, and in six cases a third researcher (SM) was consulted for the final decision. In twelve of the 39 publications the primary outcome was based on PC; in five (42%) of these corrected score values were used, five (42%) used uncorrected scores, and in two (17%) the method remained unclear. The majority of the articles found presented PCs as secondary outcomes:4 (15%) with correction, 19 (70%) without, and 4 (15%) articles with unknown status. There was no significant difference between outcome classification and method (p = 0.09).
The influence of the PC calculation method on the results of double blind placebo controlled trials has already been described and quantified in detail in our previous article . There are two main issues, which need to be considered: (1) Results of studies without correction cannot be compared to studies with correction. A 50% response criterion, for example, denotes two different facts: With corrected scores it corresponds to a 50% reduction of the measured symptoms, whereas without correction it corresponds to a 50% reduction of the score value, which is something very different. (2) Results are not only incomparable, but could even lead to different conclusions: While one method might reveal a significant treatment effect, the other might lead to the opposite result . In articles with PC as primary outcome this is particularly problematic, since without correction even the main conclusion might be erroneous. A special issue in this context are approval studies, which are obliged to follow guidelines like the EMEA guidelines and therefore regularly include outcome measures with PCs. For one approval study  an erroneous calculation of the PANSS PC has already been shown .
In combination with the results of the present review it becomes even more apparent that there is a strong need for clarification in terms of the PANSS calculation: Although some authors use corrected scores, in the majority of cases the correction is not performed. Most importantly, the non-awareness of this problem is mirrored by the fact that only in two articles the score correction was described in the Methods section. This suggests that most researchers conducting schizophrenia trials are not even aware of this pitfall. Considering the fact that we probably did not identify all relevant articles in our literature search by focussing on the searching term of "response" one could assume that there are even more publications with incorrect PANSS calculations.
This is even more remarkable keeping in mind that the papers reviewed were published in high impact journals. So we can answer the question posed at the beginning of this article: Yes, the PANSS is used incorrectly!
What solutions can be made? First of all, it would be helpful to recalculate studies which have used the PANSS PC as primary outcome without correction.
For future work with the PANSS a consensus in the psychiatric research field is needed: Is it enough to correct the score every time PCs are used or should the PANSS be rescaled? Leucht et al., in their comment on our previous paper, prefer the radical solution: The PANSS items should be rescaled into a scale ranging from 0 to 6. This would be the most straightforward solution and could avoid future problems with PCs. Additionally, renaming the scale as e.g. "PANSS-0" or "PANSS (ratio version)", as suggested previously, could prevent new confusion, which might otherwise arise with different scale versions.
Again, we emphasize the necessity of further discussion and a broad consensus on future action in the psychiatric community. Until this is achieved we recommend that, for PANSS PC calculations, all researchers at least use the scale correction and include a short statement in the description of methods.
We would like to thank T. Coutts for the linguistic revision of the manuscript.
- Kay SR, Fiszbein A, Opler LA: The positive and negative syndrome scale (PANSS) for schizophrenia. Schizophr Bull. 1987, 13: 261-276.View ArticlePubMedGoogle Scholar
- Obermeier M, Mayr A, Schennach-Wolff R, Seemuller F, Moller HJ, Riedel M: Should the PANSS Be Rescaled?. Schizophr Bull. 2010, 36: 455-460. 10.1093/schbul/sbp124.View ArticlePubMedGoogle Scholar
- Leucht S, Kissling W, Davis JM: The PANSS Should Be Rescaled. Schizophr Bull. 2010, 36: 461-462. 10.1093/schbul/sbq016.View ArticlePubMedPubMed CentralGoogle Scholar
- Leucht S, Davis JM, Engel RR, Kane JM, Wagenpfeil S: Defining 'response' in antipsychotic drug trials: recommendations for the use of scale-derived cutoffs. Neuropsychopharmacology. 2007, 32: 1903-1910. 10.1038/sj.npp.1301325.View ArticlePubMedGoogle Scholar
- Leucht S, Davis JM, Engel RR, Kissling W, Kane JM: Definitions of response and remission in schizophrenia: recommendations for their use and their presentation. Acta Psychiatr Scand Suppl. 2009, 7-14.Google Scholar
- FDA: Application No.:021253. FDA. FDA. 2004, Ref Type: Electronic Citation, [http://www.accessdata.fda.gov/drugsatfda_docs/nda/2004/21253_Zyprexa.TOC.cfm]Google Scholar
- Moher D, Liberati A, Tetzlaff J, Altman DG: Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement. BMJ. 2009, 339: b2535-10.1136/bmj.b2535.View ArticlePubMedPubMed CentralGoogle Scholar
- Heresco-Levy U, Javitt DC, Ermilov M, Mordel C, Silipo G, Lichtenstein M: Efficacy of high-dose glycine in the treatment of enduring negative symptoms of schizophrenia. Arch Gen Psychiatry. 1999, 56: 29-36. 10.1001/archpsyc.56.1.29.View ArticlePubMedGoogle Scholar
- Breier A, Meehan K, Birkett M, David S, Ferchland I, Sutton V, et al: A double-blind, placebo-controlled dose-response comparison of intramuscular olanzapine and haloperidol in the treatment of acute agitation in schizophrenia. Arch Gen Psychiatry. 2002, 59: 441-448. 10.1001/archpsyc.59.5.441.View ArticlePubMedGoogle Scholar
- Emsley R, Rabinowitz J, Medori R: Time course for antipsychotic treatment response in first-episode schizophrenia. Am J Psychiatry. 2006, 163: 743-745. 10.1176/appi.ajp.163.4.743.View ArticlePubMedGoogle Scholar
- Sikich L, Frazier JA, McClellan J, Findling RL, Vitiello B, Ritz L, et al: Double-blind comparison of first- and second-generation antipsychotics in early-onset schizophrenia and schizo-affective disorder: findings from the treatment of early-onset schizophrenia spectrum disorders (TEOSS) study. Am J Psychiatry. 2008, 165: 1420-1431. 10.1176/appi.ajp.2008.08050756.View ArticlePubMedGoogle Scholar
- Marder SR, Kramer M, Ford L, Eerdekens E, Lim P, Eerdekens M, et al: Efficacy and safety of paliperidone extended-release tablets: results of a 6-week, randomized, placebo-controlled study. Biol Psychiatry. 2007, 62: 1363-1370. 10.1016/j.biopsych.2007.01.017.View ArticlePubMedGoogle Scholar
- Campbell DB, Ebert PJ, Skelly T, Stroup TS, Lieberman J, Levitt P, et al: Ethnic stratification of the association of RGS4 variants with antipsychotic treatment response in schizophrenia. Biol Psychiatry. 2008, 63: 32-41. 10.1016/j.biopsych.2007.04.018.View ArticlePubMedGoogle Scholar
- Kegeles LS, Slifstein M, Frankle WG, Xu X, Hackett E, Bae SA, et al: Dose-occupancy study of striatal and extrastriatal dopamine D2 receptors by aripiprazole in schizophrenia with PET and [18F]fallypride. Neuropsychopharmacology. 2008, 33: 3111-3125. 10.1038/npp.2008.33.View ArticlePubMedGoogle Scholar
- Sechter D, Peuskens J, Fleurot O, Rein W, Lecrubier Y: Amisulpride vs. risperidone in chronic schizophrenia: results of a 6-month double-blind study. Neuropsychopharmacology. 2002, 27: 1071-1081. 10.1016/S0893-133X(02)00375-5.View ArticlePubMedGoogle Scholar
- Kinon BJ, Chen L, scher-Svanum H, Stauffer VL, Kollack-Walker S, Zhou W, et al: Early response to antipsychotic drug therapy as a clinical marker of subsequent response in the treatment of schizophrenia. Neuropsychopharmacology. 2010, 35: 581-590. 10.1038/npp.2009.164.View ArticlePubMedGoogle Scholar
- Levine SZ, Rabinowitz J: Trajectories and Antecedents of Treatment Response Over Time in Early-Episode Psychosis. Schizophr Bull. 2008Google Scholar
- Haas M, Eerdekens M, Kushner S, Singer J, Augustyns I, Quiroz J, et al: Efficacy, safety and tolerability of two dosing regimens in adolescent schizophrenia: double-blind study. Br J Psychiatry. 2009, 194: 158-164. 10.1192/bjp.bp.107.046177.View ArticlePubMedGoogle Scholar
- Peuskens J: Risperidone in the treatment of patients with chronic schizophrenia: a multi-national, multi-centre, double-blind, parallel-group study versus haloperidol. Risperidone Study Group. Br J Psychiatry. 1995, 166: 712-726. 10.1192/bjp.166.6.712.View ArticlePubMedGoogle Scholar
- Schennach-Wolff R, Seemuller FH, Mayr A, Maier W, Klingberg S, Heuser I, et al: An early improvement threshold to predict response and remission in first-episode schizophrenia. Br J Psychiatry. 2010, 196: 460-466. 10.1192/bjp.bp.109.069328.View ArticlePubMedGoogle Scholar
- Fleischhacker WW, Eerdekens M, Karcher K, Remington G, Llorca PM, Chrzanowski W, et al: Treatment of schizophrenia with long-acting injectable risperidone: a 12-month open-label trial of the first long-acting second-generation antipsychotic. J Clin Psychiatry. 2003, 64: 1250-1257. 10.4088/JCP.v64n1017.View ArticlePubMedGoogle Scholar
- Volpi S, Potkin SG, Malhotra AK, Licamele L, Lavedan C: Applicability of a genetic signature for enhanced iloperidone efficacy in the treatment of schizophrenia. J Clin Psychiatry. 2009, 70: 801-809. 10.4088/JCP.08m04391.View ArticlePubMedGoogle Scholar
- Canuso CM, Lindenmayer JP, Kosik-Gonzalez C, Turkoz I, Carothers J, Bossie CA, et al: A randomized, double-blind, placebo-controlled study of 2 dose ranges of paliperidone extended-release in the treatment of subjects with schizoaffective disorder. J Clin Psychiatry. 2010, 71: 587-598. 10.4088/JCP.09m05564yel.View ArticlePubMedGoogle Scholar
- Meltzer HY, Bobo WV, Nuamah IF, Lane R, Hough D, Kramer M, et al: Efficacy and tolerability of oral paliperidone extended-release tablets in the treatment of acute schizophrenia: pooled data from three 6-week, placebo-controlled studies. J Clin Psychiatry. 2008, 69: 817-829. 10.4088/JCP.v69n0515.View ArticlePubMedGoogle Scholar
- Glick ID, Poyurovsky M, Ivanova O, Koran LM: Aripiprazole in schizophrenia patients with comorbid obsessive-compulsive symptoms: an open-label study of 15 patients. J Clin Psychiatry. 2008, 69: 1856-1859. 10.4088/JCP.v69n1202.View ArticlePubMedGoogle Scholar
- Kahn RS, Schulz SC, Palazov VD, Reyes EB, Brecher M, Svensson O, et al: Efficacy and tolerability of once-daily extended release quetiapine fumarate in acute schizophrenia: a randomized, double-blind, placebo-controlled study. J Clin Psychiatry. 2007, 68: 832-842. 10.4088/JCP.v68n0603.View ArticlePubMedGoogle Scholar
- Chan HY, Lin WW, Lin SK, Hwang TJ, Su TP, Chiang SC, et al: Efficacy and safety of aripiprazole in the acute treatment of schizophrenia in Chinese patients with risperidone as an active control: a randomized trial. J Clin Psychiatry. 2007, 68: 29-36. 10.4088/JCP.v68n0104.View ArticlePubMedGoogle Scholar
- Zhong KX, Sweitzer DE, Hamer RM, Lieberman JA: Comparison of quetiapine and risperidone in the treatment of schizophrenia: A randomized, double-blind, flexible-dose, 8-week study. J Clin Psychiatry. 2006, 67: 1093-1103. 10.4088/JCP.v67n0712.View ArticlePubMedGoogle Scholar
- nil Yagcioglu AE, Kivircik Akdede BB, Turgut TI, Tumuklu M, Yazici MK, Alptekin K, et al: A double-blind controlled study of adjunctive treatment with risperidone in schizophrenic patients partially responsive to clozapine: efficacy and safety. J Clin Psychiatry. 2005, 66: 63-72. 10.4088/JCP.v66n0109.View ArticleGoogle Scholar
- Lane HY, Chang YC, Cheng YC, Liu GC, Lin XR, Chang WH: Effects of patient demographics, risperidone dosage, and clinical outcome on body weight in acutely exacerbated schizophrenia. J Clin Psychiatry. 2003, 64: 316-320. 10.4088/JCP.v64n0314.View ArticlePubMedGoogle Scholar
- Lindenmayer JP, Czobor P, Volavka J, Lieberman JA, Citrome L, Sheitman B, et al: Olanzapine in refractory schizophrenia after failure of typical or atypical antipsychotic treatment: an open-label switch study. J Clin Psychiatry. 2002, 63: 931-935. 10.4088/JCP.v63n1011.View ArticlePubMedGoogle Scholar
- Emsley RA, Roberts MC, Rataemane S, Pretorius J, Oosthuizen PP, Turner J, et al: Ethnicity and treatment response in schizophrenia: a comparison of 3 ethnic groups. J Clin Psychiatry. 2002, 63: 9-14. 10.4088/JCP.v63n0103.View ArticlePubMedGoogle Scholar
- Dossenbach MR, Kratky P, Schneidman M, Grundy SL, Metcalfe S, Tollefson GD, et al: Evidence for the effectiveness of olanzapine among patients nonresponsive and/or intolerant to risperidone. J Clin Psychiatry. 2001, 62 (Suppl 2): 28-34.PubMedGoogle Scholar
- Lindenmayer JP, Iskander A, Park M, Apergi FS, Czobor P, Smith R, et al: Clinical and neurocognitive effects of clozapine and risperidone in treatment-refractory schizophrenic patients: a prospective study. J Clin Psychiatry. 1998, 59: 521-527. 10.4088/JCP.v59n1005.View ArticlePubMedGoogle Scholar
- Koenigsberg HW, Reynolds D, Goodman M, New AS, Mitropoulou V, Trestman RL, et al: Risperidone in the treatment of schizotypal personality disorder. J Clin Psychiatry. 2003, 64: 628-634. 10.4088/JCP.v64n0602.View ArticlePubMedGoogle Scholar
- Case M, Stauffer VL, scher-Svanum H, Conley R, Kapur S, Kane JM, et al: The heterogeneity of antipsychotic response in the treatment of schizophrenia. Psychol Med. 2010, 1-10.Google Scholar
- Mizrahi R, Korostil M, Starkstein SE, Zipursky RB, Kapur S: The effect of antipsychotic treatment on Theory of Mind. Psychol Med. 2007, 37: 595-601. 10.1017/S0033291706009342.View ArticlePubMedGoogle Scholar
- Copolov DL, Link CG, Kowalcyk B: A multicentre, double-blind, randomized comparison of quetiapine (ICI 204,636, 'Seroquel') and haloperidol in schizophrenia. Psychol Med. 2000, 30: 95-105. 10.1017/S0033291799001476.View ArticlePubMedGoogle Scholar
- Musil R, Spellmann I, Riedel M, Dehning S, Douhet A, Maino K, et al: SNAP-25 gene polymorphisms and weight gain in schizophrenic patients. J Psychiatr Res. 2008, 42: 963-970. 10.1016/j.jpsychires.2007.11.003.View ArticlePubMedGoogle Scholar
- Lipkovich I, Deberdt W, Csernansky JG, Buckley P, Peuskens J, Kollack-Walker S, et al: Predictors of risk for relapse in patients with schizophrenia or schizoaffective disorder during olanzapine drug therapy. J Psychiatr Res. 2007, 41: 305-310. 10.1016/j.jpsychires.2006.07.016.View ArticlePubMedGoogle Scholar
- Bozina N, Kuzman MR, Medved V, Jovanovic N, Sertic J, Hotujac L: Associations between MDR1 gene polymorphisms and schizophrenia and therapeutic response to olanzapine in female schizophrenic patients. J Psychiatr Res. 2008, 42: 89-97. 10.1016/j.jpsychires.2006.10.002.View ArticlePubMedGoogle Scholar
- Prikryl R, Ceskova E, Kasparek T, Kucerova H: Neurological soft signs, clinical symptoms and treatment reactivity in patients suffering from first episode schizophrenia. J Psychiatr Res. 2006, 40: 141-146. 10.1016/j.jpsychires.2005.03.008.View ArticlePubMedGoogle Scholar
- Peet M, Horrobin DF: A dose-ranging exploratory study of the effects of ethyl-eicosapentaenoate in patients with persistent schizophrenic symptoms. J Psychiatr Res. 2002, 36: 7-18. 10.1016/S0022-3956(01)00048-6.View ArticlePubMedGoogle Scholar
- Girgis RR, Merrill DB, Vorel SR, Edward K, Kimberly P, You M, et al: Aripiprazole versus haloperidol treatment in early-stage schizophrenia. J Psychiatr Res. 2010Google Scholar
- Giegling I, Drago A, Schafer M, Moller HJ, Rujescu D, Serretti A: Interaction of haloperidol plasma level and antipsychotic effect in early phases of acute psychosis treatment. J Psychiatr Res. 2010, 44: 487-492. 10.1016/j.jpsychires.2009.11.004.View ArticlePubMedGoogle Scholar
- McEvoy JP, Daniel DG, Carson WH, McQuade RD, Marcus RN: A randomized, double-blind, placebo-controlled, study of the efficacy and safety of aripiprazole 10, 15 or 20 mg/day for the treatment of patients with acute exacerbations of schizophrenia. J Psychiatr Res. 2007, 41: 895-905. 10.1016/j.jpsychires.2007.05.002.View ArticlePubMedGoogle Scholar
- The pre-publication history for this paper can be accessed here:http://www.biomedcentral.com/1471-244X/11/113/prepub
This article is published under license to BioMed Central Ltd. This is an Open Access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.