Vol. 14 No. 1 (2023): Peer review: a process undergoing a required transformation
Articles

Towards peer review as a group engagement

English PDF
  • Andrea Bonaccorsi
Andrea Bonaccorsi
University of Pisa
DOI: https://doi.org/10.36253/jlis.it-511

Published 2022-12-19

Keywords

  • Peer review,
  • Open access,
  • Prosocial behavior

How to Cite

Bonaccorsi, Andrea. 2022. “Towards Peer Review As a Group Engagement”. JLIS.It 14 (1):46-59. https://doi.org/10.36253/jlis.it-511.

Copyright (c) 2022 Andrea Bonaccorsi

Creative Commons License

This work is licensed under a Creative Commons Attribution 4.0 International License.

Crossref
Scopus
Google Scholar
Europe PMC

Abstract

I discuss from an economic perspective two of the most recent suggestions to reform the peer review system: (a) payment to referees; (b) ex post peer review. I show that strong economic arguments militate against these ideas.

With respect to payment to referees I use results from the economic analysis of prosocial behavior and the private production of public goods, which show that the supply of monetary incentives has the paradoxical effect of reducing the willingness of agents to collaborate, insofar as they substitute intrincic motivation with extrinsic motivation.

With respect to ex post peer review, I show that it fails to offer sufficient incentives to researchers, since it is anonymous, depersonalized, and weak in its marginal impact on publishing decisions. I take this argument to criticize the lack of theorizing, in the side of radical proponents of Open access, about the conditions for transition from the subscription model to the Open model. It is this lack of critical attention to economic arguments that has led to the unintended but dramatic outcome of a net increase in the cost of scientific publishing, as documented in very recent papers.

Finally, I advance a proposal for admitting payments to referees, but not as individuals but as groups of researchers. I offer this idea to open discussion.

English PDF

Metrics

Metrics Loading ...

References

  1. Aczel, Balazs, Barnabas Szaszi, and Alex O. Holcombe. (2021). “A billion-dollar donation: estimating the cost of researchers’ time spent on peer review.” Research Integrity and Peer Review 6 (14). https://doi.org/10.1186/s41073-021-00118-2. DOI: https://doi.org/10.1186/s41073-021-00118-2
  2. Asai, Sumiko. (2020). “Market power of publishers in setting article processing charges for open access journals.” Scientometrics 123: 1037–1049. https://doi.org/10.1007/s11192-020-03402-y. DOI: https://doi.org/10.1007/s11192-020-03402-y
  3. Breuning, Marijke, Jeremy Backstrom, Jeremy Brannon, Benjamin Isaak Gross, and Michael Widmeier. (2015). “Reviewer Fatigue? Why Scholars Decline to Review their Peers' Work.” PS: Political Science & Politics 48 (4): 595-600. https://doi.org/10.1017/S1049096515000827. DOI: https://doi.org/10.1017/S1049096515000827
  4. Chalmers, Beverley, and Dana L. Solomon. (2022). “Academic exploitation.” Lancet 400 July 16. https://doi.org/10.1016/S0140-6736(22)00922-9. DOI: https://doi.org/10.1016/S0140-6736(22)00922-9
  5. Cheah, Phaik Yeong, and Jan Piasecki. (2022). “Should peer reviewers be paid to review academic papers?” Lancet 399 April 23. https://doi.org/10.1016/S0140-6736(21)02804-X. DOI: https://doi.org/10.1016/S0140-6736(21)02804-X
  6. Chetty, Ray, Emmanuel Saez, and Laszlo Sandor. (2014). “What Policies Increase Prosocial Behavior? An Experiment with Referees at the Journal of Public Economics.” Journal of Economic Perspectives 28 (3): 169-188. https://doi.org/10.1257/jep.28.3.169. DOI: https://doi.org/10.1257/jep.28.3.169
  7. cOAlition S. (2018). “Accelerating the transition to full and immediate Open Access to scientific publications.” Available at: https://www.coalition-s.org/about/.
  8. Copiello, Sergio. (2018). “On the money value of peer review.” Scientometrics 115: 613–620. https://doi.org/10.1007/s11192-018-2664-3. DOI: https://doi.org/10.1007/s11192-018-2664-3
  9. Deci, Edward L. (1971). “Effects of Externally Mediated Rewards on Intrinsic Motivation.” Journal of Personality and Social Psychology 18 (1): 105-115. https://doi.org/10.1037/h0030644. DOI: https://doi.org/10.1037/h0030644
  10. Deci, Edward L., Richard Koestner, and Richard M. Ryan. (1999). “A Meta-analytic Review of Experiments Examining the Effects of Extrinsic Rewards on Intrinsic Motivation.” Psychological Bulletin 125 (6): 627-668. https://doi.org/10.1037/0033-2909.125.6.627. DOI: https://doi.org/10.1037/0033-2909.125.6.627
  11. DellaVigna, Stefano, John A. List, and Ulrike Malmendier. (2012). “Testing for Altruism and Social Pressure in Charitable Giving.” Quarterly Journal of Economics 127 (1): 1-56. https://www.jstor.org/stable/41337205. DOI: https://doi.org/10.1093/qje/qjr050
  12. Diamandis, Eleftherios P. (2015). “Peer review as a business transaction.” Nature 517 (145). https://doi.org/10.1038/517145a. DOI: https://doi.org/10.1038/517145a
  13. Fox, Jeremy, and Owen L. Petchey. (2010). “Pubcreds: fixing the peer review process by “privatizing” the reviewer commons.” Bulletin of the Ecological Society of America 91 (3): 325–333. https://doi.org/10.1890/0012-9623-91.3.325. DOI: https://doi.org/10.1890/0012-9623-91.3.325
  14. Frey, Bruno S., and Reto Jegen. (2001). “Motivation Crowding Theory: A Survey of Empirical Evidence.” Journal of Economic Surveys 15 (5): 589-611. https://doi.org/10.1111/1467-6419.00150. DOI: https://doi.org/10.1111/1467-6419.00150
  15. Frijters, Paul, and Benno Torgler. (2019). “Improving the peer review process: a proposed market system.” Scientometrics 119: 1285–1288. https://doi.org/10.1007/s11192-019-03076-1. DOI: https://doi.org/10.1007/s11192-019-03076-1
  16. García, Jose A., Rosa Rodriguez‑Sanchez, and J. Fdez‑Valdivia. (2022). “Can a paid model for peer review be sustainable when the author can decide whether to pay or not?.” Scientometrics 127: 1491–1514. https://doi.org/10.1007/s11192-021-04248-8. DOI: https://doi.org/10.1007/s11192-021-04248-8
  17. Garcia, Jose A., Rosa Rodriguez-Sanchez, and J. Fdez-Valdivia. (2021). “The interplay between the reviewer’s incentives and the journal’s quality standard.” Scientometrics 126: 3041–3061. https://doi.org/10.1007/s11192-020-03839-1. DOI: https://doi.org/10.1007/s11192-020-03839-1
  18. Gasparyan, Armen Yuri, Alexey N. Gerasimov, Alexander A. Voronov, and George D. Kita. (2015). “Rewarding Peer Reviewers: Maintaining the Integrity of Science Communication.” Journal of Korean Medical Science 30 (4): 360-364. https://doi.org/10.3346/jkms.2015.30.4.360. DOI: https://doi.org/10.3346/jkms.2015.30.4.360
  19. Gerhart, Barry, and Meiyu Fang. (2015). “Pay, Intrinsic Motivation, Extrinsic Motivation, Performance, and Creativity in the Workplace: Revisiting Long-Held Beliefs.” Annual Review of Organizational Psychology and Organizational Behavior 2: 489-521. DOI: https://doi.org/10.1146/annurev-orgpsych-032414-111418
  20. Gneezy, Uri, and Aldo Rustichini. (2000). “Pay enough or don’t pay at all.” Quarterly Journal of Economics 115: 791–810. https://doi.org/10.1162/003355300554917. DOI: https://doi.org/10.1162/003355300554917
  21. Gneezy, Uri, Stephan Meier, and Pedro Rey-Biel. (2011). “When and Why Incentives (Don’t) Work to Modify Behavior.” Journal of Economic Perspectives 25 (4): 191-210. https://doi.org/10.1257/jep.25.4.191. DOI: https://doi.org/10.1257/jep.25.4.191
  22. Golden M., and David M. Schultz. (2012). “Quantifying the Volunteer Effort of Scientific Peer Reviewing.” American Meteorological Society 93 (3): 337-345. https://doi.org/10.1175/BAMS-D-11-00129.1. DOI: https://doi.org/10.1175/BAMS-D-11-00129.1
  23. Grossmann, Alexander, and Björn Brembs. (2019). “Assessing the size of the affordability problem in scholarly publishing.” PeerJ Preprints. June 18. https://doi.org/10.7287/peerj.preprints.27809v1. DOI: https://doi.org/10.7287/peerj.preprints.27809v1
  24. Guinart, Daniel, Victor Pérez-Sola, and Benedikt Lorenz Amann. (2022). “A fair day’s wage for a fair day’s work.” Lancet 400 July 16, 2022. https://doi.org/10.1016/S0140-6736(22)00920-5. DOI: https://doi.org/10.1016/S0140-6736(22)00920-5
  25. Hauser Marc and Ernst Fehr. (2007). “An incentive solution to the peer review problem.” PLoS Biology 5 (4): e107. https://doi.org/10.1371/journal.pbio.0050107. DOI: https://doi.org/10.1371/journal.pbio.0050107
  26. Horbach, Serge P.J.M., and Willem Halffman. (2020). “Innovating editorial practices: academic publishers at work.” Research Integrity and Peer Review 5 (11). https://doi.org/10.1186/s41073-020-00097-w. DOI: https://doi.org/10.1186/s41073-020-00097-w
  27. Humphreys, Hilary. (2022). “Payment and progress in peer review.” Lancet, 400 July 16, 2022. https://doi.org/10.1016/S0140-6736(22)00921-7. DOI: https://doi.org/10.1016/S0140-6736(22)00921-7
  28. Hunter, Jane. (2012). “Post-publication peer review: opening up scientific conversation.” Frontiers in Computational Neuroscience, 30 August 2012. https://doi.org/10.3389/fncom.2012.00063. DOI: https://doi.org/10.3389/fncom.2012.00063
  29. Kamenica, Emir. (2012). “Behavioral Economics and Psychology of Incentives.” Annual Review of Economics 4 (1): 427-452. https://doi.org/10.1146/annurev-economics-080511-110909. DOI: https://doi.org/10.1146/annurev-economics-080511-110909
  30. Kovanis, Michail, Raphaël Porcher, Philippe Ravaud, and Ludovic Trinquart. (2016). “The Global Burden of Journal Peer Review in the Biomedical Literature: Strong Imbalance in the Collective Enterprise.” PloS ONE 11 (11): e0166387. https://doi.org/10.1371/journal.pone.0166387. DOI: https://doi.org/10.1371/journal.pone.0166387
  31. Kriegeskorte, Nikolaus, Alexander Walther, and Diana Deca. (2012). “An emerging consensus for open evaluation: 18 visions for the future of scientific publishing.” Frontiers in Computational Neuroscience 6 (94). https://doi.org/10.3389/fncom.2012.00094. DOI: https://doi.org/10.3389/fncom.2012.00094
  32. Kriegeskorte, Nikolaus. (2012). “Open evaluation: a vision for entirely transparent post-publication peer review and rating for science.” Frontiers in Computational Neuroscience, 17 October 2012. https://doi.org/10.3389/fncom.2012.00079. DOI: https://doi.org/10.3389/fncom.2012.00079
  33. Lacetera, Nicola, Mario Macis, and Robert Slonim. (2013). “Economic Rewards to Motivate Blood Donations.” Science 340 (6135): 927-928. https://doi.org/10.1126/science.12322. DOI: https://doi.org/10.1126/science.1232280
  34. Larivière, Vincent, Stefanie Haustein, and Philippe Mongeon. (2015). “The oligopoly of academic publishers in the digital era.” PloS One. 10 (6): e0127502. https://doi.org/10.1371/journal.pone.0127502. DOI: https://doi.org/10.1371/journal.pone.0127502
  35. Macháček, Vít, and Martin Srholec. (2021). “Predatory publishing in Scopus: evidence on cross‑country differences.” Scientometrics, 126, 1897–1921. DOI: https://doi.org/10.1007/s11192-020-03852-4
  36. Maddi, Abdelghani, and David Sapinho. (2022). “Article processing charges, altmetrics and citation impact: Is there an economic rationale?” Scientometrics. https://doi.org/10.1007/s11192-022-04284-y. DOI: https://doi.org/10.1007/s11192-022-04284-y
  37. Moustafa, Khaled. (2022). “No to paid peer review.” Lancet 400 July 16, 2022. https://doi.org/10.1016/S0140-6736(22)01057-1. DOI: https://doi.org/10.1016/S0140-6736(22)01057-1
  38. Nature. (2014). Review rewards. Nature 514 (274). https://doi.org/10.1038/514274a. DOI: https://doi.org/10.1038/514274a
  39. Northcraft, Gregory B., and Ann E. Tenbrunsel. (2011). “Effective matrices, decision frames, and cooperation in volunteer dilemmas: a theoretical perspective on academic peer review.” Organization Science 22: 1277–1285. https://www.jstor.org/stable/41303119. DOI: https://doi.org/10.1287/orsc.1100.0607
  40. Ott, Sascha, and Daniel Hebenstreit. (2014). “Apply market forces to peer review.” Nature 506 (295). https://doi.org/10.1038/506295b. DOI: https://doi.org/10.1038/506295b
  41. Oviedo-García, M. Ángeles. (2021). “Journal citation reports and the definition of a predatory journal: The case of the Multidisciplinary Digital Publishing Institute (MDPI).” Research Evaluation, 30 (3): 420. https://doi.org/10.1093/reseval/rvab030. DOI: https://doi.org/10.1093/reseval/rvab030
  42. Reichardt, Juergen K.W., George P. Patrinos, Poh San Lai, and Giuseppe Novelli. (2022). “J’Accuse… Or The Plight of pro‑bono Volunteer Scientists in Academic Publishing.” Human Genomics, 16 (44). https://doi.org/10.1186/s40246-022-00413-z. DOI: https://doi.org/10.1186/s40246-022-00413-z
  43. Righi, Simone, and Károly Takacs. (2017). “The miracle of peer review and development in science: an agent-based model.” Scientometrics 113: 587–607. https://doi.org/10.1007/s11192-017-2244-y. DOI: https://doi.org/10.1007/s11192-017-2244-y
  44. Rose-Wiles, Lisa M. (2011). “The High Cost of Science Journals: A Case Study and Discussion.” Journal of Electronic Resources Librarianship 23 (3): 219-241. https://doi.org/10.1080/1941126X.2011.601225. DOI: https://doi.org/10.1080/1941126X.2011.601225
  45. Ryan, Richard M., and Edward L. Deci. (2000). “Intrinsic and Extrinsic Motivations: Classic Definitions and New Directions.” Contemporary Educational Psychology 25: 54-67. https://doi.org/10.1006/ceps.1999.1020. DOI: https://doi.org/10.1006/ceps.1999.1020
  46. Schimmer, Ralf, Kai Karin Geschuhn, and Andreas Vogler. (2015). “Disrupting the subscription journals’ business model for the necessary large-scale transformation to open access.” A Max Planck Digital Library Open Access Policy White Paper. https://doi.org/10.17617/1.3.
  47. Seeber, Marco. (2022). “Efficacy, efficiency, and models of journal peer review: the known and unknown. In the social sciences.” In Handbook on Research Assessment in the Social Sciences, edited by Tim C.E. Engels and Emanuel Kulczycki, 67–82. Cheltenham: Edward Elgar Publishing. DOI: https://doi.org/10.4337/9781800372559.00011
  48. Smith, Richard. (2006). “Peer review: a flawed process at the heart of science and journals.” Journal of the Royal Society of Medicine 99: 178–182. https://doi.org/10.1258/jrsm.99.4.178. DOI: https://doi.org/10.1258/jrsm.99.4.178
  49. Squazzoni, Flaminio, Giangiacomo Bravo, and Károly Takács. (2013). “Does incentive provision increase the quality of peer review? An experimental study.” Research Policy 42 (1): 287-294. https://doi.org/10.1016/j.respol.2012.04.014. DOI: https://doi.org/10.1016/j.respol.2012.04.014
  50. Squazzoni, Flaminio, and Claudio Gandelli. (2012). “Saint Matthew strikes again: An agent-based model of peer review and the scientific community structure.” Journal of Informetrics 6: 265– 275. https://doi.org/10.1016/j.joi.2011.12.005. DOI: https://doi.org/10.1016/j.joi.2011.12.005
  51. Squazzoni, Flaminio, Francisco Grimaldo, and Ana Marušić. (2017). “Journals could share peer-review data.” Nature 546, 15 June 2017. https://doi.org/10.1038/546352a. DOI: https://doi.org/10.1038/546352a
  52. Stern, Scott. (2004). “Do scientists pay to be scientists?” Management Science 60 (6): 835–853. https://doi.org/10.1287/mnsc.1040.0241. DOI: https://doi.org/10.1287/mnsc.1040.0241
  53. Tennant Jonathan P., and Tony Ross-Hellauer. (2020). “The limitations to our understanding of peer review.” Research Integrity and Peer Review 5 (6). https://doi.org/10.1186/s41073-020-00092-1. DOI: https://doi.org/10.1186/s41073-020-00092-1
  54. Tite, Leanne, and Sara Schroter. (2007). “Why do peer reviewers decline to review? A survey.” Epidemiology and Community Health 61: 9–12. https://doi.org/10.1136/jech.2006.049817. DOI: https://doi.org/10.1136/jech.2006.049817
  55. Treviño, Linda K. (2008). “Why review? Because reviewing is a professional obligation.” Academy of Management Review 33 (1): 8-10. https://doi.org/10.5465/amr.2008.27744831. DOI: https://doi.org/10.5465/amr.2008.27744831
  56. Van Noorden, Richard. (2013). “Open access: The true cost of science publishing.” Nature News, 495 (7442): 426. https://doi.org/10.1038/495426a. DOI: https://doi.org/10.1038/495426a
  57. Walker, Richard, and Pascal Rocha da Silva. (2015). “Emerging trends in peer review—a survey.” Frontiers in Neuroscience 27 May 2015. https://doi.org/10.3389/fnins.2015.00169. DOI: https://doi.org/10.3389/fnins.2015.00169
  58. Zaharie, Monica Aniela, and Marco Seeber. (2018). “Are non-monetary rewards effective in attracting peer reviewers? A natural experiment.” Scientometrics 117: 1587–1609. https://doi.org/10.1007/s11192-018-2912-6. DOI: https://doi.org/10.1007/s11192-018-2912-6
  59. Zhang, Lin, Yahui Wei, Ying Huang, and Gunnar Sivertsen. (2022). “Should open access lead to closed research? The trends towards paying to perform research.” Scientometrics. https://doi.org/10.1007/s11192-022-04407-5. DOI: https://doi.org/10.31235/osf.io/z97re