Helping journals to improve their publishing standards: a data analysis of DOAJ new criteria effects


In 2013, Directory of Open Access Journals (DOAJ) expanded and updated its inclusion criteria and its journal evaluation process, ultimately removing a large number of journals that failed to submit an updated application. The present study examined the results of the new process and its capability to improve the quality of the directory and the reliability of the information contained in it. A dataset of 12.595 journals included in DOAJ, since its launch in 2003 until May 15th 2016, was examined and compared to other data. The number of journals deleted from DOAJ during this period is 3776; the majority of them (2851 journals) were excluded because publishers failed to complete the reapplication on time; 490 had ceased publication or were otherwise inactive; 375 were excluded for ethical issues; 53 because they were no longer open access or the content was embargoed, the final 7 were removed for other reasons. The top five countries in terms of the percentage of journals removed are: Japan (74% of journals removed); Pakistan (60%); Canada (51%); United States (50%); and Mexico (49%). Our study has shown that 158 of the removed journals are included in Beall’s lists; 1130 journals indexed in DOAJ are included in Scopus and/or JCR. Our analysis demonstrates that, thanks to the new acceptance criteria, to the improved screening process performed by national groups under the direction of the new management, there is a noticeable quality improvement of the journals indexed in DOAJ.


DOAJ; Editorial quality; E-publishing; Open access; Predatory publishers

Full Text:




Archambault, Éric, Didier Amyot, Philippe Deschamps, Aurore Nicol, Françoise Provencher, Lise Rebout, and Guillaume Roberge. Proportion of Open Access Papers Published in Peer-Reviewed Journals at the European and World Levels—1996–2013, 2014.

Baker, Monya. “Open-Access Index Delists Thousands of Journals.” Nature, May 2016. doi:10.1038/nature.2016.19871.

Beall, Jeffrey. “Predatory Publishers Are Corrupting Open Access.” Nature 489, no. 7415 (September 12, 2012): 179–179. doi:10.1038/489179a.

Berger, Monica, and Jill Cirasella. “Beyond Beall’s List Better Understanding Predatory Publishers.” College & Research Libraries News 76, no. 3 (March 2015): 132–135.

Bloudoff-Indelicato, Mollie. “Backlash after Frontiers Journals Added to List of Questionable Publishers.” Nature 526, no. 7575 (October 2015): 613–613. doi:10.1038/526613f.

Bohannon, John. “Who’s Afraid of Peer Review?” Science 342, no. 6154 (October 2013): 60–65. doi:10.1126/science.342.6154.60.

Coyle, Karen. “Predatory Publishers | Peer to Peer Review.” Library Journal, April 2013.

Crawford, Walt. “Ethics and Access 1: The Sad Case of Jeffrey Beall.” Cites & Insights 14, no. 4 (2014).

———. Gold Open Access Journals 2011-2015. Cites & Insights Books, 2016.

———. Gold Open Access Journals 2011-2015 [Dataset], 2016.

———. “Journals and Wannabes: Investigating The List.” Cites & Insights 14, no. 7 (2014).

Djuric, Dragan. “Penetrating the Omerta of Predatory Publishing: The Romanian Connection.” Sci Eng Ethics 21, no. 1 (February 2014): 183–202. doi:10.1007/s11948-014-9521-4.

Emery, Jill. “Heard on the Net.” The Charleston Advisor 15, no. 2 (October 2013): 67–68. doi:10.5260/chara.15.2.67.

Olijhoek, Tom, Dominic Mitchell, and Lars Bjornshauge. “Criteria for Open Access and Publishing.” ScienceOpen Research, January 2016. doi:10.14293/s2199-1006.1.sor-edu.amhuhv.v1.

Shen, Cenyu, and Bo-Christer Björk. “`Predatory’ Open Access: A Longitudinal Study of Article Volumes and Market Characteristics.” BMC Medicine 13, no. 1 (October 2015). doi:10.1186/s12916-015-0469-2.

Van Noorden, Richard. “Open-Access Website Gets Tough.” Nature 512, no. 7512 (August 2014): 17–17. doi:10.1038/512017a.

Xia, Jingfeng. “Predatory Journals and Their Article Publishing Charges.” Learned Publishing 28, no. 1 (January 2015): 69–74. doi:10.1087/20150111.

Article Metrics

Metrics Loading ...

Metrics powered by PLOS ALM


  • There are currently no refbacks.

Copyright (c) 2017 Andrea Marchitelli, Paola Galimberti, Andrea Bollini, Dominic Mitchell

Creative Commons License
This work is licensed under a Creative Commons Attribution 4.0 International License.

Logo Università di is published by Università di Firenze, Dipartimento di Storia, Archeologia, Geografia, Arte e Spettacolo (SAGAS) and hosted by University of Macerata, CSIA (Italy).

ISSN: 2038-1026

DOAJ seal