What is a predatory journal? A scoping review

Kelly D. Cobey; Manoj M Lalu; Becky Skidmore; Nadera Ahmadzai; Agnes Grudniewicz; David Moher

doi:10.12688/f1000research.15256.2

Home Browse What is a predatory journal? A scoping review

ALL Metrics

Views

Downloads

Get PDF

Get XML

Export

▬

✚

Research Article

Revised

What is a predatory journal? A scoping review

[version 2; peer review: 3 approved]

Kelly D. Cobey ^1-3, Manoj M Lalu^1,4, Becky Skidmore¹, Nadera Ahmadzai¹, Agnes Grudniewicz⁵, David Moher ^1,2

Kelly D. Cobey ^1-3, Manoj M Lalu^1,4, [...] Becky Skidmore¹, Nadera Ahmadzai¹, Agnes Grudniewicz⁵, David Moher ^1,2

PUBLISHED 23 Aug 2018

Author details Author details

¹ Department of Psychology, School of Natural Sciences, University of Stirling, Stirling, FK9 4LA, UK
² School of Epidemiology and Public Health, Faculty of Medicine, University of Ottawa, Ottawa, K1G 5Z3, Canada
³ Centre for Journalology, Clinical Epidemiology Program, Ottawa Hospital Research Institute, Ottawa, K1H 8L6, Canada
⁴ Department of Anesthesiology and Pain Medicine, The Ottawa Hospital, Faculty of Medicine, University of Ottawa; Regenerative Medicine Program, Ottawa Hospital Research Institute, Ottawa, K1H 8L6, Canada
⁵ Telfer School of Management, University of Ottawa, Ottawa, K1N 6N5, Canada

Kelly D. Cobey
Roles: Conceptualization, Data Curation, Formal Analysis, Investigation, Methodology, Project Administration, Supervision, Validation, Writing – Original Draft Preparation, Writing – Review & Editing

Manoj M Lalu
Roles: Conceptualization, Methodology, Writing – Review & Editing

Becky Skidmore
Roles: Conceptualization, Investigation, Methodology, Writing – Review & Editing

Nadera Ahmadzai
Roles: Conceptualization, Data Curation, Formal Analysis, Investigation, Methodology, Writing – Review & Editing

Agnes Grudniewicz
Roles: Data Curation, Formal Analysis, Investigation, Validation, Writing – Review & Editing

David Moher
Roles: Conceptualization, Funding Acquisition, Methodology, Project Administration, Supervision, Writing – Review & Editing

OPEN PEER REVIEW

REVIEWER STATUS

This article is included in the Research on Research, Policy & Culture gateway.

Abstract

Background: There is no standardized definition of what a predatory journal is, nor have the characteristics of these journals been delineated or agreed upon. In order to study the phenomenon precisely a definition of predatory journals is needed. The objective of this scoping review is to summarize the literature on predatory journals, describe its epidemiological characteristics, and to extract empirical descriptions of potential characteristics of predatory journals.
Methods: We searched five bibliographic databases: Ovid MEDLINE, Embase Classic + Embase, ERIC, and PsycINFO, and Web of Science on January 2^nd, 2018. A related grey literature search was conducted March 27^th, 2018. Eligible studies were those published in English after 2012 that discuss predatory journals. Titles and abstracts of records obtained were screened. We extracted epidemiological characteristics from all search records discussing predatory journals. Subsequently, we extracted statements from the empirical studies describing empirically derived characteristics of predatory journals. These characteristics were then categorized and thematically grouped.
Results: 920 records were obtained from the search. 344 of these records met our inclusion criteria. The majority of these records took the form of commentaries, viewpoints, letters, or editorials (78.44%), and just 38 records were empirical studies that reported empirically derived characteristics of predatory journals. We extracted 109 unique characteristics from these 38 studies, which we subsequently thematically grouped into six categories: journal operations, article, editorial and peer review, communication, article processing charges, and dissemination, indexing and archiving, and five descriptors.
Conclusions: This work identified a corpus of potential characteristics of predatory journals. Limitations of the work include our restriction to English language articles, and the fact that the methodological quality of articles included in our extraction was not assessed. These results will be provided to attendees at a stakeholder meeting seeking to develop a standardized definition for what constitutes a predatory journal.

Keywords

scholarly publishing, open access, predatory journals, predatory publishers, illegitimate journals, peer review, reporting quality

Corresponding authors: Kelly D. Cobey, David Moher

Competing interests: No competing interests were disclosed.

Grant information: The authors declared that no grants were involved in supporting this work. DM is funded by a University Research Chair. MML is supported by The Ottawa Hospital Anesthesia Alternate Funds Association and the Scholarship Protected Time Program, Department of Anesthesiology and Pain Medicine, uOttawa.
The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.

Copyright: © 2018 Cobey KD et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

How to cite: Cobey KD, Lalu MM, Skidmore B et al. What is a predatory journal? A scoping review [version 2; peer review: 3 approved]. F1000Research 2018, 7:1001 (https://doi.org/10.12688/f1000research.15256.2) First published: 04 Jul 2018, 7:1001 (https://doi.org/10.12688/f1000research.15256.1) Latest published: 23 Aug 2018, 7:1001 (https://doi.org/10.12688/f1000research.15256.2)

Revised Amendments from Version 1

We have changed journals to “publishers” in the introduction.
We have noted the global south issue of journals using “international” or “global” in their titles.

We have more cleared described scoping reviews and added an additional reference.
We believe we have given some examples in Table 3. For example, in response to the query as to the use and meaning of “persuasive language”, we state “Language that targets; Language that attempts to convince the author to do or believe something”.
We have made some modifications to the limitations section of our paper. We now state “Thirdly, our focus was on the biomedical literature. Whether the publication (e.g., having an IMRAD (Introduction Methods Results And Discussion) and peer review norms we’ve used apply across other disciplines is likely an important topic for further investigation.”
We have further indicated the limitations of Beall’s lists for this type of research.
We have fixed the broken link to the full search strategy (Supplementary File 1).

See the authors' detailed response to the review by Joanna Chataway
See the authors' detailed response to the review by Johann Mouton
See the authors' detailed response to the review by Monica Berger

Introduction

The term ‘predatory journal’ was coined less than a decade ago by Jeffrey Beall¹. Predatory journals have since become a hot topic in the scholarly publishing landscape. A substantial body of literature discussing the problems created by predatory journals, and potential solutions to stop the flow of manuscripts to these journals, has rapidly accumulated^2–6. Despite increased attention in the literature and related educational campaigns⁷, the number of predatory journals, and the number of articles these journals publish, continues to increase rapidly⁸. Some researchers may be tricked into submitting to predatory journals⁹, while others may do so dubiously to pad their curriculum vitae for career advancement¹⁰.

One factor that may be contributing to the rise of predatory journals is that there is currently no agreed upon definition of what constitutes a predatory journal. The characteristics of predatory journals have not been delineated, standardized, nor broadly accepted. In the absence of a clear definition, it is difficult for stakeholders such as funders and research institutions to establish explicit policies to safeguard work they support from being submitted to and published in predatory journals. Likewise, if characteristics of predatory journals have not been delineated and accepted, it is difficult to take an evidence-based approach towards educating researchers on how to avoid them. Establishing a consensus definition has the potential to inform policy and to significantly strengthen educational initiatives such as Think, Check, Submit⁷.

The challenge of defining predatory journals has been recognized¹¹, and recent discussion in the literature highlights a variety of potential definitions. Early definitions by Beall describe predatory publishers as outlets “which publish counterfeit journals to exploit the open-access model in which the author pays” and publishers that were “dishonest and lack transparency”¹. Others have since suggested that we move away from using the term ‘predatory journal’, in part because the term neglects to adequately capture journals that fail to meet expected professional publishing standards, but do not intentionally act deceptively^12–15. This latter view suggests that the rise of so-called predatory journals is not strictly associated with dubious journal operations that use the open-access publishing model (e.g., publishing virtually anything to earn an article processing charge (APC)), but represents a wider spectrum of problems. For example, there is the conundrum that some journals hailing from the global south may not have the knowledge, resources, or infrastructure to meet best practices in publishing although some of them have ‘international’ or ‘global’ in their title. Devaluing or black-listing such journals may be problematic as they serve an important function in ensuring the dissemination of research on topics of regional significance.

Other terms to denote predatory journals such as “illegitimate journals^9,16”, “deceptive journals¹⁵”, “dark” journals¹⁷, and “journals operating in bad faith¹³” have appeared in the literature, but like the term “predatory journal” they are reductionist¹¹ and may not adequately reflect the varied spectrum of quality present in the scholarly publishing landscape and the distinction between low-quality and intentionally dubious journals. These terms have also not garnered widespread acceptance, and it is possible that the diversity in nomenclature leads to confusion for researchers and other stakeholders.

Here, we seek to address the question “what is a predatory journal?” by conducting a scoping review^18,19 of the literature. Scoping reviews are a type of knowledge synthesis that follow a systematic approach to map the literature on a topic, and identify the main concepts, theories and sources, and determine potential gaps in that literature. Guidance on their conduct is available^18–20 and guidance on their reporting is forthcoming. Our aims are twofold. Firstly, in an effort to provide an overview of the literature on the topic, we seek to describe epidemiological characteristics of all records discussing predatory journals. Secondly, we seek to synthesize the existing empirically derived characteristics of predatory journals. The impetus for this work is to establish a list of evidence-based traits and characteristics of predatory journals. This corpus of possible characteristics of predatory journals is one source that could be considered by an international stakeholders meeting to generate a consensus definition of predatory journals. Other sources will be included (e.g.,⁸).

Methods

Transparency statement

Prior to initiating this study, we drafted a protocol that was posted on the Open Science Framework prior to data analysis (please see: https://osf.io/gfmwr/). We did not register our review with PROSPERO as the registry does not accept scoping reviews. Other than the protocol deviations described below, the authors affirm that this manuscript is an honest, accurate, and transparent account of the study being reported; that no important aspects of the study have been omitted; and that discrepancies from the study as planned have been explained. We briefly re-state our study methods here. Large sections of the methods described here are taken directly from the original protocol. We used the PRISMA statement²¹ to guide our reporting of this scoping review.

Search strategy

For our full search strategy please see Supplementary File 1. An experienced medical information specialist (BS) developed and tested the search strategy using an iterative process in consultation with the review team. Another senior information specialist peer reviewed the strategy prior to execution using the PRESS Checklist²². We searched a range of databases in order to achieve cross-disciplinary coverage. These included: Web of Science and four Ovid databases: Ovid MEDLINE®, including Epub Ahead of Print and In-Process & Other Non-Indexed Citations, Embase Classic + Embase, ERIC, and PsycINFO. We performed all searches on January 2, 2018.

There were no suitable controlled vocabulary terms for this topic in any of the databases. We used various free-text phrases to search, including multiple variations of root words related to publishing (e.g., edit, journal, publication) and predatory practices (e.g., bogus, exploit, sham). We adjusted vocabulary and syntax across the databases. We limited results to the publication years 2012 to the present, since 2012 is the year in which the term “predatory journal” reached the mainstream literature¹.

We also searched abstracts of relevant conferences (e.g., The Lancet series and conference “Increasing Value, Reducing Waste”, International Congresses on Peer Review and Scientific Publication) and Google Scholar to identify grey literature. For the purposes of our Google Scholar search, we conducted an advanced search (on March 27, 2018) using the keywords: predatory, journal, and publisher. We restricted this search to content published from 2012 onward. A single reviewer (KDC) reviewed the first 100 hits and extracted all potentially relevant literature encountered for review, based on title. We did not review content from file sources that were from mainstream publishers (e.g., Sage, BMJ, Wiley), as we expected these to be captured in our broader search strategy.

Study population and eligibility criteria

Our study population included articles, reports, and other digital documents that discuss, characterize, or describe predatory journals. We included all study designs from any discipline captured by our search that were reported in English. This included experimental and observational research, as well as commentaries, editorials and narrative summaries in our epidemiological extraction. For extraction of characteristics of predatory journals we restricted our sample to studies that specifically provided empirically derived characteristics of predatory journals.

Screening and data extraction

Data extraction forms were developed and piloted prior to data extraction. Details of the forms used are provided in the Open Science Framework, see here: https://osf.io/p5y2k/. We first screened titles and abstracts against the inclusion criteria. We verified full-text articles met the inclusion criteria and we extracted information on corresponding author name, corresponding author country, year of publication (we selected the most recent date stated), study design (as assessed by the reviewers), and journal name. We also extracted whether or not the paper provided a definition of a predatory journal. This was coded as yes/no and included both explicit definitions (e.g. “Predatory journals are…”) as well as implicit definitions.

When extracting data, we restricted our sample of articles to those that provided a definition of predatory journals, or described characteristics of predatory journals, based on empirical work (i.e., not opinion, not definitions which referenced previous work). Specifically, we restricted our sample of articles to those classed as having an empirical study design and then re-vetted each article to ensure that the study addressed defining predatory journals or their characteristics. For those articles included, we extracted sections of text statements describing the traits/characteristics of predatory journals. Extraction was done by a single reviewer, with verification conducted by a second reviewer. Conflicts were resolved via consensus. In instances where an empirically derived trait/characteristic of predatory journals was mentioned in several sections of the article, we extracted only a single representative statement.

Data analysis

Our data analysis involved both quantitative (i.e., frequencies and percentages) and qualitative (i.e., thematic analysis) methods. First, a list of potential characteristics of predatory journals was generated collaboratively by the two reviewers who conducted data extraction (KDC, NA). Subsequently, each of the statements describing characteristics of predatory journals that were extracted from the included articles were categorized using the list generated. During the categorization of the extracted statements, if a statement did not apply to a category already on the list, a new category was added. Where duplicate statements were inadvertently extracted from a single record we categorized these only once. During the categorization and grouping process, details on the specific wording of statements from specific included records were not retained (i.e., our categories and our themes do not preserve the original wording of the extracted text).

Subsequently, in line with Galipeau and colleagues²³, after this initial categorization, we collated overlapping or duplicate categories into themes. Then, two reviewers (KDC, AG) evaluated recurring themes in the work to synthesize the data. A coding framework was iteratively developed by KDC and AG by coding each characteristic statement independently and inductively (i.e., without using a theory or framework a priori). The two reviewers met to discuss these codes, and through consensus decided on the final themes and their definitions. The reviewers then went back to the data and recoded with the agreed-upon themes. Lastly, the reviewers met to compare assignment of themes to statements. Discrepancies were resolved by consensus. Two types of themes emerged: categories (i.e., features of predatory journals to which the statements referred) and descriptors (i.e., statements which described these features, usually with either a positive or negative value).

Deviations from study protocol

We conducted data extraction of epidemiological characteristics of papers discussing predatory journals in duplicate. The original protocol indicated this would be done by a single reviewer with verification. The original protocol stated we would extract information on the discipline of the journals publishing our articles included for epidemiological data extraction (as defined by MEDLINE). Instead, we used SCIMAGOJR (SJR) (https://www.scimagojr.com/) to determine journal subject areas post-hoc and only extracted this information for the included empirical articles describing empirically derived characteristics of predatory journals. For included articles, post-hoc, we decided to extract information on whether or not the record reported on funding.

Results

Search results and epidemiological characteristics

Please see Figure 1 for record and article flow during the review. The original search captured 920 records. We excluded 19 records from initial screening because they were not in English (N = 13), we could not access a full-text document (N = 5; of which one was behind a paywall at a cost of greater than $25 CAD), or the reference referred to a conference proceeding containing multiple documents (N = 1).

Figure 1. Preferred Reporting Items for Systematic reviews and Meta-Analyses (PRISMA) flow diagram summarizing study selection.

We screened a total of 901 title and abstract records obtained from the search strategy. Of these, 402 were included for full-text screening. 499 records were excluded for not meeting our study inclusion criteria. After full-text screening of the 402 studies, 334 were determined to have full texts and to discuss predatory journals. The remaining 68 records were excluded because: they were not about predatory journals (N = 36), did not have full texts (N = 19), were abstracts (N = 12), or were published in a language other than English (N = 1). The 334 articles included for epidemiological data extraction were published between 2012 and 2018 with corresponding authors from 43 countries. The number of publications mentioning predatory journals increased each year from 2012 to 2017 (See Table 1). The vast majority of these publications took the form of commentaries, viewpoints, letters, or editorials (262/334; 78.44%).

Table 1. Epidemiological characteristics of all articles mentioning predatory journals and those included empirical articles describing empirically derived characteristics of predatory journals included in our scoping review.

	Articles mentioning predatory journals (N=334)	Empirical articles included in systematic scoping review (N=38)
Nationality of corresponding authors (Top 3)	USA: 78 India: 34 Canada: 22ⁱ	USA: 11 Italy: 5 Canada: 4ⁱⁱ
Publication year of articlesⁱⁱⁱ	2012: 5 2013: 8 2014: 22 2015: 71 2016: 78 2017:140 2018: 5 Not reported: 5	2012: 0 2013: 0 2014: 2 2015: 9 2016: 10 2017: 16 2018: 1 Not reported: 0
Study design	Commentary/Viewpoint/Editorial/Letter: 262 Observational Study: 34 Narrative Review: 20 Case report/Case series: 13 Systematic Review: 1 Other: 4	Commentary/Viewpoint/Editorial/Letter: 0 Observational Study: 26 Narrative Review: 0 Case report/Case series: 11 Systematic Review: 1 Other: 0

ⁱ 61 articles did not clearly state the corresponding authors’ nationality, and 1 stated they wished to remain anonymous

ⁱⁱ 1 article did not clearly state the corresponding author’s nationality

ⁱⁱⁱ Note this is truncated data for 2018 since we conducted out search on January 2nd, 2018

Of the articles discussing predatory journals, only 38 specifically described a study that reported empirically derived characteristics or traits of predatory journals. These studies were published between 2014 and 2018 and produced by corresponding authors from 19 countries. The majority of these included studies were observational studies (26/38; 68.4%) (See Table 1 and Table 2).

Table 2. Included empirical records (N=38). For full citations see Supplementary File 2.

RefId	Corresponding author	County of corresponding author	Year of publication	Journal Title	Subject Area (from SJR)	Study design	Number of extracted characteristics (N=350)
1	Marilyn H. Oermann	USA	2017	Nursing Outlook	Nursing	Observational	14
8	Terence V. McCann	Australia	2017	Journal of Advanced Nursing.	Nursing	Systematic Review	10
13	Eric Mercier	Canada	2017	Postgraduate Medical Journal	Medicine	Observational	14
35	Pravin Bolshete	India	2018	Current Medical Research and Opinion	Medicine	Case report/ Case Series	13
99	Franca Deriu	Italy	2017	Neuroscience	Neuroscience	Observational	8
121	Mary M. Christopher	USA	2015	Frontiers in Veterinary Science	N/A	Observational	34
150	Marilyn H. Oermann	USA	2016	Journal of Nursing Scholarship	Nursing	Observational	14
165	Katarzyna Pisanski	UK	2017	Nature	Multidisciplinary	Observational	8
168	Andrea Manca	Italy	2017	Archives of Physical Medicine and Rehabilitation	Health Professions/ Medicine	Observational	9
176	Bhakti Hansoti	USA	2016	Western Journal of Emergency Medicine	Medicine	Observational	2
181	Victor Grech	Malta	2016	Journal of Visual Communication in Medicine	Arts and Humanities/ Health Professions	Case report/ Case Series	5
203	Jelte M. Wicherts	The Netherlands	2016	PLOS ONE	Agriculture and Biological Sciences/ Biochemistry, Genetics and Molecular Biology; Medicine	Observational	1
209	Cenyu Shen	Finland	2015	BMC Medicine	Medicine	Observational	6
275	Dragan Djuric	Serbia	2015	Science and Engineering Ethics	Buisness, Management and Accounting; Medicine; Nursing; Social Sciences	Case report/ Case Series	5
299	Larissa Shamseer	Canada	2017	BMC Medicine	Medicine	Observational	27
362	Mark Clemons	Canada	2017	The Oncologist	N/A	Case report/ Case Series	15
384	David Moher	Canada	2015	BMC Medicine	Medicine	Case report/ Case Series	11
462	Lynn E. McCutcheon	USA	2016	North American Journal of Psychology	Psychology; Social Sciences	Observational	6
489	Anonymous	Anonymous	2015	Journal of Developmental & Behavioral Pediatrics	Medicine; Psychology	Case report/ Case Series	12
525	Tove Faber Frandsen	Denmark	2017	Scientometrics	Computer Science; Social Science	Observational	1
548	Jaimie A. Teixeira Da Silva	Japan	2017	Current Science	Multidisciplinary	Case report/ Case Series	6
561	P. de Jager	South Africa	2017	South African Journal of Business Management	Business, Management and Accounting	Observational	13
586	Krystal E. Noga-Styron	USA	2017	Journal of Criminal Justice Education	Social Science	Observational	9
596	John H. McCool	USA	2017	The Scientist Magazine	N/A	Case report/ Case Series	4
654	Filippo Eros Pani	Italy	2017	Library Review	Social Science	Observational	2
660	Marco Cosentino	Italy	2017	Plagiarism Across Europe and Beyond 2017- Conference Proceedings	N/A	Case report/ Case Series	2
686	Andrea Marchitelli	Italy	2017	Italian Journal of Library, Archives & Information Science	N/A	Observational	1
701	G. S. Seethapathy	Norway	2016	Current Science	Multidisciplinary	Observational	3
728	Alexandre Martin	USA	2016	Learned Publishing	Social Sciences	Case report/ Case Series	4
736	Marta Somoza- Fernández	Spain	2016	El profesional de la información	Computer Science; Social Sciences	Observational	6
755	Marcin Kozak	Poland	2016	Journal of the Association for Information Science and Technology	Computer Science; Decision Sciences; Social Sciences	Case report/ Case Series	19
812	Alexandru-Ionuţ Petrişor	Romania	2016	Malaysian Journal of Library & Information Science	Social Sciences	Observational	23
900	Jingfeng Xia	USA	2015	Journal of the Association for Information Science and Technology	Computer Science; Decision Sciences; Social Sciences	Observational	3
904	Mehrdad Jalalian	Iran	2015	Geographica Pannonica	Business, Managements and Accounting; Earth and Planetary Sciences; Social Sciences	Observational	8
975	Williams Ezinwa Nwagwu	South Africa	2015	Learned Publishing	Social Sciences	Observational	11
976	Jingfeng Xia	USA	2015	Learned Publishing	Social Sciences	Observational	6
1012	Ayokunle Olumuyiwa Omobowale	Nigeria	2014	Current Sociology	Social Sciences	Observational	5
1068	David Matthew Markowitz	USA	2014	121st ASEE Annual Conference & Exposition	N/A	Observational	10

Five additional records obtained from the grey literature search were excluded. These records were either duplicates of studies captured in the main search or they did not provide empirically derived characteristics of predatory journals.

Mapping the data into emergent themes

The list generated to categorize the extracted statements describing characteristics of predatory journals had 109 categories. Two types of themes were identified using qualitative thematic analysis: categories and descriptors. Each statement addressed at least one of the following categories: journal operations, article, editorial and peer review, communication, article processing charges, and dissemination, indexing, and archiving. Within these categories, statements used descriptors including: deceptive or lacking transparency, unethical research or publication practices, persuasive language (), poor quality standards, or high quality standards. Statements that did not include a descriptive component (i.e., were neutral) were coded as not applicable (See Table 3 for themes and definitions). Statements addressing more than one category or using more than one descriptor were coded multiple times. Below we briefly summarize the qualitative findings by category (For full results, see Table 4).

Table 3. Themes and Definitions used to Code Characteristics of Predatory Journals.

Theme	Definition
Category
1. Journal Operations	Features related to how the journal conducts its business operations
2. Article	Features related to articles appearing in the journal
3. Editorial and Peer Review	Any aspect of the internal or external review of submitted articles and decisions on what to publish
4. Communication	How the journal interacts with (potential) authors, editors, and readers
5. Article Processing Charges	Fees taken in by journal as part of their business model
6. Dissemination, Indexing, and Archiving	Information on how the journal disseminates articles and use of indexing and archiving tools
Descriptor
1. Deceptive or Lacking Transparency	Intentionally deceitful practice; Practices or processes that are not made clear to the reader; Missing information
2. Unethical Research or Publication Practices	Violations of accepted publication and research ethics standards (e.g., Committee on Publication Ethics guidelines)
3. Persuasive Language	Language that targets; Language that attempts to convince the author to do or believe something
4. Poor Quality Standards	Lack of rigour in journal operations; Lack of professional standards/ practices; missing information; Poor quality writing or presentation (e.g., grammatical or spelling errors)
5. High Quality Standards	Evidence of rigour in journal operations; Evidence that professional standards/practices are being met; Clear information
6. Not Applicable	Neutral or non-descriptive statement

Table 4. Characteristics extracted, including article reference and frequency, and their thematic categorization and descriptor.

Characteristics	Frequency	RefIDs	Category	Descriptor
Article authors not listed with credentials/contact info	1	1	Article	Poor Quality Standards; Deceptive or Lacking Transparency
Articles follow Introduction, Methods, Results, and Discussion (IMRaD) structure for reporting	1	1	Article	NA
Articles have logical presentation and organization	1	1	Article	High Quality Standards
Items expected to be reported were reported most of the time (e.g. research question, sampling procedure)	1	1	Article	High Quality Standards
Many studies failed to report REB/ethics approval	1	1	Article	Unethical Research or Publication Practices
Wide range of lengths of articles	1	1	Article	NA
Wide range of reference styles used	1	1	Article	Poor Quality Standards
Articles contain statistical and methods errors	1	462	Article	Poor Quality Standards
Journals contain articles with plagiarized content	3	1, 121, 275	Article	Unethical Research or Publication Practices; Deceptive or Lacking Transparency
Grammatical errors in articles	4	1, 121, 462, 561	Article	Poor Quality Standards
Quality of articles rated as poor	5	1,8, 121, 462, 1012	Article	Poor Quality Standards
Articles are poorly cited	5	525, 561, 654 900, 975	Article	Poor Quality Standards
Journal offers discounts on the standard open access charges e.g. for specific members, a fee waiver to authors from low-income economies	1	812	Article Processing Charges	NA
Journals highlight easy methods of payment e.g. PayPal, credit card, debit card, net banking, cash card online 24/7.	1	812	Article Processing Charges	Persuasive Language
Journal APCs clearly stated	4	561, 755, 812, 976	Article Processing Charges	High Quality Standards
Journal does not specify APCs	9	35, 99, 121, 150, 181, 299, 489, 755, 976	Article Processing Charges	Deceptive or Lacking Transparency
Journal has hidden APCs or hidden information on APCs	9	121, 150, 168, 181, 299, 362, 489, 586, 976	Article Processing Charges	Deceptive or Lacking Transparency; Poor Quality Standards; Unethical Research or Publication Practices
APCs are lower than at legitimate journals	9	99, 168, 181, 209, 299, 362, 561, 812, 976	Article Processing Charges	NA
E-mail invitations explicitly noted they were not spam	1	13	Communication	Persuasive Language
E-mail solicitations don't contain contact information	1	13	Communication	Poor Quality Standards; Deceptive or Lacking Transparency
E-mail solicitations note acceptance of all manuscript types	1	13	Communication	Persuasive Language
Journal preys on junior researchers	1	121	Communication	Deceptive or Lacking Transparency
Journal language targets authors	1	299	Communication	Persuasive Language
E-mail invitations addressed inappropriately	1	384	Communication	Poor Quality Standards
Journals use the same strategies as internet-based scams to identify their prey	1	812	Communication	Deceptive or Lacking Transparency
Journal or e-mail invitations stress ability to publish in a special issue	2	13, 35	Communication	Persuasive Language
E-mails invitations specified a deadline to submit	2	13, 362	Communication	Persuasive Language
E-mail invitations had an unsubscribe option	2	13, 384	Communication	High Quality standards
E-mail solicitations have grammar errors	2	13, 384	Communication	Poor Quality Standards
E-mail solicitations referenced researchers previous work	2	13, 384	Communication	Persuasive Language
Journal uses business advertisement terminology	2	561, 812	Communication	Persuasive Language
Journals use positive emotions, linguistic qualifiers, or few casual words to accomplish their goal of selling the publication	2	975, 1068	Communication	Persuasive Language
Journals solicit editors via aggressive e-mail tactics	3	8, 13, 586	Communication	Persuasive Language
E-mail solicitations are not relevant to researcher expertise	3	362, 384, 755	Communication	Poor Quality Standards
E-mail invites were overly formal or used praise	3	362, 384, 812	Communication	Persuasive language
Journals solicit papers via aggressive e-mail tactics	13	8, 13, 121, 150, 181, 362, 384, 489, 548, 586, 596, 755, 904	Communication	Persuasive Language
E-mail solicitations don't mention APCs	3	13, 489, 586	Communication; Article Processing Charges	Deceptive or Lacking Transparency
E-mail solicitations or journal note special discounts	4	13, 755, 812, 904	Communication; Article Processing Charges	Persuasive Language
Journals contain extreme variability in article quality	1	462	Dissemination, Indexing, Archiving	Poor Quality Standards
Journal notes fake abstracting and indexing	1	812	Dissemination, Indexing, Archiving	NA
Journals can be found on Google Scholar	1	975	Dissemination, Indexing, Archiving	NA
Journals have Digital Object Identifiers (DOIs)	1	975	Dissemination, Indexing, Archiving	High Quality Standards
Journals are not archived	2	8, 150	Dissemination, Indexing, Archiving	Poor Quality Standards
Journals tend not to mention Committee of Publication Ethics (COPE)	2	299, 384	Dissemination, Indexing, Archiving	Poor Quality Standards
Journals state they are open access but are not openly available	2	362, 755	Dissemination, Indexing, Archiving	Unethical Research or Publication Practices; Deceptive or Lacking Transparency
Journals have a subscription based model	2	755, 561	Dissemination, indexing, archiving	NA
Journals may have International Standard Serial Number (ISSN)	3	701, 736, 975	Dissemination, Indexing, Archiving	NA
Articles may be in PubMed	4	99, 150, 168, 654	Dissemination, Indexing, Archiving	NA
Journals may not be in the Directory of Open Access Journals	5	35, 99, 561, 755, 975	Dissemination, Indexing, Archiving	Poor Quality Standards
Journals are not indexed	7	8,121,150,181,561, 736, 975	Dissemination, Indexing, Archiving	Poor Quality Standards
Journal may be listed in Directory of Open Access Journals	8	168, 209, 299, 561, 686, 736, 755, 975	Dissemination, Indexing, Archiving	NA
Journals state they are open access	11	8, 13, 99, 165, 168,176, 181, 209, 299, 362, 755	Dissemination, Indexing, Archiving	NA
Journals entice big name scientists to lend name (only) to editorial board	1	121	Editorial/Peer Review	Persuasive Language; Unethical Research or Publication Practices
Journal describes peer review process clearly	1	150	Editorial/Peer Review	High Quality Standards
Journal conducts fake reviews or editorial review	2	121, 165	Editorial/Peer Review	Deceptive or Lacking Transparency; Poor Quality Standards; Unethical Research or Publication Practices
Editorial board has an agenda to publish certain articles (from certain authors)	2	121, 561	Editorial/Peer Review	Deceptive or Lacking Transparency; Unethical Research or Publication Practices
Journals have poor editorial oversight/review	2	462, 755	Editorial/Peer Review	Poor Quality Standards
Editorial board repeats in multiple journals	3	35, 362, 812	Editorial/Peer Review	Unethical Research or Publication Practices
Journal conducts peer review	6	150, 299, 362, 384, 489, 586	Editorial/Peer Review	High Quality Standards
Editorial board is not stated or incomplete	7	35, 150, 299, 548, 755, 812, 1068	Editorial/Peer Review	Deceptive or Lacking Transparency
Journals have short peer review times	7	548, 586, 596, 728, 755, 812, 1068	Editorial/Peer Review	NA
Journals conduct poor quality peer review	8	8, 121,165,489, 596, 728,1012,1068	Editorial/Peer Review	Poor Quality Standards
Authors more likely to come from second-tier academic institutes	1	701	Editorial/Peer Review; Article	NA
Editor inserts plagiarized content into article	1	728	Editorial/Peer Review; Article	Deceptive or Lacking Transparency; Unethical Research or Publication Practices
Editors or websites listed on page may not even be affiliated to journal	1	121	Editorial/Peer Review; Journal Operations	Deceptive or Lacking Transparency
Journal does not look at all submitting authors fairly	1	121	Editorial/Peer Review; Journal Operations	Deceptive or Lacking Transparency; Unethical Research or Publication Practices
Journal requests fees to sit on editorial board	1	165	Editorial/Peer Review; Journal Operations	Poor Quality Standards; Unethical Research or Publication Practices
Editorial board lacks legitimacy (appointed without knowledge, wrong skillset)	7	121, 150, 165, 299, 489, 812, 1068	Editorial/Peer Review; Journal Operations	Deceptive or Lacking Transparency; Poor Quality Standards; Unethical Research or Publication Practices
Journal has no article preparation instructions	1	35	Journal Operations	Poor Quality Standards
A journal that collects information for less-than-honorable purposes	1	121	Journal Operations	Deceptive or Lacking Transparency
A journal that commercially encroaches on existing journals	1	121	Journal Operations	Deceptive or Lacking Transparency
A journal that seeks to discredit another journal	1	121	Journal Operations	Deceptive or Lacking Transparency
Journal actively seeks manuscripts to prevent other journals from publishing	1	121	Journal Operations	Deceptive or Lacking Transparency; Unethical Research or Publication Practices
Journal shuts down ideas and results of submitted articles	1	121	Journal Operations	NA
Journal will publish non-academic research	1	121	Journal Operations	Poor Quality Standards; Unethical Research or Publication Practices
Grammatical errors on journal webpage	1	299	Journal Operations	Poor Quality Standards
Journal uses distorted or unauthorized images	1	299	Journal Operations	Deceptive or Lacking Transparency; Unethical Research or Publication practices
Journals indicate they retain copyright in spite of stating journal was OA	1	299	Journal Operations	Unethical research or publication practices; Deceptive or Lacking Transparency
Journal contains articles that should be combined into one (e.g., salami publishing)	1	561	Journal Operations	Unethical Research or Publication Practices
Journals tend to publish in high quantity without regard for quality to earn profit	1	561	Journal Operations	Unethical Research or Publication Practices; Poor Quality Standards
Journals may publish work funded by national governments	1	701	Journal Operations	NA
Journal names change with trends	1	812	Journal Operations	Poor Quality Standards
Journal is not very readable	1	1068	Journal Operations	Poor Quality Standards
Journal has no authorship policy	2	35, 121	Journal Operations	Unethical Research or Publication Practices
Journal offer authors incentives to publish	2	121, 165	Journal Operations	Persuasive Language
Article submission occurs via email	2	35, 299	Journal Operations	Poor Quality Standards
Journal has hidden publishing contract information	2	121, 362	Journal Operations	Deceptive or Lacking Transparency
Journals have the goal to make money without regard for quality	2	121, 462	Journal Operations	Deceptive or Lacking Transparency; Poor Quality Standards
Journals solicit papers under false pretenses	2	121, 489	Journal Operations	Deceptive or Lacking Transparency
Journal contains duplicate publications	2	121, 561	Journal Operations	Unethical Research or Publication Practices
Journals do not mention reporting guidelines	2	299, 384	Journal Operations	Poor Quality Standards
Journals tend not to have legitimate impact factor	2	299, 904	Journal Operations	Deceptive or Lacking Transparency
Description of the manuscript handling process is lacking	2	299, 1012	Journal Operations	Deceptive or Lacking Transparency; Poor Quality Standards
Journal has no plagiarism policy/duplicate publication policy	3	35, 121, 299	Journal Operations	Unethical Research or Publication Practices
Journals do not have retraction/correction policies	3	299, 489, 660	Journal Operations	Unethical research or publication practices
Journal publishes studies without authors’ agreement	4	121, 150, 489, 660	Journal Operations	Unethical Research or Publication Practices
Journal names specify 'worldly' or 'global' nature of journal	4	275, 362, 812, 1068	Journal Operations	Persuasive Language
Journals contact information is not professional (e.g., Gmail accounts)	4	299, 812, 904, 1068	Journal Operations	Poor Quality Standards
Journals display fake metrics	4	13,275,299,812	Journal Operations	Deceptive or Lacking Transparency
Present content unrelated to the journal readership/scope/ journal title	5	1, 121, 150, 275, 299	Journal Operations	Poor Quality Standards
Journal contains broken links/domain for sale	5	35, 168, 299, 586, 755	Journal Operations	Poor Quality Standards
Journals have short/rapid publication times	7	150, 548,586,596, 812, 975, 1068	Journal Operations	NA
Journals do not contain any articles	8	1, 35, 99, 168, 209, 299, 548, 586, 755	Journal Operations	Poor Quality Standards
Journal lists few articles	8	35, 99, 150, 168, 362, 900, 975, 976	Journal Operations	NA
Journals closely copy/plagiarize names or websites of legitimate journals/publishers	8	1,18,165,299,548,736,812,904	Journal Operations	Deceptive or Lacking Transparency
Journals display deceptive information or misleading claims about their practices	8	121,165,489,736,755,812,904, 1068	Journal Operations	Deceptive or Lacking Transparency
Contact details of publisher absent or not easily verified	11	35, 99, 121, 168, 209, 299, 362, 489, 755, 812, 904	Journal Operations	Deceptive or Lacking Transparency
Journals display low levels of transparency, integrity, poor quality practices of journal operations	14	8, 121, 203, 275, 299, 362, 384, 728, 736, 755, 812, 904, 1012, 1068	Journal Operations	Poor Quality Standards; Deceptive or Lacking Transparency
Journals are published by/in predominantly by authors from specific countries	10	1,8, 121, 209, 299, 755, 812, 900, 975, 976	Journal Operations; Article	NA

Journal Operations. Predatory journal operations were described as: being deceptive or lacking transparency (19 statements), demonstrating poor quality standards (17 statements), demonstrating unethical research or publication practices (14 statements), using persuasive language (two statements). Five statements were neutral or non-descriptive. The most common characteristics of the journal operations category were “Journals display low levels of transparency, integrity, poor quality practices of journal operations” (N=14 articles); “Contact details of publisher absent or not easily verified” (N=11 articles); and “Journals are published by/in predominantly by authors from specific countries” (N=10 articles).

Article. Articles in predatory journals were described as: demonstrating poor quality standards (six statements), demonstrating high quality standards (two statements), being deceptive or lacking transparency (three statements), and demonstrating unethical research of publication practices (three statements). Four statements were neutral or non-descriptive. The most common characteristics of the article category were: “Journals are published by/in predominantly by authors from specific countries” (N=10 articles); “Quality of articles rated as poor” (N=5 articles); and “Articles are poorly cited” (N=5 articles).

Editorial and Peer Review. The editorial and peer review process was described as: demonstrating unethical or research practices (eight statements), being deceptive or lacking transparency (seven statements), demonstrating poor quality standards (five statements), demonstrating high quality standards (two statements), and using persuasive language (one statement). Two statements were neutral or non-descriptive. The most common characteristics of the editorial and peer review category were: “Journals conduct poor quality peer review” (N=8 articles) and “Journals have short peer review times”; “Editorial board is not stated or incomplete”; “Editorial broad lacks legitimacy (appointed without knowledge, wrong skillset)” (N=7 articles each).

Communication. Communication by predatory journals was described as: using persuasive language (12 statements), demonstrating poor quality standards (four statements), being deceptive or lacking transparency (four statements), and demonstrating high quality standards (one statement). All communication statements were descriptive. The most common characteristic of the communications category was: “Journals solicit papers via aggressive e-mail tactics” (N=13 articles).

Article Processing Charges. Article processing charges in predatory journals were described as: being deceptive or lacking transparency (three statements), using persuasive language (two statements), demonstrating poor quality standards (one statement), demonstrating unethical research or publication practices (one statement), and demonstrating high quality standards (one statement). Two statements were neutral or non-descriptive. The most common characteristics of the article processing charges category were: “APCs are lower than at legitimate journals”; “Journal does not specify APCs”; and “Journal has hidden APCs or hidden information on APCs” (N=9 articles each).

Dissemination, Indexing, and Archiving. Dissemination, indexing, and archiving were described as: demonstrating poor quality standards (five statements), demonstrating unethical research or publication practices (one statement), and as being deceptive or lacking transparency (one statement). Seven statements were neutral or non-descriptive. The most common characteristics of the dissemination, indexing, and archiving category were: “Journals state they are open access” (N=11 articles); “Journal may be listed in DOAJ” (N=8 articles); and “Journals are not indexed” (N=7 articles).

Discussion

This scoping review identified 334 articles mentioning predatory journals, with corresponding authors from more than 40 countries. The trajectory of articles on this topic is increasing rapidly. As an example, our search captured five articles from 2012 and 140 articles from 2017. The majority of articles captured took the form of a commentary, editorial or letter; just 38 had relevant empirically derived characteristics of predatory journals. One possibility for why there is little empirical work on this topic may be that most funding agencies have not set aside funding for journalology or a related field of enquiry–research on research. There are recent exceptions to this²⁴, but in general such funds are not widely available. Of the 38 studies from which we extracted data, post-hoc we examined the percentage that reported funding, and found that just 13.16% (5/38) did, 21.05% (8/38) did not, and 65.79% (25/38) did not report information on funding. Even among the five studies that reported funding, several of these were not project funding specific to the research, but rather broader university chair or fellowship support.

A total of 109 unique characteristics were extracted from the 38 empirical articles. When examining these unique characteristics some clear contrasts emerge. For example, we extracted the characteristic “Journal APCs clearly stated” (N = 4 articles) as well as the characteristics “Journal does not specify APCs” (N = 9 articles) and “Journal has hidden APCs or hidden information on APCs” (N = 9 articles). Potential inconsistencies of the importance of epidemiological characteristics will make it difficult to define predatory journals. Without a (consensus) definition it will be difficult to study the construct in a meaningful manner. It also makes policy initiatives and educational outreach imprecise and potentially less effective.

We believe a cogent next move is to invite a broad spectrum of stakeholders to a summit. Possible objectives could be to develop a consensus definition of a predatory journal, discuss how best to examine the longitudinal impact of predatory journals, and develop collaborative policy and educational outreach to minimize the impact of predatory publishers on the research community. As a starting point for defining predatory journals, those involved in a global stakeholder meeting to establish a definition for predatory journals may wish to exclude all characteristics that are common to legitimate journals. Further, one could exclude all characteristics that are conflicting, or which directly oppose one another. Another fruitful approach may be to focus on characteristics that can easily be audited to determine if journals do or do not meet the expected standards.

The unique characteristics we extracted were thematically grouped into six categories and five descriptors. Although we did identify one positive descriptor, high quality standards, the majority of descriptors were negative. Most categories (all but ‘Communication’) also included neutral or non-descriptive statements. The presence of both positive and neutral descriptors points to an overlap between characteristics that describe predatory journals and those that are viewed as ‘legitimate’, further emphasizing the challenges in defining predatory journals. The category with the most statements was ‘Journal Operations’ with 19 statements describing operations as deceptive or lacking transparency. The ‘Communication’ category had the most statements described as persuasive (11 statements), highlighting the targeted language predatory journals may use to convince the reader toward a certain action. Unethical or unprofessional publication practices described statements in all but the ‘Communication’ category and were most frequent in ‘Journal Operations’ and ‘Editorial and Peer Review’. These findings point to issues of great concern in research and publishing and an urgency to develop interventions and education to protect researchers, funders, and knowledge users.

There are a number of relevant limitations of this work that should be acknowledged. Firstly, while we endeavoured to ensure our systematic search and grey literature appraisal was comprehensive, it is possible that we missed some relevant documents that would have contributed additional empirically derived characteristics of predatory journals. As an example, several authors of this manuscript recently published a paper containing relevant empirical data and predatory characteristics²; however, because this work was published in a commentary format, which did not include an abstract or use the search terms in the article title, it was not picked up in our search. Indeed, part of the challenge of systematically searching on this topic is the lack of agreement and diversity of terms used to describe predatory journals. Further, reviewers deciding which articles to include based on our inclusion criteria had to make judgements on study designs and methods used. Due to inconsistent reporting and terminology, this was not always straightforward and may have resulted in inadvertent exclusions. Secondly, in keeping with accepted scoping review methodology, we did not appraise the methodological quality of the articles that were included in our extraction. This means that the characteristics extracted have not been considered in context to the study design or methodological rigour of the work. In addition, we only extracted definitions from empirical studies describing characteristics of predatory journals. It is possible that further characteristics would have been included in our results if non-empirical research articles were not excluded. We chose to exclude these types of articles as they are more likely to be based on opinion or individual experience rather than evidence. Thirdly, our focus was on the biomedical literature. Whether the publication (e.g., having an IMRAD (Introduction Methods Results And Discussion) and peer review norms we’ve used apply across other disciplines is likely an important topic for further investigation. Fourthly, some of the studies included in our review are confounded by being identified through Beall’s lists, and journal publisher websites, which are considered controversial. Finally, we limited our study to English articles. It is possible that work published in other languages may have provided additional characteristics of predatory journals.

Reaching a consensus on what defines predatory journals, and what features reflect these, may be particularly useful to stakeholders (e.g., funders, research institutions) with a goal of establishing a list of vetted journals to recommend to their researchers. Such lists could be updated annually. Lists which attempt to curate predatory journals rather than legitimate journals are unlikely to achieve success given the reactive nature of this type of curation and the issue that new journals cannot easily be systematically discovered for evaluation²⁵. The development and use of digital technologies to provide information about journal publication practices (e.g., membership in the Committee on Publication Ethics (https://publicationethics.org/), listing in the Directory of Open Access Journals (https://doaj.org/)) may also prove to be a fruitful approach in reducing researchers’ submissions to predatory journals; empowering authors with knowledge is an important step in decision-making. Currently, researchers receive little education or support about navigating journal selection and submission processes. We envision a plug-in tool that researchers could click to get immediate feedback about a journal page they are visiting and whether it has characteristics of predatory journals. This feedback could provide them with the relevant information to determine if the journal suits their needs and/or meets any policy requirements to which they must adhere (e.g., digital preservation, indexing).

Data availability

Study data and tables are available on the Open Science Framework, see: https://osf.io/4zm3t/.

Data are available under the terms of the Creative Commons Attribution 4.0 International license (CC-BY 4.0).

Grant information

The authors declared that no grants were involved in supporting this work. DM is funded by a University Research Chair. MML is supported by The Ottawa Hospital Anesthesia Alternate Funds Association and the Scholarship Protected Time Program, Department of Anesthesiology and Pain Medicine, uOttawa.

The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.

Acknowledgements

We are grateful to Raymond Daniel (Knowledge Synthesis Group, Ottawa Hospital Research Institute) who assisted with the acquisition and import of study files into the DSR platform.

Supplementary material

Supplementary file 1: Search Strategy

Click here to access the data.

Supplementary file 2: Full citations of included articles.

Click here to access the data.

Faculty Opinions recommended

References

1. Beall J: Predatory publishers are corrupting open access. Nature. 2012; 489(7415): 179. PubMed Abstract | Publisher Full Text
2. Moher D, Shamseer L, Cobey KD, et al.: Stop this waste of people, animals and money. Nature. 2017; 549(7670): 23–5. PubMed Abstract | Publisher Full Text
3. Lalu M, Shamseer L, Cobey KD, et al.: How stakeholders can respond to the rise of predatory journals. Nat Hum Behav. 2017; 1: 852–5. Publisher Full Text
4. Clark J, Smith R: Firm action needed on predatory journals. BMJ. 2015; 350(1): h210. PubMed Abstract | Publisher Full Text
5. Bartholomew RE: Science for sale: the rise of predatory journals. J R Soc Med. 2014; 107(10): 384–385. PubMed Abstract | Publisher Full Text | Free Full Text
6. Smart P: Predatory journals and researcher needs. Learn Publ. 2017; 30(2): 103–5. Publisher Full Text
7. Bjørnshauge L: Think Check Submit. In: OASPA Conference. 2015. Reference Source
8. Shen C, Björk BC: 'Predatory' open access: a longitudinal study of article volumes and market characteristics. BMC Med. 2015; 13: 230. PubMed Abstract | Publisher Full Text | Free Full Text
9. Cobey K: Illegitimate journals scam even senior scientists. Nature. 2017; 549(7670): 7. PubMed Abstract | Publisher Full Text
10. Kolata G: Many Academics Are Eager to Publish in Worthless Journals. New York Times. 2017; 1–4. Reference Source
11. Berger M: Everything you ever wanted to know about predatory publishing but were afraid to ask. ACRL. 2017; 206–7. Reference Source
12. Wager E: Why we should worry less about predatory publishers and more about the quality of research and training at our academic institutions. J Epidemiol. 2017; 27(3): 87–8. PubMed Abstract | Publisher Full Text | Free Full Text
13. Anderson R: Should we retire the term “predatory publishing”? Scholarly Kitchen. Accessed August 9, 2018. 2015. Reference Source
14. Shamseer L, Moher D: Thirteen ways to spot a ‘predatory journal’ (and why we shouldn’t call them that). Times Higher Education.2017. Reference Source
15. Eriksson S, Helgesson G: Time to stop talking about ‘predatory journals.’ Learn Publ. 2018; 31(2): 181–3. Publisher Full Text
16. Moher D, Moher E: Stop Predatory Publishers Now: Act Collaboratively. Ann Intern Med. 2016; 164(9): 616–7. PubMed Abstract | Publisher Full Text
17. Butler D: Investigating journals: The dark side of publishing. Nature. 2013; 495(7442): 433–5. PubMed Abstract | Publisher Full Text
18. Arksey H, O'Malley L: Scoping studies: Towards a methodological framework. Int J Soc Res Methodol. 2005; 8(1): 19–32. Publisher Full Text
19. Levac D, Colquhoun H, O'Brien KK: Scoping studies: advancing the methodology. Implement Sci. 2010; 5: 69. PubMed Abstract | Publisher Full Text | Free Full Text
20. Peters MDJ, Godfrey C, McInerney P, et al.: Chapter 11: Scoping Reviews. In: Aromataris E, Munn Z, (Editors). Joanna Briggs Institute Reviewer's Manual. The Joanna Briggs Institute, 2017. Reference Source
21. Moher D, Liberati A, Tetzlaff J, et al.: Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement. PLoS Med. 2009; 6(7): e1000097. PubMed Abstract | Publisher Full Text | Free Full Text
22. Mcgowan J, Sampson M, Salzwedel DM, et al.: PRESS Peer Review of Electronic Search Strategies: 2015 Guideline Statement. J Clin Epidemiol. 2016; 75: 40–46. PubMed Abstract | Publisher Full Text
23. Galipeau J, Barbour V, Baskin P, et al.: A scoping review of competencies for scientific editors of biomedical journals. BMC Med. 2016; 14(1): 16. PubMed Abstract | Publisher Full Text | Free Full Text
24. Matthews D: Netherlands to survey every researchers on misconduct. Times Higher Education. 2016. Reference Source
25. Patwardhan B, Nagarkar S, Gadre SR, et al.: A critical analysis of the ‘UGC-approved list of journals’. Curr Sci. 2018; 114(6): 1299–1303. Publisher Full Text

Comments on this article Comments (3)

Version 2

VERSION 2 PUBLISHED 23 Aug 2018

Revised

Comment

Version 1

VERSION 1 PUBLISHED 04 Jul 2018

Discussion is closed on this version, please comment on the latest version above.

Reader Comment 19 Jul 2018

Valerie Matarese, Authors' editor and editorial consultant, Vidor, Italy

19 Jul 2018

Reader Comment

The definition of predatory journals attributed, in the Introduction, to Beall (Nature 2012) is imprecise. In that article, Beall defined predatory publishers, not journals, and he used the term "predatory ... Continue reading The definition of predatory journals attributed, in the Introduction, to Beall (Nature 2012) is imprecise. In that article, Beall defined predatory publishers, not journals, and he used the term "predatory journal" just once.

The correct quotation is "... predatory publishers ... publish counterfeit journals to exploit the open-access model in which the author pays. These predatory publishers are dishonest and lack transparency." This is a single definition, not "definitions" as suggested here.

Given that the purpose of this new article is to define predatory journals, it is important that the starting point be correct. I hope this change will be made in the revised version.
The definition of predatory journals attributed, in the Introduction, to Beall (Nature 2012) is imprecise. In that article, Beall defined predatory publishers, not journals, and he used the term "predatory journal" just once.

The correct quotation is "... predatory publishers ... publish counterfeit journals to exploit the open-access model in which the author pays. These predatory publishers are dishonest and lack transparency." This is a single definition, not "definitions" as suggested here.

Given that the purpose of this new article is to define predatory journals, it is important that the starting point be correct. I hope this change will be made in the revised version.
Competing Interests: No competing interests were disclosed. Close
Report a concern
Reader Comment 13 Jul 2018

Ross Mounce, University of Cambridge, UK

13 Jul 2018

Reader Comment

This is a literature review of opinion, and as such, one wonders what the value of the exercise is. Surely it would be better to examine the phenomenon itself, not ... Continue reading This is a literature review of opinion, and as such, one wonders what the value of the exercise is. Surely it would be better to examine the phenomenon itself, not just a synthesis of many studies of deeply variable quality about it?

One should not just treat all papers equally. The quality of evidence offered by some of the included 38 papers (Supp file 2) is extremely poor if one actually reads them. Yet they appear to be all equally weighted in terms of their evidentiary contribution. It would be tremendously interesting to examine how many of the 38 papers were actually peer-reviewed - some clearly are just correspondances and editorials. Others such Sorowski et al. (2017) are published at a well-establised commercial journal in a “commission-only section” (source: https://www.nature.com/nature/for-authors/other-subs) where the publisher has a clear commercial conflict-of-interest in sowing fear about publishing in less well known journals and has published a whole series of commissioned, not-peer-reviewed, pieces doing just that.

So let’s have a close look at some of the 38 studies included in this paper:

E.g. in “1. [RefID: 1]. Oermann MH, Nicoll LH, Chinn PL, et al. Quality of articles published in predatory nursing journals. Nurs Outlook 2018 Jan;66(1):4-10. “

In which it is written: “This constellation of problems has led to the identification of predatory journals and predatory publishers (International Academy of Nursing Editors [INANE] Predatory Publishing Collaborative, 2014). Greco (2016) reported that predatory publishers made approximately $75 million US dollars in 2015, mainly through the collection of APCs.” (emphasis mine)

The claim that predatory publishers have made approximately $75 million US dollars in 2015 (alone) simply isn’t credible but it is indicative of the kind of wild extrapolation and rumour that is undertaken when some people write about predatory publishing. The figure of 75 million I presume derives from Shen & Bjork (2015) in which they write “Using our [estimated] data for the number of articles and average APC for 2014, our estimate for the size of the market is 74 million USD”. This is an estimate from extrapolation and in no way indicative of 74 million USD actually being paid out to predatory publishers - that distinction is subtle but important. Shen & Bjork (2015) did not claim that publishers made approximately $75 million US dollars in 2015, yet through poor peer review at “non-predatory” journals the trumped-up form of the claim still enters the literature and is misleadingly repeated in headlines e.g. “Predatory publishers earned $75 million last year, study finds” (Bohannon 2015).

Shen & Bjork (2015) itself was in many ways a good, well-reported study, with transparent open peer review, but even here if one digs into the details there are problems. The most notable is that for their definition of predatory publishers and inclusion in the study, Shen & Bjork (2015) used Beall’s List at a particular point in time when the publisher MDPI (http://www.mdpi.com/) was on the list. In subsequent years (MDPI was removed from Mr. Beall's list on 28 October 2015), the world and Beall himself realised that MDPI was not a predatory publisher, despite one or two papers with peer review problems - most journals have famous ‘clangers’ even Nature, Science and PNAS. MDPI unfortunately happens to be a large publisher in terms of article volume with ‘high’ APCs relative to publishers that are genuinely are ‘predatory’ by most people’s understanding. This therefore explains well how Shen & Bjork (2015) came to their astoundingly high 74 million USD estimate (which included MDPI) but similarly shows that the study is flawed because of that and needs to be revisited with MDPI and other now understood not-to-be-predatory publishers taken out of the analysis. Specific differences in minutiae like this between the 38 studies might make each of them completely incomparable to each other! More detailed work is needed to assess comparability and quality of evidence before a synthesis is made, otherwise the analysis is garbage-in, garbage-out.

The irony is that much of the literature writing about predatory publishing is of itself also poor quality, as the examples demonstrate. The issue at hand here is poor or non-existent peer review and unfortunately this is rather more widespread than many would like to admit.

There also seems to be no acknowledgement whatsoever in this manuscript that ‘predatory journals’ and Jeffrey Beall are controversial subjects.

Beall’s views (e.g. Esposito, 2013; Bivens-Tatum 2014; Reyes-Galindo 2015; Velterop 2015; Houghton 2017), his website and list of journals (e.g. Bloudoff-Indelicato, 2015; Silver, 2017), are highly controversial.

If one is to write a manuscript on this topic, one must acknowledge upfront that Jeffrey Beall has a “distasteful political ideology” (Esposito, 2013) which this manuscript is in danger of promoting if the authors do not take steps to carefully consider the framing of this research.

The framing of the phenomena as “predatory journals” throughout the manuscript is misjudged and imprecise use of language. For a start, ‘journal’ is simply the wrong unit of assessment. To demonstrate, imagine this hypothetical scenario:

A legitimate journal has been publishing articles legitimately with high editorial and peer review standards for 5 years. The ownership and management of the journal changes and suddenly all articles in the 6th year of publication are low quality and simply haven’t been peer reviewed (despite the journal overtly claiming them to be peer-reviewed). The content of the 6th year is perhaps to be considered ‘predatory’ under Beall’s choice of naming system [which for the record I don’t really endorse], whereas the content published in the first 5 years is still legitimate. It would be absurd to call the entirety of journal ‘predatory’ – it would besmirch the first 5 years of legitimate research published there. It is only part of the content (the 6th year) that is deceptively published.

‘Predatory publishing’ as the phenomena is more widely known, is a (lack of) process and as such it can apply to individual articles, issues of journals, whole journals, or whole publishers of many journals, as well as to academic book publishing too. The narrowness of framing as “predatory journals” just isn’t appropriate if one really wants to understand the phenomena. If one wanted to scope it only to examine predatory publishing in journals that’s fine but it’s ‘predatory publishing _in_ journals’, not ‘predatory journals’.

There are also numerous problems with the search strategy employed in this manuscript. It is acknowledged that there are other ways to denote predatory publishing:

“Other terms to denote predatory journals such as “illegitimate journals”, “deceptive journals”, “dark” journals, and “journals operating in bad faith” have appeared in the literature”

But the manuscript has missed a major alternative synonym for the same phenomena and not included it in the search strategy: in China the problem is known as “Trash journals” (Hrvistendahl 2011 ; Lin & Zhan, 2014 ; Jones 2015 ; Ren and Montgomery 2015). The word “trash” is not included in the search strategy which might mean Chinese opinions on the phenomenon are missed. Relatedly the arbitrary starting point of publications from 2012 onwards, clearly excludes earlier relevant literature such as Hrvistendahl (2011).

It is generally accepted that the Global North isn’t as affected by predatory publishing (Jones 2015; ““I get these emails every day but I don’t know a single academic who would fall for these obviously fake journals. I only publish in journals that I know about. Honestly, nobody would fall for this, it’s not a problem for me.”), instead it is mainly authors from the Global South that are more detectably caught-up in the problem for a variety of reasons (Xia 2015).

Why then does this manuscript specifically exclude opinions on the phenomenon that are not written in English? The problem is most detectable in the Global South (Xia 2015) and thus in order to have true understanding of it one must include non-Anglophone contributions on the matter. It is tremendously problematic and compounds the problem the authors are studying to be monolingual in focus - the phenomenon exists at least partially because of the feeling of exclusion in the Global South by gatekeeping of Global North journals.

Although because it is recently published it falls outside of the search strategies applied, I would strongly suggest the manuscript authors read and cite Bell et al. (2018) “Predatory Publishing” and Eve & Priego (2017) “Who is Actually Harmed by Predatory Publishers?” to provide some more balance in this discussion.

As it stands, the manuscript is significantly methodologically flawed and needs serious reconsideration.

Azeez, Abdul E.P. 2017. “Academic Writing and Publishing in India: Is Quality a Touchstone?” Journal of Community Positive Practices 17, no. 1: 13–24.
Anderson, Rick. 2015. “Should We Retire the Term Predatory Publishing?” The Scholarly Kitchen, May 11. https://scholarlykitchen.sspnet.org/2015/05/11/should-we-retire-the-term-predatory-publishing/
Bohannon, J. 2015. Predatory publishers earned $75 million last year, study finds. Science http://www.sciencemag.org/news/2015/09/predatory-publishers-earned-75-million-last-year-study-finds
Bivens-Tatum, Wayne 2014. “Reactionary Rhetoric Against Open Access Publishing”
Bloudoff-Indelicato, Mollie 2015. “Backlash after Frontiers journals added to list of questionable publishers” https://www.nature.com/news/backlash-after-frontiers-journals-added-to-list-of-questionable-publishers-1.18639
Crawford, Walt. 2014. Ethics and Access 1: The Sad Case of Jeffrey Beall. Cites & Insights 14 (4): 1-14. https://citesandinsights.info/civ14i4.pdf
Esposito, Joe 2013. “Parting Company with Jeffrey Beall” https://scholarlykitchen.sspnet.org/2013/12/16/parting-company-with-jeffrey-beall/
Eve, Martin and Priego, Ernesto. 2017. “Who is Actually Harmed by Predatory Publishers?” TripleC 15, no. 2: 755–70. https://doi.org/10.31269/triplec.v15i2.867
Houghton, F. (2017). Ethics in academic publishing: a timely reminder. Journal of the Medical Library Association: JMLA, 105(3), 282. http://jmla.mlanet.org/ojs/jmla/article/view/122/415
Hrvistendahl, M. 2011. China Looks to Purge Academia of 'Trash Journals' http://doi.org/10.1126/science.334.6054.301
Jones, Phil 2015. “Predatory Publishing Isn’t The Problem, It’s a Symptom of Information Inequality” https://www.digital-science.com/blog/perspectives/predatory-publishing-isnt-the-problem-its-a-symptom-of-information-inequality/
Lin, S. 2013. Why serious academic fraud occurs in China. Learned Publishing, 26(1): 24–27. http://doi.org/10.1087/20130105
Lin, S. and Zhan, L. 2014. Trash journals in China. Learned Publishing vol 27 issue 2 http://doi.org/10.1087/20140208
Kirsten Bell, Jill Claassen, Lena Nyahodza, Reggie Raju, Vaclav Stetka 2018. Predatory Publishing http://dx.doi.org/10.17613/M6N58CK3D
Ren and Montgomery 2015. Open access and soft power: Chinese voices in international scholarship. Media, Culture & Society https://doi.org/10.1177/0163443714567019
Ren, Xiang 2013. Open and Networked Initiatives and the Digital Transformation of Academic Publishing in China [Phd Thesis] https://eprints.qut.edu.au/63337/1/Xiang_Ren_Thesis.pdf
Reyes-Galindo, Luis 2015. “On SciELO and RedALyC” https://blogs.cardiff.ac.uk/luisreyes/on-scielo-and-redalyc/
Silver, Andrew 2017. “Controversial website that lists ‘predatory’ publishers shuts down” https://www.nature.com/news/controversial-website-that-lists-predatory-publishers-shuts-down-1.21328
Sorokowski P, Kulczycki E, Sorokowska A, et al. Predatory journals recruit fake editor. Nature 2017;543(7646):481-3.Sorokowski P, Kulczycki E, Sorokowska A, et al. Predatory journals recruit fake editor. Nature 2017;543(7646):481-3.
Velterop, Jan 2015. “The fenced-off ‘nice’ publication neighbourhoods of Jeffrey Beall”
https://blog.scielo.org/en/2015/08/01/the-fenced-off-nice-publication-neighbourhoods-of-jeffrey-beall/#.W0S3T9JKhhE
Xia, J. 2014. An imbalanced journal publishing market. Learned Publishing. http://doi.org/10.1087/20140309
Xia, J. et al. 2015. Who publishes in “predatory” journals? Journal of the Association for Information Science and Technology, 66(7), 1406–1417. http://doi.org/10.1002/asi.23265
This is a literature review of opinion, and as such, one wonders what the value of the exercise is. Surely it would be better to examine the phenomenon itself, not just a synthesis of many studies of deeply variable quality about it?

One should not just treat all papers equally. The quality of evidence offered by some of the included 38 papers (Supp file 2) is extremely poor if one actually reads them. Yet they appear to be all equally weighted in terms of their evidentiary contribution. It would be tremendously interesting to examine how many of the 38 papers were actually peer-reviewed - some clearly are just correspondances and editorials. Others such Sorowski et al. (2017) are published at a well-establised commercial journal in a “commission-only section” (source: https://www.nature.com/nature/for-authors/other-subs) where the publisher has a clear commercial conflict-of-interest in sowing fear about publishing in less well known journals and has published a whole series of commissioned, not-peer-reviewed, pieces doing just that.

So let’s have a close look at some of the 38 studies included in this paper:

E.g. in “1. [RefID: 1]. Oermann MH, Nicoll LH, Chinn PL, et al. Quality of articles published in predatory nursing journals. Nurs Outlook 2018 Jan;66(1):4-10. “

In which it is written: “This constellation of problems has led to the identification of predatory journals and predatory publishers (International Academy of Nursing Editors [INANE] Predatory Publishing Collaborative, 2014). Greco (2016) reported that predatory publishers made approximately $75 million US dollars in 2015, mainly through the collection of APCs.” (emphasis mine)

The claim that predatory publishers have made approximately $75 million US dollars in 2015 (alone) simply isn’t credible but it is indicative of the kind of wild extrapolation and rumour that is undertaken when some people write about predatory publishing. The figure of 75 million I presume derives from Shen & Bjork (2015) in which they write “Using our [estimated] data for the number of articles and average APC for 2014, our estimate for the size of the market is 74 million USD”. This is an estimate from extrapolation and in no way indicative of 74 million USD actually being paid out to predatory publishers - that distinction is subtle but important. Shen & Bjork (2015) did not claim that publishers made approximately $75 million US dollars in 2015, yet through poor peer review at “non-predatory” journals the trumped-up form of the claim still enters the literature and is misleadingly repeated in headlines e.g. “Predatory publishers earned $75 million last year, study finds” (Bohannon 2015).

Shen & Bjork (2015) itself was in many ways a good, well-reported study, with transparent open peer review, but even here if one digs into the details there are problems. The most notable is that for their definition of predatory publishers and inclusion in the study, Shen & Bjork (2015) used Beall’s List at a particular point in time when the publisher MDPI (http://www.mdpi.com/) was on the list. In subsequent years (MDPI was removed from Mr. Beall's list on 28 October 2015), the world and Beall himself realised that MDPI was not a predatory publisher, despite one or two papers with peer review problems - most journals have famous ‘clangers’ even Nature, Science and PNAS. MDPI unfortunately happens to be a large publisher in terms of article volume with ‘high’ APCs relative to publishers that are genuinely are ‘predatory’ by most people’s understanding. This therefore explains well how Shen & Bjork (2015) came to their astoundingly high 74 million USD estimate (which included MDPI) but similarly shows that the study is flawed because of that and needs to be revisited with MDPI and other now understood not-to-be-predatory publishers taken out of the analysis. Specific differences in minutiae like this between the 38 studies might make each of them completely incomparable to each other! More detailed work is needed to assess comparability and quality of evidence before a synthesis is made, otherwise the analysis is garbage-in, garbage-out.

The irony is that much of the literature writing about predatory publishing is of itself also poor quality, as the examples demonstrate. The issue at hand here is poor or non-existent peer review and unfortunately this is rather more widespread than many would like to admit.

There also seems to be no acknowledgement whatsoever in this manuscript that ‘predatory journals’ and Jeffrey Beall are controversial subjects.

Beall’s views (e.g. Esposito, 2013; Bivens-Tatum 2014; Reyes-Galindo 2015; Velterop 2015; Houghton 2017), his website and list of journals (e.g. Bloudoff-Indelicato, 2015; Silver, 2017), are highly controversial.

If one is to write a manuscript on this topic, one must acknowledge upfront that Jeffrey Beall has a “distasteful political ideology” (Esposito, 2013) which this manuscript is in danger of promoting if the authors do not take steps to carefully consider the framing of this research.

The framing of the phenomena as “predatory journals” throughout the manuscript is misjudged and imprecise use of language. For a start, ‘journal’ is simply the wrong unit of assessment. To demonstrate, imagine this hypothetical scenario:

A legitimate journal has been publishing articles legitimately with high editorial and peer review standards for 5 years. The ownership and management of the journal changes and suddenly all articles in the 6th year of publication are low quality and simply haven’t been peer reviewed (despite the journal overtly claiming them to be peer-reviewed). The content of the 6th year is perhaps to be considered ‘predatory’ under Beall’s choice of naming system [which for the record I don’t really endorse], whereas the content published in the first 5 years is still legitimate. It would be absurd to call the entirety of journal ‘predatory’ – it would besmirch the first 5 years of legitimate research published there. It is only part of the content (the 6th year) that is deceptively published.

‘Predatory publishing’ as the phenomena is more widely known, is a (lack of) process and as such it can apply to individual articles, issues of journals, whole journals, or whole publishers of many journals, as well as to academic book publishing too. The narrowness of framing as “predatory journals” just isn’t appropriate if one really wants to understand the phenomena. If one wanted to scope it only to examine predatory publishing in journals that’s fine but it’s ‘predatory publishing _in_ journals’, not ‘predatory journals’.

There are also numerous problems with the search strategy employed in this manuscript. It is acknowledged that there are other ways to denote predatory publishing:

“Other terms to denote predatory journals such as “illegitimate journals”, “deceptive journals”, “dark” journals, and “journals operating in bad faith” have appeared in the literature”

But the manuscript has missed a major alternative synonym for the same phenomena and not included it in the search strategy: in China the problem is known as “Trash journals” (Hrvistendahl 2011 ; Lin & Zhan, 2014 ; Jones 2015 ; Ren and Montgomery 2015). The word “trash” is not included in the search strategy which might mean Chinese opinions on the phenomenon are missed. Relatedly the arbitrary starting point of publications from 2012 onwards, clearly excludes earlier relevant literature such as Hrvistendahl (2011).

It is generally accepted that the Global North isn’t as affected by predatory publishing (Jones 2015; ““I get these emails every day but I don’t know a single academic who would fall for these obviously fake journals. I only publish in journals that I know about. Honestly, nobody would fall for this, it’s not a problem for me.”), instead it is mainly authors from the Global South that are more detectably caught-up in the problem for a variety of reasons (Xia 2015).

Why then does this manuscript specifically exclude opinions on the phenomenon that are not written in English? The problem is most detectable in the Global South (Xia 2015) and thus in order to have true understanding of it one must include non-Anglophone contributions on the matter. It is tremendously problematic and compounds the problem the authors are studying to be monolingual in focus - the phenomenon exists at least partially because of the feeling of exclusion in the Global South by gatekeeping of Global North journals.

Although because it is recently published it falls outside of the search strategies applied, I would strongly suggest the manuscript authors read and cite Bell et al. (2018) “Predatory Publishing” and Eve & Priego (2017) “Who is Actually Harmed by Predatory Publishers?” to provide some more balance in this discussion.

As it stands, the manuscript is significantly methodologically flawed and needs serious reconsideration.

Azeez, Abdul E.P. 2017. “Academic Writing and Publishing in India: Is Quality a Touchstone?” Journal of Community Positive Practices 17, no. 1: 13–24.
Anderson, Rick. 2015. “Should We Retire the Term Predatory Publishing?” The Scholarly Kitchen, May 11. https://scholarlykitchen.sspnet.org/2015/05/11/should-we-retire-the-term-predatory-publishing/
Bohannon, J. 2015. Predatory publishers earned $75 million last year, study finds. Science http://www.sciencemag.org/news/2015/09/predatory-publishers-earned-75-million-last-year-study-finds
Bivens-Tatum, Wayne 2014. “Reactionary Rhetoric Against Open Access Publishing”
Bloudoff-Indelicato, Mollie 2015. “Backlash after Frontiers journals added to list of questionable publishers” https://www.nature.com/news/backlash-after-frontiers-journals-added-to-list-of-questionable-publishers-1.18639
Crawford, Walt. 2014. Ethics and Access 1: The Sad Case of Jeffrey Beall. Cites & Insights 14 (4): 1-14. https://citesandinsights.info/civ14i4.pdf
Esposito, Joe 2013. “Parting Company with Jeffrey Beall” https://scholarlykitchen.sspnet.org/2013/12/16/parting-company-with-jeffrey-beall/
Eve, Martin and Priego, Ernesto. 2017. “Who is Actually Harmed by Predatory Publishers?” TripleC 15, no. 2: 755–70. https://doi.org/10.31269/triplec.v15i2.867
Houghton, F. (2017). Ethics in academic publishing: a timely reminder. Journal of the Medical Library Association: JMLA, 105(3), 282. http://jmla.mlanet.org/ojs/jmla/article/view/122/415
Hrvistendahl, M. 2011. China Looks to Purge Academia of 'Trash Journals' http://doi.org/10.1126/science.334.6054.301
Jones, Phil 2015. “Predatory Publishing Isn’t The Problem, It’s a Symptom of Information Inequality” https://www.digital-science.com/blog/perspectives/predatory-publishing-isnt-the-problem-its-a-symptom-of-information-inequality/
Lin, S. 2013. Why serious academic fraud occurs in China. Learned Publishing, 26(1): 24–27. http://doi.org/10.1087/20130105
Lin, S. and Zhan, L. 2014. Trash journals in China. Learned Publishing vol 27 issue 2 http://doi.org/10.1087/20140208
Kirsten Bell, Jill Claassen, Lena Nyahodza, Reggie Raju, Vaclav Stetka 2018. Predatory Publishing http://dx.doi.org/10.17613/M6N58CK3D
Ren and Montgomery 2015. Open access and soft power: Chinese voices in international scholarship. Media, Culture & Society https://doi.org/10.1177/0163443714567019
Ren, Xiang 2013. Open and Networked Initiatives and the Digital Transformation of Academic Publishing in China [Phd Thesis] https://eprints.qut.edu.au/63337/1/Xiang_Ren_Thesis.pdf
Reyes-Galindo, Luis 2015. “On SciELO and RedALyC” https://blogs.cardiff.ac.uk/luisreyes/on-scielo-and-redalyc/
Silver, Andrew 2017. “Controversial website that lists ‘predatory’ publishers shuts down” https://www.nature.com/news/controversial-website-that-lists-predatory-publishers-shuts-down-1.21328
Sorokowski P, Kulczycki E, Sorokowska A, et al. Predatory journals recruit fake editor. Nature 2017;543(7646):481-3.Sorokowski P, Kulczycki E, Sorokowska A, et al. Predatory journals recruit fake editor. Nature 2017;543(7646):481-3.
Velterop, Jan 2015. “The fenced-off ‘nice’ publication neighbourhoods of Jeffrey Beall”
https://blog.scielo.org/en/2015/08/01/the-fenced-off-nice-publication-neighbourhoods-of-jeffrey-beall/#.W0S3T9JKhhE
Xia, J. 2014. An imbalanced journal publishing market. Learned Publishing. http://doi.org/10.1087/20140309
Xia, J. et al. 2015. Who publishes in “predatory” journals? Journal of the Association for Information Science and Technology, 66(7), 1406–1417. http://doi.org/10.1002/asi.23265
Competing Interests: I am an editor for the MDPI journal called Data (ISSN 2306-5729). I receive no financial compensation for this, it is a voluntary service. Rather being a bias, I consider this a plus as I actually have first-hand experience of the peer review and editorial process at this publisher and I can therefore better attest to it being of acceptable standards. http://www.mdpi.com/journal/data/editors Close
Report a concern
Reader Comment 10 Jul 2018

Edgardo Rolla, Sociedad Argentina de Endometriosis, Argentina

10 Jul 2018

Reader Comment

Interesting article that, somehow tries to help elucidating what is a correct scientific publication and what should be avoided. The proposal is sound, the methodology is correct, and being a ... Continue reading Interesting article that, somehow tries to help elucidating what is a correct scientific publication and what should be avoided. The proposal is sound, the methodology is correct, and being a subjective issue, the discussion/conclusions are acceptable. Publishing in medicine has turned itself a continuous race to be the first one and many times proposed articles do not fulfill what is expected from responsible authors. It is interesting to note that many of the citations are opinion articles. It would make sense if the authors could provide different qualifications for evidence based only publications and those where personal opinion is included. As far as those characteristics are noted in the articles, I would not include as "predatory" opinion based articles or the journals where they are published.
Interesting article that, somehow tries to help elucidating what is a correct scientific publication and what should be avoided. The proposal is sound, the methodology is correct, and being a subjective issue, the discussion/conclusions are acceptable. Publishing in medicine has turned itself a continuous race to be the first one and many times proposed articles do not fulfill what is expected from responsible authors. It is interesting to note that many of the citations are opinion articles. It would make sense if the authors could provide different qualifications for evidence based only publications and those where personal opinion is included. As far as those characteristics are noted in the articles, I would not include as "predatory" opinion based articles or the journals where they are published.
Competing Interests: no conflicts of interest Close
Report a concern
Discussion is closed on this version, please comment on the latest version above.

Author details Author details

Manoj M Lalu
Roles: Conceptualization, Methodology, Writing – Review & Editing

Becky Skidmore
Roles: Conceptualization, Investigation, Methodology, Writing – Review & Editing

Nadera Ahmadzai
Roles: Conceptualization, Data Curation, Formal Analysis, Investigation, Methodology, Writing – Review & Editing

Agnes Grudniewicz
Roles: Data Curation, Formal Analysis, Investigation, Validation, Writing – Review & Editing

David Moher
Roles: Conceptualization, Funding Acquisition, Methodology, Project Administration, Supervision, Writing – Review & Editing

Competing interests

No competing interests were disclosed.

Grant information

Article Versions (2)

version 2

Revised

Published: 23 Aug 2018, 7:1001

https://doi.org/10.12688/f1000research.15256.2

version 1

Published: 04 Jul 2018, 7:1001

https://doi.org/10.12688/f1000research.15256.1

© 2018 Cobey KD et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

Download

Export To

metrics

	Views	Downloads
F1000Research	-	-
PubMed Central Data from PMC are received and updated monthly.	-	-

Citations

SEE MORE DETAILS

CITE

how to cite this article

Cobey KD, Lalu MM, Skidmore B et al. What is a predatory journal? A scoping review [version 2; peer review: 3 approved] F1000Research 2018, 7:1001 (https://doi.org/10.12688/f1000research.15256.2)

NOTE: it is important to ensure the information in square brackets after the title is included in all citations of this article.

track

receive updates on this article

Track an article to receive email alerts on any updates to this article.

Open Peer Review

Current Reviewer Status: ?

Key to Reviewer Statuses VIEW HIDE

ApprovedThe paper is scientifically sound in its current form and only minor, if any, improvements are suggested

Approved with reservations A number of small changes, sometimes more significant revisions are required to address specific details and improve the papers academic merit.

Not approvedFundamental flaws in the paper seriously undermine the findings and conclusions

Version 2

VERSION 2

PUBLISHED 23 Aug 2018

Revised

Views

Reviewer Report 13 Sep 2018

Johann Mouton, Centre for Research on Evaluation, Science and Technology (CREST), DST/NRF Centre of Excellence in Scientometrics and Science, Technology and Innovation Policy, Stellenbosch University, Stellenbosch, South Africa

Approved

https://doi.org/10.5256/f1000research.17518.r37491

It is clear from the response of the authors that they disagree that my criticism is relevant. And therefore they have chosen not to change their paper to accommodate these. In fairness this would have required a substantial rewrite. I believe my comments remain appropriate. However, I have also taken into account the more positive responses from the other reviewers. I have hence decided to agree that the paper can be indexed - even though it makes less of a contribution than what I would have liked.

Competing Interests: No competing interests were disclosed.

I confirm that I have read this submission and believe that I have an appropriate level of expertise to confirm that it is of an acceptable scientific standard.

CITE

Report a concern

Respond or Comment

Version 1

VERSION 1

PUBLISHED 04 Jul 2018

Views

Reviewer Report 13 Aug 2018

Joanna Chataway, SPRU—Science Policy Research Unit, University of Sussex, Brighton, UK

Approved

https://doi.org/10.5256/f1000research.16618.r36291

This is an interesting and very useful article on a subject which, as the authors note, is widely discussed but under researched.

The article sets out to examine data derived from a scoping review in an effort to contribute to a definition of the term 'predatory journal'. Disagreements about whether the term should be used at all are summarised early on in the article and this provides a useful backdrop to the multiple difficulties involved in defining the term.

Questions and issues raised by the article

In considering the issue of predatory journals, the authors raise questions of what might be considered characteristics of a legitimate journal might. The article discusses this only partially and mainly in relation to the difficulties of distinguishing between journals which set out to mislead and which abandon the aims of publish high quality science entirely and, on the other hand, those which are poorly managed and run. It is not necessary I think to give this further and detailed consideration here, but it is important to note that there may well other issues to consider here. For example, there may well be complex relationships between the practices of legitimate journals, and the unintended consequences of impact factor metrics (as noted in The Lancet special issue on 'Increasing value, reducing waste' cited by the authors for example) and the expansion of bad as well as good journals and publication platforms which offer alternatives. The Lancet and other critiques point to intense competition involved in publishing in high impact journals, the need to publish for promotion and employment and so on as factors which drive bad practice in general and may also play a role in the rise of predatory journals.

Another issue which is only briefly mentioned in the article is whether the norms of publishing and peer review differ across different disciplines. Perhaps give the characteristics of existing literature it is not possible to say much about this currently, but the authors could raise more clearly this as an issue to be considered in future research. And I think the point should be made that whilst it is common for health research articles to follow the reporting convention of 'Introduction, methods, results, discussion', this is not the case in other fields. Thus having this as a criteria for judging the quality of a journal could be misleading.

Clarification of terminology

I would encourage the authors to explain terms such as 'epidemiological characteristics' and 'scoping review' which may be familiar to those who work in health research but not perhaps to others.

Some examples?

Some of the results would have been clearer to me if examples had been included. This is particularly the case with regard to 'persuasive language'. It is unclear to me what is being referred to by that term.

Missing link?

I couldn't get the link to further details about the search strategy to work. That accounts for the 'partial' score for source data question but that may just be a problem for me and not for others.

Is the work clearly and accurately presented and does it cite the current literature?

Yes
Is the study design appropriate and is the work technically sound?

Yes
Are sufficient details of methods and analysis provided to allow replication by others?

Yes
If applicable, is the statistical analysis and its interpretation appropriate?

Not applicable
Are all the source data underlying the results available to ensure full reproducibility?

Partly
Are the conclusions drawn adequately supported by the results?

Yes

Competing Interests: No competing interests were disclosed.

Reviewer Expertise: Science policy

I confirm that I have read this submission and believe that I have an appropriate level of expertise to confirm that it is of an acceptable scientific standard.

CITE

Report a concern

Author Response (F1000Research Advisory Board Member) 23 Aug 2018

David Moher, School of Epidemiology and Public Health, Faculty of Medicine, University of Ottawa, Ottawa, K1G 5Z3, Canada

23 Aug 2018

Author Response F1000Research Advisory Board Member
In response to Joanna Chataway’s review:
- We have made some modifications to the limitations section of our paper. We now state (version 2) “Thirdly, our focus was on
... Continue reading
In response to Joanna Chataway’s review:

We have made some modifications to the limitations section of our paper. We now state (version 2) “Thirdly, our focus was on the biomedical literature. Whether the publication (e.g., having an IMRAD (Introduction Methods Results And Discussion) and peer review norms we’ve used apply across other disciplines is likely an important topic for further investigation.”

We have more cleared described scoping reviews (version 2).

We believe we have given some examples in Table 3. For example, in response to the query as to the use and meaning of “persuasive language”, we state (Version 1 and version 2) “Language that targets; Language that attempts to convince the author to do or believe something”.

We have fixed the broken link to the full search strategy (version 2).
In response to Joanna Chataway’s review:

We have made some modifications to the limitations section of our paper. We now state (version 2) “Thirdly, our focus was on the biomedical literature. Whether the publication (e.g., having an IMRAD (Introduction Methods Results And Discussion) and peer review norms we’ve used apply across other disciplines is likely an important topic for further investigation.”

We have more cleared described scoping reviews (version 2).

We believe we have given some examples in Table 3. For example, in response to the query as to the use and meaning of “persuasive language”, we state (Version 1 and version 2) “Language that targets; Language that attempts to convince the author to do or believe something”.

We have fixed the broken link to the full search strategy (version 2).
Competing Interests: We have no competing interests to declare. Close
Report a concern
Respond or Comment

COMMENTS ON THIS REPORT

Author Response (F1000Research Advisory Board Member) 23 Aug 2018

David Moher, School of Epidemiology and Public Health, Faculty of Medicine, University of Ottawa, Ottawa, K1G 5Z3, Canada

23 Aug 2018

Author Response F1000Research Advisory Board Member
In response to Joanna Chataway’s review:
- We have made some modifications to the limitations section of our paper. We now state (version 2) “Thirdly, our focus was on
... Continue reading
In response to Joanna Chataway’s review:

We have made some modifications to the limitations section of our paper. We now state (version 2) “Thirdly, our focus was on the biomedical literature. Whether the publication (e.g., having an IMRAD (Introduction Methods Results And Discussion) and peer review norms we’ve used apply across other disciplines is likely an important topic for further investigation.”

We have more cleared described scoping reviews (version 2).

We believe we have given some examples in Table 3. For example, in response to the query as to the use and meaning of “persuasive language”, we state (Version 1 and version 2) “Language that targets; Language that attempts to convince the author to do or believe something”.

We have fixed the broken link to the full search strategy (version 2).
In response to Joanna Chataway’s review:

We have made some modifications to the limitations section of our paper. We now state (version 2) “Thirdly, our focus was on the biomedical literature. Whether the publication (e.g., having an IMRAD (Introduction Methods Results And Discussion) and peer review norms we’ve used apply across other disciplines is likely an important topic for further investigation.”

We have more cleared described scoping reviews (version 2).

We believe we have given some examples in Table 3. For example, in response to the query as to the use and meaning of “persuasive language”, we state (Version 1 and version 2) “Language that targets; Language that attempts to convince the author to do or believe something”.

We have fixed the broken link to the full search strategy (version 2).
Competing Interests: We have no competing interests to declare. Close
Report a concern

Views

132

Reviewer Report 13 Aug 2018

Not Approved

https://doi.org/10.5256/f1000research.16618.r36292

The paper ultimately promises more than what it delivers. It presents the results of an analysis which has resulted in a set of characteristics of predatory journals derived from a scoping review of recent studies. However, the final discussion section is extremely disappointing. There is no attempt by the authors to add much value to the rather fragmented results found through the review. Part of the problem is that the characteristics listed are treated as equally weighted. Most of the authors who have written on the phenomenon of predatory journals in recent years have attempted to end up with a set of fairly authoritative and even 'objective' criteria that would by themselves be sufficient to classify a journal as predatory. Some of these characteristics would include referencing fake indexing, fake impact metrics, not being indexed in the DOAJ's and a few more. In order to get to a 'consensus' view of what are the key characteristics of a predatory journal, a simple listing of all possible characteristics will not take us much further.
It is perhaps then not surprising that their recommendation is for a consensus type meeting where experts could work towards a consensus definition.

More to the point: in my view to get to the kind of end goal of a consensus or more widely acceptable definition, would require a more theoretical or at least conceptual framework that is embedded in some of the work on scientific communication and publishing which stipulates what good practices in (journal) publishing are.

Unfortunately this paper does not help us much on the way to this goal.

Is the work clearly and accurately presented and does it cite the current literature?

Yes
Is the study design appropriate and is the work technically sound?

Yes
Are sufficient details of methods and analysis provided to allow replication by others?

Yes
If applicable, is the statistical analysis and its interpretation appropriate?

Yes
Are all the source data underlying the results available to ensure full reproducibility?

Yes
Are the conclusions drawn adequately supported by the results?

Partly

Competing Interests: No competing interests were disclosed.

I confirm that I have read this submission and believe that I have an appropriate level of expertise to state that I do not consider it to be of an acceptable scientific standard, for reasons outlined above.

CITE

Report a concern

Author Response (F1000Research Advisory Board Member) 23 Aug 2018

David Moher, School of Epidemiology and Public Health, Faculty of Medicine, University of Ottawa, Ottawa, K1G 5Z3, Canada

23 Aug 2018

Author Response F1000Research Advisory Board Member

We are sorry to disappoint Johann Mouton in our scoping review. We believe that a scoping review is a reasonable way to attempt to map the literature. Scoping reviews do ... Continue reading We are sorry to disappoint Johann Mouton in our scoping review. We believe that a scoping review is a reasonable way to attempt to map the literature. Scoping reviews do not typically weight included studies. We believe our review highlights some of the disagreements in the literature about presumed relevant characteristics of presumed predatory journals.
We are sorry to disappoint Johann Mouton in our scoping review. We believe that a scoping review is a reasonable way to attempt to map the literature. Scoping reviews do not typically weight included studies. We believe our review highlights some of the disagreements in the literature about presumed relevant characteristics of presumed predatory journals.
Competing Interests: We have no competing interests to declare. Close
Report a concern
Respond or Comment

COMMENTS ON THIS REPORT

Author Response (F1000Research Advisory Board Member) 23 Aug 2018

David Moher, School of Epidemiology and Public Health, Faculty of Medicine, University of Ottawa, Ottawa, K1G 5Z3, Canada

23 Aug 2018

Author Response F1000Research Advisory Board Member

We are sorry to disappoint Johann Mouton in our scoping review. We believe that a scoping review is a reasonable way to attempt to map the literature. Scoping reviews do ... Continue reading We are sorry to disappoint Johann Mouton in our scoping review. We believe that a scoping review is a reasonable way to attempt to map the literature. Scoping reviews do not typically weight included studies. We believe our review highlights some of the disagreements in the literature about presumed relevant characteristics of presumed predatory journals.
We are sorry to disappoint Johann Mouton in our scoping review. We believe that a scoping review is a reasonable way to attempt to map the literature. Scoping reviews do not typically weight included studies. We believe our review highlights some of the disagreements in the literature about presumed relevant characteristics of presumed predatory journals.
Competing Interests: We have no competing interests to declare. Close
Report a concern

Views

Reviewer Report 01 Aug 2018

Monica Berger, CUNY New York City College of Technology, Brooklyn, NY, USA

Approved

https://doi.org/10.5256/f1000research.16618.r35740

This article represents a unique contribution to what has been written on this topic.

Although the core readership for F1000Research consists of scientists, I am sure this article will be read by many non-scientists and many of my suggestions to the authors relate to this point.

A general comment: the authors should note that most of the data used for empirical studies of predatory publishing is drawn from Beall's List and Beall's List was and is still controversial [the authors' discussion of Beall's List under Introduction is balanced and articulate]. Essentially, any empirical study of predatory publishing is based on one or two sources of data: Beall's List and/or email solicitations which lead to journals and their publisher websites. This should be made explicit.

The data that underlies much of the literature is very fuzzy and subjective. Without cross-checking publisher and journal data (e.g. many predatory publishers claim inclusion in DOAJ and this data point is particularly sticky), and probing the content, the underlying literature is limited. Moher, David et al's¹ study seems to be one of the only studies to examine content and evaluates the methodological design and research protocols of articles but, as the authors note, it gets excluded because of its publication in commentary format!

The overall quantification of the literature differentiating empirical vs. editorial is extremely helpful.

I found the raw data of characteristics pretty overwhelming and I wonder if the authors could somehow aggregate or otherwise organize the information in a way that makes it easier to scan. I recognize that they have summarized their data in the body of the article.

The conclusions clearly address the limitations of the study but what I think would be most important is teasing out where the data came from: publisher emails leading to publisher websites, and/or Beall's List leading to journals and publisher websites. Both are imperfect sources.

I would like to see this data used again with more aggregation. I recognize that scoping reviews are meant to be fast so this article's data could be used for further research.

Specific comments:
Introduction: Agreed that some journals from the Global South provide important regional research but the authors should note that many of these Global South journals market themselves as "international" or "global" and do not focus on regional research because of a desire to cast a wide net. Legitimate, amateurish journals deemed as predatory from this group actually would be more likely to have a scope that is regional and specific as opposed to the multidisciplinary scope of many predatory publishers.

The authors should explain far more explicitly what a scoping review is and its purpose. Non-biomedical readers will be unfamiliar with this type of methodology/article.

I also am not entirely sure about the use of the word "epidemiological" in terms of discussing the topic at hand: non-biomedical readers may be unsure what exactly is meant.

Lastly, as much as it is very helpful to identify characteristics of predatory journals as drawn from the literature, it seems somewhat positivist to use this very limited body of literature which is by its heavy use of Beall's List data as a means to "generate a consensus definition of predatory journals." Until there is more qualitative research and more multidisciplinary and longitudinal research as was done by Shen and Bjork, there are lacunae in the research literature. The recent articles based on the research by this team is, groundbreaking but largely limits scope to biomedical literature.

Screening and data extraction
The use of implicit and explicit definitions is very important and valuable.

Search strategy
It is possible that some research from librarians and information science scholars might have been missed. There is also some concern that if the articles are open access, they may have not been indexed in traditional databases. This concern relates to the Data Analysis section as well since newer and smaller open access journals may not have a Journal Impact Factor and be excluded from SCIMAGO.

Mapping the data into emergent themes
Under the descriptor "persuasive language," the language of predatory journals targets authors and not readers. This should be explained.

What is somewhat confusing to me is separating characteristics in the literature based on the authors' perceptions or evaluations of the journals and publishers versus the actual data drawn from the journals and publishers' emails, journals, articles, and websites. Whether or not the author of the underlying articles performed cross-checking is also important.

Table 4 Characteristics
The characteristic JOURNALS HAVE SHORT PEER REVIEW TIMES isn't mapped to a descriptor but it is a very most important common characteristic of publisher appeals to authors. It typically maps to Poor Quality Standards although not in an absolute manner since obviously large, quality journals can also have quick turnaround. It is unclear to me if because this characteristic lacks a descriptor, it may lose weight in the analysis. I note that JOURNALS HAVE SHORT/RAPID PUBLICATION TIMES is also a NA descriptor. These two facets are closely related.

ARTICLE SUBMISSION OCCURS VIA EMAIL this may be a signal of poor standards but is often more a reflection of low budgets and the many amateurish journals that have been lumped into Beall's List.

JOURNALS DO NOT CONTAIN ANY ARTICLES the high number of predatory journals without articles is a very important data point that should be emphasized.

Is the work clearly and accurately presented and does it cite the current literature?

Yes
Is the study design appropriate and is the work technically sound?

Yes
Are sufficient details of methods and analysis provided to allow replication by others?

Yes
If applicable, is the statistical analysis and its interpretation appropriate?

I cannot comment. A qualified statistician is required.
Are all the source data underlying the results available to ensure full reproducibility?

Yes
Are the conclusions drawn adequately supported by the results?

Yes

References

1. Moher D, Shamseer L, Cobey K, Lalu M, et al.: Stop this waste of people, animals and money. Nature. 2017; 549 (7670): 23-25 Publisher Full Text

Competing Interests: No competing interests were disclosed.

Reviewer Expertise: My knowledge as a scholarly communications librarian who has devoted considerable effort to writing about the topic at hand is extensive but I am not a trained researcher in either the sciences or social sciences so my ability to genuinely judge the methodology is limited.

I confirm that I have read this submission and believe that I have an appropriate level of expertise to confirm that it is of an acceptable scientific standard.

CITE

Report a concern

Author Response (F1000Research Advisory Board Member) 23 Aug 2018

David Moher, School of Epidemiology and Public Health, Faculty of Medicine, University of Ottawa, Ottawa, K1G 5Z3, Canada

23 Aug 2018

Author Response F1000Research Advisory Board Member
We thank Monica Berger for her thoughtful peer review of our manuscript. We have made revisions throughout (version 2):
- We have further indicated the limitations of Beall’s lists
... Continue reading
We thank Monica Berger for her thoughtful peer review of our manuscript. We have made revisions throughout (version 2):

We have further indicated the limitations of Beall’s lists for this type of research;

We have noted the global south issue of journals using “international” or “global” in their titles; we have provided some clarity as to scoping reviews; i

It is possible we’ve missed some relevant literature from our review (as is the potential in any review exercise) although we believe in its current form it is both broad and multidisciplinary. As a follow-up exercise we will reach out to library/expert listservs related to this field of enquiry;

We agree that the two statements without a descriptor are important, however, the length of time for a peer-review or publication cannot be classified as either a positive or negative statement and hence were not given a descriptor term. While it could be mapped to Poor Quality Standards, we cannot assume that a short peer-review time is indicative of poor quality.
We thank Monica Berger for her thoughtful peer review of our manuscript. We have made revisions throughout (version 2):

We have further indicated the limitations of Beall’s lists for this type of research;

We have noted the global south issue of journals using “international” or “global” in their titles; we have provided some clarity as to scoping reviews; i

It is possible we’ve missed some relevant literature from our review (as is the potential in any review exercise) although we believe in its current form it is both broad and multidisciplinary. As a follow-up exercise we will reach out to library/expert listservs related to this field of enquiry;

We agree that the two statements without a descriptor are important, however, the length of time for a peer-review or publication cannot be classified as either a positive or negative statement and hence were not given a descriptor term. While it could be mapped to Poor Quality Standards, we cannot assume that a short peer-review time is indicative of poor quality.
Competing Interests: I have no competing interests to declare. Close
Report a concern
Author Response (F1000Research Advisory Board Member) 23 Aug 2018

David Moher, School of Epidemiology and Public Health, Faculty of Medicine, University of Ottawa, Ottawa, K1G 5Z3, Canada

23 Aug 2018

Author Response F1000Research Advisory Board Member

In response to Valerie Ann Matarese’s comment we have changed two words in the introduction (version 2).

Ross Mounce is misinformed. This is not a “literature review of opinion, ... Continue reading In response to Valerie Ann Matarese’s comment we have changed two words in the introduction (version 2).

Ross Mounce is misinformed. This is not a “literature review of opinion, and as such, one wonders what the value of the exercise is.”. As stated in the screening and data extortion section of the Methods of this scoping review (version 1) “we restricted our sample of articles to those that provided a definition of predatory journals, or described characteristics of predatory journals, based on empirical work (i.e., not opinion, not definitions which referenced previous work).”.

We thank Edgardo Rolla for his comments on our scoping review (version 1).

In response to Valerie Ann Matarese’s comment we have changed two words in the introduction (version 2).

Ross Mounce is misinformed. This is not a “literature review of opinion, and as such, one wonders what the value of the exercise is.”. As stated in the screening and data extortion section of the Methods of this scoping review (version 1) “we restricted our sample of articles to those that provided a definition of predatory journals, or described characteristics of predatory journals, based on empirical work (i.e., not opinion, not definitions which referenced previous work).”.

We thank Edgardo Rolla for his comments on our scoping review (version 1).
Competing Interests: I have no competing interests to declare. Close
Report a concern
Respond or Comment

COMMENTS ON THIS REPORT

Author Response (F1000Research Advisory Board Member) 23 Aug 2018

David Moher, School of Epidemiology and Public Health, Faculty of Medicine, University of Ottawa, Ottawa, K1G 5Z3, Canada

23 Aug 2018

Author Response F1000Research Advisory Board Member
We thank Monica Berger for her thoughtful peer review of our manuscript. We have made revisions throughout (version 2):
- We have further indicated the limitations of Beall’s lists
... Continue reading
We thank Monica Berger for her thoughtful peer review of our manuscript. We have made revisions throughout (version 2):

We have further indicated the limitations of Beall’s lists for this type of research;

We have noted the global south issue of journals using “international” or “global” in their titles; we have provided some clarity as to scoping reviews; i

It is possible we’ve missed some relevant literature from our review (as is the potential in any review exercise) although we believe in its current form it is both broad and multidisciplinary. As a follow-up exercise we will reach out to library/expert listservs related to this field of enquiry;

We agree that the two statements without a descriptor are important, however, the length of time for a peer-review or publication cannot be classified as either a positive or negative statement and hence were not given a descriptor term. While it could be mapped to Poor Quality Standards, we cannot assume that a short peer-review time is indicative of poor quality.
We thank Monica Berger for her thoughtful peer review of our manuscript. We have made revisions throughout (version 2):

We have further indicated the limitations of Beall’s lists for this type of research;

We have noted the global south issue of journals using “international” or “global” in their titles; we have provided some clarity as to scoping reviews; i

It is possible we’ve missed some relevant literature from our review (as is the potential in any review exercise) although we believe in its current form it is both broad and multidisciplinary. As a follow-up exercise we will reach out to library/expert listservs related to this field of enquiry;

We agree that the two statements without a descriptor are important, however, the length of time for a peer-review or publication cannot be classified as either a positive or negative statement and hence were not given a descriptor term. While it could be mapped to Poor Quality Standards, we cannot assume that a short peer-review time is indicative of poor quality.
Competing Interests: I have no competing interests to declare. Close
Report a concern
Author Response (F1000Research Advisory Board Member) 23 Aug 2018

David Moher, School of Epidemiology and Public Health, Faculty of Medicine, University of Ottawa, Ottawa, K1G 5Z3, Canada

23 Aug 2018

Author Response F1000Research Advisory Board Member

In response to Valerie Ann Matarese’s comment we have changed two words in the introduction (version 2).

Ross Mounce is misinformed. This is not a “literature review of opinion, ... Continue reading In response to Valerie Ann Matarese’s comment we have changed two words in the introduction (version 2).

Ross Mounce is misinformed. This is not a “literature review of opinion, and as such, one wonders what the value of the exercise is.”. As stated in the screening and data extortion section of the Methods of this scoping review (version 1) “we restricted our sample of articles to those that provided a definition of predatory journals, or described characteristics of predatory journals, based on empirical work (i.e., not opinion, not definitions which referenced previous work).”.

We thank Edgardo Rolla for his comments on our scoping review (version 1).

In response to Valerie Ann Matarese’s comment we have changed two words in the introduction (version 2).

Ross Mounce is misinformed. This is not a “literature review of opinion, and as such, one wonders what the value of the exercise is.”. As stated in the screening and data extortion section of the Methods of this scoping review (version 1) “we restricted our sample of articles to those that provided a definition of predatory journals, or described characteristics of predatory journals, based on empirical work (i.e., not opinion, not definitions which referenced previous work).”.

We thank Edgardo Rolla for his comments on our scoping review (version 1).
Competing Interests: I have no competing interests to declare. Close
Report a concern

Comments on this article Comments (3)

Version 2

VERSION 2 PUBLISHED 23 Aug 2018

Revised

Comment

Version 1

VERSION 1 PUBLISHED 04 Jul 2018

Discussion is closed on this version, please comment on the latest version above.

Reader Comment 19 Jul 2018

Valerie Matarese, Authors' editor and editorial consultant, Vidor, Italy

19 Jul 2018

Reader Comment

The definition of predatory journals attributed, in the Introduction, to Beall (Nature 2012) is imprecise. In that article, Beall defined predatory publishers, not journals, and he used the term "predatory ... Continue reading The definition of predatory journals attributed, in the Introduction, to Beall (Nature 2012) is imprecise. In that article, Beall defined predatory publishers, not journals, and he used the term "predatory journal" just once.

The correct quotation is "... predatory publishers ... publish counterfeit journals to exploit the open-access model in which the author pays. These predatory publishers are dishonest and lack transparency." This is a single definition, not "definitions" as suggested here.

Given that the purpose of this new article is to define predatory journals, it is important that the starting point be correct. I hope this change will be made in the revised version.
The definition of predatory journals attributed, in the Introduction, to Beall (Nature 2012) is imprecise. In that article, Beall defined predatory publishers, not journals, and he used the term "predatory journal" just once.

The correct quotation is "... predatory publishers ... publish counterfeit journals to exploit the open-access model in which the author pays. These predatory publishers are dishonest and lack transparency." This is a single definition, not "definitions" as suggested here.

Given that the purpose of this new article is to define predatory journals, it is important that the starting point be correct. I hope this change will be made in the revised version.
Competing Interests: No competing interests were disclosed. Close
Report a concern
Reader Comment 13 Jul 2018

Ross Mounce, University of Cambridge, UK

13 Jul 2018

Reader Comment

This is a literature review of opinion, and as such, one wonders what the value of the exercise is. Surely it would be better to examine the phenomenon itself, not ... Continue reading This is a literature review of opinion, and as such, one wonders what the value of the exercise is. Surely it would be better to examine the phenomenon itself, not just a synthesis of many studies of deeply variable quality about it?

One should not just treat all papers equally. The quality of evidence offered by some of the included 38 papers (Supp file 2) is extremely poor if one actually reads them. Yet they appear to be all equally weighted in terms of their evidentiary contribution. It would be tremendously interesting to examine how many of the 38 papers were actually peer-reviewed - some clearly are just correspondances and editorials. Others such Sorowski et al. (2017) are published at a well-establised commercial journal in a “commission-only section” (source: https://www.nature.com/nature/for-authors/other-subs) where the publisher has a clear commercial conflict-of-interest in sowing fear about publishing in less well known journals and has published a whole series of commissioned, not-peer-reviewed, pieces doing just that.

So let’s have a close look at some of the 38 studies included in this paper:

E.g. in “1. [RefID: 1]. Oermann MH, Nicoll LH, Chinn PL, et al. Quality of articles published in predatory nursing journals. Nurs Outlook 2018 Jan;66(1):4-10. “

In which it is written: “This constellation of problems has led to the identification of predatory journals and predatory publishers (International Academy of Nursing Editors [INANE] Predatory Publishing Collaborative, 2014). Greco (2016) reported that predatory publishers made approximately $75 million US dollars in 2015, mainly through the collection of APCs.” (emphasis mine)

The claim that predatory publishers have made approximately $75 million US dollars in 2015 (alone) simply isn’t credible but it is indicative of the kind of wild extrapolation and rumour that is undertaken when some people write about predatory publishing. The figure of 75 million I presume derives from Shen & Bjork (2015) in which they write “Using our [estimated] data for the number of articles and average APC for 2014, our estimate for the size of the market is 74 million USD”. This is an estimate from extrapolation and in no way indicative of 74 million USD actually being paid out to predatory publishers - that distinction is subtle but important. Shen & Bjork (2015) did not claim that publishers made approximately $75 million US dollars in 2015, yet through poor peer review at “non-predatory” journals the trumped-up form of the claim still enters the literature and is misleadingly repeated in headlines e.g. “Predatory publishers earned $75 million last year, study finds” (Bohannon 2015).

Shen & Bjork (2015) itself was in many ways a good, well-reported study, with transparent open peer review, but even here if one digs into the details there are problems. The most notable is that for their definition of predatory publishers and inclusion in the study, Shen & Bjork (2015) used Beall’s List at a particular point in time when the publisher MDPI (http://www.mdpi.com/) was on the list. In subsequent years (MDPI was removed from Mr. Beall's list on 28 October 2015), the world and Beall himself realised that MDPI was not a predatory publisher, despite one or two papers with peer review problems - most journals have famous ‘clangers’ even Nature, Science and PNAS. MDPI unfortunately happens to be a large publisher in terms of article volume with ‘high’ APCs relative to publishers that are genuinely are ‘predatory’ by most people’s understanding. This therefore explains well how Shen & Bjork (2015) came to their astoundingly high 74 million USD estimate (which included MDPI) but similarly shows that the study is flawed because of that and needs to be revisited with MDPI and other now understood not-to-be-predatory publishers taken out of the analysis. Specific differences in minutiae like this between the 38 studies might make each of them completely incomparable to each other! More detailed work is needed to assess comparability and quality of evidence before a synthesis is made, otherwise the analysis is garbage-in, garbage-out.

The irony is that much of the literature writing about predatory publishing is of itself also poor quality, as the examples demonstrate. The issue at hand here is poor or non-existent peer review and unfortunately this is rather more widespread than many would like to admit.

There also seems to be no acknowledgement whatsoever in this manuscript that ‘predatory journals’ and Jeffrey Beall are controversial subjects.

Beall’s views (e.g. Esposito, 2013; Bivens-Tatum 2014; Reyes-Galindo 2015; Velterop 2015; Houghton 2017), his website and list of journals (e.g. Bloudoff-Indelicato, 2015; Silver, 2017), are highly controversial.

If one is to write a manuscript on this topic, one must acknowledge upfront that Jeffrey Beall has a “distasteful political ideology” (Esposito, 2013) which this manuscript is in danger of promoting if the authors do not take steps to carefully consider the framing of this research.

The framing of the phenomena as “predatory journals” throughout the manuscript is misjudged and imprecise use of language. For a start, ‘journal’ is simply the wrong unit of assessment. To demonstrate, imagine this hypothetical scenario:

A legitimate journal has been publishing articles legitimately with high editorial and peer review standards for 5 years. The ownership and management of the journal changes and suddenly all articles in the 6th year of publication are low quality and simply haven’t been peer reviewed (despite the journal overtly claiming them to be peer-reviewed). The content of the 6th year is perhaps to be considered ‘predatory’ under Beall’s choice of naming system [which for the record I don’t really endorse], whereas the content published in the first 5 years is still legitimate. It would be absurd to call the entirety of journal ‘predatory’ – it would besmirch the first 5 years of legitimate research published there. It is only part of the content (the 6th year) that is deceptively published.

‘Predatory publishing’ as the phenomena is more widely known, is a (lack of) process and as such it can apply to individual articles, issues of journals, whole journals, or whole publishers of many journals, as well as to academic book publishing too. The narrowness of framing as “predatory journals” just isn’t appropriate if one really wants to understand the phenomena. If one wanted to scope it only to examine predatory publishing in journals that’s fine but it’s ‘predatory publishing _in_ journals’, not ‘predatory journals’.

There are also numerous problems with the search strategy employed in this manuscript. It is acknowledged that there are other ways to denote predatory publishing:

“Other terms to denote predatory journals such as “illegitimate journals”, “deceptive journals”, “dark” journals, and “journals operating in bad faith” have appeared in the literature”

But the manuscript has missed a major alternative synonym for the same phenomena and not included it in the search strategy: in China the problem is known as “Trash journals” (Hrvistendahl 2011 ; Lin & Zhan, 2014 ; Jones 2015 ; Ren and Montgomery 2015). The word “trash” is not included in the search strategy which might mean Chinese opinions on the phenomenon are missed. Relatedly the arbitrary starting point of publications from 2012 onwards, clearly excludes earlier relevant literature such as Hrvistendahl (2011).

It is generally accepted that the Global North isn’t as affected by predatory publishing (Jones 2015; ““I get these emails every day but I don’t know a single academic who would fall for these obviously fake journals. I only publish in journals that I know about. Honestly, nobody would fall for this, it’s not a problem for me.”), instead it is mainly authors from the Global South that are more detectably caught-up in the problem for a variety of reasons (Xia 2015).

Why then does this manuscript specifically exclude opinions on the phenomenon that are not written in English? The problem is most detectable in the Global South (Xia 2015) and thus in order to have true understanding of it one must include non-Anglophone contributions on the matter. It is tremendously problematic and compounds the problem the authors are studying to be monolingual in focus - the phenomenon exists at least partially because of the feeling of exclusion in the Global South by gatekeeping of Global North journals.

Although because it is recently published it falls outside of the search strategies applied, I would strongly suggest the manuscript authors read and cite Bell et al. (2018) “Predatory Publishing” and Eve & Priego (2017) “Who is Actually Harmed by Predatory Publishers?” to provide some more balance in this discussion.

As it stands, the manuscript is significantly methodologically flawed and needs serious reconsideration.

Azeez, Abdul E.P. 2017. “Academic Writing and Publishing in India: Is Quality a Touchstone?” Journal of Community Positive Practices 17, no. 1: 13–24.
Anderson, Rick. 2015. “Should We Retire the Term Predatory Publishing?” The Scholarly Kitchen, May 11. https://scholarlykitchen.sspnet.org/2015/05/11/should-we-retire-the-term-predatory-publishing/
Bohannon, J. 2015. Predatory publishers earned $75 million last year, study finds. Science http://www.sciencemag.org/news/2015/09/predatory-publishers-earned-75-million-last-year-study-finds
Bivens-Tatum, Wayne 2014. “Reactionary Rhetoric Against Open Access Publishing”
Bloudoff-Indelicato, Mollie 2015. “Backlash after Frontiers journals added to list of questionable publishers” https://www.nature.com/news/backlash-after-frontiers-journals-added-to-list-of-questionable-publishers-1.18639
Crawford, Walt. 2014. Ethics and Access 1: The Sad Case of Jeffrey Beall. Cites & Insights 14 (4): 1-14. https://citesandinsights.info/civ14i4.pdf
Esposito, Joe 2013. “Parting Company with Jeffrey Beall” https://scholarlykitchen.sspnet.org/2013/12/16/parting-company-with-jeffrey-beall/
Eve, Martin and Priego, Ernesto. 2017. “Who is Actually Harmed by Predatory Publishers?” TripleC 15, no. 2: 755–70. https://doi.org/10.31269/triplec.v15i2.867
Houghton, F. (2017). Ethics in academic publishing: a timely reminder. Journal of the Medical Library Association: JMLA, 105(3), 282. http://jmla.mlanet.org/ojs/jmla/article/view/122/415
Hrvistendahl, M. 2011. China Looks to Purge Academia of 'Trash Journals' http://doi.org/10.1126/science.334.6054.301
Jones, Phil 2015. “Predatory Publishing Isn’t The Problem, It’s a Symptom of Information Inequality” https://www.digital-science.com/blog/perspectives/predatory-publishing-isnt-the-problem-its-a-symptom-of-information-inequality/
Lin, S. 2013. Why serious academic fraud occurs in China. Learned Publishing, 26(1): 24–27. http://doi.org/10.1087/20130105
Lin, S. and Zhan, L. 2014. Trash journals in China. Learned Publishing vol 27 issue 2 http://doi.org/10.1087/20140208
Kirsten Bell, Jill Claassen, Lena Nyahodza, Reggie Raju, Vaclav Stetka 2018. Predatory Publishing http://dx.doi.org/10.17613/M6N58CK3D
Ren and Montgomery 2015. Open access and soft power: Chinese voices in international scholarship. Media, Culture & Society https://doi.org/10.1177/0163443714567019
Ren, Xiang 2013. Open and Networked Initiatives and the Digital Transformation of Academic Publishing in China [Phd Thesis] https://eprints.qut.edu.au/63337/1/Xiang_Ren_Thesis.pdf
Reyes-Galindo, Luis 2015. “On SciELO and RedALyC” https://blogs.cardiff.ac.uk/luisreyes/on-scielo-and-redalyc/
Silver, Andrew 2017. “Controversial website that lists ‘predatory’ publishers shuts down” https://www.nature.com/news/controversial-website-that-lists-predatory-publishers-shuts-down-1.21328
Sorokowski P, Kulczycki E, Sorokowska A, et al. Predatory journals recruit fake editor. Nature 2017;543(7646):481-3.Sorokowski P, Kulczycki E, Sorokowska A, et al. Predatory journals recruit fake editor. Nature 2017;543(7646):481-3.
Velterop, Jan 2015. “The fenced-off ‘nice’ publication neighbourhoods of Jeffrey Beall”
https://blog.scielo.org/en/2015/08/01/the-fenced-off-nice-publication-neighbourhoods-of-jeffrey-beall/#.W0S3T9JKhhE
Xia, J. 2014. An imbalanced journal publishing market. Learned Publishing. http://doi.org/10.1087/20140309
Xia, J. et al. 2015. Who publishes in “predatory” journals? Journal of the Association for Information Science and Technology, 66(7), 1406–1417. http://doi.org/10.1002/asi.23265
This is a literature review of opinion, and as such, one wonders what the value of the exercise is. Surely it would be better to examine the phenomenon itself, not just a synthesis of many studies of deeply variable quality about it?

One should not just treat all papers equally. The quality of evidence offered by some of the included 38 papers (Supp file 2) is extremely poor if one actually reads them. Yet they appear to be all equally weighted in terms of their evidentiary contribution. It would be tremendously interesting to examine how many of the 38 papers were actually peer-reviewed - some clearly are just correspondances and editorials. Others such Sorowski et al. (2017) are published at a well-establised commercial journal in a “commission-only section” (source: https://www.nature.com/nature/for-authors/other-subs) where the publisher has a clear commercial conflict-of-interest in sowing fear about publishing in less well known journals and has published a whole series of commissioned, not-peer-reviewed, pieces doing just that.

So let’s have a close look at some of the 38 studies included in this paper:

E.g. in “1. [RefID: 1]. Oermann MH, Nicoll LH, Chinn PL, et al. Quality of articles published in predatory nursing journals. Nurs Outlook 2018 Jan;66(1):4-10. “

In which it is written: “This constellation of problems has led to the identification of predatory journals and predatory publishers (International Academy of Nursing Editors [INANE] Predatory Publishing Collaborative, 2014). Greco (2016) reported that predatory publishers made approximately $75 million US dollars in 2015, mainly through the collection of APCs.” (emphasis mine)

The claim that predatory publishers have made approximately $75 million US dollars in 2015 (alone) simply isn’t credible but it is indicative of the kind of wild extrapolation and rumour that is undertaken when some people write about predatory publishing. The figure of 75 million I presume derives from Shen & Bjork (2015) in which they write “Using our [estimated] data for the number of articles and average APC for 2014, our estimate for the size of the market is 74 million USD”. This is an estimate from extrapolation and in no way indicative of 74 million USD actually being paid out to predatory publishers - that distinction is subtle but important. Shen & Bjork (2015) did not claim that publishers made approximately $75 million US dollars in 2015, yet through poor peer review at “non-predatory” journals the trumped-up form of the claim still enters the literature and is misleadingly repeated in headlines e.g. “Predatory publishers earned $75 million last year, study finds” (Bohannon 2015).

Shen & Bjork (2015) itself was in many ways a good, well-reported study, with transparent open peer review, but even here if one digs into the details there are problems. The most notable is that for their definition of predatory publishers and inclusion in the study, Shen & Bjork (2015) used Beall’s List at a particular point in time when the publisher MDPI (http://www.mdpi.com/) was on the list. In subsequent years (MDPI was removed from Mr. Beall's list on 28 October 2015), the world and Beall himself realised that MDPI was not a predatory publisher, despite one or two papers with peer review problems - most journals have famous ‘clangers’ even Nature, Science and PNAS. MDPI unfortunately happens to be a large publisher in terms of article volume with ‘high’ APCs relative to publishers that are genuinely are ‘predatory’ by most people’s understanding. This therefore explains well how Shen & Bjork (2015) came to their astoundingly high 74 million USD estimate (which included MDPI) but similarly shows that the study is flawed because of that and needs to be revisited with MDPI and other now understood not-to-be-predatory publishers taken out of the analysis. Specific differences in minutiae like this between the 38 studies might make each of them completely incomparable to each other! More detailed work is needed to assess comparability and quality of evidence before a synthesis is made, otherwise the analysis is garbage-in, garbage-out.

The irony is that much of the literature writing about predatory publishing is of itself also poor quality, as the examples demonstrate. The issue at hand here is poor or non-existent peer review and unfortunately this is rather more widespread than many would like to admit.

There also seems to be no acknowledgement whatsoever in this manuscript that ‘predatory journals’ and Jeffrey Beall are controversial subjects.

Beall’s views (e.g. Esposito, 2013; Bivens-Tatum 2014; Reyes-Galindo 2015; Velterop 2015; Houghton 2017), his website and list of journals (e.g. Bloudoff-Indelicato, 2015; Silver, 2017), are highly controversial.

If one is to write a manuscript on this topic, one must acknowledge upfront that Jeffrey Beall has a “distasteful political ideology” (Esposito, 2013) which this manuscript is in danger of promoting if the authors do not take steps to carefully consider the framing of this research.

The framing of the phenomena as “predatory journals” throughout the manuscript is misjudged and imprecise use of language. For a start, ‘journal’ is simply the wrong unit of assessment. To demonstrate, imagine this hypothetical scenario:

A legitimate journal has been publishing articles legitimately with high editorial and peer review standards for 5 years. The ownership and management of the journal changes and suddenly all articles in the 6th year of publication are low quality and simply haven’t been peer reviewed (despite the journal overtly claiming them to be peer-reviewed). The content of the 6th year is perhaps to be considered ‘predatory’ under Beall’s choice of naming system [which for the record I don’t really endorse], whereas the content published in the first 5 years is still legitimate. It would be absurd to call the entirety of journal ‘predatory’ – it would besmirch the first 5 years of legitimate research published there. It is only part of the content (the 6th year) that is deceptively published.

‘Predatory publishing’ as the phenomena is more widely known, is a (lack of) process and as such it can apply to individual articles, issues of journals, whole journals, or whole publishers of many journals, as well as to academic book publishing too. The narrowness of framing as “predatory journals” just isn’t appropriate if one really wants to understand the phenomena. If one wanted to scope it only to examine predatory publishing in journals that’s fine but it’s ‘predatory publishing _in_ journals’, not ‘predatory journals’.

There are also numerous problems with the search strategy employed in this manuscript. It is acknowledged that there are other ways to denote predatory publishing:

“Other terms to denote predatory journals such as “illegitimate journals”, “deceptive journals”, “dark” journals, and “journals operating in bad faith” have appeared in the literature”

But the manuscript has missed a major alternative synonym for the same phenomena and not included it in the search strategy: in China the problem is known as “Trash journals” (Hrvistendahl 2011 ; Lin & Zhan, 2014 ; Jones 2015 ; Ren and Montgomery 2015). The word “trash” is not included in the search strategy which might mean Chinese opinions on the phenomenon are missed. Relatedly the arbitrary starting point of publications from 2012 onwards, clearly excludes earlier relevant literature such as Hrvistendahl (2011).

It is generally accepted that the Global North isn’t as affected by predatory publishing (Jones 2015; ““I get these emails every day but I don’t know a single academic who would fall for these obviously fake journals. I only publish in journals that I know about. Honestly, nobody would fall for this, it’s not a problem for me.”), instead it is mainly authors from the Global South that are more detectably caught-up in the problem for a variety of reasons (Xia 2015).

Why then does this manuscript specifically exclude opinions on the phenomenon that are not written in English? The problem is most detectable in the Global South (Xia 2015) and thus in order to have true understanding of it one must include non-Anglophone contributions on the matter. It is tremendously problematic and compounds the problem the authors are studying to be monolingual in focus - the phenomenon exists at least partially because of the feeling of exclusion in the Global South by gatekeeping of Global North journals.

Although because it is recently published it falls outside of the search strategies applied, I would strongly suggest the manuscript authors read and cite Bell et al. (2018) “Predatory Publishing” and Eve & Priego (2017) “Who is Actually Harmed by Predatory Publishers?” to provide some more balance in this discussion.

As it stands, the manuscript is significantly methodologically flawed and needs serious reconsideration.

Azeez, Abdul E.P. 2017. “Academic Writing and Publishing in India: Is Quality a Touchstone?” Journal of Community Positive Practices 17, no. 1: 13–24.
Anderson, Rick. 2015. “Should We Retire the Term Predatory Publishing?” The Scholarly Kitchen, May 11. https://scholarlykitchen.sspnet.org/2015/05/11/should-we-retire-the-term-predatory-publishing/
Bohannon, J. 2015. Predatory publishers earned $75 million last year, study finds. Science http://www.sciencemag.org/news/2015/09/predatory-publishers-earned-75-million-last-year-study-finds
Bivens-Tatum, Wayne 2014. “Reactionary Rhetoric Against Open Access Publishing”
Bloudoff-Indelicato, Mollie 2015. “Backlash after Frontiers journals added to list of questionable publishers” https://www.nature.com/news/backlash-after-frontiers-journals-added-to-list-of-questionable-publishers-1.18639
Crawford, Walt. 2014. Ethics and Access 1: The Sad Case of Jeffrey Beall. Cites & Insights 14 (4): 1-14. https://citesandinsights.info/civ14i4.pdf
Esposito, Joe 2013. “Parting Company with Jeffrey Beall” https://scholarlykitchen.sspnet.org/2013/12/16/parting-company-with-jeffrey-beall/
Eve, Martin and Priego, Ernesto. 2017. “Who is Actually Harmed by Predatory Publishers?” TripleC 15, no. 2: 755–70. https://doi.org/10.31269/triplec.v15i2.867
Houghton, F. (2017). Ethics in academic publishing: a timely reminder. Journal of the Medical Library Association: JMLA, 105(3), 282. http://jmla.mlanet.org/ojs/jmla/article/view/122/415
Hrvistendahl, M. 2011. China Looks to Purge Academia of 'Trash Journals' http://doi.org/10.1126/science.334.6054.301
Jones, Phil 2015. “Predatory Publishing Isn’t The Problem, It’s a Symptom of Information Inequality” https://www.digital-science.com/blog/perspectives/predatory-publishing-isnt-the-problem-its-a-symptom-of-information-inequality/
Lin, S. 2013. Why serious academic fraud occurs in China. Learned Publishing, 26(1): 24–27. http://doi.org/10.1087/20130105
Lin, S. and Zhan, L. 2014. Trash journals in China. Learned Publishing vol 27 issue 2 http://doi.org/10.1087/20140208
Kirsten Bell, Jill Claassen, Lena Nyahodza, Reggie Raju, Vaclav Stetka 2018. Predatory Publishing http://dx.doi.org/10.17613/M6N58CK3D
Ren and Montgomery 2015. Open access and soft power: Chinese voices in international scholarship. Media, Culture & Society https://doi.org/10.1177/0163443714567019
Ren, Xiang 2013. Open and Networked Initiatives and the Digital Transformation of Academic Publishing in China [Phd Thesis] https://eprints.qut.edu.au/63337/1/Xiang_Ren_Thesis.pdf
Reyes-Galindo, Luis 2015. “On SciELO and RedALyC” https://blogs.cardiff.ac.uk/luisreyes/on-scielo-and-redalyc/
Silver, Andrew 2017. “Controversial website that lists ‘predatory’ publishers shuts down” https://www.nature.com/news/controversial-website-that-lists-predatory-publishers-shuts-down-1.21328
Sorokowski P, Kulczycki E, Sorokowska A, et al. Predatory journals recruit fake editor. Nature 2017;543(7646):481-3.Sorokowski P, Kulczycki E, Sorokowska A, et al. Predatory journals recruit fake editor. Nature 2017;543(7646):481-3.
Velterop, Jan 2015. “The fenced-off ‘nice’ publication neighbourhoods of Jeffrey Beall”
https://blog.scielo.org/en/2015/08/01/the-fenced-off-nice-publication-neighbourhoods-of-jeffrey-beall/#.W0S3T9JKhhE
Xia, J. 2014. An imbalanced journal publishing market. Learned Publishing. http://doi.org/10.1087/20140309
Xia, J. et al. 2015. Who publishes in “predatory” journals? Journal of the Association for Information Science and Technology, 66(7), 1406–1417. http://doi.org/10.1002/asi.23265
Competing Interests: I am an editor for the MDPI journal called Data (ISSN 2306-5729). I receive no financial compensation for this, it is a voluntary service. Rather being a bias, I consider this a plus as I actually have first-hand experience of the peer review and editorial process at this publisher and I can therefore better attest to it being of acceptable standards. http://www.mdpi.com/journal/data/editors Close
Report a concern
Reader Comment 10 Jul 2018

Edgardo Rolla, Sociedad Argentina de Endometriosis, Argentina

10 Jul 2018

Reader Comment

Interesting article that, somehow tries to help elucidating what is a correct scientific publication and what should be avoided. The proposal is sound, the methodology is correct, and being a ... Continue reading Interesting article that, somehow tries to help elucidating what is a correct scientific publication and what should be avoided. The proposal is sound, the methodology is correct, and being a subjective issue, the discussion/conclusions are acceptable. Publishing in medicine has turned itself a continuous race to be the first one and many times proposed articles do not fulfill what is expected from responsible authors. It is interesting to note that many of the citations are opinion articles. It would make sense if the authors could provide different qualifications for evidence based only publications and those where personal opinion is included. As far as those characteristics are noted in the articles, I would not include as "predatory" opinion based articles or the journals where they are published.
Interesting article that, somehow tries to help elucidating what is a correct scientific publication and what should be avoided. The proposal is sound, the methodology is correct, and being a subjective issue, the discussion/conclusions are acceptable. Publishing in medicine has turned itself a continuous race to be the first one and many times proposed articles do not fulfill what is expected from responsible authors. It is interesting to note that many of the citations are opinion articles. It would make sense if the authors could provide different qualifications for evidence based only publications and those where personal opinion is included. As far as those characteristics are noted in the articles, I would not include as "predatory" opinion based articles or the journals where they are published.
Competing Interests: no conflicts of interest Close
Report a concern
Discussion is closed on this version, please comment on the latest version above.

Open Peer Review

Reviewer Status

Reviewer Reports

	Invited Reviewers
	1	2	3
Version 2 (revision) 23 Aug 18		read
Version 1 04 Jul 18	read	read	read

Monica Berger, CUNY New York City College of Technology, Brooklyn, USA
Johann Mouton, Stellenbosch University, Stellenbosch, South Africa
Joanna Chataway, University of Sussex, Brighton, UK

Comments on this article

All Comments(3)

Add a comment

Browse by related subjects

Back to all reports

Reviewer Report

57 Views

13 Sep 2018 | for Version 2

57 Views Cite this report Responses(0)

Approved

Competing Interests

No competing interests were disclosed.

I confirm that I have read this submission and believe that I have an appropriate level of expertise to confirm that it is of an acceptable scientific standard.

Respond to this report

Responses (0)

Back to all reports

Reviewer Report

70 Views

13 Aug 2018 | for Version 1

Joanna Chataway, SPRU—Science Policy Research Unit, University of Sussex, Brighton, UK

70 Views Cite this report Responses(1)

Approved

Is the work clearly and accurately presented and does it cite the current literature?

Yes
Is the study design appropriate and is the work technically sound?

Yes
Are sufficient details of methods and analysis provided to allow replication by others?

Yes
If applicable, is the statistical analysis and its interpretation appropriate?

Not applicable
Are all the source data underlying the results available to ensure full reproducibility?

Partly
Are the conclusions drawn adequately supported by the results?

Yes

Competing Interests

No competing interests were disclosed.

Reviewer Expertise

Science policy

I confirm that I have read this submission and believe that I have an appropriate level of expertise to confirm that it is of an acceptable scientific standard.

Respond to this report

Responses (1)

Author Response F1000Research Advisory Board Member

23 Aug 2018

David Moher, School of Epidemiology and Public Health, Faculty of Medicine, University of Ottawa, Ottawa, K1G 5Z3, Canada

In response to Joanna Chataway’s review:

We have made some modifications to the limitations section of our paper. We now state (version 2) “Thirdly, our focus was on the biomedical literature. Whether the publication (e.g., having an IMRAD (Introduction Methods Results And Discussion) and peer review norms we’ve used apply across other disciplines is likely an important topic for further investigation.”
We have more cleared described scoping reviews (version 2).
We believe we have given some examples in Table 3. For example, in response to the query as to the use and meaning of “persuasive language”, we state (Version 1 and version 2) “Language that targets; Language that attempts to convince the author to do or believe something”.
We have fixed the broken link to the full search strategy (version 2).

View more View less

Competing Interests

We have no competing interests to declare.

Back to all reports

Reviewer Report

132 Views

13 Aug 2018 | for Version 1

132 Views Cite this report Responses(1)

Not Approved

Is the work clearly and accurately presented and does it cite the current literature?

Yes
Is the study design appropriate and is the work technically sound?

Yes
Are sufficient details of methods and analysis provided to allow replication by others?

Yes
If applicable, is the statistical analysis and its interpretation appropriate?

Yes
Are all the source data underlying the results available to ensure full reproducibility?

Yes
Are the conclusions drawn adequately supported by the results?

Partly

Competing Interests

No competing interests were disclosed.

Respond to this report

Responses (1)

Author Response F1000Research Advisory Board Member

23 Aug 2018

David Moher, School of Epidemiology and Public Health, Faculty of Medicine, University of Ottawa, Ottawa, K1G 5Z3, Canada

We are sorry to disappoint Johann Mouton in our scoping review. We believe that a scoping review is a reasonable way to attempt to map the literature. Scoping reviews do not typically weight included studies. We believe our review highlights some of the disagreements in the literature about presumed relevant characteristics of presumed predatory journals.

View more View less

Competing Interests

We have no competing interests to declare.

Back to all reports

Reviewer Report

91 Views

01 Aug 2018 | for Version 1

Monica Berger, CUNY New York City College of Technology, Brooklyn, NY, USA

91 Views Cite this report Responses(2)

Approved

Is the work clearly and accurately presented and does it cite the current literature?

Yes
Is the study design appropriate and is the work technically sound?

Yes
Are sufficient details of methods and analysis provided to allow replication by others?

Yes
If applicable, is the statistical analysis and its interpretation appropriate?

I cannot comment. A qualified statistician is required.
Are all the source data underlying the results available to ensure full reproducibility?

Yes
Are the conclusions drawn adequately supported by the results?

Yes

References

1. Moher D, Shamseer L, Cobey K, Lalu M, et al.: Stop this waste of people, animals and money. Nature. 2017; 549 (7670): 23-25 Publisher Full Text

Competing Interests

No competing interests were disclosed.

Reviewer Expertise

My knowledge as a scholarly communications librarian who has devoted considerable effort to writing about the topic at hand is extensive but I am not a trained researcher in either the sciences or social sciences so my ability to genuinely judge the methodology is limited.

I confirm that I have read this submission and believe that I have an appropriate level of expertise to confirm that it is of an acceptable scientific standard.

Respond to this report

Responses (2)

Author Response F1000Research Advisory Board Member

23 Aug 2018

David Moher, School of Epidemiology and Public Health, Faculty of Medicine, University of Ottawa, Ottawa, K1G 5Z3, Canada

We thank Monica Berger for her thoughtful peer review of our manuscript. We have made revisions throughout (version 2):

We have further indicated the limitations of Beall’s lists for this type of research;
We have noted the global south issue of journals using “international” or “global” in their titles; we have provided some clarity as to scoping reviews; i
It is possible we’ve missed some relevant literature from our review (as is the potential in any review exercise) although we believe in its current form it is both broad and multidisciplinary. As a follow-up exercise we will reach out to library/expert listservs related to this field of enquiry;
We agree that the two statements without a descriptor are important, however, the length of time for a peer-review or publication cannot be classified as either a positive or negative statement and hence were not given a descriptor term. While it could be mapped to Poor Quality Standards, we cannot assume that a short peer-review time is indicative of poor quality.

View more View less

Competing Interests

I have no competing interests to declare.

Author Response F1000Research Advisory Board Member

23 Aug 2018

David Moher, School of Epidemiology and Public Health, Faculty of Medicine, University of Ottawa, Ottawa, K1G 5Z3, Canada

In response to Valerie Ann Matarese’s comment we have changed two words in the introduction (version 2).

Ross Mounce is misinformed. This is not a “literature review of opinion, and as such, one wonders what the value of the exercise is.”. As stated in the screening and data extortion section of the Methods of this scoping review (version 1) “we restricted our sample of articles to those that provided a definition of predatory journals, or described characteristics of predatory journals, based on empirical work (i.e., not opinion, not definitions which referenced previous work).”.

We thank Edgardo Rolla for his comments on our scoping review (version 1).

View more View less

Competing Interests

I have no competing interests to declare.

Alongside their report, reviewers assign a status to the article:

Approved - the paper is scientifically sound in its current form and only minor, if any, improvements are suggested

Approved with reservations - A number of small changes, sometimes more significant revisions are required to address specific details and improve the papers academic merit.

Not approved - fundamental flaws in the paper seriously undermine the findings and conclusions

[1] 1. Beall J: Predatory publishers are corrupting open access. Nature. 2012; 489(7415): 179. PubMed Abstract | Publisher Full Text

[2] 2. Moher D, Shamseer L, Cobey KD, et al.: Stop this waste of people, animals and money. Nature. 2017; 549(7670): 23–5. PubMed Abstract | Publisher Full Text

[3] 3. Lalu M, Shamseer L, Cobey KD, et al.: How stakeholders can respond to the rise of predatory journals. Nat Hum Behav. 2017; 1: 852–5. Publisher Full Text

[4] 4. Clark J, Smith R: Firm action needed on predatory journals. BMJ. 2015; 350(1): h210. PubMed Abstract | Publisher Full Text

[5] 5. Bartholomew RE: Science for sale: the rise of predatory journals. J R Soc Med. 2014; 107(10): 384–385. PubMed Abstract | Publisher Full Text | Free Full Text

[6] 6. Smart P: Predatory journals and researcher needs. Learn Publ. 2017; 30(2): 103–5. Publisher Full Text

[7] 7. Bjørnshauge L: Think Check Submit. In: OASPA Conference. 2015. Reference Source

[8] 8. Shen C, Björk BC: 'Predatory' open access: a longitudinal study of article volumes and market characteristics. BMC Med. 2015; 13: 230. PubMed Abstract | Publisher Full Text | Free Full Text

[9] 9. Cobey K: Illegitimate journals scam even senior scientists. Nature. 2017; 549(7670): 7. PubMed Abstract | Publisher Full Text

[10] 10. Kolata G: Many Academics Are Eager to Publish in Worthless Journals. New York Times. 2017; 1–4. Reference Source

[11] 11. Berger M: Everything you ever wanted to know about predatory publishing but were afraid to ask. ACRL. 2017; 206–7. Reference Source

[12] 12. Wager E: Why we should worry less about predatory publishers and more about the quality of research and training at our academic institutions. J Epidemiol. 2017; 27(3): 87–8. PubMed Abstract | Publisher Full Text | Free Full Text

[13] 13. Anderson R: Should we retire the term “predatory publishing”? Scholarly Kitchen. Accessed August 9, 2018. 2015. Reference Source

[14] 14. Shamseer L, Moher D: Thirteen ways to spot a ‘predatory journal’ (and why we shouldn’t call them that). Times Higher Education.2017. Reference Source

[15] 15. Eriksson S, Helgesson G: Time to stop talking about ‘predatory journals.’ Learn Publ. 2018; 31(2): 181–3. Publisher Full Text

[16] 16. Moher D, Moher E: Stop Predatory Publishers Now: Act Collaboratively. Ann Intern Med. 2016; 164(9): 616–7. PubMed Abstract | Publisher Full Text

[17] 17. Butler D: Investigating journals: The dark side of publishing. Nature. 2013; 495(7442): 433–5. PubMed Abstract | Publisher Full Text

[18] 18. Arksey H, O'Malley L: Scoping studies: Towards a methodological framework. Int J Soc Res Methodol. 2005; 8(1): 19–32. Publisher Full Text

[19] 19. Levac D, Colquhoun H, O'Brien KK: Scoping studies: advancing the methodology. Implement Sci. 2010; 5: 69. PubMed Abstract | Publisher Full Text | Free Full Text

[20] 20. Peters MDJ, Godfrey C, McInerney P, et al.: Chapter 11: Scoping Reviews. In: Aromataris E, Munn Z, (Editors). Joanna Briggs Institute Reviewer's Manual. The Joanna Briggs Institute, 2017. Reference Source

[21] 21. Moher D, Liberati A, Tetzlaff J, et al.: Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement. PLoS Med. 2009; 6(7): e1000097. PubMed Abstract | Publisher Full Text | Free Full Text

[22] 22. Mcgowan J, Sampson M, Salzwedel DM, et al.: PRESS Peer Review of Electronic Search Strategies: 2015 Guideline Statement. J Clin Epidemiol. 2016; 75: 40–46. PubMed Abstract | Publisher Full Text

[23] 23. Galipeau J, Barbour V, Baskin P, et al.: A scoping review of competencies for scientific editors of biomedical journals. BMC Med. 2016; 14(1): 16. PubMed Abstract | Publisher Full Text | Free Full Text

[24] 24. Matthews D: Netherlands to survey every researchers on misconduct. Times Higher Education. 2016. Reference Source

[25] 25. Patwardhan B, Nagarkar S, Gadre SR, et al.: A critical analysis of the ‘UGC-approved list of journals’. Curr Sci. 2018; 114(6): 1299–1303. Publisher Full Text

What is a predatory journal? A scoping review

Abstract

Keywords

Revised Amendments from Version 1

Introduction

Methods

Transparency statement

Search strategy

Study population and eligibility criteria

Screening and data extraction

Data analysis

Deviations from study protocol

Results

Search results and epidemiological characteristics

Figure 1. Preferred Reporting Items for Systematic reviews and Meta-Analyses (PRISMA) flow diagram summarizing study selection.

Table 1. Epidemiological characteristics of all articles mentioning predatory journals and those included empirical articles describing empirically derived characteristics of predatory journals included in our scoping review.

Table 2. Included empirical records (N=38). For full citations see Supplementary File 2.

Mapping the data into emergent themes

Table 3. Themes and Definitions used to Code Characteristics of Predatory Journals.

Table 4. Characteristics extracted, including article reference and frequency, and their thematic categorization and descriptor.

Discussion

Data availability

Grant information

Acknowledgements

Supplementary material

References

Comments on this article Comments (3)

Open Peer Review

Comments on this article Comments (3)

Open Peer Review

Reviewer Status

Reviewer Reports

Comments on this article

Browse by related subjects

Competing Interests Policy

Stay Updated