Paper Mills in Systematic Reviews and Clinical Guidelines: What the 2026 Evidence Means for Authors

Most researchers who think about paper mills picture the problem at one level: fabricated primary studies slipping past peer review into journals, inflating retraction counts, and muddying specific research areas. That picture is accurate, but it misses what is now the more serious downstream effect. Fabricated papers get cited. When a paper mill article ends up in a systematic review, the fraud does not stop at that review. It enters the evidence base that guideline panels draw on. It gets cited in national policy documents. It informs clinical practice recommendations for patients who will never know where the underlying evidence came from.

Two analyses published in 2025 and 2026 have put numbers on this for the first time. The first, a cross-sectional study published in JAMA Network Open in June 2025, screened 200,000 systematic reviews from 2013 to 2024. The second, a preprint posted on arXiv on August 31, 2026 by Federica Silvi and Leslie McIntosh of Digital Science in London, traced the downstream footprint of papers affiliated with the Pharmakon Neuroscience Network, a known authorship-for-sale scheme. Taken together, the two papers describe a contamination problem that is larger than most authors conducting evidence synthesis have accounted for, and they point to specific steps that can reduce the risk before a compromised review reaches publication.

What the JAMA Network Open Study Found

The JAMA Network Open study, published in June 2025 under the title “Citation Contamination by Paper Mill Articles in Systematic Reviews of the Life Sciences,” is the largest analysis of its kind. The researchers cross-referenced 200,000 systematic reviews indexed in Web of Science between 2013 and 2024 against the Retraction Watch database, specifically looking for reviews that had incorporated papers later identified as paper mill products and subsequently retracted.

They found 299 contaminated reviews, a rate of 0.15 percent. That number sounds small until you consider that systematic reviews are not ordinary primary papers. They synthesize evidence, and the downstream readership of a single systematic review cited in clinical practice guidelines can run into the tens of thousands of clinicians. Of the 385 total citations these 299 reviews contained to paper mill articles, 124 (32.2 percent) occurred after the paper mill article had already been retracted. Thirteen of those post-retraction citations occurred more than 500 days after the retraction date, meaning the review authors had a substantial window in which they could have caught the problem and did not.

Oncology was the most contaminated field, accounting for 16.1 percent of affected reviews. That finding echoes other analyses of paper mill distribution, including the BERT-based AI screening of cancer literature published as a bioRxiv preprint in August 2025, which flagged roughly 9.87 percent of cancer research papers as potentially originating from paper mills. The concentration in oncology is not coincidental. Cancer research is a high-citation field with strong commercial incentives for authorship, and the journals that accepted paper mill articles in large numbers from the mid-2010s through the early 2020s had a pronounced concentration in biomedical and clinical oncology.

The 1,350 percent increase in contaminated reviews between 2016 and 2023 is the figure that should concern authors most. The growing number of paper mill retractions over this period means more retracted papers in circulation at any moment, and systematic reviewers who searched the literature in 2020 or 2021 may have incorporated papers that were only retracted in 2023 or 2024. If those reviews have not been corrected, they remain in circulation with contaminated evidence bases.

When Paper Mill Articles Reach Clinical Guidelines

The second analysis, which Retraction Watch covered on September 17, 2026, traced where papers affiliated with the Pharmakon Neuroscience Network ended up after publication. The Pharmakon Neuroscience Network is not a real research institution. It is an authorship-for-sale scheme identified by Retraction Watch in 2022 and since the subject of multiple journal investigations. Researchers at the network sell co-authorship positions on papers, with the papers themselves written to formula and submitted to journals accepting lower-quality submissions.

Silvi and McIntosh searched the Dimensions scholarly database for downstream citations to 1,975 PNN-affiliated papers. The numbers are sobering in context. Of those papers, 480 were cited in patent filings, 57 were cited in policy documents, and 12 were cited in clinical guidelines. To be clear about what this means: researchers writing clinical practice recommendations consulted papers from what amounts to a commercial fraud operation and treated them as evidence. Whether those researchers knew the papers were suspicious or simply failed to verify the source is not known, but the practical effect is the same.

The Pharmakon Neuroscience Network is only one of many paper mill operations that have been identified. Others, including the Ayhan Dincol Network and the clusters of Moldovan and Egyptian institutional addresses that have appeared repeatedly in retraction records, have generated many hundreds of papers that may follow similar downstream trajectories. The PNN analysis reveals a pattern, not an exception.

How Contamination Travels Up the Evidence Pyramid

The traditional evidence pyramid places randomized controlled trials at or near the apex and systematic reviews above individual primary studies because synthesis provides a more complete picture than any single study. Paper mills understood this dynamic before most integrity researchers did. A fabricated result that lands in a systematic review gains a kind of laundered credibility. The review itself may be legitimate, written by real researchers with no knowledge that one of their sources was fraudulent. But when that review is cited in a guideline, the fraudulent data has effectively passed three filters without detection.

The process is made easier by the scale of modern systematic reviews. A typical meta-analysis in oncology or cardiology may incorporate dozens to hundreds of individual studies. Authors conducting large network meta-analyses might screen thousands of abstracts before narrowing to an included set. In that environment, verifying each included study against a retraction database is genuinely burdensome, and in practice it rarely happens systematically. Searches against Retraction Watch's database are not built into major reference managers by default, and automated tools that check reference lists for retracted papers are not yet standard in systematic review platforms like Covidence or Rayyan.

The post-retraction citation problem compounds this. When the JAMA Network Open study found 124 citations occurring after retraction, including 13 more than 500 days after, it was documenting the time lag between retraction and the updating of reference databases. Papers retracted after formal notification often remain visible in search results without prominent retraction notices for months. PubMed has improved its retraction tagging significantly over the past few years, but the improvement is recent and not uniform across all databases. Authors who search only Embase or Scopus may not see a retraction notice that has been added to PubMed but not yet propagated elsewhere.

Fields with Elevated Risk

Not all areas of medicine carry the same paper mill contamination risk, and authors writing systematic reviews should calibrate their verification intensity accordingly. Oncology has consistently shown the highest exposure in multiple analyses. Molecular biology, biochemistry, and cell biology also show elevated rates, likely because these fields have large journal markets with lower average citation thresholds for acceptance. Studies of traditional Chinese medicine and herbal preparations have been flagged in multiple analyses as disproportionately contaminated, in part because a cluster of paper mills operated primarily through journals in that niche for several years.

Cardiovascular medicine, immunology, and diabetes research have also produced contaminated systematic reviews. A January 2026 analysis published in PubMed Central examining retractions in cardiovascular literature from the Retraction Watch database identified multiple instances of paper mill-adjacent retractions, though that study focused primarily on clinical cardiovascular research rather than basic science.

For systematic reviewers working in lower-risk areas, the practical implication is not necessarily to screen every citation at the same level of intensity as an oncology network meta-analysis. But it does mean developing a working sense of which papers in your included set come from journal types or institutional affiliations that have been associated with paper mills in the past, and applying closer scrutiny to those specific inclusions.

What Journal Editors and Guideline Panels Are Doing

The response from the major publishers has been incremental. Springer Nature, Elsevier, and Wiley have all expanded their post-publication integrity teams and developed internal tools for flagging paper mill signals. But these tools focus on detecting paper mills in new submissions, not on retroactively auditing the citations in previously published systematic reviews. The COPE September 2025 retraction guideline update, Version 3, added new provisions for batch retractions and paper mill products, but COPE guidelines govern how journals respond to identified problems, not how to find them prospectively in the existing literature.

Clinical guideline panels, including those producing NICE guidelines in the UK and many AHA/ACC guidelines in the United States, rely on systematic reviews as their evidence base without typically having independent capacity to verify the integrity of every primary study cited in every included review. That is structurally a gap. When a paper mill article enters a systematic review that is then incorporated into a guideline, the guideline panel almost certainly has no visibility into the chain. Improving this requires either better upstream screening at the systematic review stage or clearer communication from journals to guideline bodies when a retraction may affect an evidence base.

There have been early discussions at bodies like the Guidelines International Network about whether systematic review platforms should mandate retraction screening as a condition of inclusion in guideline databases, but as of October 2026 these are discussions rather than requirements. The field is moving, but it is moving slowly relative to the scale of contamination already present in the evidence base.

Tools for Screening Systematic Review Bibliographies

Practical tools exist for authors willing to use them, and their use should now be considered standard practice for any systematic review being prepared in a field with known paper mill exposure. The Crossref Retraction Watch API is the most comprehensive free option. It allows batch querying of DOIs against the Retraction Watch database and returns retraction status and reasons. For a systematic review with a final included set of 50 to 200 studies, running all DOIs through the API at the end of the review process adds one to two hours of work and provides documentary evidence that the retraction check was performed.

Scite.ai provides a paid option that integrates retraction signals and contrasting citations into reference manager exports. For teams with institutional subscriptions, scite can flag included papers that have been disputed, corrected, or retracted with greater contextual detail than a simple retraction database lookup. Lens.org and Semantic Scholar both expose retraction metadata, though less comprehensively than Retraction Watch.

For researchers in high-risk fields, or those conducting systematic reviews with meta-analytic components where a single fraudulent data point can distort the pooled estimate, a more thorough verification approach is worth considering. The Open Science Framework hosts several tools built by the research integrity community for systematic screening of paper mill signals, including image duplication checkers for studies where figures are a concern and template pattern detectors that flag studies with the characteristic stylistic regularities of paper mill products.

What to Do When You Find a Problem

Finding a retracted or suspected paper mill article in your included set after the review is written is not an emergency, but it does require a decision. If the study was retracted before your search date and you missed it, that is a quality control failure that should be corrected in the manuscript. Remove the study from the meta-analysis, note the removal in a sensitivity analysis, and update the methods to describe when you performed the retraction check and what it found.

If the retraction occurred after your search date but before submission, you are not strictly obligated to have caught it, but updating your included set at that point is still good practice, especially if the retracted study contributed meaningfully to a pooled estimate or a key recommendation. If the review has already been published and a included study is subsequently retracted, most journals now expect a post-publication correction or update notice. COPE guidelines are explicit that retraction of a source study can require action from authors of secondary publications that relied on that study.

If you find a paper mill article that has not yet been retracted but shows strong indicators of being a paper mill product (characteristic phrasing, impossible data values, authors with identical institutional names that turn out to be non-existent, image duplications in figures), the appropriate step is to report the concern to the publishing journal through their editorial integrity contact, typically via the relevant publisher's research integrity form. You can then note in your systematic review that one included study has been flagged to the journal and is under investigation, and either exclude it pending investigation or run a sensitivity analysis showing the estimate with and without the flagged study.

Disclosing Your Screening Process in the Methods

The PRISMA 2020 checklist, and the newer PRISMA 2026 update published in August 2026, include items on data source searching and quality assessment that can accommodate retraction screening as a methodological step. But there is no single PRISMA item that specifically mandates retraction checking. Until such a requirement is codified, authors have some flexibility in how they describe the process, and many currently omit it entirely even when they have performed a check.

Describing your retraction check in the Methods is worth one or two sentences. A typical statement might read: “All included studies were checked against the Retraction Watch database using the Crossref Retraction Watch API (check performed October 2026) prior to final analysis. No retracted studies were identified in the included set,” or, if relevant: “One included study was identified as retracted after the initial search date; sensitivity analysis excluding this study did not materially alter the pooled estimate.”

Some journals are beginning to ask about this explicitly in peer review. Cochrane now recommends retraction checking as part of review preparation, and several oncology journals have added a question to their reviewer forms asking whether included studies in a systematic review have been verified as non-retracted. This is still the exception rather than the rule, but the direction of travel is clear.

The Longer Problem: Updating the Existing Literature

The analyses published in 2025 and 2026 describe the contamination that currently exists in already-published systematic reviews. Correcting that is a different and harder problem than preventing new contamination. Of the 299 contaminated reviews identified in the JAMA Network Open study, the researchers found no evidence that more than a small fraction had been corrected or updated after the paper mill retractions occurred. The reviews exist in PubMed, are cited in downstream work, and in some cases have contributed to clinical guidelines that are themselves updated infrequently.

This is partly a workflow problem. There is no automatic mechanism by which a journal that publishes a retraction notifies every journal that published a systematic review citing the retracted paper. Retraction Watch's Crossref integration is the closest thing to that mechanism, but it depends on downstream authors and editors checking proactively rather than being notified passively. Until that infrastructure exists, the best available tool is the growing body of published guidance encouraging systematic reviewers to treat retraction checking as routine rather than exceptional.

For authors of living systematic reviews, there is a practical advantage here. A living review with scheduled updates already has a mechanism for incorporating new retraction information as part of each update cycle. The challenge is for the much larger stock of published systematic reviews that are not living reviews and whose authors have no standing update obligations. Many of those reviews were published by teams that have since dispersed, with no practical way to enforce a correction even when the need for one is clear.

What This Means for Authors Right Now

If you are currently writing or updating a systematic review, the practical message from this evidence is simple: build a retraction check into your workflow before submission, not after. The tools exist, the process is not technically difficult, and the cost of not checking has become harder to justify as the contamination evidence accumulates. For reviews in oncology, molecular biology, or any of the other fields with documented paper mill exposure, a retraction check should be considered as mandatory as a search strategy documentation or a risk-of-bias assessment.

If you have a previously published systematic review that you know has not been checked against Retraction Watch, consider running that check now. If you find a problem, COPE guidelines are clear that authors can file voluntary correction requests with the publishing journal. Most journals respond positively to authors who proactively identify and report issues in their own published work. The reputational cost of a voluntary correction is far lower than the reputational cost of having a problem identified post-hoc by someone else.

The harder recommendation is for anyone involved in clinical guideline development. The September 2026 preprint and the broader evidence of systematic review contamination suggest that guideline panels should treat provenance verification as a step in the evidence evaluation process, particularly for reviews published before 2023 in high-risk fields. This is asking guideline panels to take on additional work without additional resources, which is a real constraint. But it is a constraint that the evidence now makes difficult to ignore.

Further Reading