On Dec. 18, 2024, A huge collaboration has confirmed growing concerns that fake or flawed research is polluting medical systematic reviews, which summarize evidence from multiple clinical trials and shape treatment guidelines worldwide. The study is part of an effort to address the problem by creating a short checklist that will help researchers to spot untrustworthy trials. Combined with automated integrity tools, this could help those conducting systematic reviews to filter out flawed work — in medicine and beyond.

In the study, which took two years and was posted to the medRxiv preprint server, a team of more than 60 researchers trawled through 50 systematic reviews published under the aegis of Cochrane, an organization renowned for its gold-standard reviews of medical evidence. After applying a barrage of checks, the authors — many of whom are themselves editors or authors of Cochrane reviews — reported that they had “some concerns” about 25% of the clinical trials in the reviews, and “serious concerns” about 6% of them.

The results echo previous concerns about rising numbers of problematic studies corrupting systematic reviews in medicine and other research fields, probably owing to paper mills that produce fake science.