Collections > Scholarly Posters and Presentations > Knowing When to Stop: Final Results vs. Work Involved in Systematic Review Database Searching
pdf

Objectives: To examine the relationship between the amount of work involved for the systematic review (SR) team for each additional database searched in an SR versus the number of unique articles included in the final review that were retrieved from each additional database searched. Methods: We reviewed SRs and meta-analyses published in JAMA, Lancet, and Annals of Internal Medicine from January 2012 to December 2015, selecting those journals as high impact factor journals that have published over more than 50 SRs per year since 2012. We required that all papers in our analysis report a complete search strategy, with the total number of citations retrieved from each database and a complete list of articles included in the final analysis. We then used lists of included journals from each database to determine in which database(s) each of the included articles were indexed, and we calculated the percentage of articles included in each SR that could be located by searching each database. We then compared the number of results added by searching each additional database to the total number of papers added to the review phase to estimate the amount of work required for each additional paper identified. Results: Ninety-seven SRs met our inclusion criteria. These SRs included an average of 48.13 journal articles and searched an average of 4.43 databases each. Of these, the journal articles included in 16 SRs could all be found in one database; the articles included in 58 SRs could all be found in two databases. For 20 SRs, all included articles could be found in three databases, and the remaining three SRs included articles that could be found in four databases. For 96 SRs, over 90% of articles could be found in two databases. Conclusions: In total, 99% of articles included in each SR were found in two databases, with the majority being found in PubMed/MEDLINE, Embase, or Cochrane. SRs that found articles in three or more databases screened an additional 923 records in order to find one additional included article, plus an additional 2410 records from databases that did not return any additional included articles, adding an average of 756 hours of work to each SR. SRs for which articles were found in four databases screened an average of 1440 records in order to find one additional included article, plus an additional 8963 records from databases that did not return any additional included articles.