User testing of an adaptation of fishbone diagrams to depict results of systematic reviews

Public Deposited
Analytics

Downloadable Content

Download PDF
Request Version for Screen Reader
Creator
  • Gartlehner, Gerald
    • Other Affiliation: RTI International, Durham, USA; Department for Evidence-based Medicine and Clinical Epidemiology, Krems, Austria
  • Schultes, Marie-Therese
    • Affiliation: Gillings School of Global Public Health, Department of Maternal and Child Health
    • Other Affiliation: Department for Evidence-based Medicine and Clinical Epidemiology, Krems, Austria
  • Titscher, Viktoria
    • Other Affiliation: Department for Evidence-based Medicine and Clinical Epidemiology, Krems, Austria
  • Morgan, Laura C
    • Other Affiliation: RTI International, Durham, USA
  • Bobashev, Georgiy V
    • Other Affiliation: RTI International, Durham, USA
  • Williams, Peyton
    • Other Affiliation: RTI International, Durham, USA
  • West, Suzanne L
    • Other Affiliation: RTI International, Durham, USA
Abstract
  • Abstract Background Summary of findings tables in systematic reviews are highly informative but require epidemiological training to be interpreted correctly. The usage of fishbone diagrams as graphical displays could offer researchers an effective approach to simplify content for readers with limited epidemiological training. In this paper we demonstrate how fishbone diagrams can be applied to systematic reviews and present the results of an initial user testing. Methods Findings from two systematic reviews were graphically depicted in the form of the fishbone diagram. To test the utility of fishbone diagrams compared with summary of findings tables, we developed and pilot-tested an online survey using Qualtrics. Respondents were randomized to the fishbone diagram or a summary of findings table presenting the same body of evidence. They answered questions in both open-ended and closed-answer formats; all responses were anonymous. Measures of interest focused on first and second impressions, the ability to find and interpret critical information, as well as user experience with both displays. We asked respondents about the perceived utility of fishbone diagrams compared to summary of findings tables. We analyzed quantitative data by conducting t-tests and comparing descriptive statistics. Results Based on real world systematic reviews, we provide two different fishbone diagrams to show how they might be used to display complex information in a clear and succinct manner. User testing on 77 students with basic epidemiological training revealed that participants preferred summary of findings tables over fishbone diagrams. Significantly more participants liked the summary of findings table than the fishbone diagram (71.8% vs. 44.8%; p < .01); significantly more participants found the fishbone diagram confusing (63.2% vs. 35.9%, p < .05) or indicated that it was difficult to find information (65.8% vs. 45%; p < .01). However, more than half of the participants in both groups were unable to find critical information and answer three respective questions correctly (52.6% in the fishbone group; 51.3% in the summary of findings group). Conclusions Fishbone diagrams are compact visualizations that, theoretically, may prove useful for summarizing the findings of systematic reviews. Initial user testing, however, did not support the utility of such graphical displays.
Date of publication
DOI
Identifier
Resource type
  • Article
Rights statement
  • In Copyright
Rights holder
  • The Author(s).
Language
  • English
Bibliographic citation
  • BMC Medical Research Methodology. 2017 Dec 12;17(1):169
Publisher
  • BioMed Central

Relations

Parents:

This work has no parents.

Items

Thumbnail Title Date Uploaded Visibility Actions
file details 12874_2017_article_452.pdf 2019-05-06 Public Download
file details 12874_2017_452_moesm2_esm.docx 2019-05-06 Public Download
file details 12874_2017_452_moesm4_esm.docx 2019-05-06 Public Download
file details 12874_2017_452_moesm1_esm.docx 2019-05-06 Public Download
file details 12874_2017_452_moesm3_esm.docx 2019-05-06 Public Download