The Society for Implementation Research Collaboration Instrument Review Project: A methodology to promote rigorous evaluation

Public Deposited
Analytics

Downloadable Content

Download PDF
Request Version for Screen Reader
Creator
  • Lewis, Cara C
    • Other Affiliation: Department of Psychological and Brain Sciences, Indiana University, 1101 E. 10th St., Bloomington, IN 47405, USA; Department of Psychiatry and Behavioral Sciences, Harborview Medical Center, School of Medicine, University of Washington, Box 359911, 325 9th Ave, Seattle, WA 98104, USA
  • Stanick, Cameo F
    • Other Affiliation: Department of Psychology, University of Montana, 32 Campus Dr., Skaggs Bldg. 362, Missoula, MT 59812, USA
  • Martinez, Ruben G
    • Other Affiliation: Department of Psychological and Brain Sciences, Indiana University, 1101 E. 10th St., Bloomington, IN 47405, USA; Department of Psychology, Virginia Commonwealth University, 806 West Franklin St, Richmond, VA 23220, USA
  • Weiner, Bryan
    • Other Affiliation: Department of Psychological and Brain Sciences, Indiana University, 1101 E. 10th St., Bloomington, IN 47405, USA; Department of Psychiatry and Behavioral Sciences, Harborview Medical Center, School of Medicine, University of Washington, Box 359911, 325 9th Ave, Seattle, WA 98104, USA
  • Kim, Mimi M
    • Other Affiliation: North Carolina Translational and Clinical Sciences Institute
  • Barwick, Melanie
    • Other Affiliation: Child Health Evaluative Sciences, Research Institute, The Hospital for Sick Children, 555 University Avenue, Toronto, ON M5G 1X8, Canada
  • Comtois, Katherine A
    • Other Affiliation: Department of Psychiatry and Behavioral Sciences, Harborview Medical Center, School of Medicine, University of Washington, Box 359911, 325 9th Ave, Seattle, WA 98104, USA
Abstract
  • Abstract Background Identification of psychometrically strong instruments for the field of implementation science is a high priority underscored in a recent National Institutes of Health working meeting (October 2013). Existing instrument reviews are limited in scope, methods, and findings. The Society for Implementation Research Collaboration Instrument Review Project’s objectives address these limitations by identifying and applying a unique methodology to conduct a systematic and comprehensive review of quantitative instruments assessing constructs delineated in two of the field’s most widely used frameworks, adopt a systematic search process (using standard search strings), and engage an international team of experts to assess the full range of psychometric criteria (reliability, construct and criterion validity). Although this work focuses on implementation of psychosocial interventions in mental health and health-care settings, the methodology and results will likely be useful across a broad spectrum of settings. This effort has culminated in a centralized online open-access repository of instruments depicting graphical head-to-head comparisons of their psychometric properties. This article describes the methodology and preliminary outcomes. Methods The seven stages of the review, synthesis, and evaluation methodology include (1) setting the scope for the review, (2) identifying frameworks to organize and complete the review, (3) generating a search protocol for the literature review of constructs, (4) literature review of specific instruments, (5) development of an evidence-based assessment rating criteria, (6) data extraction and rating instrument quality by a task force of implementation experts to inform knowledge synthesis, and (7) the creation of a website repository. Results To date, this multi-faceted and collaborative search and synthesis methodology has identified over 420 instruments related to 34 constructs (total 48 including subconstructs) that are relevant to implementation science. Despite numerous constructs having greater than 20 available instruments, which implies saturation, preliminary results suggest that few instruments stem from gold standard development procedures. We anticipate identifying few high-quality, psychometrically sound instruments once our evidence-based assessment rating criteria have been applied. Conclusions The results of this methodology may enhance the rigor of implementation science evaluations by systematically facilitating access to psychometrically validated instruments and identifying where further instrument development is needed.
Date of publication
DOI
Identifier
Resource type
  • Article
Rights statement
  • In Copyright
Rights holder
  • Lewis et al.; licensee BioMed Central.
Language
  • English
Bibliographic citation
  • Implementation Science. 2015 Jan 08;10(1):2
Publisher
  • BioMed Central

Relations

Parents:

This work has no parents.

Items

Thumbnail Title Date Uploaded Visibility Actions
file details 13012_2014_article_193.pdf 2019-05-06 Public Download
file details 13012_2014_193_moesm1_esm.docx 2019-05-06 Public Download