Bias in trials comparing paired continuous tests can cause researchers to choose the wrong screening modality

Public Deposited
Analytics

Downloadable Content

Download PDF
Request Version for Screen Reader
Creator
  • Glueck, Deborah H.
    • Other Affiliation: Department of Biostatistics, Colorado School of Public Health, University of Colorado, Denver, Aurora, CO, USA
  • Lamb, Molly M.
    • Other Affiliation: Department of Epidemiology, Colorado School of Public Health, University of Colorado, Denver, Aurora, CO, USA
  • O'Donnell, Colin I.
    • Other Affiliation: Department of Biostatistics, Colorado School of Public Health, University of Colorado, Denver, Aurora, CO, USA
  • Ringham, Brandy M.
    • Other Affiliation: Department of Biostatistics, Colorado School of Public Health, University of Colorado, Denver, Aurora, CO, USA
  • Brinton, John T.
    • Other Affiliation: Department of Biostatistics, Colorado School of Public Health, University of Colorado, Denver, Aurora, CO, USA
  • Muller, Keith E.
    • Other Affiliation: Division of Biostatistics, Department of Epidemiology and Health Policy Research, University of Florida, Gainesville, FL, USA
  • Lewin, John M.
    • Other Affiliation: Diversified Radiology of Colorado, Denver, CO, USA
  • Alonzo, Todd A.
    • Other Affiliation: Department of Preventive Medicine, University of Southern California, Arcadia, CA, USA
  • Pisano, Etta D.
    • Affiliation: School of Medicine, Department of Radiology
Abstract
  • Abstract Background To compare the diagnostic accuracy of two continuous screening tests, a common approach is to test the difference between the areas under the receiver operating characteristic (ROC) curves. After study participants are screened with both screening tests, the disease status is determined as accurately as possible, either by an invasive, sensitive and specific secondary test, or by a less invasive, but less sensitive approach. For most participants, disease status is approximated through the less sensitive approach. The invasive test must be limited to the fraction of the participants whose results on either or both screening tests exceed a threshold of suspicion, or who develop signs and symptoms of the disease after the initial screening tests. The limitations of this study design lead to a bias in the ROC curves we call paired screening trial bias. This bias reflects the synergistic effects of inappropriate reference standard bias, differential verification bias, and partial verification bias. The absence of a gold reference standard leads to inappropriate reference standard bias. When different reference standards are used to ascertain disease status, it creates differential verification bias. When only suspicious screening test scores trigger a sensitive and specific secondary test, the result is a form of partial verification bias. Methods For paired screening tests with bivariate normally distributed scores, we give formulae and programs to quantify the effect of paired screening trial bias on a paired comparison of area under the curves. We fix the prevalence of disease, and the chance a diseased subject manifests signs and symptoms. We derive the formulas for true sensitivity and specificity, and those for the sensitivity and specificity observed by the study investigator. Results The observed area under the ROC curves is quite different from the true area under the ROC curves. The typical direction of the bias is a strong inflation in sensitivity, paired with a concomitant slight deflation of specificity. Conclusion In paired trials of screening tests, when area under the ROC curve is used as the metric, bias may lead researchers to make the wrong decision as to which screening test is better.
Date of publication
DOI
Identifier
Resource type
  • Article
Rights statement
  • In Copyright
Rights holder
  • Deborah H Glueck et al.; licensee BioMed Central Ltd.
License
Journal title
  • BMC Medical Research Methodology
Journal volume
  • 9
Journal issue
  • 1
Page start
  • 4
Language
  • English
Is the article or chapter peer-reviewed?
  • Yes
ISSN
  • 1471-2288
Bibliographic citation
  • BMC Medical Research Methodology. 2009 Jan 20;9(1):4
Publisher
  • BioMed Central Ltd
Access right
  • Open Access
Date uploaded
  • July 15, 2016

Relations

Parents:

This work has no parents.

Items

Thumbnail Title Date Uploaded Visibility Actions
file details 12874_2008_article_320.pdf 2019-05-06 Public Download
file details 12874_2008_320_moesm1_esm.pdf 2019-05-06 Public Download
file details 12874_2008_320_moesm3_esm.pdf 2019-05-06 Public Download
file details 12874_2008_320_moesm4_esm.pdf 2019-05-06 Public Download
file details 12874_2008_320_moesm2_esm.pdf 2019-05-06 Public Download