Reinforcement learning design for cancer clinical trials

Public Deposited
Analytics

Downloadable Content

Download PDF
Request Version for Screen Reader
Last Modified
  • March 21, 2019
Creator
  • Zhao, Yufan
    • Affiliation: Gillings School of Global Public Health, Department of Biostatistics
Abstract
  • There has been significant recent research activity in developing therapies that are tailored to each individual. Finding such therapies in treatment settings involving multiple decision times is a major challenge. In this dissertation, we develop reinforcement learning trials for discovering these optimal regimens for life-threatening diseases such as cancer. A temporal-difference learning method called Q-learning is utilized which involves learning an optimal policy from a single training set of finite longitudinal patient trajectories. Approximating the Q-function with time-indexed parameters can be achieved by using support vector regression or extremely randomized trees. Within this framework, we demonstrate that the procedure can extract optimal strategies directly from clinical data without relying on the identification of any accurate mathematical models, unlike approaches based on adaptive design. We show that reinforcement learning has tremendous potential in clinical research because it can select actions that improve outcomes by taking into account delayed effects even when the relationship between actions and outcomes is not fully known. To support our claims, the methodology's practical utility is firstly illustrated in a virtual simulated clinical trial. We then apply this general strategy with significant refinements to studying and discovering optimal treatments for advanced metastatic stage IIIB/IV non-small cell lung cancer (NSCLC). In addition to the complexity of the NSCLC problem of selecting optimal compounds for first and second-line treatments based on prognostic factors, another primary scientific goal is to determine the optimal time to initiate second-line therapy, either immediately or delayed after induction therapy, yielding the longest overall survival time. We show that reinforcement learning not only successfully identifies optimal strategies for two lines of treatment from clinical data, but also reliably selects the best initial time for second-line therapy while taking into account heterogeneities of NSCLC across patients.
Date of publication
DOI
Resource type
Rights statement
  • In Copyright
Advisor
  • Kosorok, Michael
Degree granting institution
  • University of North Carolina at Chapel Hill
Language
Access right
  • Open access
Date uploaded
  • October 11, 2010

Relations

Parents:

This work has no parents.

Items

Thumbnail Title Date Uploaded Visibility Actions
file details Reinforcement learning design for cancer clinical trials 2019-04-11 Public Download