Abstracts

2024 Annual Symposium on Risks and Opportunities of AI in Pharmaceutical Medicine

Day 2 Plenary Session 1

Title:

Empowering Insights in the All of Us Research Program: A Statistical Perspective on the Transformational Role of AI and ML, Qingxia ‘Cindy’ Chen, Biomedical Informatics, and Ophthalmology & Visual Sciences at Vanderbilt University Medical Center

Abstract:

The All of Us Research Program is building a diverse cohort of a million or more Americans engaged longitudinally in data sharing including biospecimens, electronic health records, surveys, and digital health technology to advance precision medicine research and fuel new insights into human health. This data is at the fingertips of researchers through the Researcher Workbench—a cloud-based analytical platform equipped with graphical interface tools and bolstered by Jupyter notebooks. Underpinned by the FAIR principles, these resources render data and discoveries Findable, Accessible, Interoperable, and Reproducible in the domain of biomedical research. The All of UsResearch Program leverages cutting-edge AI and ML methods. These endeavors include devising models to enhance missing data with external resources and various data types, forecasting participant engagement, and refining polygenetic risk scores for underrepresented populations within biomedicine. At its core, the All of Us Researcher Workbench stands as a dynamic hub, fostering methodological innovation and catalyzing outcome research.

Title:

“The Generalist Medical AI Will See You Now”, Pranav Rajpurkar, Ph.D., Harvard University

Abstract:

Accurate interpretation of medical images is crucial for disease diagnosis and treatment, and AI has the potential to minimize errors, reduce delays, and improve accessibility. The focal point of this presentation lies in a grand ambition: the development of ‘Generalist Medical AI’ systems that can closely resemble doctors in their ability to reason through a wide range of medical tasks, incorporate multiple data modalities, and communicate in natural language. Starting with pioneering algorithms that have already demonstrated their potential in diagnosing diseases from chest X-rays or electrocardiograms, matching the proficiency of expert radiologists and cardiologists, I will delve into the core challenges and advancements in the field. The discussion will navigate towards the topic of label-efficient AI models: with a scarcity of meticulously annotated data in healthcare, the development of AI systems capable of learning effectively from limited labels has become a key concern. In this vein, I’ll delve into how the innovative use of self-supervision and pre-training methods has led to algorithmic advancements that can perform high-level diagnostic tasks using significantly less annotated data. Additionally, I will talk about initiatives in data curation, human-AI collaboration, and the creation of open benchmarks to evaluate the generalizability of medical AI algorithms. In summary, this talk aims to deliver a comprehensive picture of the state of ‘Generalist Medical AI,’ the advancements made, the challenges faced, and the prospects lying ahead.

Title:

“Generative AI for Case Adjudication in OHDSI”, Marc Suchard, UCLA

Abstract:

Observational Health Data Science and Informatics (OHDSI) aims to improve health by empowering an open-science community to collaboratively generate the evidence that promotes better health decisions and better care. OHDSI is actively exploring ways in which large language models (LLMs) can help deliver this aim through the Assessment of Pre-trained Observational Large Longitudinal models in OHDSI (APOLLO) project. This talk highlights two APOLLO directions. First, medical case validation stands as a necessary element of regulatory-grade evidence, but currently requires human adjudication that is time and resource-intensive. APOLLO provides open-source development of LLMs to improve the reliability and scalability of case validation. Across 10 diseases in 2 different data sources, we find substantial heterogeneity in agreement between human reviewers and that LLMs agree with humans as much as humans agree with each other. Second, APOLLO explores the use of LLMs pre-trained to the Observational Medical Outcomes Partnership (OMOP) Common Data Model (CDM) data sources to perform patient-level prediction, where a pre-trained model may prove more accurate with less training data than current non-pre-trained models, missing value imputation, almost identical to a bidirectional pre-training task, and counterfactual prediction. In this latter task, the LLM provides a prediction of what happens to a patient in the future given both treatment options. We suspect further applications will continue to emerge across the OHDSI community.

Day 2 Plenary Session 2

Title:

“Generative Mixed-Response State-Space Model for Analyzing Multi-Dimensional Digital Phenotypes”, Yuanjia Wang, Columbia University

Abstract:

Digital technologies (e.g., mobile phones) can be used to obtain objective, frequent, and real-world digital phenotypes from individuals. However, modeling these data poses substantial challenges since observational data are subject to confounding and various sources of variabilities. For example, signals on patients’ underlying health status and treatment effects are mixed with variation due to the living environment and measurement noises. The digital phenotype data thus shows extensive variabilities between and within patient as well as across different health domains (e.g., motor, cognitive, and speaking). Motivated by a mobile health study of Parkinson’s disease (PD), we develop a mixed-response state-space (MRSS) model to jointly capture multi-dimensional, multi-modal digital phenotypes and their measurement processes by a finite number of latent state time series. These latent states reflect the dynamic health status and personalized time-varying treatment effects and can be used to adjust for informative measurements. We conduct comprehensive simulation studies and demonstrate the advantage of MRSS in modeling a mobile health study that remotely collects real-time digital phenotypes from PD patients. We discuss extensions to deep latent state-space models for generating digital phenotype time-series data to learn optimal treatment strategies.

Title:

“Machine Learning for Causal Inference”, Stefan Wager, Stanford University

Abstract:

Given advances in machine learning over the past decades, it is now possible to accurately solve difficult non-parametric prediction problems in a way that is routine and reproducible. In this talk, I’ll discuss how machine learning tools can be rigorously integrated into observational study analyses, and how they interact with classical statistical ideas around randomization, semiparametric modeling, double robustness, etc. I’ll also survey some recent advances in methods for treatment heterogeneity. When deployed carefully, machine learning enables us to develop causal estimators that reflect an observational study design more closely than basic linear regression based methods.

Title:

“Leveraging the Power of Large Language Models (LLMs) by Statisticians for Pharmaceutical Research & Development: Illustrating with an Example AI App”, Junshui Ma, Head of the Biometrics Research Department, Merck Research Lab.

Abstract:

This presentation aims to explore the transformative potential of ChatGPT-like large language models (LLMs) in the realm of pharmaceutical research and development (R&D), with a specific focus on its application by statisticians. The intricate nature of pharmaceutical R&D necessitates that statisticians in this industry consistently work within multidisciplinary teams. They are expected to rapidly acquire a comprehensive understanding of the scientific and medical domains they engage with. LLMs have showcased a broad spectrum of skills, such as drafting and refining messages, translating across various languages, elucidating and summarizing documents, programming in multiple computer languages, strategizing, and reasoning for problem-solving. They also possess knowledge in numerous fields, including statistics, science, and clinical medicine, among others. Statisticians who can harness this technology have demonstrated a significant increase in efficiency in fulfilling their roles within the team. Real-world cases showcasing the successful implementation of this technology in various settings will be presented. The presentation will conclude with a discussion on strategies to address general issues related to the use of LLMs.

Day 2 Plenary Session 3

Title:

The Role of Targeted Machine Learning in a Causal Roadmap for Generating High-Quality Real-World Evidence, Lauren Elizabeth Eyler Dang, National Institute of Allergy and Infectious Diseases Biostatistics Research Branch

Abstract:

A broad spectrum of studies use real-world data (RWD) to produce real-world evidence (RWE), ranging from randomized controlled trials with outcomes assessed using RWD to fully observational studies. The statistical target parameter that answers the research question in such studies is usually not equivalent to a coefficient in a potentially misspecified parametric model; machine learning algorithms can help to avoid parametric modeling assumptions but must be implemented in a way that provides reliable inference. The Targeted Machine Learning approach is aimed at efficiently and consistently estimating target parameters that directly answer research questions through a rigorous, pre-specified process that minimizes statistical assumptions. This approach involves estimating components of the observed data likelihood that are relevant for the selected parameter using data-adaptive ensemble machine learning (Super Learning). Targeted maximum likelihood estimation (TMLE) is then used to target initial estimates to optimize the bias-variance tradeoff for the target parameter of interest while providing accurate inference. Petersen and van der Laan’s Roadmap for Causal Inference provides a unifying framework for translating causal questions and true knowledge into a statistical parameter that may be estimated using a targeted approach. This talk will use real and simulated data examples to introduce fundamental concepts in Targeted Learning, including the Causal Roadmap, Super Learning, and TMLE applied to estimation of a causal average treatment effect. Examples of more complex target parameters and a brief overview of software for user-friendly implementation of these methods through the tlverse software ecosystem will be given.

Title:

“The multitude of group affiliations: Algorithmic Fairness, Loss Minimization and Outcome Indistinguishability Omer Reingold, Stanford University

Abstract:

We will discuss a rather recent and very fruitful line of research in algorithmic fairness, coined multi-group fairness. We will focus on risk prediction, where a machine learning algorithm tries to learn a predictor to answer questions of the form “what is the probability that patient x will have a particular adverse reaction to a drug?” Training a risk predictor to minimize a loss function fixed in advance is the dominant paradigm in machine learning. However, global loss minimization may create predictions that are mis-calibrated on sub-populations, causing harm to individuals of these populations. Multi-group fairness tries to prevent forms of discrimination to a rich (possibly exponential) collection of arbitrarily intersecting groups. In a sense, it provides a computational perspective on the meaning of individual risks and the classical tension between clinical prediction, which uses individual-level traits, and actuarial prediction, which uses group-level traits. Surprisingly, it also provides unique robustness properties to the learned risk predictors, such as robustness to distributional shifts and to changing objectives. Multi-group fairness has found practical usages in health care, which we will discuss as well.

Based on a sequence of works joint with (subsets of) Cynthia Dwork, Shafi Goldwasser, Parikshit Gopalan, Úrsula Hébert-Johnson, Lunjia Hu, Adam Kalai, Christoph Kern, Michael P. Kim, Frauke Kreuter, Guy N. Rothblum, Vatsal Sharan, Udi Wieder, Gal Yona and others.

Modeling Covid-19 Immunological Reactions and Clinical Susceptibility in the Context of Long-Term Follow-up of a Prospective Cohort of Healthcare Workers, Noam Barda, Ben-Gurion University. (25 min)

Title:

“Designs for Observational Vaccine Effectiveness Studies”, Noam Barda, Ben-Gurion University

Abstract:

The Sheba healthcare worker cohort is a prospective cohort of ~15,000 healthcare workers that intensively monitors Covid-19 infections, symptoms, and vaccinations. The cohort’s most unique characteristic is, however, its frequent tracking of participants’ immune response with monthly serological tests of antibody levels, neutralizing antibody response, and cell-mediated immunity markers. Many studies based on the cohort’s data have been published in prestigious journals, and have helped direct public health policy. Previous studies based on the cohort’s data have shown that neutralizing antibodies are strong correlates for protection, and that immune response differs substantially between individuals. We now hypothesize, that with over two years of clinical and immunological data, spanning multiple SARS-CoV-2 variants and vaccine types, data from the cohort can be used to predict an individual’s immune response to vaccination and infection, and to recommend optimal times for additional vaccination.

Title:

“Drugs in clinical trials out of generative AI”, Petrina Kamya , Insilico Medicine.

Abstract:

Abstract. Generative AI is not new. GPT3 was published in mid-2020. In biomedicine, the generative AI revolution started around 2017 with generative adversarial networks and in 2018 with transformers. Around 2019, the technology matured enough to achieve molecular precision and companies started the first discovery and development programs. Some of these therapeutics now reached human clinical trials and some are in Phase II.