Causal Data Science Meeting 2025 Keynote – Stefan Feuerriegel (LMU Munich)

Introduction to Professor Stefan Foyer and His Work in AI

Welcome and Background

• The speaker introduces Professor Stefan Foyer, a keynote speaker from Munich, highlighting his dual affiliation with LMU Munich School of Management and the Faculty of Mathematics, Informatics, and Statistics.

• Professor Foyer has been recognized as a Cambridge Center for AI Medicine scholar and has previously visited Stanford University as a visiting scholar.

• He has an impressive academic record with over 70 journal articles and 80 peer-reviewed conference papers, indicating a highly productive research group.

Keynote Focus: Applications vs. Methods in Machine Learning

• Professor Foyer emphasizes the tension between applications of machine learning and the need for new methods tailored to specific settings.

• The first half of his talk will focus on applications while the second half will address methods necessary for improving decision-making processes.

Causal Machine Learning: Importance and Challenges

Necessity of Causal Perspectives

• Professor Foyer discusses the importance of causal machine learning by using an analogy involving predicting drowning based on sunscreen use, illustrating how correlation does not imply causation.

• He argues that relying solely on easily measurable variables can lead to misleading conclusions due to confounding factors.

Treatment Effect Estimation

• The main argument is that many decision-making tasks require causal methods for reliable outcomes rather than traditional machine learning approaches which may yield biased estimates.

Traditional vs. Causal Machine Learning

• A distinction is made between traditional machine learning (associative tasks predicting relationships between variables) versus causal machine learning (understanding effects of interventions).

• The "do operator" is introduced as a fundamental concept in causal inference, emphasizing its role in determining outcomes from specific actions or interventions.

Understanding Causal Machine Learning in Practice

The Misconception of Causal Machine Learning

• The speaker emphasizes the need to clarify what causal machine learning is, noting that it is often perceived as a "magic silver bullet" for data analysis.

• It is highlighted that causal relationships are not automatically learned from data; assumptions must be made to derive valid conclusions.

Counterfactual Prediction and Decision Making

• The concept of counterfactual prediction, or "what if modeling," is introduced as crucial for managerial decision-making between options A and B.

• Managers seek predictions on outcomes based on different actions rather than a single future forecast.

Applications of Causal Machine Learning in Healthcare

• The speaker presents four applications from their group, starting with healthcare where predicting treatment outcomes can guide decisions.

• Most regulated medical devices focus on risk scoring and diagnostics rather than aiding in treatment option decisions.

Enhancing Traditional Statistics with Causal Machine Learning

• Causal machine learning can help predict patient trajectories under various treatment options, enhancing personalized medicine.

• Combining traditional statistics with causal machine learning can yield more reliable inferences and foster trust in conclusions.

Case Studies Demonstrating Practical Applications

• A study by Stefan Vaga's group used observational data to create individualized decision rules for hospitalization benefits among suicidal patients.

• Another example involves off-label medication use, aiming to understand its effectiveness without prior rigorous trials.

Monitoring Health Trends During the Pandemic

• Research focused on monitoring vitamin D levels during COVID-19 utilized observational data while accounting for sociodemographic confounders.

• This approach allows policymakers to track health trends across different societal groups effectively.

Future Directions and Integration of Data Sources

• There is potential for further exploration of methods combining randomized controlled trials (RCTs) with real-world observational data for enhanced validity.

• The speaker mentions ongoing work integrating RCT and observational data strengths to improve external validity.

Understanding Human Behavior and Climate Change Interventions

The Complexity of Human Behavior

• Human behavior is inherently heterogeneous, influenced by factors such as social status and personality traits.

• There is a need for tools in psychology and behavioral science to understand the variability in human responses to interventions.

• A reanalysis of large-scale studies on behavioral interventions aimed at increasing belief in climate change highlights this complexity.

Behavioral Interventions and Their Effectiveness

• Psychological interventions, like advertisements or tasks prompting reflection on climate change, aim to trigger psychological mechanisms (e.g., negativity bias).

• Existing literature suggests that these interventions are often marginally effective, showing statistically significant but minimal improvements.

• The assumption is that finding the right intervention for specific populations could enhance effectiveness beyond average treatment effects.

Causal Machine Learning Applications

• Utilizing causal machine learning methods, such as causal forests, allows for modeling heterogeneity in treatment effects across different populations.

• A study involving 60 countries with 60,000 participants measured climate change beliefs using a randomized control trial approach.

Analyzing Treatment Effects

• The analysis revealed that while 80% benefited from the intervention promoting psychological proximity to climate change, 20% showed no improvement.

• Other interventions demonstrated varied effectiveness; some were ineffective for up to 40% of individuals despite benefiting others significantly.

Global Differences in Intervention Effectiveness

• Analysis indicated substantial differences in intervention effectiveness between global north and south regions.

• Policymakers may require tailored communication strategies based on regional differences identified through causal machine learning insights.

Impact of Personalized Ads and Counterfactual Predictions

Use of Variables in Ad Effectiveness

• The study utilized two variables, age and gender, due to privacy restrictions from Google Ads, which limited the use of a more extensive dataset.

• Personalized ads showed a significant increase in effectiveness, boosting click-through rates by approximately 30%, indicating a substantial treatment effect in marketing.

Counterfactual Predictions for Development Goals

• The discussion highlights the potential for counterfactual predictions related to sustainable development goals (SDGs), particularly concerning climate change.

• Decision-makers can simulate outcomes based on varying levels of aid to countries, helping identify where resources can have the most significant impact under budget constraints.

Analysis of HIV/AIDS Pandemic Aid

• An analysis was conducted on how changes in development aid budgets could affect HIV infection rates globally.

• A tool was developed for decision-makers to visualize how funding allocations could lead to specific health outcomes, promoting data-driven resource allocation.

Methodology for Data Analysis

• The methodology involved using high-dimensional covariates with an emphasis on predicting HIV infection rates based on development aid volumes.

• An autoencoder was employed to create lower-dimensional representations of data, addressing challenges posed by wide datasets with numerous covariates.

Business Applications: News Promotion Strategies

• Collaboration with a Swiss newspaper aimed at optimizing news article promotion without relying on personalization strategies.

• The strategy focused on placing less popular content higher up on the page to encourage user engagement through scrolling rather than simply prioritizing frequently clicked topics like sports or weather.

Content Optimization Strategies

Balancing Clicks and Subscriptions

• The company evaluates a score that balances clicks, subscriptions, and other indicators for content items while considering time input as covariates.

• There are capacity constraints when selecting articles to promote during specific hours, necessitating an optimization process to choose the most effective content.

High-Dimensional Covariates and Uncertainty

• The complexity arises from high-dimensional covariates; simply optimizing based on point estimates (like the Kate model) is insufficient.

• A combined approach is used that incorporates risk awareness by integrating Kate estimates with upper confidence bounds to create a more robust strategy.

Learning from Experts

• The effectiveness of the Kate-based targeting strategy aligns with previous research, allowing insights into expert strategies and revealing new opportunities for improvement.

• Notably, it was discovered that promoting content from the editor-in-chief could significantly increase click rates, which had been underutilized by experts.

Q&A Session Insights

Addressing Observed Heterogeneity

• A question raised about observed average treatment effects suggests meaningful heterogeneity in research findings. This highlights potential biases in understanding data.

• The analysis of heterogeneity stems from a randomized experiment, simplifying the evaluation process and enhancing reliability.

Propensity Score Considerations

• The use of causal forests in analyzing propensity scores is discussed; estimating these scores can provide deeper insights compared to merely using observed values.

Moving Beyond Average Treatment Effects

• Emphasis is placed on shifting focus from average treatment effects to individual-level analyses to better understand intervention effectiveness across different populations.

• Scholars advocate for this "heterogeneity revolution" in behavioral science to refine understanding of when interventions work best.

Challenges in Research Design

Concerns About Study Power

• Acknowledgment of challenges such as underpowered studies in both RCT and observational settings raises questions about data interpretation validity.

Rigorous Evidence Collection

• To address concerns regarding identification problems, three key actions were taken: utilizing RCT data for rigorous evidence collection.

Discussion on Methodological Effectiveness

Importance of External Validation

• The speaker emphasizes the power of their method in challenging settings, aiming to demonstrate its effectiveness compared to other approaches.

• Acknowledges that smaller data settings may not yield equally good results, highlighting the importance of external validation beyond merely finding patterns.

Inspiration for Future Research

• The speaker hopes their approach will inspire others in behavioral science to pursue external validation in follow-up studies and explore heterogeneity in human behavior.

Open Challenges and Flexibility in Research

Current State and Future Directions

• The discussion shifts towards identifying open challenges, focusing on flexibility, efficiency, robustness, and decision-making within research methodologies.

Development of Flexible Models

• Highlights the need for increased flexibility to handle non-standardized settings as many practical applications do not conform to tabular datasets.

Innovative Approaches: Diffusion Models

Introduction to Diffusion Models

• Describes a flexible diffusion model developed by the group for learning potential outcomes, which combines various advantages into a single robust model.

Performance Insights

• The diffusion model shows strong performance not only in predicting potential outcomes but also in K estimation tasks.

Causal Inference Techniques

Addressing Missing Values

• Discusses how diffusion models can address missing value problems inherent in causal inference by framing them appropriately based on observed variables.

Causal Masking Strategy

• Introduces a causal masking technique that informs the diffusion model about which variables are observed or missing, enhancing data filling accuracy.

Combining Learning Approaches

Merging Orthogonal Learners with Representation Learning

• Explores combining orthogonal learners with representation learning to leverage their respective strengths while addressing confounding biases that may arise from dimensionality reduction.

Framework Development

• Proposes a principled framework called "O learner" that integrates representation learning with orthogonal learning methods effectively.

Balancing Representations

Investigating Representation Effects

• Examines how balancing representations between treated groups can enhance understanding and application within causal inference contexts.

Understanding the Limitations of Balancing in Treatment Groups

The Ineffectiveness of Balancing

• The speaker discusses how balancing treatment groups often fails to improve outcomes, emphasizing that specific assumptions are rarely met.

• It is suggested that balancing should not be a primary step in analysis, indicating its limited utility.

Challenges in Time Series Settings

• The complexity of time series settings is highlighted, where historical data informs future interventions and outcomes.

• Issues arise from discrete time steps and the overlap problem when multiple treatments are applied over time.

Time-Varying Confounding

• A visual representation illustrates the challenges posed by time-varying confounding variables affecting treatment outcomes.

• Addressing these confounders requires principled adjustment strategies, with various learners developed to tackle this issue.

Exploring Advanced Learning Methods

Overview of Learner Types

• Different types of learners (model-based and meta-learners) are categorized based on their adjustment strategies derived from statistics and biostatistics.

• The complexity of the field necessitates ongoing research to understand which methods work best under varying conditions.

Application in Complex Settings

• Examples include spillover effects and constraints related to machine learning models, such as interpretability and privacy concerns.

• Recent advancements in causal machine learning highlight emerging methodologies for addressing complex scenarios effectively.

The Future of Causal Inference Techniques

Foundation Models for Co-Inference

• New foundation models have emerged that facilitate co-inference without requiring fine-tuning, leveraging techniques similar to prompting used in large language models (LLMs).

Valid Inference with Identifiability Guarantees

• Some models provide valid inference capabilities while ensuring identifiability guarantees; however, practical application remains an area needing further exploration.

Addressing Overlap Problems in Treatment Analysis

Low Overlap Challenges

• Low overlap refers to situations where patients with similar characteristics receive different treatments, complicating analysis. This is typically measured using propensity scores.

Innovative Solutions for Adaptive Learning

• An adaptive learner model has been proposed that incorporates dropout techniques from neural networks to address low overlap issues effectively.

Understanding Causal Inference and Regularization Techniques

Introduction to Metal Learner Steps

• The process begins with estimating nuisance parameters, followed by estimating pseudo outcomes. The oracle potential outcome is depicted as a curvy line.

• The simulated setting shows a zero treatment effect, represented by the absence of treatment effect despite challenging conditions. Data points are illustrated in yellow, orange, and blue.

Observed Outcomes and Regularization

• A gray area indicates observed pseudo outcomes; these estimates are close to the ground truth due to good observations. However, some pseudo outcomes deviate significantly from the oracle.

• Introducing regularization helps reduce weights on less reliable areas of the covariant space, enhancing model robustness.

Addressing Unobserved Confounding

• Unobserved confounding is a significant issue in practical applications where experiments cannot be conducted. Partial identification relaxes assumptions about unobserved confounders.

• This method provides bounds on causal effects (KATE), allowing for informed decision-making even when direct measurements are unavailable.

Application of Sensitivity Analysis Methods

• Dennis from the research group has developed methods for mapping sensitivity analysis techniques into an unsupervised framework to account for unobserved confounding.

• Real-world data often lacks complete information on confounders; thus, partial identification can yield insights into how variables like development aid affect HIV rates.

Balancing Interpretability and Decision-Making Performance

• There exists a trade-off between interpretability (important in fields like medicine) and decision-making performance when using KATE models.

• Research aims to balance KATE error with decision-making error to optimize both aspects effectively.

Visualizing Model Performance Trade-offs

• A visual representation shows that while finite sample estimates may not perfectly align with ground truth (red line), alternative models can provide better approximations through trade-offs.

• Models that prioritize either KATE or decision-making errors may lead to incorrect treatments; however, combining strengths can yield more accurate predictions (green line).

Future Directions in Causal Machine Learning

• Integrating causal machine learning methods into traditional decision-making frameworks presents new research opportunities. Recommended further reading includes works by Shen Shani and Nathan Kos from LSE.