Stanford Convenes Researchers to Rethink Empirical Methods for the AI Era

If you work with data for a living, you've probably noticed the ground shifting under your feet. The tools we use to measure things, run experiments, and draw causal conclusions are getting rewired by AI, and not always in ways that are well understood yet.

That's the premise behind the Empirical Methods in the Age of AI Conference, hosted by Stanford's Causal Science Center and the Graduate School of Business. The event brings together academics and industry practitioners to dig into how AI is changing empirical workflows: the pipelines researchers use to collect data, analyze it, and turn it into reliable evidence.

This isn't just an academic exercise. AI systems are now embedded in scientific discovery, business decision-making, and policy analysis, often sitting right in the middle of the measurement process itself. When a language model helps you label data, summarize results, or even generate hypotheses, you've introduced a new layer of uncertainty into your pipeline. The conference is set up to ask: how do we account for that?

Why This Matters for Practitioners

The organizing team reads like a who's-who of causal inference and econometrics at Stanford. Ramesh Johari, a professor of Management Science and Engineering and co-director of the Causal Science Center, is joined by economists Lihua Lei and Jann Spiess, along with Vasilis Syrgkanis and Stefan Wager, who co-directs the Causal Science Center alongside Johari. That's a lot of statistical firepower in one room, and it signals the conference is aiming for technical depth, not just high-level AI hype.

The agenda covers topics that should sound familiar to anyone doing applied research right now:

• AI-assisted research workflows, where models help with literature review, data cleaning, or hypothesis generation
• New approaches to measurement, especially when AI-generated signals substitute for or augment traditional data collection
• Causal inference and econometrics in settings where AI tools introduce their own biases or noise
• The shifting division of labor between human expertise and machine intelligence in the research process

Here's the practical tension at the heart of all this. Causal inference depends on clean assumptions about how data was generated. If an AI model is doing part of that generation, whether through synthetic data, automated labeling, or even just summarizing survey responses, those assumptions get murkier. Researchers need new statistical frameworks to handle that, or at minimum, new ways to quantify the uncertainty AI introduces.

This is where the econometrics crowd and the ML crowd actually need each other. Economists have decades of rigor around identification strategies and bias correction. ML researchers have the tools to build and deploy the models doing the heavy lifting. Neither group alone is equipped to fully solve the problem of trustworthy AI-assisted empirical work.

Think about a concrete example: using an LLM to classify open-ended survey responses at scale. It's fast and cheap compared to human coders. But if the model has systematic blind spots, maybe it misreads sarcasm, or struggles with domain-specific jargon, your downstream causal estimates inherit that bias silently. Catching that requires the kind of careful measurement thinking that econometricians specialize in, combined with an understanding of how the underlying model actually behaves.

The conference format, keynotes, panels, and open discussions, suggests the organizers want cross-pollination rather than siloed presentations. That's probably the right call. A lot of the most useful insights in this space come from people who've hit the same wall from different directions: an ML engineer frustrated by lack of statistical guarantees, or an economist frustrated by the black-box nature of modern models.

Worth noting: tickets are already sold out and the waitlist is closed. That's a strong signal of demand. Empirical researchers across disciplines are clearly hungry for a venue to hash out these questions together, rather than each field reinventing the wheel in isolation.

For practitioners who can't attend, the broader takeaway is still useful. If you're building or using AI tools anywhere near your data pipeline, you need to start treating that integration as a methodological choice, not a free lunch. The convenience of AI-assisted data work comes with hidden costs in validity and bias that classical empirical methods were designed to catch.

That doesn't mean abandoning AI tools. It means being more deliberate about where and how you use them. A model that helps you draft code faster is a different risk profile than a model that's generating the outcome variable in your regression. Conflating the two is an easy mistake to make when everything just feels like "using AI."

Key Takeaways

The core message from this conference, even before it happens, is clear: AI isn't just a new tool bolted onto old research methods. It's forcing a rethink of how we define reliable evidence in the first place.

• Stanford's Causal Science Center and GSB are convening top researchers in causal inference, econometrics, and AI to address this directly
• The organizing faculty, Johari, Lei, Spiess, Syrgkanis, and Wager, bring deep expertise in both classical statistics and modern ML
• Core topics include AI-assisted workflows, measurement under AI-generated signals, and the evolving human-machine division of labor in research
• Demand for this kind of cross-disciplinary venue is high; the event sold out with waitlists now closed

If you're doing empirical work anywhere near AI tools, keep an eye on what comes out of this gathering. The frameworks that emerge here will likely shape how rigorous research gets done for the next decade, not just in academia, but in any industry that depends on trustworthy data analysis.