Stanford HAI Convenes Researchers to Rethink Empirical Methods for the AI Era

If you've done any applied research lately, you've probably noticed the ground shifting under your feet. The tools we use to collect data, run analyses, and draw conclusions are changing fast, and a lot of the old assumptions about what counts as "rigorous" empirical work don't map cleanly onto a world where large language models can draft hypotheses, clean datasets, or even simulate survey responses.

That's the premise behind the Empirical Methods in the Age of AI Conference, hosted jointly by the Stanford Causal Science Center and the Graduate School of Business. It's not a product launch or a benchmark drop. It's a working session for people who build and apply statistical methods, trying to get ahead of a problem that's already here: AI is changing not just what questions researchers can ask, but how they validate the answers.

The organizing list reads like a who's who of causal inference and econometrics at Stanford. Ramesh Johari, who co-directs the Causal Science Center and serves as a senior fellow at HAI, is joined by economists Lihua Lei and Jann Spiess, along with Vasilis Syrgkanis and Stefan Wager, the center's other co-director. This isn't a group chasing hype. These are researchers whose day jobs involve making sure statistical claims actually hold up, which is exactly the lens the field needs right now.

Why Measurement and Causal Inference Need a Rethink

Here's the practical problem. Empirical research has always relied on a chain of trust: you collect data, you apply a method, you interpret the result, and each link in that chain has known failure modes that statisticians have spent decades characterizing. Insert an AI model anywhere in that chain, as a data labeler, a feature extractor, a text summarizer feeding into a regression, and you've introduced a new source of error that classical methods weren't designed to catch.

Causal inference is especially exposed here. The whole point of techniques like instrumental variables or difference-in-differences is to isolate a causal effect from noise and confounding. If an AI system is doing part of the measurement work, say, scoring sentiment from text or extracting structured variables from messy records, any bias in that model quietly propagates into every downstream estimate. You don't get a clean error term anymore. You get a correlated one, baked in by whatever training data and architecture choices produced the model.

Econometrics has mechanisms for this kind of problem in theory: measurement error correction, bounds analysis, sensitivity checks. But most of that machinery was built assuming the measurement error comes from human coders or survey instruments with reasonably well-understood properties. A foundation model fine-tuned on who-knows-what, with failure modes that shift across prompts and versions, is a different animal. The conference's focus on "the evolving relationship between human expertise and machine intelligence" is really pointing at this: where does a researcher's judgment still need to sit in the loop, and where can it safely step back?

There's also the opportunity side, which is why this isn't just a cautionary tale. AI-assisted research can genuinely expand what's measurable. Text-as-data methods have been around for years, but large language models make it possible to extract richer, more nuanced variables from unstructured sources, think legal documents, customer reviews, or policy text, at a scale human coders never could. That's a real methodological unlock, not just a speed-up. The hard part is building the validation layer that lets researchers trust those extracted variables the way they'd trust a hand-coded one.

Scheduled sessions run on a seminar format, Wednesdays from 4:30 to 5:30 PM in CoDa E160, with dates set for October 21, October 28, and November 4, 2026. Each week features a different Stanford speaker, though specific talk titles are still being finalized by the organizers. That rolling structure suggests this is being treated less as a one-off event and more as an ongoing seminar series, which tracks with how fast the underlying tooling is moving. A conclusion drawn from today's model generation might need revisiting by spring.

Worth noting: tickets are already sold out, and the waitlist has closed. For a methods-focused academic seminar, that's a signal in itself. Applied researchers across economics, statistics, and business are hungry for guidance on this, and there isn't yet a settled playbook anyone can just hand them.

Key Takeaways

The core tension here isn't really about AI capability, it's about epistemic plumbing. Every empirical method rests on assumptions about where error comes from and how it behaves. AI systems introduce new error structures that don't fit the old assumptions cleanly, and that mismatch is exactly where bad conclusions sneak in if nobody's watching.

For practitioners, a few things are worth carrying forward even without attending the sessions:

• Treat AI-assisted measurement the same way you'd treat any new instrument: characterize its error before trusting its output in a causal model.
• Expect existing econometric tools (sensitivity analysis, bounds, robustness checks) to need adaptation, not replacement, for AI-in-the-loop pipelines.
• Watch for methodological output from this series. With Johari, Lei, Spiess, Syrgkanis, and Wager involved, expect papers and frameworks, not just talks, to come out of it eventually.
• The "human expertise vs. machine intelligence" framing isn't philosophical fluff. It's a design question about where human review checkpoints belong in an AI-augmented research pipeline.

None of this is solved yet, which is sort of the point. The field is still figuring out what rigor even means when a model is doing part of the analytical work. Conferences like this one are where that definition gets hashed out, slowly, by people who actually have to answer for their numbers later.