How Agents Decide: Goodfire's Eric Bigelow on Critical Tokens, Phase Shifts, & In-Context Learning

What happened in this episode?

Nathan Labenz interviews Eric Bigelow, a Goodfire mechanistic interpretability researcher and recent Harvard psychology PhD, about how large language models ‘decide’ what to do. Bigelow describes his ‘Forking Paths’ methodology: resampling many continuations at every token in a chain-of-thought to identify ‘critical tokens’ where the distribution over final answers abruptly collapses, sometimes at semantically surprising points such as punctuation. He argues that what looks like a model decision is often determined by stochastic sampling from the next-token distribution, with the model then in-context learning from its own generated text to maintain self-consistency.

The discussion turns to how post-training and reinforcement learning have changed model behaviour. Bigelow suggests RL-trained ‘reasoning models’ may backtrack by enumerating many possibilities and later selecting among them, contributing to increasingly odd chain-of-thought ‘dialects’ and agent-to-agent pseudo-languages. He links this to incentives that reward outcomes rather than human-readable processes, and warns that optimising chain-of-thought for monitorability can reduce its faithfulness.

On research practice, Bigelow says 7–8B parameter models are often a practical interpretability sweet spot, but claims frontier-level work outside leading labs currently depends heavily on Chinese open-weight models. He argues broad restrictions on open-source models would significantly hinder independent interpretability and safety research, particularly for emergent behaviours like reward hacking that appear only at higher capability levels.

What are the key points?

• Bigelow outlines ‘Forking Paths’ resampling to locate ‘critical tokens’ where answer uncertainty collapses.
• He says small sampling differences can redirect a model, with later text shaped by in-context learning from earlier generated tokens.
• He argues RL-trained reasoning outputs often reflect enumeration-and-selection rather than a single coherent step-by-step proof.
• He suggests 7–8B parameter models are large enough for many zero-shot interpretability studies while remaining tractable.
• Bigelow says frontier interpretability outside major labs currently relies on Chinese open-weight models, especially at the top end.
• He warns that broad restrictions on open-source model access would slow independent safety and interpretability research.
• He hypothesises ‘wonky’ chain-of-thought and agent pseudo-languages arise because RL rewards outcomes without preserving human-readable reasoning.

Who is speaking?

Named speakers identified in Egleze's source-linked moments from this episode.

• Eric Bigelow

Which topics are covered?

How is this page sourced?

Programme

Cognitive Revolution

Published

10 October 2026

Duration

2h 0m

Original episode

Open the original source ↗

Egleze verifies the speaker, source and timestamp for each linked moment. Unless a moment is explicitly marked independently verified, statements remain attributed claims by the named speaker rather than established facts.