AI & Tech
Goodfire’s Eric Bigelow says 7–8bn-parameter models suit interpretability research
Cognitive Revolution
How Agents Decide: Goodfire's Eric Bigelow on Critical Tokens, Phase Shifts, & In-Context Learning
What was said
"I usually I I tend towards like the scale of 7 to8 billion or more being a sweet spot where like you can start to do good interpretability research."
Eric Bigelow argues he often chooses models around 7–8 billion parameters for interpretability experiments. He says they are large enough to show interesting zero-shot behaviours while still being tractable to study.
Verification status
Attributed speaker claim
Egleze verified who said this and where. The underlying claim has not been independently established by Egleze.
Source and context
Egleze verified who said this and where. The underlying claim has not been independently established by Egleze.
- Speaker
- Eric Bigelow · guest
- Programme
- Cognitive Revolution
- Episode
- How Agents Decide: Goodfire's Eric Bigelow on Critical Tokens, Phase Shifts, & In-Context Learning
- Clip time
- 49:32
- Original source
- Watch the source segment →
From this episode
Cognitive Revolution