Emmanuel Ameisen says interpretability suggests language models can form hidden plans during generation

What was said

"we find that models will like in their intermediate results sometimes formulate a plan. Um, the example we have for this is poetry. It's hard to rhyme uh without planning."

Emmanuel Ameisen says interpretability work indicates some language models create and reuse latent “plans” stored in intermediate representations, even though they output text one token at a time. He gives poetry generation as an example, saying the model may internally identify a rhyme target and later tokens follow that plan without explicitly stating it.