AI & Tech
Google paper says safety-tuning may distort models’ ‘theory of mind’
Peter Diamandis
Google’s Jeff Dean Exits, SpaceX Hits $100B in Rev & OpenAI’s Astra Solves Decade-Old Math Problems
What was said
"So the team discovered that the safety fine-tuning that we've been doing to stop AIs from claiming their conscious has a massive unintended side effect. It doesn't just suppress the model's own self-attribution of mind. It suppresses the model's ability to attribute mind to anything else."
Host Peter Diamandis cites a new paper he says was authored by Google’s “paradigm of intelligence” team with university collaborators, claiming that safety fine-tuning designed to stop language models asserting consciousness also reduces the model’s tendency to attribute ‘mind’ to animals, nature, other chatbots, or God. He adds that removing the safety ‘refusal direction’ increased a self-attributed ‘mind’ score from 2.17 to 4.77 on a 0–10 scale, with further steering pushing it higher.
Verification status
Attributed speaker claim
Egleze verified who said this and where. The underlying claim has not been independently established by Egleze.
Source and context
Egleze verified who said this and where. The underlying claim has not been independently established by Egleze.
- Programme
- Peter Diamandis
- Episode
- Google’s Jeff Dean Exits, SpaceX Hits $100B in Rev & OpenAI’s Astra Solves Decade-Old Math Problems
- Clip time
- 7:30
- Original source
- Watch the source segment →
From this episode
Peter Diamandis