AI & Tech
Google paper says safety-tuning may distort models’ ‘theory of mind’
Peter Diamandis
Google’s Jeff Dean Exits, SpaceX Hits $100B in Rev & OpenAI’s Astra Solves Decade-Old Math Problems
"So the team discovered that the safety fine-tuning that we've been doing to stop AIs from claiming their conscious has a massive unintended side effect. It doesn't just suppress the model's own self-attribution of mind. It suppresses the model's ability to attribute mind to anything else."
Host Peter Diamandis cites a new paper he says was authored by Google’s “paradigm of intelligence” team with university collaborators, claiming that safety fine-tuning designed to stop language models asserting consciousness also reduces the model’s tendency to attribute ‘mind’ to animals, nature, other chatbots, or God. He adds that removing the safety ‘refusal direction’ increased a self-attributed ‘mind’ score from 2.17 to 4.77 on a 0–10 scale, with further steering pushing it higher.
From this episode
Peter Diamandis