Handling artificial intelligence (AI) is dealing with a “real and mysterious creature, not a simple and predictable machine,” Jack Clark, co-founder of AI company Anthropic, said during a conference at Berkley, with the speech posted to Substack on Oct. 13.
“My own experience is that as these AI systems get smarter and smarter, they develop more and more complicated goals. When these goals aren’t absolutely aligned with both our preferences and the right context, the AI systems will behave strangely,” said Clark said, who admitted to being “deeply afraid” of the tech.
Clark recounted an incident from 2016 when he was working at OpenAI, where an AI agent was trained to navigate a boat on a race course in a video game. Instead of piloting the boat to the finish line, the AI made the boat run over a barrel to score points. The boat was then bounced off walls and eventually set on fire so it could run over the barrel again for points….