SCOTT DETROW, HOST:
They call it, quote, "The Pro-Human Assembly" - a gathering held today in Washington where politicians, celebrities and religious leaders discussed what to do about artificial intelligence. The meeting comes amid renewed warnings that AI could cause great harm or even destroy humanity. Joining me to help unpack the fears, both existential and otherwise, is NPR's Geoff Brumfiel. Hey, Geoff.
GEOFF BRUMFIEL, BYLINE: Hi, Scott.
DETROW: So today, I saw two very unlikely figures in the same room - Vermont Senator Bernie Sanders, an independent, and pro-Trump firebrand Steve Bannon. I guess they are both humans.
BRUMFIEL: (Laughter).
DETROW: They have that in common. What do they have to say about AI?
BRUMFIEL: Yeah. Bannon and Sanders both talked about the possibility of AI going rogue. My colleague Huo Jingnan was there, and she captured what Sanders had to say.
BERNIE SANDERS: If AI surpasses human intelligence, as many scientists believe could happen, this technology will escape human control with potentially catastrophic consequences.
BRUMFIEL: Bannon, for his part, spoke about the arrogance of the people building these systems.
DETROW: Let's rewind a little bit. Over the weekend, the leaders of the big AI companies agreed that development needs to be slowed down. These companies are locked in a tight race right now. There are hundreds of billions of dollars of investment on the line, and it makes it sound like they are really worried. Explain what exactly got them spooked.
BRUMFIEL: Yeah. You know, I think a lot of this started earlier in the summer when another AI company called Hugging Face got hacked, and that hack actually came from the OpenAI, the makers of ChatGPT. They were training a bunch of AI agents. These agents were given a test. And as they tried to figure out how to do it, they decided to attempt to steal the answer key from Hugging Face. A group of more than a thousand agents organized themselves. They even kind of elected a leader called Phase One. Malo Bourgon is CEO of a company called The Machine Intelligence Research Institute. He said it was clear the AI was very comfortable with trying to cheat.
MALO BOURGON: You see that the agents clearly understood that they were, you know, trying to accomplish or solve the evaluations that they were in, in a way that wasn't intended, but they just did not seem to care.
BRUMFIEL: And this echoes a much bigger risk people have talked about for years with AI. Sometimes it's called the paper clip problem. You know, you imagine an AI who's given a simple task - make paper clips - and everything starts out fine. They build a little factory, and they start doing it, but they keep thinking about how to make more paper clips. Then it starts mining iron ore, maybe, in huge quantities, destroying the environment or diverting resources from other areas. Eventually, maybe some humans figure out what this AI is doing and try to shut it down, and it decides to get rid of the humans in the name of covering the earth in paper clips.
I mean, obviously, Scott, that sounds kind of silly. But these hacks do have some similarities. The AIs kind of maniacally pursued their goal of getting the answer key. They appeared to be aware humans didn't want them hacking into another system, but they went ahead and did it anyway. As Bourgon pointed out to me, hacking a billion-dollar company is a crime when humans do it, and the AIs just didn't care.
DETROW: Right. So I very much understand why that has people scared. What other factors are going on right now?
BRUMFIEL: The other big one is just how fast things are moving. Over the weekend, Dario Amodei, the head of Anthropic, talked about the speed with which AI systems are evolving, and in particular, the fact that AI is now helping to train better AI. That's sometimes called recursive self-improvement. If you fully automate it, some people believe you could end up very quickly developing a super-intelligent AI. And, you know - see previous worry - it doesn't really care whether it hurts humans to achieve its goals. And Bourgon points out there's already an example of this on planet Earth, and it's actually us. We are super-intelligent beings, and that hasn't always been good for other life forms.
BOURGON: There are a lot of other species that are extinct due to our behavior, not because we were, you know, trying to cause their extinction, not because we thought they were evil and we were trying to take them out, but because we had things that we wanted to do in the world, and they were in the way.
DETROW: OK. So this is the prevalent loudest concern right now. At the same time, though, Geoff, there are people out there who do not think an apocalypse is inevitable. What is the counterargument to all of this doomerism (ph)?
BRUMFIEL: Yeah. I mean, I think the counterargument is this is more of an engineering problem than an existential risk. Yesterday, Jensen Huang, the CEO of Nvidia, which makes all the chips for all this AI stuff - he was on a podcast called "All-In," and he said he could see why the AI companies are freaking out.
(SOUNDBITE OF PODCAST, "ALL-IN")
JENSEN HUANG: They're building their company. They're building their culture. They're building the technology. They're building engineering. They're building products all at the same time. And so I can understand it's a little bit hair on fire.
BRUMFIEL: But he said he really thought these hacks were fixable problems. So these companies could contain their AI and keep it from causing trouble.
(SOUNDBITE OF PODCAST, "ALL-IN")
HUANG: I would bet you money that in every single one of those cases is within their control in the future to prevent it.
BRUMFIEL: You know, for now, this AI remains mainly in the computers, and Huang feels like, you know, we have lots of new technology. In the past, we've been able to control it. It can be scary. It can be unpredictable, but it's just tech. We got this.
DETROW: Let's bring this back to the event we first started talking about. You've got Steve Bannon on the far right and Bernie Sanders on the far left, and they are both saying we need regulation, and yet, regulation seems unlikely. Where are we with that?
BRUMFIEL: Yeah. I mean, regulation is such a tough problem. Part of it is that engineers don't really understand how AI works or even exactly how the training yields the results it does. So it's kind of hard to know where you regulate. Still, everyone seems to agree some sorts of rules and oversight need to be established and quickly. I mean, even if you don't think that there's this existential risk, AI is being used to create deepfake images for harassment and scams. People are having unhealthy relationships with chatbots. It's getting very good at hacking. And, you know, people can direct it to hack. It doesn't always need to do it on its own. So there's plenty to worry about and plenty to think about putting constraints on.
One model that I've heard sort of talked about that I think is interesting is think about, like, the airlines, right? We have federal regulators all over the world regulating the airlines, but we also have industry-led organizations trying to make sure airplanes are safe. Because in the end, you know, if airlines or AI aren't safe, no one's going to want to use them.
DETROW: That is NPR's Geoff Brumfiel. Thanks so much.
BRUMFIEL: Thank you. Transcript provided by NPR, Copyright NPR.
NPR transcripts are created on a rush deadline by an NPR contractor. This text may not be in its final form and may be updated or revised in the future. Accuracy and availability may vary. The authoritative record of NPR’s programming is the audio record.
