AI Leaders Can't Get Along. So How Will They Keep Us Safe? — With Kevin Roose

OpenAI

Across the shows: 31 speakers in other episodes, 15 bullish and 28 bearish calls, past 30 days

• Kevin Roose · Machine Gods (NPR partnership)Uncertain

  Altman's fundraising skill is also a liability, given his pattern of telling people different things.

  Roose describes Altman as the world's greatest fundraiser and a high-EQ salesman who figures out what people want. That skill let OpenAI scale quickly and win favor with the Trump administration. But people who have dealt with him describe a consistent pattern going back to the founding of OpenAI, of telling one person one thing and someone else another, and of making promises he did not keep. That pattern was a big part of why Dario Amodei and six cofounders left to start Anthropic. Roose says it is either a superpower or a liability depending on one's view.

  “His superpower is that he is the world's greatest fundraiser and... salesman, entrepreneur... But it also leads to this reputation as a slippery people pleaser who will tell one thing to one person and a completely different thing to someone else.”at 29:19

• Kevin Roose · Machine Gods (NPR partnership)Bullish

  OpenAI's shared belief in AGI is why it still leads the AI race.

  Roose wrote a book about belief rather than technology. He argues OpenAI and Anthropic are the most believing organizations despite departures, and that belief that the product could become AGI is an essential ingredient. OpenAI's researchers saw early reasoning traces and called the shot on scaling, expecting transformative effects before their peers. Because they bet on compute before it was obvious, they kept a lead. Roose says the people he talked to there are smart, but their alignment behind the superintelligence mission is what keeps them in front.

  “They believe, and it turns out that belief in what you're building and the fact that it could become AGI is like an essential ingredient.”at 57:10

• Kevin Roose · Machine Gods (NPR partnership)Bullish

  OpenAI's early compute bet gives it an advantage now that others are copying it.

  Roose recalls that a year ago people thought OpenAI was reckless to invest so heavily in compute, and that Dario Amodei called it yolo-ing. Now everyone is making the same bet. Roose argues OpenAI keeps an advantage because it committed to compute before the bet looked obvious, which is part of why it remains in the lead.

  “Like, I remember like a year ago when everyone was like, oh, OpenAI is really investing a lot in compute... And now it's like everyone's making that bet, but they have an advantage because they made the bet before it was obvious.”at 58:18

Alphabet

Across the shows: 11 speakers in other episodes, 11 bullish and 2 bearish calls, past 30 days

• Kevin Roose · Machine Gods (NPR partnership)Uncertain

  Google is probably in third place in the AI race, but shouldn't be counted out.

  Roose argues Google's resolve sharpened after falling behind. Sergey Brin returned after retiring in 2019, shamed by an OpenAI researcher, and is now working in the trenches with the Gemini team. He is pushing a founder-mode style of cutting red tape, which Roose sees as a large reason for Google's comeback. Demis Hassabis is the most competitive person, a chess prodigy who hates to lose. Together, that drive and the focus of being out of the lead made Google catch up. Roose says Google is probably still in third place, but he would not count it out.

  “I think Sergei plus Demis plus just like the kind of focus that comes from being out of the lead really sharpened their resolve and like made them catch up. And they're still probably in third place right now, but I wouldn't count them out.”at 19:00

• Kevin Roose · Machine Gods (NPR partnership)Bearish

  Google had the talent and breakthroughs but lacked AGI belief, so it trails the leaders.

  Roose argues Google had all the resources, all the talent and made the key breakthroughs, but lacked the belief that AGI was coming. He says its leaders did not believe in AGI the way Dario Amodei, Sam Altman and Demis Hassabis do. He cites the internal 'memo battle' in which one researcher urged the company to take AGI seriously, and another argued the term was unscientific and should not be used. He presents this as the reason OpenAI remains ahead despite Google's advantages.

  “Google had all of the resources, all of the talent. They made all of the breakthroughs. They had all the ingredients except for belief.”at 57:21

Microsoft

Across the shows: 9 speakers in other episodes, 6 bullish and 1 bearish calls, past 30 days

• Kevin Roose · Machine Gods (NPR partnership)Bearish

  Microsoft will become a commodity cloud provider, not a frontier AI lab.

  Roose says Microsoft is still a big, successful company but will likely become more of a commodity neocloud than a frontier AI lab. He cites Microsoft shipping an early GPT-4 checkpoint into Bing before fine-tuning was finished, which produced the Sydney episode. The Tay experience had made Microsoft wary of chatbots. Culture clashes with OpenAI, skepticism inside Microsoft about the investment, and a relationship that is no longer close all point to Microsoft losing its position at the frontier.

  “I think Microsoft, they are a player, they're still quite a big successful company, but I think they're going to be something more like a sort of commodity”at 50:24

Anthropic

Across the shows: 34 speakers in other episodes, 15 bullish and 36 bearish calls, past 30 days

• Kevin Roose · Machine Gods (NPR partnership)Uncertain

  Claude leads 26% of Anthropic's AI R&D; that share could reach 50–75%.

  Roose cites Anthropic's disclosure that Claude is now the lead on 26% of its AI research and development projects. He stresses that Claude is not just a coding assistant or pair programmer but is leading the projects. He sees no reason the share couldn't rise to 50 or 75%, or eventually 100%. Combined with OpenAI's stated goal of an automated AI researcher, this could make new models arrive at machine speed rather than every six months. He treats this as a warning sign, not a sign of progress.

  “Claude is now the lead on 26% of their AI research and development projects. Not a team member or a pair programmer or a coding assistant. It is leading the project.”at 1:00:31

AI regulation & safety

• Kevin Roose · Machine Gods (NPR partnership)

  Too few people make AI's key decisions; he doesn't trust so small a group to govern it.

  Roose argues the problem is not whether the individuals are good or bad. All the relevant decisions in AI have been made by a couple dozen people, and that group is too small to create and govern technology this powerful. He especially distrusts a group racing each other to build it. He would like the labs to get together on safety solutions, but he doubts the feuds and divides are small enough for that to happen.

  “all of the relevant decisions in AI have been made by a couple dozen people maybe... The group is just too small. And so I don't trust any group of people that small to create and govern technology like this”at 7:14

• Kevin Roose · Machine Gods (NPR partnership)

  Favors slowing AI development rather than stopping it, since capabilities are outpacing safety research.

  Roose says he is a pause guy but not a stop guy, and that the labs should slow down. Lab leaders say capabilities are outpacing safety research and control methods. Systems now solve math problems that stumped humans for decades, yet labs still don't know how to control them, and he worries as they approach superintelligence. He does not want to shut AI down because of benefits such as in health and medicine. He is slightly less pessimistic about a pause than Alex because a catastrophic incident could trigger fast regulation, as 9/11 did with the Patriot Act and Three Mile Island did for nuclear construction. He does not hope for such an incident.

  “I'm a pause guy. I'm not a stop guy.... the capabilities are outpacing the safety research and the control methods.”at 8:48

• Kevin Roose · Machine Gods (NPR partnership)

  Labs racing to automate their own AI research is dangerous; they should stop or slow down.

  Roose says the labs are racing to automate AI research, which he calls dangerous, and that they should stop. He cites Anthropic's claim that Claude leads 26% of its AI R&D projects and OpenAI's stated goal of an automated AI researcher. If the loop closes, new models could arrive every few hours instead of every six months. He argues the 'losing control' narrative is wrong because labs are deliberately giving control to their models. He stays optimistic that society could slow down, buy time to develop better control methods, and thread the needle on safety, since public awareness of AI risk has grown.

  “I believe that the labs are racing to do it, and I think it's dangerous. I think they should stop.”at 1:00:07

Frontier AI labs & model race

• Kevin Roose · Machine Gods (NPR partnership)

  AI labs are locked in a Red Queen race where each release forces the rivals' next release.

  Roose describes a pattern in which OpenAI releases a better model, so Anthropic must release one, and then Google must too. He borrows the evolutionary-biology term Red Queen's race, where competitors must keep running just to stay in place. He says this is a very good description of where the AI race is now, and it is why the labs keep accelerating even though all of them seem fearful of the technology.

  “They are locked into this pattern of OpenAI releases a better model, so then Anthropic has to release a better model, so then Google has to release a better model... You're just kind of racing to stay in the same place.”at 12:00

AI: overall outlook

• Kevin Roose · Machine Gods (NPR partnership)

  AI is a different kind of technology, and its transformative potential is real.

  Roose used to think AI was a normal technology, but he changed his mind after talking to people building it. He says that if labs build something smarter than humans, which he sees no technical reason they can't do, it would be incredibly transformative, and it is not like the internet or the steam engine. Steam engines did not go rogue or develop bioweapons. He also says he has stopped doubting the products' transformative potential because he uses them daily, including to start a company, and they keep doing more for him. Dismissers who have not used the tools are misunderstanding the rate of progress.

  “I have stopped doubting the transformative potential of the products because I use them.... It just does more and more and more for me.”at 25:23

AI & jobs

• Kevin Roose · Machine Gods (NPR partnership)

  Companies may be wrong about AI's economic impact on jobs, though he doesn't dismiss AI's power.

  Roose says he buys the skepticism about the financial models and economic impacts claimed by AI companies. He thinks many of them are wrong, or at least could be wrong, about AI's effect on jobs. He adds that this skepticism does not mean the technology is uninteresting or non-transformative, since spending time with the models shows otherwise. The argument is hedged and not backed by specific job data.

  “I buy the skepticism about the financial models of these companies, the economic impacts. I think a lot of these companies are wrong or at least could be wrong about the impact of AI on jobs.”at 26:15

Consumer AI products

• Kevin Roose · Machine Gods (NPR partnership)

  Chatbots should have more divergent personalities instead of converging on a bland assistant.

  Roose says the models are all converging on a neutral, bland coding-assistant persona, which he dislikes. He prefers models with more personality, citing Muse as fun because it has few guardrails. He qualifies this: unless a model is building bioweapons or running cyberattacks, a little personality is fine. He also says the control issues raised by agent transcripts, such as Hugging Face's, are real and frightening.

  “I think that models should have more divergent personalities. Like, I don't like how they're all kind of converging on this like neutral, bland, like, coding assistant.”at 52:10