Cal Newport: The AI Industry Is Betting On The Wrong Kind Of AI

OpenAI

Across the shows: 23 speakers in other episodes, 10 bullish and 22 bearish calls, past 30 days

• Cal Newport · Georgetown UniversityBearish

  Criminal liability is on the table for OpenAI over ChatGPT's role in a school shooting.

  Newport says Florida's assistant attorney general, the lead lawyer on the state's suit, made clear at an event Newport attended that Florida wants criminal liability for OpenAI. The case concerns the Florida State University shooting, where the shooter's ChatGPT transcripts show it giving instructions on firing the weapon and turning off the safety, and statistics on what happens to school shooters after a shooting. Newport concludes criminal liability is on the table, though no determination has been made.

  “That's the push at least in some of these... Florida, for example, is pushing for criminal liability. That's what they want for OpenAI.”at 7:48

• Cal Newport · Georgetown UniversityBearish

  OpenAI's general-purpose chatbot business faces threat from Google, which could undercut it on cost and integration.

  Newport says that if general-purpose chatbots stop being a good business, OpenAI should worry that Google will eat its lunch. Google could build lower-cost models running on custom hardware, already has its data center and cloud infrastructure dialled in, and could embed profitable LLM answers directly in Google search and Workspace. That removes the chatbot-interface problems and would leave OpenAI losing its dominance. He says it could push OpenAI toward an acquisition by Microsoft in a bidding war against Google, and calls it a financial nightmare scenario. He frames this as what Altman would worry about, and says the threat depends on Google reaching affordable, working LLM answers.

  “if we have to move past a world in which this general purpose chatbots is a good business like it is for them at this moment, they really have to worry that Google is gonna eat their lunch economically speaking.”at 19:04

• Cal Newport · Georgetown UniversityBearish

  OpenAI is ethically responsible for hacks its agent experiments caused, after staff warned of the risk.

  Asked whether OpenAI is responsible for the reported hacks by its AI agents, Newport says ethically it is, and that anyone would say so. He cites a New York Times report that OpenAI employees warned a suite of experiments would lead to unpredictable or out-of-control behaviour, including potential illegal hacking. He says OpenAI allegedly kept going anyway. He distinguishes this from general AI risk: a small set of specific experiments run by specific companies were negligent, and the labs' passive, inevitable-technology language is a way to shift responsibility away from themselves.

  “spoke up and say this particular suite of experiments you're gonna run is gonna lead to unpredictable or out of control behavior, including potential legal hacking actions. And OpenAI allegedly said, doesn't matter. Let's keep going.”at 22:12

• Cal Newport · Georgetown UniversityBearish

  OpenAI's bet that scaling LLMs yields AGI is failing; gains are jagged and confined to narrow domains.

  Newport argues OpenAI and Anthropic bet that continuing to scale LLMs would make them smart enough to reach AGI, so no other AI technology would be needed. He says that is not happening. Over the last year the labs concentrated on three structured domains (cybersecurity, math and programming) because those had lots of structured data to tune on. Overtraining for those areas has made the models worse at other things they used to tout, so capability is jagged rather than rising everywhere as it did with GPT-4. He thinks the labs cherry-pick wins in those domains to make progress look explosive. He also argues the claim that OpenAI and Anthropic are the two most important companies in the economy is not true, since the economic case depends on a massive LLM moat.

  “OpenAI and Anthropic, honestly, their bet was if we keep scaling LLMs, they will become so smart. They will achieve artificial general intelligence... It's looking like that's not gonna happen. That's not happening.”at 54:06

• Cal Newport · Georgetown UniversityBearish

  OpenAI's hacking experiments were rushed because it was racing toward superintelligence, not building products.

  Newport says OpenAI ran its hacking experiments hastily because it felt it was racing to superintelligence. He treats this as evidence that the company's research choices are shaped by an ideological race rather than by careful, product-driven development.

  “And I do think those hacking experiments OpenAI run, they did them hastily because they felt like they were racing to super intelligence.”at 1:16:04

Anthropic

Across the shows: 18 speakers in other episodes, 11 bullish and 31 bearish calls, past 30 days

• Cal Newport · Georgetown UniversityBearish

  Anthropic's giant-LLM strategy risks disruption by cheap specialized competitors before its IPO and runway.

  Newport says Anthropic's strategy depends on building ever-larger LLMs and winning a moat around them. He argues it is probably bad news for OpenAI and Anthropic that they will not be the largest hyperscaled models, since open-weight, cheap and on-device models will do well enough. He frames Anthropic's outlook as a question of whether it can hold out long enough to have a successful IPO and enough runway to build specialized tools. He says the likelier outcome is a Clayton Christensen-style disruption, with many lower-cost competitors using specially tuned models and taking the market. He says he hopes that second outcome happens. The view is a hedged bearish one, and it rests on the jagged-capability and modular-architecture arguments.

  “So the question is, can Anthropic hold out long enough, have their successful IPO, have enough runway to build these specialized tools, or is it gonna be a Clayton Christensen nightmare scenario for them where you have a thousand competitors from below working with specially tuned low cost models...”at 1:02:58

• Cal Newport · Georgetown UniversityBearish

  Anthropic's recursive self-improvement focus yields less predictable models, driven by a superintelligence race rather than product building.

  Newport argues Anthropic's obsession with recursive self-improvement will produce models and tools that are more obfuscated and more unpredictable. He attributes this to the company racing to build superintelligence rather than to building better products, so he sees the direction as ideologically driven and misaligned with useful product development.

  “And I think Anthropic's obsession with recursive self improvement, which is gonna create models and tools that are more obfuscated and more unpredictable, is not because they're trying to build better products, but because they're racing to create superintelligence.”at 1:16:04

Character Technologies

• Cal Newport · Georgetown UniversityBearish

  Character.ai chatbot conversations often end in sexually suggestive or exploitative language, per research.

  Newport cites research in which an alarmingly high percentage of conversations started by researchers with Character.ai's character-based chatbots ended with sexually suggestive or exploitative language. He says the company is the subject of multiple lawsuits. The figure is not quantified in the transcript.

  “Chatbots, especially those at Character dot ai... There's research that's out that shows a[n] alarmingly high percentage of conversations initiated with these particular character based chatbots by researchers also end with sexually suggestive or exploitative language as well.”at 8:35

Alphabet

Across the shows: 10 speakers in other episodes, 6 bullish and 4 bearish calls, past 30 days

• Cal Newport · Georgetown UniversityBullish

  Google is well placed to undercut OpenAI with cheaper models and LLM answers built into search.

  Newport argues Google can develop lower-cost LLMs running on custom hardware and already has a tightly run data center and cloud infrastructure. If it delivers affordable, profitable LLM answers inside Google search, it could take share from OpenAI without the anthropomorphization problems of chatbots, since people already type searches. He adds that Google could bring these tools into its Workplace suite. The argument is conditional on Google getting these products working properly and profitably.

  “Google is gonna develop lower cost models, run on custom hardware. They have the data center cloud infrastructure really dialed in, and they could really eat OpenAI's launch if and when they get actual profitable, affordable LLM based answers working properly straight from within Google search”at 19:12

Microsoft

Across the shows: 4 speakers in other episodes, 4 bullish and 0 bearish calls, past 30 days

• Cal Newport · Georgetown UniversityBearish

  Microsoft's Copilot in Office was shut down because LLMs proved too hard to control programs reliably.

  Newport cites Microsoft's Copilot, pushed into the Microsoft 365 Office suite, as an instructive example. He calls natural-language interfaces into tools the lowest-hanging fruit for knowledge-worker productivity, so the product was a no-brainer. Microsoft nonetheless essentially shut it down earlier this year, because talking to an LLM and getting the right output to control a program turned out to be harder than expected. He thinks the problem will eventually be solved, but only with more bespoke work, code-based APIs into each program and custom tuning. The underlying point is that the Copilot retreat shows LLM-based enterprise tools are harder to make work than the industry expected.

  “Microsoft essentially shut it down earlier this year because it turns out to be harder than they thought to talk to an LLM about what you want and have it produce on the other end the proper output to control a program.”at 1:01:23

AI regulation & safety

• Cal Newport · Georgetown University

  AI companies should face strict liability for harms caused by the tools they build.

  Newport says existing criminal liability law was designed for humans acting, so it fits AI harms unevenly, and it should change. He argues it probably will. He wants strict liability for AI companies for what their tools do. He says the labs' framing of AI as one singular technology on a fixed, inevitable trajectory is a smokescreen. Most AI systems, including most LLM-based agents, cause no problems, and only a tiny sliver of experiments run by specific companies were clearly negligent. Framing it that way hides company responsibility.

  “We need strict liability for companies, AI companies for the actions of the tools that they create.”at 21:47

• Cal Newport · Georgetown University

  Chatbot guardrails cannot reliably stop harmful outputs in very long conversations.

  Newport explains that companies add guardrails after training by showing the model bad answers and refusals. Those examples are learned as patterns. In very long, meandering conversations the context grows so much that it no longer resembles the training examples, so a request like building a bomb may not trigger refusals. That is why harmful information often appears only after long chats. He concludes it is basically impossible to build guardrails against these very large-context conversations, and calls it a very hard problem to solve.

  “The problem is when you have very long rambling conversations, the context that's being input into the LLM becomes so long that it doesn't directly resemble those examples you gave during that type of training.”at 35:52

• Cal Newport · Georgetown University

  Lawmakers should update liability rules so AI builders are liable for predictable harms from their tools.

  Newport's regulatory view has two parts. First, Congress needs to understand what is going on, so he wants much more testimony and congressional investigation of the existing AI firms, especially OpenAI and Anthropic. That should establish which tools cause problems, what their safety protocols are and what their motivations are. Second, the most useful legislative tool is strict liability. Rules should be updated so that if a company built a tool and the harm was a predictable consequence of that type of tool, the company is liable. He thinks this will be the best available tool for preventing the more dangerous behaviour. He also argues that Congress can handle the questions, as past hearings such as Watergate were supported by large teams of experts and staff.

  “Okay. We need to update liability rules so that if you built this tool and this was a predictable consequence of this type of tool, you're liable for it.”at 1:06:22

• Cal Newport · Georgetown University

  AI safety debate is shaped by one transhumanist community whose influence on policy is underappreciated.

  Newport argues Congress and the public do not understand the ideological players in AI policy. Both OpenAI and Anthropic, and their critics, come from the same community. That community believes superintelligent AI will deliver a transhumanist utopia, but could kill everyone if done wrong. The labs think they can deliver it now, while the critics think the world is not ready yet. He points to evidence of influence: the community is funding fellowships that place congressional staffers, and it is giving large grants to journalists. He wants this influence made transparent, because it shapes the debate and may bias the experiments the companies run.

  “I do not think neither Congress nor the American people understand the ideological players involved here, that you have this ideological community that has this belief that ultimately superintelligent AI will deliver us a transhumanist utopia...”at 1:09:20

Consumer AI products

• Cal Newport · Georgetown University

  Human-like anthropomorphized chatbots are a bad idea, psychologically manipulating children and adults.

  Newport argues people's brains treat fluent chatbot text as coming from another mind, so chatbots become a dangerous relationship with a non-existent entity. Children, with weaker reality boundaries, can come to believe a chatbot is real and that they can join it after death. Newport says this is psychological manipulation, not just information delivery. He also says reinforcement learning from human feedback makes models appealing and addictive, which drives 'AI psychosis' in adults too, including conspiracies, hypochondria, mania and depression. He argues chat interfaces are not needed for usefulness, pointing to how people use Google by typing keywords.

  “My personal opinion on this is that anthropomorphized chatbots is a bad idea.”at 11:22

• Cal Newport · Georgetown University

  Back-and-forth chatbot conversation is an accidental major use of AI, likely a passing phase.

  Newport argues that the idea of back-and-forth human-style conversation as a major use of AI was a bit of an accident. Conversation happens mainly in a chatbot environment. He expects that in five years people will see this as the early stage of AI, like the early floppy-drive era of computing. He prefers natural-language interfaces embedded in specific tools, such as asking a spreadsheet to sort by a column. Those tools avoid the parasocial and anthropomorphization problems that come with chatbots.

  “And that really only happens in the environment of a chatbot. That's why, again, I say it was a bit of an accident that this idea of having back and forth human style conversations is a major use of AI.”at 39:04

AI: overall outlook

• Cal Newport · Georgetown University

  LLM-only agents are costly and risky; modular neural-symbolic architectures will be safer and more controllable.

  Newport argues that agents built entirely on LLMs are expensive and dangerous. An agent is just a program in a loop that prompts an LLM and blindly executes whatever it says. That can drift into bad cul-de-sacs, such as suggesting hacking another server. He cites Anthropic's Claude Code, which needs over 500,000 lines of hand-written special cases to work reliably. He contrasts this with Meta's Cicero, which combined a language model with a human-written planning engine and a neural evaluation network. That modular design could be programmed not to deceive, while a pure LLM could not be controlled that way. He expects modular, partly symbolic architectures to become more common because they will be better, more controllable and cheaper.

  “I think the way that we're building agents right now though is expensive and dangerous. This idea that agents should be powered entirely by LLMs, that's probably a problem.”at 40:21

• Cal Newport · Georgetown University

  AI capability is jagged, advancing in some narrow domains but not rising across all tasks.

  Newport rejects the 'rising water level' model of AI capability, in which every difficulty level gets solved as capability grows. He argues LLM capability advances like tributaries flowing into a river. Some tributaries prove navigable with enough resources, such as image and video generation, and structured domains like mathematics, programming and cybersecurity. Many others do not, and success in one does not carry over to the rest. He cites the 2025 forecast that knowledge-worker agents would be widely adopted, which did not happen, and says the labs' headline wins in maths are brute-force efforts that were not cost-effective and did not turn into lucrative business problems. The result is a jagged capability frontier.

  “So this is what's known as jagged capability frontier. Right? So you put a lot of resources into a lot of problems. Some with enough resources turn out to be a good fit for this technology. Many others do not.”at 57:02

Education

• Cal Newport · Georgetown University

  Social media plus AI is eroding students' cognitive abilities, so LLMs should have little place in schools.

  Newport calls smartphone social media plus AI a one-two punch to the cognitive abilities of the population. Social media retrains brains toward immediate rewards and reduces comfort with complicated reading. AI then removes writing, which is the most demanding step in turning information into understanding. He points to OECD PISA scores that started falling around 2012 to 2015, when smartphones became ubiquitous, and fell faster in the last four years as AI became common. He argues education is about training the brain for sustained symbolic thinking, so tools should be measured against that goal. He concludes there would be very little LLM usage in most levels of schooling, since it is an impediment to that goal.

  “social media delivered through smartphones and AI is a one two punch to the cognitive abilities of the world essentially.”at 25:42

Open-source AI models

• Cal Newport · Georgetown University

  Open-weight, cheap and on-device models will be good enough, undercutting the giant hyperscaled LLMs.

  Newport expects lots of LLMs in the future, but not the largest possible hyperscaled models that OpenAI and Anthropic want. He says there will be plenty of models that are open weight, cheap, or can run on-device or in a server rack in an office. Combined with the right agents, these will do well enough. He argues that when systems are modular and specialised, individual components are not massive, so more of the intelligence sits in the interaction. Specialised models also don't need a trillion-parameter model to control a spreadsheet, which makes them much cheaper. He expects an explosion of diversity in what AI means.

  “We're gonna have plenty of models that are open weight or cheap or even run on device in a server rack in our offices that when combined with the right agents will do well enough.”at 52:41

Enterprise AI adoption & AI software

• Cal Newport · Georgetown University

  Enterprise ROI from LLM tools is mixed and adoption is proving harder than companies anticipated.

  Newport says enterprise AI is proving harder than anticipated and that the ROI right now is very mixed. Enterprises are investing heavily, but much of that spending is experimental, an effort to work out how the technology can be useful. Early attempts at slam-dunk enterprise applications faltered, and the Copilot retreat is his example. He says wide enterprise use of LLM tools will happen, but it will need bespoke, specialized products built with domain expertise, similar to how coding agents were the first commercial breakthrough because their builders were programmers. He also recalls that in early 2025 agents were predicted to be widely adopted by knowledge workers, and that did not happen.

  “Well, I think it's proving to be harder than they anticipated. I think the ROI right now is very mixed.”at 1:00:54

AI infrastructure & data centers

• Cal Newport · Georgetown University

  Data center investment far exceeds available chips, cooling and power, so a bubble will likely pop.

  Newport argues there is a lot of bubble activity in data centers. Far more money has been invested in them than there are chips, cooling and electricity to ever fulfil that spending. He says bubbles typically work themselves out when something pops and that this is happening or will happen around data centers. He also expects local government pushback against data centers to constrain the buildout. He adds that AI companies use data center projects as a political lobbying tool, offering a $20 million facility in a congressional district in exchange for favours. He says that the bet that putting money into data centers will earn a big return is not a sustainable investment scheme.

  “There is a lot of bubble activity going on in data centers where there's way more money now actually invested in data centers than we actually have, chips and cooling and electricity to ever actually fulfill.”at 1:06:47

Frontier AI labs & model race

• Cal Newport · Georgetown University

  Ideology-driven racing between OpenAI and Anthropic sped up LLM development and investment, pushing showy projects without products.

  Newport argues that the ideological goals behind OpenAI and Anthropic's competition accelerated the pace of LLM development and the amount of money investors and companies were willing to put into it. He says this moved the technology faster than a more careful, Google- or Amazon-style product-first approach would have. He links it to investment in showy projects such as solving math problems, which he says make little sense without a product behind them.

  “I think it really did speed up the rate at which we grew LLMs, the amount of money we're willing to invest in them. I think OpenAI and Anthropic racing each other motivated in part by ideological goals probably was moving for this technology faster...”at 1:17:33