Chips, Memory, and Power | Pat Gelsinger

NVIDIA

Across the shows: 19 speakers in other episodes, 16 bullish and 3 bearish calls, past 30 days

• Pat Gelsinger · Playground GlobalBullish

  Nvidia will absorb Groq's architecture, hiding the heterogeneity inside its platform within two years.

  Pat says large players will own some of the chip heterogeneity, and points to Nvidia's deal with Groq as the example. Within two years, customers will not be able to tell what Groq technology sits inside Nvidia's products. He says this is how abstraction and layering work: the architectural variety is absorbed beneath a unified platform. He reads this as Nvidia's way of keeping control of AI compute as the field narrows, which is a positive for Nvidia's position.

  “we saw this with what NVIDIA did with [Groq]. Okay, we're going to sort of own some of that heterogeneity, two years from now, you're not going to tell what [Groq] is inside of Nvidia.”at 21:35

• Pat Gelsinger · Playground GlobalNeutral

  Nvidia's NVL72 rack was an engineering marvel but a manufacturing nightmare that delayed the industry.

  Gelsinger says Nvidia's NVL72 rack-scale system is an engineering marvel but a manufacturing nightmare, and that it took the industry an extra year and a half to digest. He ties this to the immaturity of the optical supply chain, which limited how quickly scale-up designs could be built. He does not give a directional view on the stock.

  “I should have never built an NVL 72. It's an engineering marvel and it's a manufacturing nightmare. It took the industry an extra year and a half to digest that beast.”at 36:42

• Pat Gelsinger · Playground GlobalBearish

  Nvidia's GPU power per teraflop has been flat for five generations; CMOS scaling has stalled.

  Gelsinger argues that CMOS scaling has stopped improving the power needed per teraflop. As evidence he cites five successive generations of Nvidia chips with a flat power-per-teraflop line. He concludes that the physics must change, and points to superconducting computing (Snowcap) as the path to a much better power profile.

  “CMOS hasn't improved the essential power per teraflop for the last five generations of NVIDIA chips, flat line. Not okay.”at 48:14

d-Matrix

• Pat Gelsinger · Playground GlobalBullish

  D-Matrix is pursuing memory-compute integration that he views as the fundamental fix for HBM bandwidth limits.

  Pat says HBM's shoreline bandwidth limit is a fundamental issue, and that the future is stacked, with memory and compute brought together in new structures. He points to D-Matrix as a company doing this, and to Cerebras with similar techniques, as examples. He ties this to his view that AI capital will fund breakthroughs in memory. His remark that 'we have a company D Matrix' suggests Playground Global is involved with the company.

  “the idea, and obviously we have a company D Matrix that's doing this, but I think others will be”at 25:07

Snowcap Compute

• Pat Gelsinger · Playground GlobalBullish

  Snowcap's superconducting approach could deliver roughly 1,000x better chip power performance than CMOS.

  Gelsinger argues that CMOS has stopped improving power per teraflop across five Nvidia generations, so the physics need to change. He presents Snowcap, a superconducting company that Playground backs, as a way to change that physics. He says this affords a thousand times better power performance. He expects these innovations to arrive in the near future, though he does not give a date for the performance claim.

  “we have a superconducting company, Snowcap, right, which affords the opportunity to fundamentally change the physics, you know, a thousand times better power performance.”at 48:22

AI chips & semiconductors

• Pat Gelsinger · Playground Global

  AI tools can now handle most logic design, though analog blocks like SerDes still resist automation.

  Pat says the 486 was the start of modern EDA, and AI could be the next such inflection. Transistor budgets are large enough and design tools are getting smart enough, with a few experts guiding them, that many logic functions can now be designed with AI tools. Analog design, with SerDes as the best example, remains hard to automate. It needs lots of silicon data, so designers either become conservative in analog or do it by hand. He sees the current moment as an early version of the 486 transition.

  “But now many of the logic functions can really be done with AI tools and techniques in pretty incredible ways.”at 8:30

• Pat Gelsinger · Playground Global

  AI design tools cut chip design to three months, but silicon processing still takes nine months.

  Pat argues AI has made the design step easy: a supercharged design flow produces a good AI chip in about three months. The binding constraint is then manufacturing. He says he still needs about nine months of silicon processing before scale, plus advanced packaging of roughly three months, plus rack-scale integration. Because the AI software being targeted can change within that year and a half, the design can be obsolete by the time it ships. He says the cycle needs to compress to a month or two, which would require new lithography and cheaper prototyping flows, since mask costs run around $50M before prototyping.

  “So I can design the thing in three months, but I can't actually get it into real silicon at scale for nine months.”at 10:19

• Pat Gelsinger · Playground Global

  Chip power modelling is poor; guard-banding wastes about 40% of the power envelope.

  Pat says current power simulation is weak. The industry lacks good 3D modelling tools for power dissipation and hotspots. Because voltage is uncertain, designers guard-band almost the full power envelope and effectively lose about 40% of power to that margin. Power dissipation techniques and power delivery also need to change. He expects better power modelling to be needed, and he sees power delivery moving into the vertical dimension of the stack, with better regulation reducing thermal hotspots.

  “My voltage in, I'm essentially guard banding almost my full power envelope. So essentially I'm losing 40% of my power in guard banding, so I got to get much better at power modeling.”at 13:09

• Pat Gelsinger · Playground Global

  Around 100 AI accelerator startups will consolidate into a few winning architectures.

  Pat argues the current glut of AI inference chips is not sustainable. Many will fail because scale requires capital, workloads and software investment. Workloads are also becoming more granular, with specialised prefill, mid-fill, decode and reasoning tiers, and then shifting again, which makes extreme specialisation unlikely to last. Capital markets and market share will pick the winning designs. He expects big buyers such as OpenAI, Nvidia and Anthropic to back particular platforms because hardware and software need to be co-designed. He concludes the field will narrow.

  “there's like 100 of them now. ... And you know they're not all going to win, and they're not all going to win because at the end of the day, you have to get scale on these things.”at 18:03

• Guido Appenzeller · Andreessen Horowitz

  AI-written software lowers the cost of programming diverse chips, so architectural heterogeneity may persist.

  Guido argues that historically the industry converged on one or two architectures because the software complexity of different instruction sets required multiple compilers, which limited hardware platforms. That constraint seems to be changing. AI agents can now write optimised kernels, so a chip with a given capability is far easier to build software for. He suggests this could make heterogeneity more durable than the consolidation Pat expects. He frames this as a question, so the view is tentative. He also argues that a transition period will produce an explosion of approaches before the field shakes out.

  “That part seems to be changing with AI a little bit, right? Today, ... but an agent swarm overnight can do it. That's all we need, right?”at 20:25

• Pat Gelsinger · Playground Global

  HBM is a poor memory and memory innovation is due, the first new memory architecture in thirty years.

  Pat calls HBM a 'hideous' memory with poor bit density. It is limited by shoreline bandwidth, runs into power and thermal issues (DRAM does not like heat), and is hard to pack next to compute. He says the industry has produced no major new memory in about thirty years, despite around 100 attempts he has seen, and that memory has been a poor business with boom-bust commoditisation. He thinks AI's large capital commitment will now fund breakthroughs. He expects new materials such as ferroelectrics and non-capacitive, high-density, stackable memories. He is sceptical of low-cost flash and of processing-in-memory, which he thinks imposes workload constraints, and prefers high-bandwidth memory close to compute. He also backs memory-compute integration and says he has funded a stealth memory company.

  “But I think memory innovation for the first time in thirty years is nigh upon us.”at 26:51

• Pat Gelsinger · Playground Global

  Memory stacks will top out around 2-4 high; 16-high stacks are unrealistic because yields compound.

  Pat says stacking gets harder with each layer. The value of a stack rises with its height, so per-layer yield must rise exponentially, not linearly. Defect management and spare-line redundancy cannot offset a bad die. He expects three to five high stacks to be the sweet spot and says a two- or four-high memory stack is probably where things settle. Compute tiles also become constrained by yield and thermal limits, and power delivery and redistribution layers add to the stack. He ends up at roughly eight layers as a practical limit for his canonical design, with optics and other materials added to that.

  “I'm not a crazy 16, 32 stack kind of guy. I think there's gonna be nice three, four, five stacks that just end up sort of being the sweet spot.”at 29:05

• Pat Gelsinger · Playground Global

  Continuous learning will merge training and inference, pressuring the trend toward many specialized AI chip architectures.

  Gelsinger argues that the separation between training and inference will shrink over time, as more algorithms need continuous learning and updating model weights during inference. He sees this as a major argument against the growing number of heterogeneous architectures. He also points to the operational cost of managing fleets with different chips for different workloads, such as fleet management, upgrades and fault domains. Together, these point toward consolidation around fewer architectures.

  “I do think the separation of training and inferencing is gonna become less so going forward, not more so. That's a little bit of the argument against more and more heterogeneous architectures is I think continuous learning.”at 40:41

• Guido Appenzeller · Andreessen Horowitz

  Different AI workloads need different chips, so specialized hardware may persist despite consolidation.

  Appenzeller argues that heterogeneity may still matter because AI workloads have different hardware needs. Some are compute-limited, while others are memory-bandwidth-limited, and a small diffusion workload needs a very different chip from a very large LLM. He sees this as a possible driver of agility in hardware choices. Gelsinger pushes back that at data-center scale the flexibility of a single architecture is worth a lot, though he concedes there is some room for specialization.

  “I still think there might be a role for heterogeneity coming from a different angle... different types of AI workloads require very different underlying [hardware]. Like, some are more compute limited, some are more memory bandwidth.”at 41:29

AI infrastructure & data centers

• Pat Gelsinger · Playground Global

  Energy capacity limits AI buildout; expects more data center project defaults as power fails to arrive.

  Pat argues that in the AI era, energy capacity equals economic capacity. There is little point building a new data center and buying a million GPUs if the power cannot be supplied. He expects a rising number of defaults on data center projects because the energy will not be there. The argument is that power, not chips, will set the pace of the buildout, and that projects financed on the expectation of power that never arrives will fail.

  “energy capacity equals economic capacity. Why build a new data center and buy the million GPUs if I can't power that? You're gonna see more and more defaults happening on many of those data center projects because the energy won't be there.”at 0:02

• Pat Gelsinger · Playground Global

  Optical I/O will replace copper in AI scale-up; NPO and CPO should arrive around 2028-29.

  Pat argues that copper is becoming a shorter waveguide as AI systems scale. The physics pushes the industry to optics, since making copper work at five metres costs more than making optics work at 100 metres. He thinks all I/O should move to optical, but not the core compute-memory complex, which he expects to stay electrical. He acknowledges that optical supply chains have never been tested at scale. He says the industry is looking at NPO or CPO in roughly the 2028-29 timeframe. He thinks optical-to-electrical conversions in the core add cost and power loss.

  “And essentially, copper is becoming a waveguide that's getting shorter and shorter. And all of a sudden, to make my copper work at five meters is more expensive than making optical work at 100 meters.”at 35:52

• Pat Gelsinger · Playground Global

  AI clusters will move to optical interconnect and optical circuit switching because AI traffic is predictable.

  Gelsinger contrasts traditional networks, built to route any packet anywhere, with AI traffic, which consists of large, predictable flows that look more like switching than packet routing. He sees optical moving into the compute complex and optical circuit switching (OCS) becoming the right architecture. He notes the bottleneck is an immature optical supply chain, with thermal, packaging and laser-capacity problems that take time to solve. He expects the path to resolve around 2028–2029 and calls the fixes hard work rather than fundamental barriers.

  “to me, I do see that, yeah, we will go all the way to bringing optical into the compute complex, and we will go all the way to OCS or some form of optical switching with more predictable flows. That is the right architecture.”at 39:19

• Pat Gelsinger · Playground Global

  Power shortages will cause data center project defaults, a headwind on the AI buildout.

  Gelsinger argues that energy constraints will somewhat limit how euphoric the AI buildout can become. Developers will not be able to justify building data centers and buying a million GPUs if they cannot power them. He expects more defaults on data center projects as energy fails to materialize. He sees Oracle's comment as an early sign. He does not expect a slowdown, but expects investors to question the capital commitments for cement, racks and GPUs, making power an upper bound on growth.

  “why build a new data center and buy the million GPUs if I can't power them? Right? And I think you're gonna see more and more defaults happening on many of those data center projects because the energy won't be there.”at 45:21

Power demand & the grid

• Pat Gelsinger · Playground Global

  US energy capacity has stalled, growing only about 4% a year, and that constrains the AI buildout.

  Gelsinger argues that for about fifteen years the US retired coal faster than it added renewables, so national energy capacity went flat. In an AI-driven economy, he says, energy capacity equals economic capacity, so this stagnation holds back growth. He cites national capacity growth of only about 4% a year despite recent investment. He adds that gas turbine lead times are about eight years, renewables depend heavily on China, and the country needs more nuclear and US-based supply chains.

  “Because in an AI digital age, energy capacity equals economic capacity. So essentially, my economic capacity as a nation flat for fifteen years if you buy that thesis, which I think is very provable.”at 43:53

• Pat Gelsinger · Playground Global

  Vertical GaN power devices will be a killer technology for 800V DC data center power conversion.

  Gelsinger argues that moving data centers to 800V DC will require better power conversion and that many conversion steps can be eliminated. He says vertical GaN devices can convert 800V directly to 48V, 12V or even 5V in one step, which saves a lot of energy. He mentions that Playground has a couple of companies in this space, one of them in vertical GaN, though he does not name it. He also expects solid-state transformers and switches to rebuild the power delivery network.

  “We have a couple of companies, one of those vertical GaN, I think will be a killer technology to be able to do 800 to 48 or 800 to 12 or even 800 to five in one conversion step.”at 47:00

Frontier AI labs & model race

• Pat Gelsinger · Playground Global

  LLMs are nearing their limits, and flat LLMs fail on 3D molecular and chemistry workloads.

  Pat argues that today's models are about to undergo evolutionary breakthroughs because the LLM is reaching its limits. Flat LLM architectures do not work well for modelling 3D structures such as molecules, chemicals and imaging. He expects these limits to force shifts in the algorithmic domain. He also expects some workloads to bring back high-precision HPC-style computation, such as 64-bit chemical algorithms run on outputs from AI models, which would change the chip mix.

  “I think today's models are about to go through some evolutionary breakthroughs as I think the LLM is sort of reaching the limits.”at 16:20

Nuclear energy

• Pat Gelsinger · Playground Global

  The US should build far more nuclear power; no new US reactor has come online in twenty years.

  Gelsinger calls nuclear a fabulous base-load source the country should build more of. He points out that the last US nuclear reactor came online about twenty years ago. He argues the US stopped building out a very capable industry, which is part of why energy capacity has stagnated.

  “I think nuclear, right, as a base load is a fabulous thing that we want to go build more of.”at 44:47

Enterprise AI adoption & AI software

• Pat Gelsinger · Playground Global

  Virtualization and management functions must be rebuilt for AI agents, creating opportunities for new companies.

  Gelsinger argues that each abstraction layer in computing leads to the next, and that the virtual machine concept still belongs in the compute stack. He says that with agent swarms, questions about who manages the agents, sets their security profiles and monitors their performance are unanswered. He argues that every element of virtualization and management must be recreated for this new computing layer. He expects many companies to emerge with solutions.

  “Essentially, every fundamental element of virtualization and management needs to get recreated in this next computing hierarchy. There's gonna be lots of wonderful companies that you'll sort of break through for how those things will be done.”at 50:30