Building the Cloud for an Agentic World | AWS CEO Matt Garman

Matt Garman · Amazon Web Services (AWS)

The binding constraint on the AI buildout rotates constantly, from power to memory, TSMC, HBM and networking.

Garman argues there is never just one constraint. Once one bottleneck is solved, another appears, first power, then possibly memory, TSMC capacity, HBM, networking components, or a supply blip such as connectors. Where capacity is located also matters, because power is not fungible across regions. He adds that construction labor is now at a premium.

“For us, it's do you have enough power? And then as soon as power is no longer the constraint, ... it might be memory. It might be TSMC capacity. It might be, you know, HBM.”at 27:36

Matt Garman · Amazon Web Services (AWS)

The data center industry must be more vocal about benefits, since a few bad operators hurt everyone.

Garman says the industry needs to be more upfront about its benefits, including renewable power, minimal water use from free-air cooling, and high-paying jobs. He cites a county where residents pay about $5,000 less in taxes because of the data centers, though residents don't know it. He says some operators ignore regulations and move fast, which damages the whole sector's reputation.

“I think we need to be more vocal and be more upfront, because we actually do a ton that's that's really beneficial, both for communities we operate in.”at 30:05

Matt Garman · Amazon Web Services (AWS)

Startups now launch with far more capital and billion-dollar valuations from day one than before.

Garman contrasts early AWS startups, which raised about $10 million and iterated slowly, with today's startups. Today's startups are often valued at a billion dollars from day one and raise around $200 million. He argues the bigger scale and ambition of these ideas require far more capital, especially to train models.

“startups started at a much smaller size than when we first started. ... Now it's from day one, they're valued at a billion dollars. They have $200,000,000 of funding.”at 5:21

Matt Garman · Amazon Web Services (AWS)

Most enterprise agents are simple and non-autonomous; trust and evals block fully autonomous workflows.

Garman says most enterprise agents built so far are relatively simple and mostly non-autonomous, with people still in the loop, though customers still get a lot of value. He says the next step is autonomy, but nervousness about trust is holding enterprises back: they need guardrails, permissions, and assurance that an agent will not delete production systems. He says enterprises often just replicate existing workflows, when they should rethink them from a greenfield approach that agents can do differently, such as trying 50 approaches in parallel. He says evals, labeled data, and drift measurement are the hardest unsolved problems and the single biggest area where customers need help. AWS is responding with forward-deployed teams that teach customers to own evals within about 45 days, rather than a consulting dependency.

“And right now, I think that nervousness is probably holding people back... You don't actually want an agent to just go crazy and accidentally deleted production database.”at 42:26

Matt Garman · Amazon Web Services (AWS)

Fine-tuning open-weight models on proprietary data can yield better, cheaper models; most do it on SageMaker.

Garman says many customers believe they hold meaningful proprietary data that could be mixed into post-training or fine-tuning of an open-weight model, which they could distill down to get a better-performing model at a lower price. He hedges that many pieces must work and customers need a good eval to prove it. He notes most people doing this today run it on SageMaker and host inference there, and says AWS is ramping up support for open-weight models and sees SageMaker getting a new lease of life as a custom model-building platform.

“a lot of customers believe that they have meaningful proprietary data... do some fine tuning to an open weights model that they could distill down, they can actually get a better performing model at a lower price.”at 47:27