20VC: Horizontal vs Vertical Agents: Who Wins | Model Routing: Commodity or Increasing in Value | The Future of the Inference Market | Why Founders Should Take Discounts for Tier 1 Firms with Harrison Chase, LangChain

OpenRouter

• Harrison Chase · LangChainNeutral

  Most OpenRouter usage is gateway model access, not smart routing, which is done less than people think.

  Chase calls OpenRouter an incredible company with great usage, but argues most people use it for single-API access to chosen models rather than for routing. Users specify the model upfront. He guesses most of its usage comes from being a gateway, not a router. He also says smart routing is done far less than people assume, and that it is mostly built into harnesses by firms like Factory.

  “I think a lot of why people use OpenRouter and Vercel and other gateways is largely for the model access. So I think there's also a difference between a router and a gateway.”at 18:00

Anthropic

Across the shows: 41 speakers in other episodes, 22 bullish and 47 bearish calls, past 30 days

• Harrison Chase · LangChainBullish

  Enterprise agent spend, beyond coding, will likely drive Anthropic's value; he is bullish.

  Chase agrees that the share of engineering pay spent on agents underpins Anthropic's value. His caveat is that agents have only had real effect in software engineering so far. He argues many other enterprise functions have yet to be affected. Software engineering spend could go either way, but spend in other domains will undoubtedly be much higher than today. That is why he is very bullish on Anthropic and other model providers.

  “I think it's probably right that that number underpins [Anthropic's] value. I think the tricky part is that the only place we've seen agents really have effect is in software engineering so far, though. And there are a ton of other places within an enterprise that I would argue have yet to kind of be affected by agents.”at 42:58

OpenAI

Across the shows: 32 speakers in other episodes, 18 bullish and 29 bearish calls, past 30 days

• Harrison Chase · LangChainBullish

  Enterprise agent spend beyond coding will drive model providers, so he is very bullish on OpenAI.

  Chase expects agent spend in enterprise domains outside software engineering to rise a lot from today's level. The main uncertainty is how software engineering spend changes. Enterprise spend outside coding will undoubtedly grow, and he says that makes him very bullish on OpenAI, Anthropic and other model providers.

  “for the other domains, it will be a lot more than it is today. And so I think from that perspective, I think that's very bullish on Anthropic and OpenAI and other model providers.”at 43:25

Enterprise AI adoption & AI software

• Harrison Chase · LangChain

  Agent context and memory should stay portable, since tying them to one model provider creates lock-in.

  Being able to switch models helps offensively, since a team can use the best model no matter who makes it. It also helps defensively, since a team avoids lock-in that lets a provider raise rates. If memory and context get built around one model's way of interacting with them, portability suffers and the business is at a disadvantage. Chase argues the context and harness should therefore stay neutral.

  “Being able to change models is very offensive and defensive. So it's offensive in the fact that you wanna be able to take advantage of the best model when it comes out no matter who it's coming from. It's defensive in the fact that you don't wanna get locked in”at 9:45

• Harrison Chase · LangChain

  Vertical agents will win because domain experts who write verifiers and evals capture the value.

  If agent value comes from verification, the winners are domain experts who can quantify what good output means in their field, which is hard work. Those experts will be best placed to write the verifiers and evals and then build harnesses and models. Chase advises starting at the harness layer with context engineering and moving to post-training once there are users, since that is far higher ROI.

  “I'm massively bullish on vertical agents for different domains for this reason, because I think the domain experts who are best suited to write those verifiers or build those evals... will be able to build harnesses and models.”at 12:18

• Harrison Chase · LangChain

  Model routing belongs in the agent harness for vertical agents; gateway routing only suits general chat and coding.

  A harness holds the state needed for good routing, such as prompt-cache status and model-specific prompting strategies. A gateway receives only a single API call with limited state. Routing also needs domain and harness awareness, so an off-the-shelf gateway router will not serve domain-specific work. Gateway routing fits only horizontal use cases like general chat or coding. Chase agrees that in two to three years the majority of routing will sit in the harness.

  “If you're building a vertical agent, I think routing needs to be in the harness. If you're thinking about a general purpose horizontal task, like just general chat or coding, it maybe be in the gateway.”at 21:13

• Harrison Chase · LangChain

  Model routing is not a commodity; it is hard to build because models change so rapidly.

  Chase accepts that many people will build routing, but says building it well is hard even today. Models change rapidly. A new frontier model can beat older models at everything, which shrinks the value of routing decisions built on past comparisons.

  “I don't think routing is a commodity. I do think people will build it, but I think building it is quite hard even today. And I think part of the reason for that is the models just change so rapidly.”at 21:22

• Harrison Chase · LangChain

  The agent context layer will be an agent wrapped around a database, since extracting memory is the hard part.

  Databases like ClickHouse, Mongo and Elastic are good at storing and querying data, but they predate AI. An agent context layer needs intelligence to turn observations into useful information before storage. Chase says the hardest part of memory is not the database, since many companies store memory in a single file. The difficulty is distilling user and software interactions into valuable information.

  “I think a key part of what exists there is, yes, there's a database, but I actually think there's a system around the database. I think, like, a context layer for agents will look more similar to, like, an agent around a database than just a database.”at 26:47

• Harrison Chase · LangChain

  Agent memory is massively unsolved but will become standard in most products within a few years.

  Memory is rare in agents today but Chase expects it to be common in the next few years, with almost every product having some memory aspect. Consumer agents like Instinct and Muse show the value through personalization and learning. LangChain tried a general memory product and shut it down for lack of product-market fit. Chase believes memory is domain-specific, since what memory looks like for a consumer app differs from legal or finance, so learning agents will need to be domain-specific.

  “I think this memory layer for agents is still massively unsolved. I think one thing about agents that is rare today but will be incredibly common in the next few years is this memory layer.”at 29:47

• Harrison Chase · LangChain

  Service providers will expose their own agents, letting third-party agents interact rather than be blocked.

  Agents multiply request volume. A concert-ticket agent polling every five minutes can generate about a thousand times the queries a human would, and Chase recalls Resy blocking a user's account for heavy agent use. He says infrastructure must become much burstier. He expects the future to be providers exposing their own vertical agents, with users' horizontal agents interacting with them. A vertical agent is more efficient with underlying systems. Ads and upsells can be included in the natural-language agent response, so the provider keeps its monetisation.

  “I think the future is providers exposing their own agent and having people either interact with that agent or having people's horizontal agents interact with that agent.”at 34:44

• Harry Stebbings · 20VC

  AI customer support is crowded and uninteresting, with about 20 well-funded providers and incumbents.

  Stebbings says the customer support space is wildly uninteresting. He cites about 20 providers, all funded with over $100 million in the last eighteen months. He adds that incumbents such as Salesforce and Atlassian are present, along with older players such as Intercom and Talkdesk. The implication is that a crowded market with big incumbents limits the opportunity for new entrants.

  “I think the the customer support space is, like, wildly uninteresting. 20 providers all funded with over 100,000,000 in the last eighteen months. You got Salesforce and Atlassian and big incumbents there too.”at 47:32

• Harrison Chase · LangChain

  Vertical AI agents will be a good layer, won by deep domain expertise rather than by model weights.

  Chase thinks vertical agents are a good layer. He says the important part is a domain-specific point of view and diving deep into the domain, ideally in an area with low AI penetration but lots of built-up domain expertise. That expertise can be captured in data or logic in the harness, and eventually possibly in model weights, though he would not start there. He adds that the teams succeeding in this area need strong technical talent, speed of execution and brand.

  “I think vertical agents are going to be a good layer.”at 48:57

Open-source AI models

• Harrison Chase · LangChain

  Post-training open-weight models is valuable but works best for simpler subtasks, not the core agent loop.

  Chase lays out a ladder of control: context first, then harness, then post-training an open-weight model, then training from scratch. Most companies post-train, and few train from scratch. Post-training pays off now for simpler tasks inside an agent, such as classification, extraction, summarization, guardrails and routing. Post-training the core loop model is unresolved, since frontier models keep improving at running that loop. Chase says it is not the right move at the start, but with heavy usage it could become realistic in about five years.

  “a lot of the post training that practically makes sense right now is probably for, like, simpler tasks that make up a larger harness.”at 14:48

• Harrison Chase · LangChain

  Enterprise fear of Chinese open models is largely unfounded if self-hosted or run via a US host.

  Chase says some enterprises fear Chinese open-source models and others do not care. He does not think the fear is justified. The concern that data will flow back to Chinese labs is ridiculous if the model runs on a host such as Fireworks or is self-hosted. A second concern is that training data will make the model respond in ways that conflict with enterprise values. He thinks that is largely unfounded, though he concedes some domains may have real nuance, which evals can address.

  “I think one concern is that the data that it runs on will somehow go back to the Chinese labs. And again, if you're using Fireworks or hosting it yourself, I think that's just ridiculous.”at 44:10

• Harrison Chase · LangChain

  US open-source models will match Chinese open models within twelve months, driven by NVIDIA's ecosystem push.

  Chase says comparable US open-source models will absolutely arrive within twelve months. NVIDIA is working hard with the open-model ecosystem and cares about it a lot. Reflection's recent model launch was still a bit behind some Chinese models on some benchmarks, but there is a lot of attention in this area, so he expects the gap to close.

  “I think in the next twelve months, we'll absolutely have comparable US open source models. I think you look at everything that NVIDIA is doing to kind of, like, work with the ecosystem.”at 44:38

• Harrison Chase · LangChain

  Open models mostly handle narrow tasks like routing and summarization, not core agentic loops.

  Chase sees heavy use of open-source and post-trained models, more than he expected a year or so ago. He still thinks they are mainly used for simpler tasks such as routing, guardrails, extraction and summarization. They are not really driving the core agentic loops of the main agents. He calls the debate divisive. Frontier models will keep getting better and cheaper, but for a high-volume use case with a few simple components, post-training models for those components can be a very good return on investment.

  “But I still think they're primarily for these simpler tasks like routing or guardrails or extraction or summarization. I don't think they're really driving the core agentic loops for a lot of these main agents.”at 45:28

AI pricing & unit economics

• Harrison Chase · LangChain

  Inference demand is far bigger than expected; LangChain's token usage keeps rising while coding-tool spend share falls.

  LangChain spends roughly 30% of its engineering salary on coding tools. That share fell over the last two months while the number of tokens LangChain uses rose. Chase says he underestimated demand for inference, that the market is very large, and that demand may not ease soon.

  “We spend probably roughly 30% of our engineering salary on coding tools. It's actually gone down the last two months, while tokens... The number of tokens that we've used has actually gone up. I think I really underestimated the demand for inference.”at 0:00

• Harrison Chase · LangChain

  Per-unit model costs are falling, but agent token volume is pushing total AI spend sharply higher.

  Chase says frontier models are getting cheaper over time, with price-per-performance much better than a year ago. Demand is rising faster, though. Multi-agent systems burn far more tokens, so total costs are rising, and coding costs have skyrocketed. At LangChain, coding-tool spend was about 30% of engineering salary, but it fell over the last two months even as token use rose. They got there through visibility into who spends what, budget caps, and control of the harness, including prompt caching and changing tools mid-flight without busting the cache. He expects org design to shift toward small teams. Agent spend per team could be about three times higher, but scarce talent costs more, so total spend may stay flat.

  “[the frontier models are relatively] getting cheaper over time... [It's] price per performance fantastic compared to a lot of things that existed a year ago. Two, there's just way more demand. ...so the costs are going up. We see this in coding primarily. Coding costs have skyrocketed”at 35:52

Frontier AI labs & model race

• Harrison Chase · LangChain

  Model labs tune their harnesses for coding, leaving many non-coding use cases that need builders' own control.

  Model labs train their harnesses to excel at tasks inside their training distribution, which is mostly coding. Coding has many use cases, but many others, such as sales, research and legal work, sit outside that distribution. For those, builders need control over the harness and context. Chase presents this as a reason to own the harness.

  “sometimes these model labs are training harnesses to be really good at the things that are in distribution for them, which oftentimes is coding, and coding is great, and there's a ton of use cases around coding, but there are a ton of use cases that aren't coding.”at 10:34

• Harrison Chase · LangChain

  Model gains will flatline on tasks like coding and summarization, while meta-learning may improve faster.

  For many tasks models are already good enough, including coding, summarization and routing, so improvement there will flatten. Harder tasks remain, and meta-learning, meaning reflection on prompts and agent architecture, is something models still do poorly. Chase thinks labs will work on it because it matters for recursive self-improvement, and that progress there could be faster than before. He stresses that without being inside a lab, a firm view is hard to form.

  “I would say I think there are a lot of tasks for which the model improvements will flatline in the sense that, like, it's already good enough.”at 28:45

Artificial Intelligence

• Harrison Chase · LangChain

  Continual learning adds an outer loop over agent traces; the core agent loop stays largely the same.

  Chase says continual learning changes the systems around the agent more than the agent itself. The core loop still runs an LLM calling tools in an environment. Continual learning adds an outer loop that takes traces, extracts learnings, fixes something, measures whether it improved and repeats. The update can target context, harness code or model weights, and the outer loop looks roughly the same for each.

  “I don't think so much changes how agents work, but I think it changes the systems that you wanna have running around the agent.”at 24:37

Consumer AI products

• Harrison Chase · LangChain

  Consumer apps will have both horizontal and vertical agents, with horizontal agents calling vertical ones.

  Chase thinks most consumer apps should have a vertical agent. Such apps hold knowledge about the user, such as music taste in Spotify or training data in Strava, and can learn how a person interacts with them better than a general agent could. Many of these apps also have many APIs. A natural-language agent wrapping those APIs gives a better experience than raw APIs, and a horizontal agent can then interact with it.

  “I think my take on the consumer space is that you'll both. You'll have both horizontal and vertical agents, and the horizontal agents will interact with the vertical agents.”at 32:30

AI infrastructure & data centers

• Harrison Chase · LangChain

  Inference and coding-agent demand are far bigger than expected, so inference providers have strong demand.

  Looking back from LangChain's start, Chase says he underestimated two things: demand for inference and demand for coding agents. He thinks both markets are tremendously bigger than he thought three or four years ago. He cites a Benchmark partner's view that the inference market is so big it works for everyone. He sees no sign that this demand will fall soon, so he argues Fireworks, AWS and Crusoe can all succeed and asks why anyone would not be investing now.

  “there's two things that I think I really underestimated from the start of [LangChain] now. One is the demand for inference, and two is the demand for, like, coding agents. I think both of those markets are just tremendously bigger than I thought three, four years ago.”at 46:17

Venture capital & startups

• Harrison Chase · LangChain

  AI startup valuations are probably above rational levels, but breakout teams justify paying up.

  Chase says some personal-assistant companies are probably at higher valuations than is rational. Some funding rounds, though, go to breakout companies with strong technical teams that have proven they can execute quickly, and in those cases he thinks investors should not focus on price. He also says being first matters more than ever in AI, citing LangChain's brand advantage. He qualifies this: being first does not guarantee success, as AutoGPT's early lead shows.

  “It would probably be at, like, maybe higher valuations than is rational. But in terms of some of these funding rounds that are getting done, where there's a breakout company with a really strong technical roster... I think you just don't look at the price and you just invest.”at 51:08