Tag: Langchain

  • Harness Engineering, Orchestration, and Compound Agents: The Enterprise AI Vocabulary You  Need To Know

    Harness Engineering, Orchestration, and Compound Agents: The Enterprise AI Vocabulary You Need To Know

    Most enterprise AI practitioners are running multiple AI models. The question was never how many models or whether they were open models.  The question that’s harder to answer — and the one that determines whether your AI investments compound or fragment — is which system those models are part of, and what you are actually evaluating when a vendor puts a platform in front of you.

    At NVIDIA‘s GTC, Jensen Huang convened a session with the CEOs of Cursor, Perplexity, LangChain, Reflection AI, Thinking Machines Lab, and several others building at the edges of the AI ecosystem. What they described isn’t a debate about models. It’s a map of how AI systems are being assembled — orchestration layers, agent harnesses, multi-component architectures, and specialized vertical systems that combine foundation models with proprietary data and domain logic. Vendors are no longer selling model access. They’re selling systems. Understanding the components of those systems is what makes the difference between a well-matched procurement decision and an expensive one.

    Here’s the updated map.

    The Model Is a Component. The System Is the Product.

    Jensen Huang opened the session with a distinction worth internalizing: a model can be a technology or not a product. He said that ChatGPT is a product. The model underneath it is a technology that someone assembled into that product.

    This reframe is useful for enterprise buyers evaluating vendor offerings. When a vendor presents an AI platform — whether that’s Perplexity Computer, an agentic desktop assistant designed to function as an autonomous coworker, or a vertical industry system built on top of foundation models — you’re not evaluating the model. You’re evaluating everything assembled around it: how the system connects to your data, what tools it can invoke, how it manages memory and context across tasks, what guardrails constrain its actions, and how it handles the handoff between automated steps and human oversight.

    The model is the engine. The system is the car. And you’re buying the car.

    Michael Truell, CEO of Cursor, added a structural observation that clarifies why vendor evaluation has gotten more complex. For years, he said, there were two groups. There were foundation-model companies that built large general-purpose models and sold API access.  Or application companies that built models into their products or on top of them. Truell argues a third category is now well established — companies that combine the best foundation models available through APIs with their own purpose-built models and proprietary domain knowledge, assembled into a specialized vertical system. It’s not a foundation model or application layer. It’s both, plus the integration work that makes them useful together.

    When you evaluate a vendor in this third category, the question isn’t just which model it runs on. It’s whether the vertical specialization, the data architecture, and the system design match the use cases you’re actually trying to solve.

    What Harness Engineering Is, and Why It’s Now a Discipline

    Harrison Chase, CEO of LangChain, introduced a term that’s worth adding to your vocabulary: harness engineering.

    Harness engineering is the discipline of building reliable, structured environments around AI agents. The agent itself — the model running in a loop, calling tools, taking action — is only part of the system. The harness is everything surrounding it: the workflows, the tool interfaces, the validation loops, the context management strategies, and the memory architecture. It’s what makes the difference between an agent that works in a demo and one that operates reliably in production.

    NOTE: This is one definition of the term. The market is still debating this term, and I will write another specific piece on harness engineering.

    Chase made the point directly: even the closed model labs practice harness engineering constantly. He gave the example of Anthropic’s Claude and Claude Code. A model may be exceptional, but the harness around it — how it connects to file systems, how it manages long tasks, what guardrails constrain its actions — is equally responsible for the results. When enterprise teams underinvest in the harness and focus only on the model, they end up running expensive experiments that don’t scale.

    For technology leaders, this has a direct implication. Technical evaluations that benchmark models against each other without assessing the harness — the orchestration framework, the tooling, the integration architecture — are incomplete. The model is one variable. The harness is where most of the enterprise-specific work lives, and where most of the deployment risk concentrates.

    Multi-Component (Multi-Agent) AI Systems: What the Term Actually Means, and What to Ask

    You will hear vendors use terms like “compound agents,” “multi-agent systems,” and “agentic platforms” to describe their offerings. These terms are not interchangeable, and the market has not settled on consistent definitions. That ambiguity is worth understanding before you evaluate vendor claims.

    The most rigorous current framing comes from UC Berkeley’s Sky Lab, which defines a compound AI system as one that combines multiple AI components — models, retrievers, tools, databases, external APIs — to complete a task, rather than relying on a single model call. The system’s behavior emerges from how those components interact, not from any one of them individually. This framing has practical logic behind it: composing specialized components often outperforms a single frontier model on both capability and cost, particularly for complex multi-step tasks.

    Where it gets murkier is in how vendors apply the label. “Compound agent” in active use can mean three different things: a compound AI system that takes actions rather than just generating output; multiple discrete agents collaborating, each with its own reasoning loop; or an orchestrator-and-subagent architecture in which a controlling agent decomposes tasks and delegates them to specialized agents. These have meaningfully different architecture, cost, and governance implications. A vendor calling their product a “compound agent platform” may mean any of the three. The label alone tells you nothing about the actual design.

    The governance implication is the one most enterprise buyers miss. Multi-component systems diffuse accountability. When a consequential decision emerges from the interaction of a retriever, a reasoning model, and a tool execution layer, which component is responsible for the output? Traditional audit trails track the system’s final action. They often don’t track which component drove the decision that led to it. Before deploying any multi-component AI system in a business-critical workflow, buyers should require component-level architecture disclosure and confirm that audit logging covers component interactions, not just system outputs.

    The practical posture: treat “compound,” “agentic,” and “multi-agent” as marketing descriptors until a vendor discloses their specific component architecture. Ask what components the system includes, how they interact, how failures in individual components surface, and where in the stack governance and audit trails are enforced. Those answers will tell you far more than the product label.

    Orchestration Is the New Core Infrastructure

    Arvind Srinivas, CEO of Perplexity, described what his company calls Perplexity Computer: a multi-model, multi-cloud orchestration system where the models themselves become tools, and the orchestration layer determines which tool to apply to which task. The goal is that an enterprise can delegate a goal without specifying which model handles each step — the system manages that routing. Here is another take he gave on the topic in February.

    The analogy he offered is useful for enterprise framing: sub-agents are musicians, models are instruments, and the orchestration system is what produces the symphony. The quality of the output depends on all three, not just the instruments.

    The practical translation: the orchestration layer is where the strategic architecture decisions live. It determines which models get used for which tasks, how context is managed across long-running workflows, where governance and guardrails are enforced, and how exposed you are to vendor dependency. If your orchestration layer is tightly coupled to a single model provider, every future model decision becomes a migration project. If it’s designed to be model-agnostic, you preserve optionality as the model market continues to evolve rapidly.

    Mira Murati, CEO of Thinking Machines Lab, added a related dimension. Her firm has focused on making post-training — the layer of model development that adapts a foundation model to specific domains and tasks — accessible to enterprises and researchers. Most enterprise AI value doesn’t come from a raw pre-trained model. It comes from a model tailored to your domain, data, and task requirements. Accessible post-training means more organizations can build the specialized models that multi-component architectures require, without depending entirely on what general-purpose foundation models provide out of the box.

    What to Do Before Your Next AI System Decision

    The architecture described in this conversation isn’t on a roadmap. Cursor, LangChain, Perplexity, Mistral, and Thinking Machines Lab are all in production with enterprise customers today. The market has already moved.

    Three things worth doing before your next AI procurement decision:

    Evaluate the system, not just the model. Ask vendors to show you the harness — the orchestration framework, the tool interfaces, the memory architecture, the governance layer — not just model benchmark scores. You’re buying the system. Evaluate it as one.

    Require component-level architecture disclosure. When a vendor describes their offering as a compound agent, an agentic platform, or a multi-agent system, ask them to specify the components the system includes and how they interact. The label is not a specification. The architecture is.

    Treat orchestration as strategic infrastructure. The orchestration layer is where vendor dependencies are created or avoided, where governance is enforced or bypassed, and where the long-term flexibility of your AI architecture resides. It deserves the same scrutiny as your data infrastructure decisions. Evaluate orchestration before you’re locked into a system that makes changing it painful.

    The model was never the moat, but perhaps the system is. Knowing how to evaluate the totality of an AI system is what separates a well-matched AI investment from an expensive lesson.