Category: AI Models

  • Harness Engineering, Orchestration, and Compound Agents: The Enterprise AI Vocabulary You Need To Know

    Harness Engineering, Orchestration, and Compound Agents: The Enterprise AI Vocabulary You Need To Know

    Most enterprise AI practitioners are running multiple AI models. The question was never how many models or whether they were open models.  The question that’s harder to answer — and the one that determines whether your AI investments compound or fragment — is which system those models are part of, and what you are actually evaluating when a vendor puts a platform in front of you.

    At NVIDIA‘s GTC, Jensen Huang convened a session with the CEOs of Cursor, Perplexity, LangChain, Reflection AI, Thinking Machines Lab, and several others building at the edges of the AI ecosystem. What they described isn’t a debate about models. It’s a map of how AI systems are being assembled — orchestration layers, agent harnesses, multi-component architectures, and specialized vertical systems that combine foundation models with proprietary data and domain logic. Vendors are no longer selling model access. They’re selling systems. Understanding the components of those systems is what makes the difference between a well-matched procurement decision and an expensive one.

    Here’s the updated map.

    The Model Is a Component. The System Is the Product.

    Jensen Huang opened the session with a distinction worth internalizing: a model can be a technology or not a product. He said that ChatGPT is a product. The model underneath it is a technology that someone assembled into that product.

    This reframe is useful for enterprise buyers evaluating vendor offerings. When a vendor presents an AI platform — whether that’s Perplexity Computer, an agentic desktop assistant designed to function as an autonomous coworker, or a vertical industry system built on top of foundation models — you’re not evaluating the model. You’re evaluating everything assembled around it: how the system connects to your data, what tools it can invoke, how it manages memory and context across tasks, what guardrails constrain its actions, and how it handles the handoff between automated steps and human oversight.

    The model is the engine. The system is the car. And you’re buying the car.

    Michael Truell, CEO of Cursor, added a structural observation that clarifies why vendor evaluation has gotten more complex. For years, he said, there were two groups. There were foundation-model companies that built large general-purpose models and sold API access.  Or application companies that built models into their products or on top of them. Truell argues a third category is now well established — companies that combine the best foundation models available through APIs with their own purpose-built models and proprietary domain knowledge, assembled into a specialized vertical system. It’s not a foundation model or application layer. It’s both, plus the integration work that makes them useful together.

    When you evaluate a vendor in this third category, the question isn’t just which model it runs on. It’s whether the vertical specialization, the data architecture, and the system design match the use cases you’re actually trying to solve.

    What Harness Engineering Is, and Why It’s Now a Discipline

    Harrison Chase, CEO of LangChain, introduced a term that’s worth adding to your vocabulary: harness engineering.

    Harness engineering is the discipline of building reliable, structured environments around AI agents. The agent itself — the model running in a loop, calling tools, taking action — is only part of the system. The harness is everything surrounding it: the workflows, the tool interfaces, the validation loops, the context management strategies, and the memory architecture. It’s what makes the difference between an agent that works in a demo and one that operates reliably in production.

    NOTE: This is one definition of the term. The market is still debating this term, and I will write another specific piece on harness engineering.

    Chase made the point directly: even the closed model labs practice harness engineering constantly. He gave the example of Anthropic’s Claude and Claude Code. A model may be exceptional, but the harness around it — how it connects to file systems, how it manages long tasks, what guardrails constrain its actions — is equally responsible for the results. When enterprise teams underinvest in the harness and focus only on the model, they end up running expensive experiments that don’t scale.

    For technology leaders, this has a direct implication. Technical evaluations that benchmark models against each other without assessing the harness — the orchestration framework, the tooling, the integration architecture — are incomplete. The model is one variable. The harness is where most of the enterprise-specific work lives, and where most of the deployment risk concentrates.

    Multi-Component (Multi-Agent) AI Systems: What the Term Actually Means, and What to Ask

    You will hear vendors use terms like “compound agents,” “multi-agent systems,” and “agentic platforms” to describe their offerings. These terms are not interchangeable, and the market has not settled on consistent definitions. That ambiguity is worth understanding before you evaluate vendor claims.

    The most rigorous current framing comes from UC Berkeley’s Sky Lab, which defines a compound AI system as one that combines multiple AI components — models, retrievers, tools, databases, external APIs — to complete a task, rather than relying on a single model call. The system’s behavior emerges from how those components interact, not from any one of them individually. This framing has practical logic behind it: composing specialized components often outperforms a single frontier model on both capability and cost, particularly for complex multi-step tasks.

    Where it gets murkier is in how vendors apply the label. “Compound agent” in active use can mean three different things: a compound AI system that takes actions rather than just generating output; multiple discrete agents collaborating, each with its own reasoning loop; or an orchestrator-and-subagent architecture in which a controlling agent decomposes tasks and delegates them to specialized agents. These have meaningfully different architecture, cost, and governance implications. A vendor calling their product a “compound agent platform” may mean any of the three. The label alone tells you nothing about the actual design.

    The governance implication is the one most enterprise buyers miss. Multi-component systems diffuse accountability. When a consequential decision emerges from the interaction of a retriever, a reasoning model, and a tool execution layer, which component is responsible for the output? Traditional audit trails track the system’s final action. They often don’t track which component drove the decision that led to it. Before deploying any multi-component AI system in a business-critical workflow, buyers should require component-level architecture disclosure and confirm that audit logging covers component interactions, not just system outputs.

    The practical posture: treat “compound,” “agentic,” and “multi-agent” as marketing descriptors until a vendor discloses their specific component architecture. Ask what components the system includes, how they interact, how failures in individual components surface, and where in the stack governance and audit trails are enforced. Those answers will tell you far more than the product label.

    Orchestration Is the New Core Infrastructure

    Arvind Srinivas, CEO of Perplexity, described what his company calls Perplexity Computer: a multi-model, multi-cloud orchestration system where the models themselves become tools, and the orchestration layer determines which tool to apply to which task. The goal is that an enterprise can delegate a goal without specifying which model handles each step — the system manages that routing. Here is another take he gave on the topic in February.

    The analogy he offered is useful for enterprise framing: sub-agents are musicians, models are instruments, and the orchestration system is what produces the symphony. The quality of the output depends on all three, not just the instruments.

    The practical translation: the orchestration layer is where the strategic architecture decisions live. It determines which models get used for which tasks, how context is managed across long-running workflows, where governance and guardrails are enforced, and how exposed you are to vendor dependency. If your orchestration layer is tightly coupled to a single model provider, every future model decision becomes a migration project. If it’s designed to be model-agnostic, you preserve optionality as the model market continues to evolve rapidly.

    Mira Murati, CEO of Thinking Machines Lab, added a related dimension. Her firm has focused on making post-training — the layer of model development that adapts a foundation model to specific domains and tasks — accessible to enterprises and researchers. Most enterprise AI value doesn’t come from a raw pre-trained model. It comes from a model tailored to your domain, data, and task requirements. Accessible post-training means more organizations can build the specialized models that multi-component architectures require, without depending entirely on what general-purpose foundation models provide out of the box.

    What to Do Before Your Next AI System Decision

    The architecture described in this conversation isn’t on a roadmap. Cursor, LangChain, Perplexity, Mistral, and Thinking Machines Lab are all in production with enterprise customers today. The market has already moved.

    Three things worth doing before your next AI procurement decision:

    Evaluate the system, not just the model. Ask vendors to show you the harness — the orchestration framework, the tool interfaces, the memory architecture, the governance layer — not just model benchmark scores. You’re buying the system. Evaluate it as one.

    Require component-level architecture disclosure. When a vendor describes their offering as a compound agent, an agentic platform, or a multi-agent system, ask them to specify the components the system includes and how they interact. The label is not a specification. The architecture is.

    Treat orchestration as strategic infrastructure. The orchestration layer is where vendor dependencies are created or avoided, where governance is enforced or bypassed, and where the long-term flexibility of your AI architecture resides. It deserves the same scrutiny as your data infrastructure decisions. Evaluate orchestration before you’re locked into a system that makes changing it painful.

    The model was never the moat, but perhaps the system is. Knowing how to evaluate the totality of an AI system is what separates a well-matched AI investment from an expensive lesson.

  • Managing a Fleet of Thousands of Agents Is the Real Problem. Google Cloud Showed Its Answer.

    Managing a Fleet of Thousands of Agents Is the Real Problem. Google Cloud Showed Its Answer.

    Enterprises spent most of 2025 figuring out what AI agents are and which tools could build them. The conversation in 2026 is different. The question now is how to govern, monitor, and scale fleets of agents across the enterprise — without losing control of what they’re doing or why. That shift was visible at Google Cloud Next. One of the headline announcements — the Gemini Enterprise Agent Platform — is more than a product consolidation. It reflects where the enterprise AI market is actually headed: away from point tools and toward platforms that manage agents at scale, with governance and observability built in rather than bolted on afterward.Here’s what was announced, and more importantly, what it means for enterprise buyers trying to move from individual pilots to production-grade agentic systems.

    What Google Actually Announced

    Google combined both Vertex AI and Agentspace into a single unified offering called the Gemini Enterprise Agent Platform. Vertex was its managed AI development platform, while Agentspace was its enterprise-focused platform for deploying and managing agents across organizational data and applications such as Jira, Salesforce, and Google Workspace. Neither Vertex AI nor Agentspace will exist as standalone products. The new platform combines what each did separately, such as  Vertex AI’s model building and generative AI development capabilities, and Agentspace’s agent deployment, workflow automation, and enterprise application integration. The platform adds new or improved capabilities for orchestration, governance, security, and observability. That last part matters most. On the model side, enterprise buyers now have access to a model garden with more than 200 options. The model garden includes Google’s own Gemini 3.1 Pro, Gemini 3.1 Flash Image, and Lyria 3, alongside third-party models such as Anthropic’s Claude Opus, Sonnet, and Haiku. The breadth is notable. A single platform that spans first-party and third-party models gives enterprises flexibility to match the model to the task — rather than locking into one provider’s output quality for every workload.On the development side, the platform spans from low-code tooling via Agent Studio to a more capable Agent Development Kit for engineering teams. A new graph-based framework organizes agents into networks of sub-agents, allowing enterprises to define reliable logic for how agents collaborate on complex tasks. An Agent Garden provides access to a curated set of agent templates that cover use cases such as code modernization, financial analysis, economic research, and invoice processing. These templates serve as building blocks for multi-agent systems. The operational layer is where the platform makes its clearest argument for enterprise buyers. Agent Runtime delivers sub-second cold starts and supports long-running agents that maintain state for days, backed by a Memory Bank for persistent context.  This allows agents to run more complex tasks. An Agent Gateway provides unified connectivity between agents and tools across environments while enforcing a consistent security policy. Agent Sandbox provides a hardened environment for executing model-generated code and browser-based automation tasks without exposing host systems. And critically, the platform includes Agent Identity and Agent Registry. Every agent — whether built internally or sourced from a third-party partner — carries a trackable identity and operates within defined guardrails. Model Armor protections guard against prompt injection and data leakage. Testing and observability round out the picture. Agent Simulation, Agent Evaluation, and Agent Observability provide execution traces and real-time insight into agent reasoning. Agent Optimizer goes further, automatically clustering real-world failures and suggesting refined system instructions rather than requiring teams to dig through logs manually.

    Why the Consolidation Matters

    Google also announced updates to its Cloud Data Cloud offering, which improves how agents access and interpret enterprise data. Better data grounding means agents work with accurate, current, enterprise-specific information rather than relying on general model knowledge, which hallucinates at rates that make it unsuitable for business-critical decisions. The Gemini Enterprise Agent Platform competes directly with Amazon’s Bedrock AgentCore and Microsoft’s Azure AI Foundry. All three hyperscalers are converging on the same recognition: enterprise buyers don’t need more ways to build a single agent. They need platforms that manage how hundreds or thousands of agents behave, interact, and scale — without requiring a dedicated engineering team to babysit each one. That’s the real shift in 2026. The challenge is no longer proof-of-concept. It’s production life cycle management with security and governance.For enterprises that had already deployed Agentspace, the consolidation raises a practical question: what happens to what you built? Google’s framing suggests continuity rather than migration — existing Agentspace capabilities carry forward into the new platform rather than requiring a rebuild. But enterprises currently running Agentspace workflows should verify specifically how their integrations, agent configurations, and user access models map to the consolidated platform before assuming a seamless transition. Consolidations that look clean on a slide often surface friction in production environments. Ask Google directly what the migration path looks like and what, if anything, requires rework.

    What’s Hard About It

    The technology being ready and your organization being ready are two different things. A few realities worth holding onto as you evaluate this platform. Model flexibility is only valuable when you define what you need for your workloads. Two hundred models in a garden is a resource if you have a framework for selecting among them. Without a clear use-case taxonomy — which tasks require higher accuracy versus lower latency, which workloads require on-premises data handling versus cloud inference — the abundance becomes a selection problem rather than a solution. Agent Identity and Agent Registry are necessary, not sufficient. Registering agents and assigning identities is a prerequisite for governance, not the governance itself. Enterprises still need to define what each agent is authorized to do, under what conditions it escalates to a human, and how they will audit agent behavior over time. The platform provides the infrastructure for those decisions. The decisions still belong to the enterprise. Observability tooling requires someone to act on what it surfaces. Agent Optimizer, which automates the suggestion of improved instructions, is genuinely useful. But organizations still need the operational capacity to evaluate those suggestions, test changes, and maintain accountability for agent behavior in production. Automation reduces the burden; it doesn’t eliminate the need for human judgment. The partner ecosystem is announced, not fully proven. Google named a broad set of integrations and ecosystem partners. As with every hyperscaler launch, the gap between announced partnerships and certified, production-ready integrations takes time to close. Before building production workflows on partner integrations, confirm what is shipping now versus what is on the roadmap.

    What Enterprise Buyers Should Do

    If you are evaluating the Gemini Enterprise Agent Platform — or any hyperscaler agent platform — three questions will tell you more than the demo. First, ask what the platform does when an agent fails. Not how it handles errors gracefully, but what happens when an agent takes a wrong action at scale, across a fleet of similar agents running the same logic. Failure modes at scale differ from those in a pilot. Platforms that offer real-time observability and automated clustering of failure patterns are meaningfully ahead of those that don’t. Second, ask how agent identities integrate with your existing identity and access management infrastructure. If agents need to be registered and governed separately from human users, the platform adds operational overhead. If agent identities extend naturally from your current IAM framework, adoption is faster, and governance is less fragile. Third, ask how data grounding works within your specific data architecture. The value of grounding AI agents in enterprise data depends entirely on the quality and accessibility of that data. If your data is scattered across disconnected systems with inconsistent formats, the grounding layer will struggle regardless of how capable the platform is. Google’s Cloud Data improvements are in the right direction. But no platform substitutes for data readiness on the enterprise side.

    The Broader Signal

    What Google announced at Cloud Next reflects something the enterprise AI market has needed for a while. Enterprise buyers need platforms from credible vendors that treat governance, security, and observability as core features rather than afterthoughts. We spent 2024 and 2025 building agents. The organizations that will succeed in 2026 are the ones that build the management layer around those agents — the identity frameworks, the observability infrastructure, the governance policies that define what agents are allowed to do and what requires human review. The Gemini Enterprise Agent Platform is a serious attempt to deliver that layer. It is not the only attempt. Amazon and Microsoft are building toward the same destination. But the consolidation of Vertex AI and Agentspace into a unified platform with built-in governance and observability signals that Google understands where the real enterprise challenge lies. That’s the right problem to be solving. Whether this platform solves it for your specific environment depends on how well it fits your data architecture, your existing security stack, and your organizational capacity to govern agents in production. 

    Subscribe to my AI with Maribel Lopez podcast on your channel of choice at https://www.buzzsprout.com/194744.Lopez Research is a market research and strategy consulting firm specializing in enterprise AI, AI infrastructure, agentic systems, AI governance, and AI-driven customer experience. Learn more at www.lopezresearch.com.

    Lorem ipsum dolor sit amet, consectetur adipiscing elit. Ut elit tellus, luctus nec ullamcorper mattis, pulvinar dapibus leo

  • Scaling AI with Proven Strategies and Frameworks

    Scaling AI with Proven Strategies and Frameworks

    Picking a use case, proving value, and expanding is the standard advice for getting started with AI. For the early stages of AI deployment, this advice is still sound. But at NVIDIA GTC, Cameron Davies, Chief Data Officer of Yum Brands, made the case that scaling AI in a large enterprise requires companies to think differently—and he had the results to back it up.

    Yum Brands is one of the world’s largest restaurant companies with 63,000 locations, processing more than 100 million transactions per day, across 155 countries through 1,500 franchisees. Davies didn’t come to GTC to talk about a pilot. He came to talk about what happens after the pilot, when the real complexity begins.

    His session, “Scaling AI Agents Globally Across Brands, Use Cases, and Restaurants,” wasn’t a product pitch. It was a framework. And it resets the conversation about what it actually takes to put AI into production at scale.

    Stop Thinking in Use Cases. Start Thinking in Skills.

    Over the past two years, enterprises have been plagued by failed AI proofs of concept. The firms that demonstrated returns from AI started with a fairly straightforward approach: identify a use case, build a proof of concept, demonstrate ROI, and expand. Davies challenged that model directly, not because it’s wrong in principle, but because it doesn’t scale.

    “Doing AI in a lab, it’s easy. We’ve all done it before. In fact, the models aren’t the problem anymore… Getting it to work in the lab, getting it to work with a phone call, that wasn’t the hard part. Getting it to scale in a messy, imprecise world, that’s hard.”

    At Yum’s scale, a use-case-first approach produces monolithic agents that are good at one thing in one context and fall apart everywhere else. The company operates over 500 different point-of-sale systems globally. Its menus differ not just across brands but across regions within the same brand. An AI that works perfectly at a Taco Bell in California may not transfer cleanly to a KFC in the UK or a Pizza Hut in India.

    Davies reframed the question. Instead of asking “what use case should we build?”, his team asks “what skills do we need, and how do we make those skills reusable?”

    He described it using a sports analogy: “I could ask you to build me a baseball pitching robot… That’s very different from me saying to my team, I need a throwing machine. This machine doesn’t care what’s in its hand, and it doesn’t care what the target is. It’s just really good at throwing things.”

    The practical implication: Yum built customer-facing agent skills and team productivity agent skills that can be deployed across brands, markets, and use cases — rather than building discrete agents for each problem. The same voice-ordering capability that operates a drive-through in one country can surface in a customer service context in another, because the underlying skill is real-time voice processing, transcription, and translation.

    Scaling AI requires a meaningful shift for large enterprises. It requires more architectural thinking upfront and more coordination between IT and business teams before a single POC is built. But the payoff is a reusable capability layer rather than a growing inventory of one-off AI deployments that each require their own maintenance, integration, and governance.

    You Can’t Scale AI Without a Governance Strategy.

    Davies was unambiguous on this point, and it is worth quoting him directly.

    “If you’re going to do AI, people often ask me, what’s the first thing you do? … It’s governance, because there is no intelligence without governance.”

    He used a vivid example from a colleague, Manoj Saxena, chair of the Responsible AI Institute: “You are all excited about these (AI) agents and what they’re going to do, but you don’t have the right governance in place, so what you’re doing is you’re all building these little Chuckies (from the movie)… in one hand’s a knife and the other hand’s a credit card, and you’re sending him loose into the system and he’s stabbing and swiping, and you don’t know what he’s doing, because you have no agent registries. You have no control.”

    At 100 million transactions per day, the risk profile of an ungoverned AI agent is not theoretical. A bad decision made at machine speed — the wrong menu item, a mishandled order, a brand-damaging interaction — can propagate across thousands of restaurants before anyone catches it. Davies is explicit that governance is not a post-deployment concern. It is the foundation.

    For most enterprise organizations, this is the piece that gets deprioritized. Governance feels like overhead when the pressure is to demonstrate AI value quickly. Davies’ experience suggests the opposite: organizations that skip governance in pursuit of speed end up constrained later, when the complexity of managing ungoverned agents becomes a bigger obstacle than moving slowly would have been.

    The practical starting point is an agent registry that tracks which agents are running, which tools they have access to, what decisions they are authorized to make, and how their outputs are monitored. That discipline doesn’t require a mature AI platform. It requires a decision to treat governance as infrastructure rather than an afterthought.

    The Practical Caveat: This Requires Knowing What You’re Trying to Accomplish

    Davies’ framework works because Yum entered this process with a clear understanding of what they were trying to do and what outcomes they needed to achieve. They have a dedicated data science team. They worked hard to create clean data, expand data sources, and define a platform strategy, and were supported by an executive mandate.

    That context matters. Smaller organizations, or those earlier in their AI journey, should not interpret “don’t think in use cases” as “don’t focus on defining a specific business value.” The opposite is true. Yum’s skill-based approach requires a sophisticated understanding of what capabilities the business needs, how those capabilities connect to customer or operational outcomes, and how the underlying architecture will support reuse.

    Davies described decomposing something as simple as a taco order into a set of discrete tool calls — finding a product, applying a modification, adding it to the cart, and submitting the order. Each of those is a separate decision point, a separate skill. Getting that decomposition right required deep knowledge of Yum’s systems, menus, and the ways customers actually interact with them.

    If you are selecting a specific use case to prove value, that is still sound advice — especially if you are still building organizational confidence in AI or don’t yet have the infrastructure to support a platform approach. The piece of Davies’ advice that applies universally is this: whatever you build, know the outcome you are trying to achieve before you start. The goal and the measurement need to come before the tool selection.

    Davies put it succinctly in his closing guidance: “Build in your measurement from day one. Build it into your systems from day one. Continually measure it and continually talk about the measurement.”

    How Yum Solved the Scaling Problem

    None of this is easy to replicate. Yum has resources, a mature data organization, and an executive mandate that made this approach possible. But the framework Davies laid out is worth capturing — not to copy it, but to understand the decisions behind it.

    Start with governance, not a pilot. Define what agents are allowed to do, how they are registered, and how their behavior is monitored before deploying anything at scale. Governance is infrastructure.

    Build an AI platform, not a collection of POCs. Yum invested in its proprietary platform — Byte — as the connective tissue for its AI work. Davies was direct: “Platforms, not pilots.” The platform enables skills to be reused. Without it, every deployment starts from scratch.

    Design for reusable skills, not one-time use cases. Break capabilities into discrete, transferable components. Each skill should be able to operate in multiple contexts without modification. This requires decomposing workflows into specific decision points and building and training against them rather than against an end-to-end scenario.

    Use synthetic data to address data gaps. Yum worked with NVIDIA’s Nemo tools to generate over 20,000 training records from limited production data. This allowed them to train small, domain-specific models on the kinds of decisions that matter in their environment — without waiting for years of production data to accumulate.

    Measure continuously and own your data. Davies was emphatic on two points that are easy to underestimate. First, always measure, always share measurements, and never stop. Second, do not give up your data or your orchestration layer to a vendor partner. “No matter who you partner with, don’t give them up. That’s your intellectual property. That’s what matters most.”

    The results Yum reported: a three-times improvement in tool call accuracy over a base model after fine-tuning, 100% function match for complex multi-step orders, and a 20-times reduction in inference cost compared to general-purpose closed models. Those numbers reflect what happens when the architecture, the training approach, and the measurement discipline are all aligned.

    Scaling AI is not just a technology problem. It is a business strategy, architecture, and governance problem that technology enables. The lesson for other organizations is not necessarily to copy Yum’s specific approach — it is to ask the same questions Davies asked before assuming a POC is a strategy.

    Define what your business outcome looks like. Start with governance. Build a platform. Design for reuse. Measure everything.

    A version of this article was originally posted in my AI Decoded with Maribel Lopez newsletter. Subscribe to my LinkedIn Newsletter here.

    Subscribe to my podcast here.

     

  • From Chatbots To Digital Workers: Why NiCE’s AI Focuses On Execution

    From Chatbots To Digital Workers: Why NiCE’s AI Focuses On Execution

    NiCE is betting that AI agents that complete tasks—not just conversations—will separate winners from pretenders in the enterprise AI race

    The artificial intelligence hype cycle has reached peak saturation, with technology vendors scrambling to slap “AI-powered” labels on everything from refrigerator recipe suggestions to chatbots that rely on simple keyword matching. Yet, for all the breathless marketing rhetoric, most business leaders are still waiting for AI that simplifies operations and improves data analysis.

    NiCE, a customer experience platform provider, is making a calculated bet that the next phase of enterprise AI won’t be about making chatbots sound more human. The next wave of business outcomes will leverage AI agents that can navigate complex business processes from start to finish with minimal human intervention. 

    NiCE’s CEO, Scott Russell said, “Optimizing knowledge is not just critical for AI to truly thrive in your environment, but it’s also the key for transforming service from reactive to proactive, identifying opportunities to solve issues and predict future needs.” NiCE’s recent product launches and strategic moves reveal a move toward enhanced automation and agentic AI.

    From AI Data Access to Intelligent Action

    The first wave of enterprise AI focused primarily on making data more accessible through conversational interfaces—essentially putting a chat layer on top of existing applications and knowledge repositories. 

    While this represented significant progress in democratizing data access, it only scratched the surface of AI’s potential business value. Organizations could ask questions and get answers, but the burden of reasoning through complex decisions and taking action remained entirely on human operators.

    Going forward, technology vendors, such as NiCE, will use AI to deliver solutions that can reason through multifaceted problems and take semi or fully autonomous action. This evolution from conversational AI to agentic AI represents the difference between AI that informs and AI that performs. Agentic AI enhances a company’s ability to analyze context, weigh multiple variables, make informed decisions based on key business performance indicators, and execute actions across interconnected systems.

    Traditional conversational AI helps a customer service representative find relevant information. Still, agentic AI can help a representative evaluate a customer’s complete history more easily, assess risk factors, determine appropriate responses based on business rules, and automatically trigger the necessary workflows to resolve issues end-to-end. “There’s a big difference between AI that talks and AI that gets things done,” explains Barry Cooper, President of NiCE’s CX Division. “While others are building agents that mimic conversations, we’re building agents that fulfill customer needs—end to end.” 

    This distinction becomes crucial when examining NiCE’s CXone Mpower Agents. Traditional AI chatbots had limited access to data, offered scripted responses, and were confined to specific areas of the business, such as front-office or back-office operations. NiCE’s AI agent platform aim to break through these constraints by operating across the entire enterprise ecosystem—from initial customer contact through mid-office approvals to back-end fulfillment systems. Admittedly, Agentic AI is the AI buzzword of 2025, but early, well-scoped use cases show promise. 

    Speeding Up Time to AI Agent Creation

    Technology companies are increasingly working to streamline AI deployment as traditional approaches require extensive technical resources, custom development, and lengthy implementation cycles. NiCE’s model simplifies AI agent creation while maintaining enterprise-grade sophistication through what they call vibe coding, allowing business users to tailor each agent’s personality and communication style without requiring technical expertise.

    While the concept of vibe coding remains ill-defined, and its merits are hotly debated within the enterprise software community, there is a broad consensus around the underlying goal of making AI agents easier to code and deploy. The specific term matters less than the fundamental shift toward empowering business users to create and customize AI functionality without requiring deep technical expertise.

    Breaking Down Data and Function Silos With Strategic Partnerships

    In a rapidly evolving tech landscape, no single vendor can deliver everything an enterprise needs to succeed with AI, cloud, data, and digital transformation. Today, companies are no longer looking for isolated solutions—they need interconnected ecosystems. That’s why strategic partnerships are essential. By working together, enterprise technology vendors can bridge data and function silos, improving workflows and accelerating innovation. Just as importantly, these alliances help enterprises extract greater value from existing technology investments by ensuring that new capabilities work in concert with the tools already in place. Over the past several months, NiCE has expanded its partnership with Amazon Web Services (AWS) and added ServiceNow and Snowflake to the mix.

    NICE and AWS Tackle AI Integration at Enterprise Scale

    At Interactions 2025, NiCE announced an expanded collaboration with AWS, bringing together NiCE’s domain expertise and rich interaction data with AWS’s cloud infrastructure and generative AI services, including Amazon Bedrock, Amazon Q, and the Amazon Nova family of large language models. The partnership addresses some of the most pressing challenges facing enterprise AI deployments: fragmented workflows, disconnected data, and inconsistent global performance. 

    The partnership focuses on three core pillars. First, content-aware automation ensures that AI-generated responses are highly relevant and context-specific. Using the Amazon Q Index, Mpower Agents are equipped with up-to-date business content—from product documentation to policy details and case histories—enabling them to respond accurately and confidently in real time.

    Second, the integration delivers enterprise-wide orchestration by bridging front, middle, and back-office operations. NiCE’s CXone Mpower Orchestrator automates workflows across functional teams, while Amazon Q Business extends this reach into a broader set of enterprise applications—eliminating silos and streamlining complex processes.

    Additionally, global scalability is made possible through AWS’s robust cloud infrastructure. With low-latency performance and high availability across regions, multinational organizations can deploy and scale AI-driven customer service experiences quickly and consistently around the world. NiCE’s partnership strategy also extends beyond AWS to include other critical enterprise platforms, such as ServiceNow and Snowflake. 

    NiCE and ServiceNow Partner to Automate the Full Customer Journey

    NiCE’s latest partnership with ServiceNow aims to eliminate long-standing service gaps by tightly integrating real-time customer engagement with enterprise workflow automation. Announced at ServiceNow’s Knowledge 2025 event, the collaboration integrates NiCE’s customer service platform with ServiceNow’s AI and Customer Service Management (CSM) tools to streamline operations across the entire organization, from the front office to the back.

    The goal: fully automated customer service fulfillment. The combined solution routes inquiries based on sentiment, intent, and service-level agreements (SLAs)—bridging siloed departments to accelerate resolution times and enhance both customer and employee experiences. Role-based AI copilots assist agents and back-office teams with real-time insights and next-best actions, while continuous optimization tools flag issues and launch workflows automatically.

    These relationships provide access to complementary technologies and customer bases, allowing NiCE to integrate with the broader enterprise software ecosystem that companies rely on for operations, data management, and workflow automation.

    NICE and Snowflake Partner to Turn Customer Interaction Data Into Enterprise Intelligence

    NiCE’s strategic collaboration with Snowflake aims to unlock the full value of customer interaction data by making it accessible, secure, and actionable across the enterprise. By integrating Snowflake’s AI Data Cloud with CXone Mpower, NICE can improve data sharing, breaking down silos that have traditionally limited the impact of customer insights. Snowflake serves as the backbone of the CXone Mpower data lake, centralizing interaction data and enriching it with information from other enterprise systems. This unified data foundation allows organizations to automate key processes—from billing to claims handling—while powering AI-driven analytics, dashboards, and decision-making. The result: faster fulfillment, greater accuracy, and a deeper, organization-wide understanding of the customer experience.

    The Strategic Imperative of Brand Evolution in an AI Era

    Apparently, 2025 is the year of the brand refresh. The technology industry has witnessed updates from Five9, Google’s G, Hitachi HPE, and Qualcomm’s introduction of Dragonwing, alongside NiCE’s own transformation. Every brand refresh has its own story to tell, but NiCE’s new logo and marketing campaign represent more than a desire for fresh typography and color schemes.

    The rebrand indicates the company’s strategic desire to expand its AI vision beyond the contact center to encompass its broader portfolio of finance and security solutions. The company describes the rebrand as positioning “NICE to empower brands to deliver AI-powered experiences that are proactive, human-centered and intuitive—whether connecting with customers, protecting communities or combatting financial crime.”

    NiCE’s solution involves partnering with actress Kristen Bell, who serves as the face of the company’s “NiCE World” brand campaign. The initiative positions Bell as the “NiCEst Person in the World,” NiCE said the campaign “builds on NiCE’s reimagined brand, championing a future where AI isn’t just intelligent – it’s connected, intuitive and working behind the scenes to make life better.

    Key Takeaways

    The enterprise AI market remains in flux, with new entrants and existing players continually repositioning themselves. NiCE’s focus on domain expertise, integration depth, strategic partnerships, and automation suggests a company that understands both the technical and implementation requirements necessary for large-scale AI adoption.

    As enterprises increasingly demand AI that delivers results, NiCE’s bet on fulfillment-focused automation may prove prescient. Of course, there’s still the matter of  cost and return on investment. Most companies struggle to understand and plan for the true product and operational costs of AI. Organizations need to work with their technology vendors to deploy well-scoped use case that deliver measurable return on investment, fast. The question isn’t whether AI will transform customer experience—it’s which companies will build AI that completes the transformation rather than just talking about it.

  • Dell AI Factory Expands with 40+ Enhancements for Enterprise AI Deployment

    Dell AI Factory Expands with 40+ Enhancements for Enterprise AI Deployment

    Dell Technologies unveiled a significant expansion of its AI Dell Factory platform at its annual Dell Technologies World conference today, announcing over 40 product enhancements designed to help enterprises deploy artificial intelligence workloads more efficiently across both on-premises environments and cloud systems.

    The Dell AI Factory is not a physical manufacturing facility but a comprehensive framework combining advanced infrastructure, validated solutions, services, and an open ecosystem to help businesses harness the full potential of artificial intelligence across diverse environments—from data centers and cloud to edge locations and AI PCs.

    The company has attracted over 3,000 AI Factory customers since launching the platform last year. In an earlier call with industry analysts, Dell shared research stating that 79% of production AI workloads are running outside of public cloud environments—a trend driven by cost, security, and data governance concerns. During the keynote, Michael Dell provided more color on the value of Dell’s AI factory concept. He said, “The Dell AI factory is up to 60% more cost effective than the public cloud, and recent studies indicate that about three-fourths of AI initiatives are meeting or exceeding expectations. That means organizations are driving ROI and productivity gains from 20% to 40% in some cases. 

    Making AI Easier to Deploy

    Organizations need the freedom to run AI workloads wherever makes the most sense for their business, without sacrificing performance or control. While IT leaders embraced the public cloud for their initial AI services, many organizations are now looking for a more nuanced approach where the company can control over their most critical AI assets while maintaining the flexibility to use cloud resources when appropriate. Over 80 percent of the companies Lopez Research interviewed said they struggled to find the budget and technical talent to deploy AI. These AI deployment challenges have only increased as more AI models and AI infrastructure services have been launched.

    Silicon Diversity and Customer Choice

    A central theme of Dell’s AI Factory message is how Dell makes AI easier to deploy while delivering choice. Dell is offering customers choice through silicon diversity in its designs, but also with ISV models. The company announced it has added Intel to its AI Factory portfolio with Intel Gaudi 3 AI accelerators and Intel Xeon processors, with a strong focus on inferencing workloads.

    Dell also announced its fourth update to the Dell AI Platform with AMD, rolling out two new PowerEdge servers—the XE9785 and the XE9785L—equipped with the latest AMD Instinct MI350 series GPUs. The Dell AI Factory with NVIDIA combines Dell’s infrastructure with NVIDIA’s AI software and GPU technologies to deliver end-to-end solutions that can reduce setup time by up to 86% compared to traditional approaches. The company also continues to strengthen its partnership with NVIDIA, announcing products leveraging NVIDIA’s Blackwell family and other updates launched at NVIDIA GTC. As of today, Dell supports choice by delivering AI solutions with all of the primary GPU and AI accelerator infrastructure providers.

    Client-Side AI Advancements

    At the edge of the AI Factory ecosystem, Dell announced enhancements to the Dell Pro Max in a mobile form factor, leveraging Qualcomm’s AI 100 discrete NPUs designed for AI engineers and data scientists who need fast inferencing capabilities. With up to 288 TOPs at 16-bit floating point precision, these devices can power up to a 70-billion parameter model, delivering 7x the inferencing speed and 4x the accuracy over a 40 TOPs NPU. Dell says the Pro Max Plus line can run a 109-billion-parameter AI model.

    The Pro Max and Plus launches follow Dell’s previous announcement of AI PCs featuring Dell Pro Max with GB 10 and GB 300 processors powered by NVIDIA’s Grace Blackwell architecture. Overall, Dell has simplified its PC portfolio but made it easier for customers to choose the right system for their workloads by providing the latest chips from AMD, Intel, Nvidia, and Qualcomm.

    On-Premise AI Deployment Gains Ecosystem Momentum

    Following the theme of choice, organizations need the flexibility to run AI workloads on-premises and in the cloud. Dell is making significant strides in enabling on-premise AI deployments with major software partners. The company announced it is the first provider to bring Cohere capabilities on-premises, combining Cohere’s generative AI models with Dell’s secure, scalable infrastructure for turnkey enterprise solutions.

    Similar partnerships with Mistral and Glean were also announced, with Dell facilitating their first on-premise deployments. Additionally, Dell is supporting Google’s Gemini on-premises with Google Distributed Cloud.

    To simplify model deployment, Dell now offers customers the ability to choose models on Hugging Face and deploy them in an automated fashion using containers and scripts. Enterprises increasingly recognize that while public cloud AI has its place, a hybrid AI infrastructure approach could deliver better economics and security for production workloads.

    The imperative for scalable yet efficient AI infrastructure at the edge is a growing need. As Michael Dell said during his Dell Technologies World keynote, “Over 75% of enterprise data will soon be created and processed at the edge, and AI will follow that data; it’s not the other way around. The future of AI will be decentralized, low latency, and hyper-efficient.”

    Dell’s ability to offer robust hybrid and fully on-premises solutions for AI is proving to be a significant advantage as companies increasingly seek on-premises support and even potentially air-gapped solutions for their most sensitive AI workloads. Key industries adopting the Dell AI Factory include finance, retail, energy, and healthcare providers.

    Scaling AI Requires a Focus on Energy Efficiency 

    Simplifying AI also requires product innovations that deliver cost-effective, energy-efficient technology. As AI workloads drive unprecedented power consumption, Dell has prioritized energy efficiency in its latest offerings. The company introduced the Dell PowerCool Enclosed Rear Door Heat Exchanger (eRDHx) with Dell Integrated Rack Controller (IRC). This new cooling solution captures nearly 100% of the heat coming from GPU-intensive workloads. This innovation lowers cooling energy requirements for a rack by 60%, allowing customers to deploy 16% more racks with the same power infrastructure.

    Dell’s new systems are rated to operate at 32 to 37 degrees Celsius, supporting significantly warmer temperatures than traditional air-cooled or water-chilled systems, further reducing power consumption for cooling. The PowerEdge XE9785L now offers Dell liquid cooling for flexible power management. Even if a company isn’t aiming for a specific sustainability goal, every organization wants to improve energy utilization. 

    Early Adopter Use Cases Highlight AI’s Opportunity

    With over 200 product enhancements to its AI Factory in just one year, Dell Technologies is positioning itself as a central player in the rapidly evolving enterprise AI infrastructure market. It offers the breadth of solutions and expertise organizations require to successfully implement production-grade AI systems in a secure and scalable fashion. However, none of this technology matters if enterprises can’t find a way to create business value by adopting it. Fortunately, examples from the first wave of enterprise early adopters highlight ways AI can deliver meaningful returns in productivity and customer experience. Let’s look at two use cases presented at Dell Tech World. 

    The Power of LLMs in Finance at JPMorgan Chase

    JPMorgan Chase took the stage to make AI real from a customer’s perspective. The financial firm uses Dell’s compute hardware, software-defined storage, client, and peripheral solutions. Larry Feinsmith, the Managing Director and Head of Global Tech Strategy, Innovation & Partnerships at JPMorgan Chase, said, “We have a hybrid, multi-cloud, multi-provider strategy. Our private cloud is an incredibly strategic asset for us. We still have many applications and data on-premises for resiliency, latency, and a variety of other benefits.” 

    Feinsmith also spoke of the company’s Large Language Model (LLM) strategy. He said, “Our strategy is to use a constellation of models, both foundational and open, which requires a tremendous amount of compute in our data centers, in the public cloud, and, of course, at the edge. The one constant thing, whether you’re training models, fine-tuning models, or finding a great use case that has large-scale inferencing, is that they all will drive compute. We think Dell is incredibly well positioned to help JPMorgan Chase and other companies in their AI journey.”

    Feinsmith noted that using AI isn’t new for JPMorgan Chase. For over a decade, JPMorgan Chase has leveraged various types of AI, such as machine learning models for fraud detection, personalization, and marketing operations. The company uses what Feinsmith called its LLM suite, which over 200,000 people at JPMorgan Chase use today. The generative AI application is used for QA summarization and content generation using JPMorgan Chase’s data in a highly secure way. Next, it has used the LLM suite architecture to build applications for its financial advisors, contact center agents, and any employee interacting with its clients. Its third use case highlighted changes in the software development area. JPMorgan Chase rolled out code generation AI capabilities to over 40,000 engineers. It has achieved as much as 20% productivity in the code creation and expects to leverage AI throughout the software development life cycle. Going forward, the financial firm expects to use AI agents and reasoning models to execute complex business processes end-to-end.

    Seemantini Godbole, EVP and Chief Digital and Information Officer at Lowe's, shared insights on designing the strategy for AI

    How AI Makes It Easier For Employees to Serve Customers at Lowe’s

    Lowe’s Home Improvement Stores provided another example of how companies are leveraging Dell Technology and AI to transform the customer and employee experience. Seemantini Godbole, EVP and Chief Digital and Information Officer at Lowe’s, shared insights on designing the strategy for AI when she said, “How should we deploy AI? We wanted to do impactful and meaningful things. We did not want to die a death of 1000 pilots, and we organized our efforts across how we sell, how we shop, and how we work. How we sell was for our associates. How we shop is for our customers, and how we work is for our headquarters employees. For whatever reason, most companies have begun with their workforce in the headquarters. We said, No, we are going to put AI in the hands of 300,000 associates.” For example, she described a generative AI companion app for store associates. “Every store associate now has on his or her zebra device a ChatGPT-like experience for home improvement.”, said Godbole. Lowe’s is also deploying computer vision algorithms at the edge to understand issues such as whether a customer in a particular aisle is waiting for help. The system will then send notifications to the associates in that department. Customers can also ask various home improvement questions, such as what paint finish to use in a bathroom, at Lowes.com/AI.

    Designing A World Where AI Delivers Human Opportunity

    Michael Dell said, “We are entering the age of ubiquitous intelligence, where AI becomes as essential as electricity, with AI, you can distill years of experience into instant insights, speeding up decisions and uncovering patterns in massive data. But it’s not here to replace humans. AI is a collaborator that frees your teams to do what they do best, to innovate, to imagine, and to solve the world’s toughest problems.” 

    While there are many AI deployment challenges ahead, the customer examples shared at Dell Technologies World provide a glimpse into a world where AI benefits both customers and employees. The challenge now is to do this sustainably and ethically at scale.  

  • Google Cloud’s Ironwood TPU Forges Better Enterprise AI

    Artificial intelligence infrastructure has emerged as the critical battleground for cloud computing dominance. At this year’s Google Cloud Next conference, the company demonstrated its intensified commitment to AI infrastructure, unveiling strategic investments, such as the Ironwood Tensor Processing Units (TPUs), designed to transform enterprise AI deployment across industries.


    “We’re investing in the full stack of AI innovation,” stated Sundar Pichai, CEO of Google and Alphabet, who outlined plans to allocate $75 billion in capital expenditure toward this vision. This substantial commitment reflects the scale of investment required to maintain competitive positioning in the rapidly evolving AI infrastructure market. Innovating in AI requires courage and deep pockets.

    Google Cloud articulated a full stack strategy focused on developing AI-optimized infrastructure spanning three integrated layers: purpose-built hardware, foundation models, and tooling for building and orchestrating multi-agent systems. During the keynote presentation, Google Cloud introduced the Ironwood TPU its seventh-generation Tensor Processing Units (TPUs), representing a significant advancement in AI computational architecture.

    Optimizing Infrastructure for AI With TPUs

    Cloud Computing infrastructure started as a method of replacing and optimizing on-premise data centers. Today, cloud computing providers are adding specific infrastructure to support new computing requirements introduced with supporting AI. TPUs are specialized processors developed by Google specifically to accelerate AI and machine learning workloads—with particular optimization for deep learning operations. TPUs deliver superior performance-per-dollar compared to general-purpose GPUs or CPUs across numerous machine learning use cases, resulting in reduced infrastructure costs or increased computational capability within existing budget constraints.

    Ironwood TPUs represent a cornerstone component of Google Cloud’s AI Hypercomputer architecture, which integrates optimized hardware and software components for high-demand AI workloads. The AI Hypercomputer platform constitutes a supercomputing system that combines performance-optimized silicon, open software frameworks, machine learning libraries, and flexible consumption models designed to enhance efficiency throughout the AI lifecycle—from training and tuning to inference and serving.

    According to Google’s technical specifications, these specialized AI processors deliver computational performance that’s 3,600 times more powerful and 29 times more energy efficient than the original TPUs launched in 2013. Ironwood also demonstrates a 4-5x performance improvement across multiple operational functions compared to the previous version 6 Trillium TPU architecture.

    Ironwood implements advanced liquid cooling systems and proprietary high-bandwidth Inter-Chip Interconnect (ICI) technology to create scalable computational units called “pods” that integrate up to 9,216 chips. At maximum pod configuration, Ironwood delivers 24 times the computational capacity of El Capitan, currently ranked as the world’s largest supercomputer.


    To maximize this infrastructure’s utility, Google Cloud has developed Pathways, a machine learning runtime created by Google DeepMind that enables efficient distributed computing across multiple TPU chips. Pathways on Google Cloud simplifies scaling beyond individual Ironwood Pods, allowing for the orchestration of hundreds of thousands of Ironwood chips for next-generation AI computational requirements. Google uses Pathways internally to train advanced models such as Gemini and now extends these same distributed computation capabilities to Google Cloud customers.

    Marrying Business Impact With Economics

    While the industry has witnessed a proliferation of smaller, specialized AI models, significant AI chip innovation remains essential to deliver the performance requirements for supporting advanced reasoning and multimodal models.

    According to Amin Vahdat, VP/GM of ML, Systems & Cloud AI at Google Cloud, “Ironwood is designed to gracefully manage the complex computation and communication demands of ‘thinking models,’ which encompass Large Language Models (LLMs), Mixture of Experts (MoEs) and advanced reasoning tasks.” This architecture addresses the market requirement for modular, scalable systems that deliver improved performance and accuracy while optimizing both cost efficiency and energy utilization.


    For enterprises implementing large-scale AI initiatives, Google’s hardware advancements translate to quantifiable benefits across three dimensions:

    1. Economic Efficiency. Google’s specialized hardware substantially increases computational density per dollar, reducing the total cost of ownership for AI infrastructure. Organizations can deploy increasingly sophisticated AI models without corresponding linear increases in computing expenditures.
    2. Sustainability Metrics. As AI model complexity increases (across systems like Gemini, ChatGPT, and advanced image generators), the underlying computational infrastructure generates significantly more heat and power consumption. Liquid cooling technology, implemented in Ironwood, delivers substantially higher thermal efficiency compared to conventional air cooling, enabling chips to operate at higher frequencies without thermal throttling. This innovation addresses power consumption—a critical consideration for both cloud providers and enterprise buyers with sustainability commitments. The enhanced performance-per-watt metrics of these TPUs help organizations address environmental impact concerns while scaling their AI capabilities.
    3. Time-to-Market Acceleration. The exponential increase in processing capacity means that AI model training and inference workflows—previously requiring weeks or months of computation—can now be completed in days or hours. This compression of development timelines enables organizations to iterate more rapidly and operationalize AI solutions with significantly reduced deployment cycles.

    Why TPUs Matters to Enterprise Buyers

    Organizations are over the phase of interesting AI proof of concept trials that never make it to production-grade systems. 2025 is the year that organizations expect to deploy use cases with quantifiable business value while laying the foundation for what’s next. Google Cloud’s enhanced AI infrastructure enables practical enterprise applications today while supporting previously constrained by computational economics or performance limitations. Consider the impact of AI today and tomorrow in:

    • Financial Services Analytics. During the Google Cloud Next keynote, Deutsche Bank shared how it uses technology from Google Cloud to power an AI-powered research agent named DB Lumina for faster data analysis. Many banking and investment firms are investigating how to use enhanced AI infrastructure to process market data streams, detect complex pattern anomalies in real-time, and enable more responsive trading strategies and comprehensive risk management frameworks.
    • Customer Experience Transformation. Retail and service organizations can implement sophisticated recommendation engines and multimodal conversational AI systems that process customer interactions with minimal latency while incorporating rich contextual understanding. For example, Verizon uses Google Cloud’s Customer Engagement Suite to enhance its customer service for over 115 million connections with AI-powered tools, like the Personal Research Assistant which accurately answers 95% of questions, helping agents provide faster, more accurate, and personalized support. The next era focuses on cracking the code for personalization, advanced marketing assets, and empathetic conversational AI.
    • Computational Medicine. Today, healthcare organizations use AI to improve patient experiences with data gathering and summarization features. For example, Seattle Children’s Hospital used Google Cloud’s generative AI to create Pathway Assistant, an AI-powered agent that improves clinicians’ access to complex information and the latest evidence-based best practices needed to treat patients. As we advance, healthcare institutions can leverage advances in AI infrastructure to accelerate the analysis of complex imaging datasets, genomic sequences, and patient records, potentially enhancing diagnostic accuracy and treatment protocol optimization.

    Get Comfortable With Change

    As competition intensifies among cloud infrastructure providers, Google’s substantial investment in AI represents a strategic assessment that enterprise computing will increasingly prioritize AI-driven workloads—and that organizations will select platforms offering the optimal combination of performance, cost efficiency, and energy sustainability.


    The only constant in the AI market will be change. Business leaders must be comfortable with continuously adapting strategies to leverage AI advancements. For CIOs and technology leaders developing their AI implementation roadmaps, Google Cloud’s hardware innovations, such as the Ironwood TPU, present technical and economic justifications to reevaluate their infrastructure strategy as AI becomes increasingly central to operational excellence and competitive differentiation.

  • Why Small AI Models Matter: The Business Case for Enterprise AI Efficiency

    In the rapidly evolving landscape of artificial intelligence, business leaders face a critical decision: which AI models will deliver the most value to their organization? While much attention has focused on massive models with billions of parameters, there’s a compelling case for considering smaller, more specialized AI models. Lopez Research interviewed Kate Soule, Director of Technical Product Management for IBM’s Granite products, to learn more about these smaller models.

    IBM Granite represents a family of open AI models specifically designed for transparency, data governance, and practical business applications. IBM’s third-generation of Granite models (3.2) offer reasoning capabilities and multimodal models that offers vision and has been optimized for document understanding.

    Big News: Small Models Will Make Enterprise AI More Efficient and Effective. Image Source: Adobe Stock

    Why Size Matters (and Why Smaller Models Can Be Better)

    Soule says, “Everything gets more difficult as the model gets larger.” This challenge manifests in several business-critical ways:

    • Higher operational costs. Larger models require more computing resources and energy, directly increasing operational expenses.
    • Increased latency. Bigger models take longer to generate responses, potentially degrading customer experience.
    • Need for powerful hardware. If using larger models in the cloud or on-premises, these models demand more powerful and expensive GPU infrastructure.
    • Limited customization. Adapting massive models to your business needs requires substantial computing resources and expertise.

    The Small Model Efficiency Advantage

    IBM’s approach with their Granite models focuses on efficiency through purpose-built smaller models. At just 2 to 8 billion parameters (compared to the largest industry models, such as Llama 3.1 from Meta, exceeding 400 billion parameters), these models deliver several key advantages, including::

    1. Cost-effectiveness. Lower computational requirements translate directly to reduced operational costs.
    2. Customization potential. Smaller models are more manageable and less expensive to fine-tune for specific business tasks.
    3. Deployment flexibility. Some models are small enough to run locally on standard hardware, such as an AI PC, eliminating cloud dependency.
    4. Faster response times. Reduced model size means quicker inference and better user experiences.

    Whether it’s Small Models or Large Models, Companies Need a “Fit-for-Purpose” Approach

    Rather than seeking a single “model to rule them all,” business and IT leaders are working together to assemble a portfolio of AI solutions that incorporate foundational LLMs with specialized AI models to support different business needs. As Soule notes, “To get value and to be able to deploy AI cost-effectively… you need to consider having fit-for-purpose models.”

    Fit-for-purpose AI model selection allows businesses to optimize performance and cost based on the specific requirements of each use case. A larger model may be the best solution for high-value, complex tasks. For routine operations, a smaller specialized model often delivers comparable results at a fraction of the cost.

    Soule shared that IBM’s Granite models embody this philosophy with their modular design. Instead of trying to create a single massive model for all tasks, IBM has developed distinct models for different enterprise functions. For example, IBM’s Granite solutions offer the models for use cases such as coding, time-series forecasting, security and language models for designed for agentic workflows, RAG etc.

    Despite their efficient design, IBM shared that Granite models don’t sacrifice performance for cost. According to IBM’s benchmarks, Granite outperforms comparable models across various enterprise tasks, achieving high scores on Hugging Face’s RAGBench Leaderboard. This targeted approach ensures organizations can select the right tools for specific business challenges without unnecessary computational overhead.

    Multimodal Models: Expanding AI’s Capabilities

    Multimodal AI models can simultaneously process and interpret multiple types of data inputs—such as text, images, audio, and video. Unlike unimodal models that work with only one data type (typically text), multimodal models can understand the relationships between different forms of information, similar to how humans process the world through multiple senses:

    IIBM’s Granite 3.2 vision models offer a practical application of multimodal capabilities in an enterprise context. Rather than focusing on image generation (creating pictures from text prompts), these models specialize in image understanding—extracting valuable information from visual content. At just 2 billion parameters, these specialized vision models can:

    • Extract data from documents, even poorly scanned PDFs
    • Analyze charts and graphs to answer specific business questions
    • Process dashboard screenshots to provide insights on performance metrics
    • Interpret receipts and other visual business documents

    Making Strategic AI Decisions: The Performance-Cost Matrix

    As AI technology evolves, organizations must balance cost, accuracy, and safety for model use. Soule shared at least four guidelines for evaluating what models to use within the enterprise, including:

    1. Right-sizing your models: Match model capabilities to business requirements rather than defaulting to the largest available.
    2. Design model selection criteria with transparency and governance in mind: Understand how models were trained and whether they align with your governance requirements.
    3. Systems-based security: Implement guardrails and safety protocols beyond relying solely on the model’s built-in safeguards.
    4. Customization needs: Assess how much adaptation a model will require for your specific use cases.

    For example, Soule discussed how different models may offer various levels of transparency and governance. One distinguishing feature of IBM Granite models is that IBM publishes detailed information about their training datasets and methodologies, allowing enterprises to understand what’s “under the hood” of these models. The Granite ecosystem includes risk and harm detection capabilities, transparency tools, and IP protection. Model transparency allows IBM to provide indemnification for Granite models, offering businesses additional protection when deploying these AI solutions.

    The Shift is Underway

    The “bigger is better” paradigm is giving way to a more nuanced approach to enterprise AI. Business leaders can achieve comparable performance by strategically implementing smaller, specialized models for appropriate use cases while significantly reducing costs and complexity. IBM’s Granite models are one example of this approach. We’ve also seen the large foundation model providers, such as Open.AI and Meta, support smaller models. 

    Looking ahead, we’re moving toward more flexible AI deployment models where businesses can dynamically allocate resources based on task importance. This could mean using models of various sizes for different tasks or enabling features like “reasoning” when demands such as accuracy justify the additional cost and latency. As AI becomes further integrated into business operations, this efficiency-focused approach will be increasingly critical for sustainable AI adoption and competitive advantage.

    You can subscribe to our video channel here and our podcast here.