Tag: Generative AI

  • AI Search Helps SAP Improve Customer Support With Coveo

    AI Search Helps SAP Improve Customer Support With Coveo

    Most companies struggle to prove AI ROI, but enterprise software provider SAP found an approach that works


    The artificial intelligence (AI) market is expanding rapidly. Enterprise AI spending surged eight-fold in 2024 according to the Menlo Ventures. This year is expected to be a banner year for AI adoption. However, many AI initiatives are not delivering measurable returns.


    Recent AI studies paint a stark picture. MIT reports that 95% of generative AI pilots fail to reach production, while RAND found that 46% of companies scrapped their AI projects before launch. Perhaps most telling, Boston Consulting Group discovered that only 24% of firms have developed the capabilities needed to move AI beyond proof-of-concepts into tangible business value. While we could debate what the actual percentages are, one thing is clear. Companies are spending money on AI but aren’t getting the returns they expected.

    What separates the AI strategy winners from the also-rans? It appears the key to AI success is selecting a set of narrowly scoped use cases with a quantifiable cost that will benefit from using AI. As Best Buy said, we don’t have AI projects, we have projects that use AI. SAP’s journey offers one success story with crucial lessons for any organization struggling to prove AI’s worth.


    Some of the best AI success stories start by addressing a simple problem. For example, Cisco discussed how leveraging AI to detect network configuration issues solved a key customer support challenge.

    The Million-Case Support Crisis That Sparked SAP’s Internal AI Journey


    For SAP, the path to AI success began with a simple but expensive problem: customer support for its over 300 million cloud subscribers. Like many companies, the German enterprise software vendor was experiencing significant outflow of money on avoidable service requests. The goal was not to remove employees, but to eliminate unnecessary inquiries.

    AI has the ability to create capacity for the existing workforce to build new customers service experiences. Customers were struggling to find answers , leading to unnecessary support tickets that customers could have avoided through self-service.
    “We weren’t hitting our self-service targets,” explains Michelle Lewis-Miller, the Head of Digital Experience and Voice of Customer for Customer Support at SAP. “We couldn’t see what was happening and couldn’t measure anything.”

    Visibility is a key issue across businesses of all sizes. Many companies deploy AI solutions without clear metrics or understanding of where the technology can create the most impact. SAP started with a measurable business problem and worked backward to find the right AI solution.

    The Foundation: Getting Knowledge Quality Right First


    Before implementing any search solution, SAP had already invested heavily in knowledge quality. Lewis-Miller uses a culinary metaphor to explain: “If you’re serving dinner with poor ingredients, it doesn’t matter how great the chef is.”

    Too many organizations rush to implement AI technology without first ensuring their underlying data and content are ready to support it. Knowledge wasn’t the company’s problem. SAP had over 10 million knowledge assets—spanning help documents, SAP Knowledge Base Articles, community posts, and videos- as a foundation for its search efforts. “SAP had already made massive investments to get the knowledge into good shape, and introduced AI-driven Incident Solution Matching recommender for knowledge. The problem was an inferior knowledge discovery experience via search,” Lewis-Miller notes.

    30% Case Reduction: The Pilot Results Nobody Expected


    Rather than launching a company-wide AI initiative, SAP began with a targeted pilot in their Concur travel and expense division. The Concur division was already using Coveo’s AI-powered search platform, making it an ideal testing ground for generative AI capabilities.


    The Coveo AI pilot results exceeded all expectations. Within six months, SAP Concur experienced a 30% drop in support case volume, translating to €8 million in annual cost avoidance. For many customers, this was their first encounter with generative AI—and it delivered immediate, tangible value.


    “When we piloted Relevance Generative Answering in Concur, it went so well it was shocking,” Lewis-Miller recalls. “We saw such a huge decrease in case submission that helped us tremendously in keeping up with business growth.”


    The Coveo-SAP Concur pilot’s success illustrates a key principle: AI ROI comes from solving specific, measurable problems rather than implementing technology for technology’s sake. The SAP Concur team had clear metrics (case volume reduction), a defined user base, and a focused use case.


    The Cultural Shift That Enabled Bold Decisions


    The cultural shift at SAP proved crucial to the initiative’s success. In a discussion with Lewis-Miller, she emphasizes that executive support went beyond budget approval: “Our executives came in and made a very clear statement. They said, You’re not going to be punished for trying.” This created an environment where teams could be decisive rather than seeking consensus from dozens of stakeholders.


    She refers to this as the “tip of the spear approach,” noting that “at an organization of any size you need to find the people who are willing to take risks, and you need to have the executive leadership being unequivocal about supporting that person who’s willing to take the risk.” Without this top-down support for calculated risk-taking, the project likely could have stalled in committee discussions and consensus-building exercises that ultimately may have prevented any meaningful progress.


    Scaling Success: The SAP for Me Challenge


    Emboldened by the Concur results, SAP faced a much larger challenge: implementing AI-powered search across SAP for Me, the central portal serving millions of customer interactions every month. Unlike the focused SAP Concur deployment, this required integrating 14 to 20 different knowledge bases while serving vastly different user types—from certified engineers with 30 years of SAP experience to small business owners with limited technical background.


    The complexity initially threw the team. When they deployed the same generative AI technology on SAP for Me, the results looked worse at first glance—higher click rates and more case submissions seemed to indicate the system wasn’t working.
    “We expected it to address the low-hanging fruit. It wasn’t at all like what we saw with Concur. We asked Coveo if something was broken in their backend,” Lewis-Miller admits.


    The breakthrough came when SAP’s analysts realized they were measuring the wrong things. While simple questions decreased on the Concur platform, SAP for Me users were asking much more complex queries. The AI wasn’t avoiding low-value cases—it was helping resolve sophisticated problems that would have required expensive expert support.


    “We were handling a more valuable subset of cases,” Lewis-Miller explains. “When we changed our perspective and looked at the overall numbers, we saw that submissions had actually gone down.”


    Here’s where many AI projects fail: teams panic when initial metrics don’t match expectations and abandon promising initiatives. SAP took a different approach, partnering with their technology vendor to dig deeper into the analytics. Successful AI implementations require a company to deploy, monitor, analyze and iterate. SAP’s story highlights how theory rarely matches reality.


    The Results


    Since launching in 2023, the AI-powered search has enabled intelligent scaling of the support function to drive business growth. SAP’s AI strategy ensures that support costs grow slower than revenue, thereby expanding profit margins, and allowing its business to absorb significant market expansion. As a result, customer self-service success from search is now at over 80%, and the technology spans 47 different sources and indexes 11.2 million documents across the SAP ecosystem.


    The Technical Foundation That Makes It Work


    SAP’s success stems from choosing the right technical approach for their specific needs. Rather than building custom AI models from scratch, which is labor and time intensive, SAP leveraged a “hybrid search” system that combines multiple AI techniques:


    • Lexical search (traditional keyword matching) finds documents that contain the exact words you type—like searching for “invoice processing” and getting results that include those specific terms.
    • Semantic search (context-aware search) understands what you actually mean, not just the words you use. It recognizes that “payment issues” and “billing problems” refer to similar concepts, even when the exact words don’t match.
    • Behavioral machine learning (learning from user patterns) tracks which search results people actually click and use, then automatically surfaces the most helpful content for similar future searches.
    • Generative AI (AI-powered answer creation) provides direct responses to questions in plain English while showing exactly which company documents or sources the information came from, similar to how ChatGPT works but trained on your organization’s specific knowledge base.


    This hybrid approach addresses common search frustrations: lexical systems miss relevant results when you don’t use the exact right keywords, while pure generative AI can sometimes provide inaccurate information. The combination delivers both precision and intelligence.


    Advice For Building ROI with AI


    SAP’s re-invested cost avoidance didn’t happen by accident. Behind those impressive numbers lies a methodical approach that any organization can follow—if they’re willing to resist the allure of flashy AI demos and focus on fundamentals. Key steps that every organization can take include:


    • Start with measurable problems rather than exciting technology possibilities
    • Ensure your knowledge foundation is solid before implementing AI search solutions—garbage in, garbage out applies to AI just as much as any other technology
    • Pilot in controlled environments where you can isolate variables and measure impact
    • Secure executive support for risk-taking and empower decision-makers to act without requiring consensus from dozens of stakeholders
    • Document rigorously for compliance and architecture
    • Evaluate enterprise-ready AI solutions instead of jumping first into extensive custom AI development
    • Invest in proper analytics and measurement to understand true impact versus surface-level metrics


    While all of these points are important, Lopez Research has seen many companies skip the governance and compliance step. Lewis-Miller identifies meticulous documentation as a critical but often overlooked success factor.

    “We invested a lot of time in finding employees who had the soft skills necessary to interpret the technical detail for the legal people and the legal detail back to the technical detail.” This documentation became essential reference material that prevented the project from stalling at multiple compliance checkpoints. At enterprise scale, having detailed outlines for every architectural and legal decision proves invaluable.


    Beyond Cost Avoidance: Proactive Customer Experience


    SAP isn’t stopping at self-service enablement. The company is now using behavioral analytics to intervene before customers encounter problems. Lewis-Miller describes the vision: “As a user is engaging in one of our support journeys, our goal is to intervene in real time to get ahead of issues before they actually emerge.”


    The approach involves using customer behavior patterns to identify when someone is struggling to find information, then proactively routing them to chat support before frustration sets in. “We’re not trying to give everybody a white glove experience. We’re specifically intervening only in those moments where we can see it’s going down the wrong path,” Lewis-Miller explains.


    This includes predicting system issues before they occur and routing insights to product teams to fix confusing features before they generate support cases. By analyzing search patterns and support interactions, SAP can identify friction points in their products and address them proactively—turning support data into product improvement fuel.


    AI is doing more for internal innovation: For the cases that do reach a support engineer, SAP focuses on agent augmentation. AI-based case summarization enables faster handoffs, and intelligent routing connects a case to the correct expert instantly.
    The AI revolution is about implementing the right technology to solve real business challenges. SAP found their formula. Now it’s time for other enterprises to find theirs.

  • Google Cloud’s Ironwood TPU Forges Better Enterprise AI

    Artificial intelligence infrastructure has emerged as the critical battleground for cloud computing dominance. At this year’s Google Cloud Next conference, the company demonstrated its intensified commitment to AI infrastructure, unveiling strategic investments, such as the Ironwood Tensor Processing Units (TPUs), designed to transform enterprise AI deployment across industries.


    “We’re investing in the full stack of AI innovation,” stated Sundar Pichai, CEO of Google and Alphabet, who outlined plans to allocate $75 billion in capital expenditure toward this vision. This substantial commitment reflects the scale of investment required to maintain competitive positioning in the rapidly evolving AI infrastructure market. Innovating in AI requires courage and deep pockets.

    Google Cloud articulated a full stack strategy focused on developing AI-optimized infrastructure spanning three integrated layers: purpose-built hardware, foundation models, and tooling for building and orchestrating multi-agent systems. During the keynote presentation, Google Cloud introduced the Ironwood TPU its seventh-generation Tensor Processing Units (TPUs), representing a significant advancement in AI computational architecture.

    Optimizing Infrastructure for AI With TPUs

    Cloud Computing infrastructure started as a method of replacing and optimizing on-premise data centers. Today, cloud computing providers are adding specific infrastructure to support new computing requirements introduced with supporting AI. TPUs are specialized processors developed by Google specifically to accelerate AI and machine learning workloads—with particular optimization for deep learning operations. TPUs deliver superior performance-per-dollar compared to general-purpose GPUs or CPUs across numerous machine learning use cases, resulting in reduced infrastructure costs or increased computational capability within existing budget constraints.

    Ironwood TPUs represent a cornerstone component of Google Cloud’s AI Hypercomputer architecture, which integrates optimized hardware and software components for high-demand AI workloads. The AI Hypercomputer platform constitutes a supercomputing system that combines performance-optimized silicon, open software frameworks, machine learning libraries, and flexible consumption models designed to enhance efficiency throughout the AI lifecycle—from training and tuning to inference and serving.

    According to Google’s technical specifications, these specialized AI processors deliver computational performance that’s 3,600 times more powerful and 29 times more energy efficient than the original TPUs launched in 2013. Ironwood also demonstrates a 4-5x performance improvement across multiple operational functions compared to the previous version 6 Trillium TPU architecture.

    Ironwood implements advanced liquid cooling systems and proprietary high-bandwidth Inter-Chip Interconnect (ICI) technology to create scalable computational units called “pods” that integrate up to 9,216 chips. At maximum pod configuration, Ironwood delivers 24 times the computational capacity of El Capitan, currently ranked as the world’s largest supercomputer.


    To maximize this infrastructure’s utility, Google Cloud has developed Pathways, a machine learning runtime created by Google DeepMind that enables efficient distributed computing across multiple TPU chips. Pathways on Google Cloud simplifies scaling beyond individual Ironwood Pods, allowing for the orchestration of hundreds of thousands of Ironwood chips for next-generation AI computational requirements. Google uses Pathways internally to train advanced models such as Gemini and now extends these same distributed computation capabilities to Google Cloud customers.

    Marrying Business Impact With Economics

    While the industry has witnessed a proliferation of smaller, specialized AI models, significant AI chip innovation remains essential to deliver the performance requirements for supporting advanced reasoning and multimodal models.

    According to Amin Vahdat, VP/GM of ML, Systems & Cloud AI at Google Cloud, “Ironwood is designed to gracefully manage the complex computation and communication demands of ‘thinking models,’ which encompass Large Language Models (LLMs), Mixture of Experts (MoEs) and advanced reasoning tasks.” This architecture addresses the market requirement for modular, scalable systems that deliver improved performance and accuracy while optimizing both cost efficiency and energy utilization.


    For enterprises implementing large-scale AI initiatives, Google’s hardware advancements translate to quantifiable benefits across three dimensions:

    1. Economic Efficiency. Google’s specialized hardware substantially increases computational density per dollar, reducing the total cost of ownership for AI infrastructure. Organizations can deploy increasingly sophisticated AI models without corresponding linear increases in computing expenditures.
    2. Sustainability Metrics. As AI model complexity increases (across systems like Gemini, ChatGPT, and advanced image generators), the underlying computational infrastructure generates significantly more heat and power consumption. Liquid cooling technology, implemented in Ironwood, delivers substantially higher thermal efficiency compared to conventional air cooling, enabling chips to operate at higher frequencies without thermal throttling. This innovation addresses power consumption—a critical consideration for both cloud providers and enterprise buyers with sustainability commitments. The enhanced performance-per-watt metrics of these TPUs help organizations address environmental impact concerns while scaling their AI capabilities.
    3. Time-to-Market Acceleration. The exponential increase in processing capacity means that AI model training and inference workflows—previously requiring weeks or months of computation—can now be completed in days or hours. This compression of development timelines enables organizations to iterate more rapidly and operationalize AI solutions with significantly reduced deployment cycles.

    Why TPUs Matters to Enterprise Buyers

    Organizations are over the phase of interesting AI proof of concept trials that never make it to production-grade systems. 2025 is the year that organizations expect to deploy use cases with quantifiable business value while laying the foundation for what’s next. Google Cloud’s enhanced AI infrastructure enables practical enterprise applications today while supporting previously constrained by computational economics or performance limitations. Consider the impact of AI today and tomorrow in:

    • Financial Services Analytics. During the Google Cloud Next keynote, Deutsche Bank shared how it uses technology from Google Cloud to power an AI-powered research agent named DB Lumina for faster data analysis. Many banking and investment firms are investigating how to use enhanced AI infrastructure to process market data streams, detect complex pattern anomalies in real-time, and enable more responsive trading strategies and comprehensive risk management frameworks.
    • Customer Experience Transformation. Retail and service organizations can implement sophisticated recommendation engines and multimodal conversational AI systems that process customer interactions with minimal latency while incorporating rich contextual understanding. For example, Verizon uses Google Cloud’s Customer Engagement Suite to enhance its customer service for over 115 million connections with AI-powered tools, like the Personal Research Assistant which accurately answers 95% of questions, helping agents provide faster, more accurate, and personalized support. The next era focuses on cracking the code for personalization, advanced marketing assets, and empathetic conversational AI.
    • Computational Medicine. Today, healthcare organizations use AI to improve patient experiences with data gathering and summarization features. For example, Seattle Children’s Hospital used Google Cloud’s generative AI to create Pathway Assistant, an AI-powered agent that improves clinicians’ access to complex information and the latest evidence-based best practices needed to treat patients. As we advance, healthcare institutions can leverage advances in AI infrastructure to accelerate the analysis of complex imaging datasets, genomic sequences, and patient records, potentially enhancing diagnostic accuracy and treatment protocol optimization.

    Get Comfortable With Change

    As competition intensifies among cloud infrastructure providers, Google’s substantial investment in AI represents a strategic assessment that enterprise computing will increasingly prioritize AI-driven workloads—and that organizations will select platforms offering the optimal combination of performance, cost efficiency, and energy sustainability.


    The only constant in the AI market will be change. Business leaders must be comfortable with continuously adapting strategies to leverage AI advancements. For CIOs and technology leaders developing their AI implementation roadmaps, Google Cloud’s hardware innovations, such as the Ironwood TPU, present technical and economic justifications to reevaluate their infrastructure strategy as AI becomes increasingly central to operational excellence and competitive differentiation.

  • Microsoft Demonstrates Generative AI Customer Momentum

    Image of Emily He at Microsoft
    Emily He, CVP Microsoft

    The entire software market is going through a rather substantial change based on advancements such as generative AI. After Microsoft’s Build, Charles Lamana and Emily He of Microsoft shared an update with industry analysts on the state of AI across Microsoft’s portfolio. Many remember Marc Andreessen’s famous quote saying that software was eating the world. Lamana described how the software evolved from green screens to browser-based SaaS applications but that, as an industry, we hadn’t fundamentally changed the software industry’s economics, workflow, and operations. In a revised take on Andreessen’s quote, Lamana said, “AI is eating software.” It aptly describes how we’re entering a new era of software and technology rapidly changing by adding AI capabilities.


    The market experienced a seismic shift in AI innovation in the past six months focused on how foundation models change what’s possible in software through natural language understanding, image, and video generation, and simplification of software programming. These large-scale transformer models can also deliver new functions, such as Windows UI automation.


    AI has the potential to change the way a company tracks its financials, delivers goods and how it services to its customers. AI will also launch new applications, experiences, and data-driven products. To that end, Microsoft announced a plethora of assistive tools that it calls Copilots across its suite of products. The idea behind a copilot is that it will assist, not replace, an individual by providing the right data within the task context. AI copilots accelerate an individual’s productivity by collecting information, automating workflows, and accelerating task completion.



    Microsoft shared many examples of how it anticipates generative AI will change workflows and processes across the business. For example, a copilot can assist a customer service agent by surfacing the latest information found across a company from a vast set of internal documents, including previously resolved case notes and published knowledge articles. It shared a real-world example based on its use of the technology.



    Today all of Microsoft Azure’s thousands of support engineers use the Dynamics 365 customer service Copilot to provide Azure support to Azure customers. With this software, the agent doesn’t have to spend time searching for information. Dynamics 365 customer service knows the context from the company’s internal data, including customer names and relevant knowledge articles. It can surface all the relevant information and timelines for the case an agent is working on. The Copilot expands on the right side of an agent’s screen and can help the agent draft a reply to a customer with just one click using generative AI.





    In sales, a copilot can summarize a call with a prospect and generate a draft of a follow-up email. Across the general business, an AI-enhanced workspace suite can help employees draft emails, prepare for meetings, and explore data. Marketing teams will benefit from content ideas, data inspiration, and social media post suggestions. For supply chain and finance teams, it can assist with items such as status reports.





    Lopez Research describes these as right-time experiences because it provides the right information and insights at the point of need. Today’s artificial intelligence services are different from what we discussed three years ago, which had focused more on the automation of routine. Three years ago, the technology wasn’t there to build copilots from an infrastructure and AI foundation model perspective. Lamana described this as “AI before foundation models, and AI after foundation models’ as the pivot point for why we can offer AI-assisted applications today.



    Emily He shared a business update demonstrating that the concepts resonated with customers. While the technology had been in limited preview, Microsoft said that it had over 40,000 organizations using these generative AI capabilities in Microsoft business applications, which surprised me. At one point, the company said 600 customers were trialing various features. Whether it’s prompt engineering or even figuring out how a company can create custom-tune versions of chat GPT, Microsoft said its latest announcements were driven by customer feedback.



    One item that has surprised the market is the rapid cadence of Microsoft’s co-pilot releases across its software portfolio. During Microsoft Build, we learned that Microsoft created a co-pilot framework to scale the design of new copilots. It has also invested heavily in engineering, AI infrastructure, Microsoft-created AI models, and partnering with Open AI for several years. Coupling its AI investments with OpenAI’s foundation models has proven a successful strategy, given the myriad announcements Microsoft has made since March of 2023.



    Yet, AI is not without its challenges for organizations. Companies must design data strategies that allow them to use a combination of open source and more private generative AI models without compromising security, privacy, and regulatory compliance. It’s a crucial question that many IT buyers have asked Lopez Research. To alleviate these concerns, Microsoft offers Azure OpenAI Services allowing an organization to use its data with OpenAI and Azure storage encrypted at rest with Microsoft Managed Keys within the same region as the resource. This structure preserves all the security and permissions without placing proprietary data directly into open-source models.



    For years I’ve been discussing the potential of AI for business, but it always lacked the connection to the applications and the workflow. Today, Microsoft and other technology vendors are pulling these core capabilities into their applications to seamlessly integrate AI into business workflows to make individuals more productive without needing to learn how to use AI. This is a great step for the industry.