
Share
Parloa and Metric Sherpa surveyed 700+ CX operators and executives and found a widening gap: teams keep deploying AI agents while confidence in metrics rises, but visibility into what those agents actually say does not.
Here's a number that should make anyone deploying conversational AI pause: 78.5% of operators using AI in customer experience report no metric for hallucination or factual accuracy. Not a soft metric, not an imperfect one. None.
That's the headline finding from a new study by Parloa and Metric Sherpa, which surveyed more than 500 CX operators and 200 CX executives about how they measure AI agent performance, what they're missing, and what happens to the insights once they have them. The short version: organizations are scaling AI deployments faster than they're building the instrumentation to monitor them, and the people running these systems know it.
If you work in ML infra or observability tooling, this will sound familiar. It's the same pattern you see when teams ship a model to production without proper logging, drift detection, or eval harnesses, except here it's customer-facing conversations at scale, with real compliance and trust implications attached.
The confidence paradox is the part that should worry you most. Both operators and executives reported increased confidence in their CX metrics as AI maturity advanced, moving from evaluating to piloting to scaling to fully embedded deployment. Operator confidence sat around 38.0%, executive confidence around 52.0%. But the blind-spot rate, the share of respondents who admit there's something they can't currently see, stayed flat across every single stage: 76.9% for operators, a staggering 92.1% for executives.
In other words, confidence climbs while visibility doesn't move at all. That's not a maturity curve, it's a false sense of security compounding over time. You don't get more observability just by running more volume through a system you already can't fully see into.
This blind spot isn't abstract. 89.5% of operators said there's at least one area of customer experience they currently can't see into. When asked what they'd want more of, both operators and executives ranked the same two things at the top: real-time visibility into agent behavior, and the ability to evaluate every conversation rather than a sample.
That second point matters a lot if you've worked with eval pipelines before. Sampling-based QA, where a human reviews some fraction of conversations after the fact, is the industry default in most CX orgs today. It's also fundamentally mismatched to how LLM-based agents fail. Hallucinations and policy violations don't show up predictably; they can cluster in edge cases that a 2% sample will simply never catch. You need something closer to full-coverage, automated evaluation if you actually want to catch factual drift or compliance breaches before a customer, or a regulator, does.
Parloa's framing for this is "CX observability": complete visibility into every customer interaction, human or AI-handled, that lets teams monitor, understand, and act on performance and compliance issues in real time. When researchers presented that definition to respondents, 56.8% of operators and 52.8% of executives said it would be very or extremely valuable to their organization. That's a strong signal that the concept resonates, even if almost nobody has built it yet.

There's also a structural ownership problem sitting underneath all of this. 42.4% of operators say no single person owns CX monitoring at their organization. 48.5% of executives say the same, that ownership is either spread across teams or has no designated owner at all. When you ask who's primarily responsible for monitoring and reporting CX performance, the answers split awkwardly: a slim majority point to a named central owner (CX ops, CX analytics, or IT/data teams), but a meaningful chunk say it's individual team leads acting independently, or nobody in particular.
That matters because unowned systems don't get instrumented. If you've ever worked on a service with no clear on-call rotation or no single team accountable for its SLOs, you know what happens: monitoring becomes an afterthought, incidents get discovered by customers instead of dashboards, and nobody has the authority to force a fix. CX observability has the same dependency. Someone needs clear authority over the experience and a KPI tied directly to company goals, or the blind spots just persist indefinitely.
Then there's the translation problem. Among operators who do share CX insights with other teams, 55.1% say the insight stalled, met resistance, or got ignored outright once it left the CX dashboard. Only 32.2% of operators prioritize financial and business impact linkage in their metrics, even though 46.4% of executives say that linkage is important to them.
That's a classic cross-team communication failure, the kind you see whenever one group's dashboard doesn't map to another group's incentives. CX teams track CSAT and resolution time; the rest of the business thinks in revenue, churn, and cost-to-serve. If a CX insight can't be translated into one of those terms, it dies on arrival in a product or exec review, no matter how good the underlying data is.
The pattern across all four findings is consistent: teams are confident in what they track, aware it's incomplete, and short on both the tooling and the organizational structure to close the gap.
If you're building or buying observability tooling for conversational AI, the implication is pretty direct. Sampling-based QA and vibes-based confidence aren't going to cut it as AI agents take on more customer-facing volume. The fix looks a lot like what mature ML teams already do for any production model: full-coverage evaluation, real-time monitoring, a clear owner accountable for the metric, and a way to express what you find in terms the rest of the business actually cares about. Right now, most CX orgs are missing at least two of those four, and often all four.
Tags
Original Sources
The 2026 CX Observability Study | Parloa x Metric Sherpa
↗ http://go.parloa.com/cx-observability-2026
About the author
Kai built ML infrastructure at a Bay Area startup before developing an obsession with transformer architectures and inference optimisation that eventually pulled him out of product work entirely. A stint at a compute research lab sharpened his instinct for what actually matters in a model release versus what is marketing. He writes from the inside — from the perspective of someone who has debugged the systems he is describing at three in the morning. He is allergic to hype and instinctively drawn to the unglamorous plumbing questions that everyone else skips over.
More from The Engineer →This Week's Edition
20 November 2026
29 articles
Related Articles

National Academies Report Calls for Faster, Tougher Retraction Standards
Models & Research · 5 min

FAMU and Sandia National Labs Sign New Five-Year Research Agreement Spanning AI and Quantum Computing
Models & Research · 5 min

Who Decides If Your Treatment Is Medically Necessary? Industry Leaders Say It Shouldn't Be AI Alone
Policy & Regulation · 6 min
Related Articles

National Academies Report Calls for Faster, Tougher Retraction Standards
Models & Research · 5 min

FAMU and Sandia National Labs Sign New Five-Year Research Agreement Spanning AI and Quantum Computing
Models & Research · 5 min

Who Decides If Your Treatment Is Medically Necessary? Industry Leaders Say It Shouldn't Be AI Alone
Policy & Regulation · 6 min
More Stories
© 2026 Cedar & Bloom. All rights reserved.