Salesforce spent much of 2024 telling the world that AI agents would transform enterprise operations. Now their SVP of Product Marketing has said publicly: “All of us were more confident about large language models a year ago.” The industry has called this a retreat. I think it’s something more useful — if we ask the right question about it.

The Admission

CEO Marc Benioff reduced the company’s support function from 9,000 to 5,000 employees — approximately 4,000 roles — on the back of confidence in AI agent deployment. Agentforce was the flagship expression of a bet that autonomous AI agents could reliably handle complex customer operations at scale.

Then came the production reality. Salesforce’s CTO, Muralidhar Krishnaprasad, acknowledged that models begin omitting instructions when given more than eight — a serious flaw for precision-dependent business tasks. Vivint, a home security company using Agentforce to support 2.5 million customers, discovered that despite clear instructions to send satisfaction surveys after every customer interaction, the AI sometimes simply didn’t. No error. No log. Just drift.

The company is now pivoting toward “deterministic” automation — rules-based, auditable, predictable — as the corrective to LLM unreliability. Their messaging has shifted to emphasise that Agentforce can help “eliminate the inherent randomness of large models.”

Most of the commentary on this has framed it as a retreat. An embarrassing walk-back from a company that overreached.

That framing misses what’s actually important here.

The Wrong Binary

Salesforce’s proposed correction — lean toward deterministic automation, reduce reliance on probabilistic LLMs — sounds like operational pragmatism. In a narrow sense, it is.

But framed as a strategic direction, it sets up a false choice that will send enterprises down the wrong path.

Pure deterministic automation is what enterprise software looked like before AI. Rule engines, decision trees, scripted workflows. These are powerful in bounded, well-defined contexts — a returns policy with three outcomes, a routing rule triggered by account type, an escalation flag based on ticket age. Deterministic systems are excellent at these problems.

They are also brittle the moment real-world complexity falls outside the rulebook. In customer operations, that happens constantly. The edge cases aren’t edge cases at scale. They’re a substantial and growing proportion of actual interaction volume — the conversations that require judgment, context, empathy, and a decision that no rule anticipated.

Pure probabilistic — LLMs with broad autonomy and minimal structure — is what failed at Vivint. The capability is real. The failure mode is equally real: unpredictable omissions, instruction drift as context grows complex, and outputs that are plausible but wrong in ways that are hard to detect until a customer complains or churns quietly.

The Actual Problem: Nobody Is Designing the Boundary

Salesforce’s CTO described the survey omission problem as a model reliability issue. It isn’t. It’s a workflow architecture issue.

When an AI agent is given eight or more instructions and starts dropping some, the appropriate response is not to reduce the instructions or replace the AI with a rule engine. The appropriate response is to redesign the workflow — so that the AI is never holding eight instructions simultaneously, so the task is decomposed into bounded units with defined checkpoints, and with explicit handoffs between what the AI reasons about and what the system enforces deterministically.

This is not an exotic idea. It is how every other complex operational system is designed.

A financial trading system doesn’t ask its algorithm to simultaneously optimise for returns, monitor risk limits, check regulatory compliance, and handle exception reporting in a single probabilistic pass. These are separate processes — with defined boundaries, explicit constraints, and oversight mechanisms at the handoff points.

The question every enterprise needs to be asking — and almost none are asking deliberately — is:

That last question is the most revealing. In most enterprise AI deployments right now, nobody designed the boundary. The AI was given a broad scope, connected to production systems, and asked to perform. The boundary between AI autonomy and human oversight — where it exists at all — is implicit, untested, and discovered only when something fails visibly enough to get noticed.

Vivint’s fix was instructive: they worked with Salesforce to implement “deterministic triggers” to ensure consistent survey delivery. They didn’t replace the AI. They designed the system properly around it. A deterministic rule enforcing a non-negotiable step. An AI handling the rest. A boundary between them that was explicit rather than assumed.

That is the pattern that scales. It just requires someone to actually design it.

When the Market Leader Says It Out Loud

After a year of aggressive AI-first positioning, Salesforce’s own spokesperson issued a statement worth reading carefully:

Set aside the defensive context in which this was said. Read it as a design principle. It is the right one.

The phrase that matters most is “business logic and governance.” Not better models. Not more training data. Not improved prompting. The gap between raw LLM capability and production-grade enterprise deployment is filled by governance infrastructure — the explicit design of what the AI can decide, what it cannot, who reviews consequential actions, and how the system learns from its own production behaviour.

This is not a new insight. It is, however, an insight the industry is only now saying out loud — because it took a year of production deployments, visible failures, and quietly disappointing ROI to make the cost of avoiding it impossible to ignore.

When the company that spent 2024 telling the world AI agents would transform enterprise operations arrives at this conclusion publicly, the conversation has genuinely shifted. The question now is whether the rest of the market takes the lesson — or spends the next year discovering it independently.

The Opportunity in the Correction

Salesforce’s partial retreat is not a signal that AI agents don’t work in enterprise operations. They do. Vivint’s survey problem was solved — not by abandoning AI, but by adding the right deterministic layer in the right place. The capability survived. The architecture improved.

The enterprises that get this right in the next 18 months will not be the ones with the most powerful AI. They will be the ones that build the operational infrastructure to deploy it with defined scope, measurable outcomes, and the governance layer that makes it trustworthy in production.

Not AI-only. Not rules-only. A deliberately designed operational layer that specifies what each component is responsible for — where AI provides the intelligence and flexibility, where rules provide the guardrails and consistency, and where humans provide the judgment that neither can reliably replicate.

Salesforce has now told us, at scale and in public, what happens when you skip that step.

4,000 people absorbed the cost of that confidence. The lesson deserves to be taken seriously.

A developer told an AI agent to migrate his infrastructure to AWS. The AI found duplicate resources. Decided they needed to go. Issued a destroy command. Two and a half years of production data — student submissions, course projects, leaderboards — gone in seconds. AWS restored it within a day. The internet had opinions.

The developer, Alexey Grigorev, founder of DataTalks.Club, recounted the incident publicly. His candour was commendable. He admitted to over-relying on the AI, bypassing manual review of destructive commands, and ignoring Claude’s own warning flags about the approach. Within hours, the verdict on social media had been rendered:

Fair. But here’s what bothers me about that framing: it treats this as a user error story.

It isn’t.

The Problem With “He Should Have Known Better”

Yes — the developer over-relied on AI. Yes, he proceeded despite Claude’s own warnings. Yes, in hindsight, the failure points are clear and individually avoidable.

All of that is also largely irrelevant to the bigger structural question.

Because in the next 18 months, millions of enterprises will put AI agents into production — customer operations, sales workflows, IT automation, financial processing. These agents will have real permissions, real access, real consequences.

And most of those organizations will not have a thoughtful developer who reads the flags, understands the risk model, and makes a deliberate judgment call — however flawed that call turns out to be. They’ll have something much more dangerous:

The question isn’t why one developer made a bad call under pressure.

The question is: what happens when this is the default operating model at enterprise scale?

The Quiet Version Is Already Happening

Most organizations won’t get an incident as clean as this one — with a clear cause, a recoverable outcome, and a developer willing to explain exactly what went wrong.

I’ve watched the quieter version play out in customer operations more times than I can count.

An AI handles an interaction it wasn’t designed for. Nobody catches it — there’s no oversight layer, just a dashboard showing 74% deflection. The customer churns. The ops team doesn’t connect it to the AI. The model gets credited for the deflection rate; it never gets debited for the damage.

No dramatic story. No AWS rescue. Just slow, invisible erosion of customer relationships — attributed to churn, to market conditions, to anything except the AI system that nobody is actually watching.

This is already the production reality in most enterprises that have deployed AI with any meaningful autonomy. The Grigorev incident is notable because it was sudden and recoverable. The version happening inside customer operations is slow and isn’t.

Sandbox AI and Production AI Are Fundamentally Different Problems

In a sandbox: a mistake is a learning. The failure is contained, reversible, instructive.

In production: a mistake is a consequence. A customer is lost. A database is gone. A policy exception has been made that cannot be unmade.

The difference isn’t the AI’s capability level. It’s what is built around it.

Moving an AI system from sandbox to production without changing the operational architecture around it — the oversight model, the authorization boundaries, the escalation paths — is like deploying any other powerful tool without the safety infrastructure that makes it safe to operate at scale. You wouldn’t do it with a surgical team. You wouldn’t do it with a financial trading desk. You don’t do it because the cost of failure in production is categorically different from the cost of failure in testing.

AI agents in production aren’t different. They’re just new. And their newness has become cover for avoiding a very old and well-understood design discipline: operational control architecture.

What Governance Actually Means in a Hybrid World

When I say governance in this context, I don’t mean compliance checklists. I mean something more fundamental: designing who decides what, and making that design explicit.

The developer in this story didn’t have that layer. He had the tool, the access, and good intentions. What he lacked was the operational architecture that would have caught the failure mode before it became irreversible.

Most enterprises deploying AI agents right now are in exactly the same position — just at larger scale, with more distributed accountability, and less visibility into where the failure mode lives.

Human-AI Collaboration Requires More Governance, Not Less

There’s a particular kind of optimism about AI that I encounter frequently in enterprise contexts: the belief that as models get better, governance requirements go down. That as the AI becomes more capable and more aligned, the need for oversight infrastructure reduces.

I think this gets it precisely backwards.

As AI agents get more capable — as they handle more consequential decisions, with more autonomy, in more production contexts — the governance infrastructure required to keep them operating safely gets more important, not less. The more powerful the agent, the more important the control layer.

The solution isn’t more careful prompting. It isn’t better models. It isn’t more cautious developers.

It’s treating AI agents like every other powerful operational resource: defined scope, audit trails, human review gates for irreversible actions, escalation paths that function before the crisis, and continuous feedback loops that make the system smarter over time.

We have always known this for surgical teams, financial traders, pilots, nuclear plant operators. The discipline exists. The methodology exists.

What’s lagging is the willingness to apply it to AI — because applying it requires admitting that the “just deploy it” mentality that works in sandbox environments has a real cost ceiling in production.

The Moment We’re In

Grigorev’s incident will be remembered, if it is remembered at all, as an early and recoverable example of AI production failure. He was lucky: his data was recoverable, his vendor was responsive, and he had the presence of mind to document and share what happened.

The next version of this story won’t be as clean. The organization will be larger. The AI will have access to more. The damage will be more distributed and harder to trace. The post-mortem — if one gets done — will point at human error, change management failures, vendor limitations. Everything except the absence of operational control architecture.

Every enterprise moving AI from pilot to production right now is navigating the same gap. The governance infrastructure for hybrid human-AI operations is not a future problem. It is a present one.

The question for every organization is simply whether they build it deliberately — before the incident — or scramble to reconstruct it after.