Any enterprise will tell you that the chasm between a chatbot in production and one on demo is where budgets are squandered. The demo version will handle three sample questions with ease. But put it in the field for six months and the same bot is quietly sending customers in circles, exposing documents it has no business showing, and handing your support staff an entirely new set of tickets to sort out. That is the crux of any frank discussion on the topic.
We are not talking about model quality as a limiting factor anymore. Take Klarna’s OpenAI-driven assistant: 2.3 million conversations in month one and average handling time trimmed from 12 minutes to less than 2. Or IBM’s AskHR, which has seen 11.5 million interactions and a 94% containment rate. The reason for those figures is that the teams behind them view the bot as part of their operational fabric, not some chat window tacked onto a site. Skip that discipline and the results are what John Iosifov describes on X, with voice agent containment rates falling from the 50-87% range in a pilot to 20-35% at scale.
For the product leader or operator who is done asking “should we do AI?” and is faced with the tougher questions – should we build or buy? What must the bot be barred from doing? – this is meant for you.
Where an Enterprise Chatbot Actually Earns Its Keep
There is a pattern to the deployments that make it into the public record. Klarna focused on customer service. Sephora on product discovery via Messenger. Barking & Dagenham council on constituent matters and turned in a 533% ROI in nine months. The successful ones do not try to be an assistant for everything; they pick a high-volume workflow the business is already familiar with and put boundaries around it.
Find a process that is repetitive, whose answer is in systems you have, and where there is an obvious path for escalation if the bot is unsure. Tier-one support, IT helpdesk, order status and HR policy lookups all qualify. A general “ask us anything” proposition does not. The economics are clear from published data: $0.50 for a bot versus $6-$8 for a human, with a 30-40% drop in support costs for a well-scoped rollout. Of course, that assumes actual integration. If you cannot point to the specific workflow and the metric that will move, then you are ready for scoping, not for a purchase.
If the bot cannot take a repeated task off a real queue, it is nothing but decoration.
The purpose of a good chatbot is to let people get on with the work only they can do, much as the customer-obsessed support teams still do. It is there to eliminate the repeats so the human is free for the customer who wants to reach one.
What Makes an Enterprise Chatbot Different From a Chat Widget
A public FAQ bot can run from a script. An enterprise one has to pull off three distinct jobs. Knowing the difference is how you avoid making the wrong investment.
Then there is the conversation layer – the interface, the tone, intent detection. In 2026 it is the least of your worries; every vendor does it well enough.
More important is the knowledge layer. This is where retrieval-augmented generation comes in and where the “add your docs” sales pitch tends to break down. You want the bot to ground its response in approved material, with a citation to the passage. Most fail when the retriever does not respect permissions. A semantic search is not permission-aware. If the bot can access an HR file, it had better follow the same rules as the HR system. Do not or you have engineered a leak with a pleasant face.
And the action layer, where the bot opens a ticket or triggers a request inside your systems. That is where the risk profile shifts. An incorrect answer is one thing; a wrong action is another. For those considering tool-using agents, our AI agent development guide goes into the architecture.
What to ask in a vendor demo
Put a few hard questions to a vendor and the sales talk will evaporate. Where did that answer come from, show me the source. What is the protocol when a user is not cleared to see the document? And if the bot is to act, what is the rollback story? If they consider an “escalate to human” a failure, the product is not fit for an enterprise.
Build, Buy, or a Deliberate Mix
You will see the build-versus-buy debate mismanaged on both sides. Some teams will buy quickly for a problem unique to their business and end up customising it for a year into something inferior. Others will build from scratch for a common issue that would have been solved by an off-the-shelf platform, wasting a year to put together a conversation layer. Forget the old build versus buy debate. The more practical way to look at it is to buy what is ordinary in shape and put your own work into the parts that set you apart.
| Decision factor | Buy a platform | Build custom |
|---|---|---|
| Time to first value | Weeks for standard support use cases | Months, because discovery and integration are real work |
| Integration depth | Fine when your CRM, ticketing, and IAM match the vendor’s connectors | Necessary when systems are custom, messy, or regulated |
| Control over behavior | Constrained by the vendor’s prompt, model, and evals | You own the guardrails, escalation, and confidence thresholds |
| Compliance posture | Depends on vendor’s certifications and data handling | Tailored to your access model and audit requirements |
| Long-term cost shape | Predictable subscription, unpredictable at scale | Higher upfront, cheaper per interaction as volume grows |
| Vendor lock-in | Real, and painful when the model or pricing changes | Lower, if you keep prompts, evals, and retrieval separable |
Many teams find a good rhythm in this: let someone else provide the conversation and orchestration layer, then build out the retrieval and action layers on top of your systems of record. That keeps the commodity elements as they should be and leaves the business logic where it belongs with you. Should you be looking for outside hands on the custom side? We have put down some thoughts on what to look for (and what to avoid) when hiring an AI chatbot development company.
You will also notice the nomenclature is changing in a hurry; “chatbot” is giving way to “agent” so quickly vendors are rebranding their wares. Some of it is warranted, some is not. When a sales deck gets vague on the point, our AI terminology cheat sheet is handy to have on hand.
The Governance Work That Actually Prevents Incidents
Do not make the error of thinking governance is just compliance paperwork. For an enterprise bot, it is the very design of the system. It is the thing that prevents a document from being leaked or a prompt injection in a retrieved file from coaxing the bot into breaking its rules.
These five controls need to be there in v1, not v4:
- Identity-aware retrieval. What the user can read in the source system is what the bot can read. Enforce that at the retrieval level.
- Action boundaries that are explicit. Make a list of every tool and argument available to the bot. Any write action with legal or financial consequences – a refund, a change in permissions – should be drafted, not executed, without human sign-off.
- Defences against prompt injection. A piece of malicious text in a document can override the system prompt if you are not careful. Sanitize your inputs and isolate tool calls from anything the user can control. Test for it before you go live.
- Audit trails. You need to see every tool call and source cited in every conversation. This is what allows for tuning and makes incident response bearable.
- An evaluation harness for real. Put together some offline test sets with adversarial cases and use production telemetry to check your hallucination and containment rates. A demo is not the same as production.
The figures for voice agents are plain about it. A pilot might boast 50 to 87 per cent containment, but put real traffic on it and you will see that fall to 20–35 per cent. Add in the 40–60 per cent failure rate on CRM lookups and IVR handoffs, and context lost in an escalation two-thirds of the time. That is not a model issue. It is one of integration and evaluation.
A Roadmap That Survives Contact With Reality
Be narrow at first. Instrument everything. Only expand when the data supports it.
1. Discovery and scoping. Choose a single workflow and identify the systems involved and the current cost per interaction. Put a baseline on the metric you intend to improve, be it resolution rate or average handling time. If you have no baseline you can never claim the bot was any good.
2. An unexciting pilot. There is value in building the smallest useful version with a clean path for escalation and genuine integrations to a system of record. Put it in front of a small audience. We did just that with the journalism training tool we made for El Colectivo 506; the aim was a well-defined coaching workflow within a methodology, not to be a generalist assistant.
3. Integration and rollout. Get the systems connected and train the people who will be responsible for the bot. An AI rollout is an organisational project as much as a software one. Do it in phases with a KPI and an owner. As we found with our enterprise workflow automation work, automation has a habit of exposing bad process before it can fix it.
4. Post-launch. Go through the transcripts and patch up the weak points. Do not add another workflow until the first has been through its paces. Those that do not bother end up with bots that become nothing more than churn drivers. And if the bot is to operate inside your operational systems, treat it accordingly; our ERP AI bot guide goes into that discipline in greater detail.
The Decision, Written Down
Put pen to paper before your next planning session. One page should have the workflow, the systems, the metric to be moved and the person in charge. Leave any of those blank and the project is not ready, vendor demo notwithstanding.
Then be honest with yourself as to which pieces are yours and which are not. Buy the platform if it suits your integrations. Build what reflects your risk tolerance and your permissions. A pretty conversation layer does not make a useful bot; the bot has to earn its keep in the action and retrieval.
Refact’s AI chatbot development practice is for those who want a partner to make those early decisions on what to build and what to leave out, well before production code is written.
Saeedreza Abbaspour is the CEO of Refact, where he works across product, engineering, and sales. He sets the studio’s direction while staying closely involved in the work itself, from shaping product strategy and UX architecture to helping define the technical systems behind Refact’s projects. His role connects business thinking with hands-on product execution, giving him a practical view of how software should be planned, built, launched, and improved. At Refact, Saeedreza focuses on building a studio that can move quickly, solve real client problems, and turn ideas into reliable digital products.
More from Saeedreza Abbaspour
