The bot is live. After three weeks nobody uses it any more.
That is the most common trajectory, and it rarely has anything to do with the model. The technology works. Answers come back quickly, they read well, and too often they are unusable. After that people return to what worked before: messaging a colleague.
A chatbot gets two attempts
That is the number to plan around. Users test a new system twice, maybe three times. If it produces an answer in that window that later turns out to be wrong, it is burnt. Not as a tool, but as a habit. Winning that back is close to impossible, because nobody tries a fourth time.
The cause usually sits upstream of the project. A chatbot answers from what you give it, and in most companies that is the accumulated state of the repository: partly stale, partly contradictory, and in essential parts not present at all. 42 percent of job-related knowledge is documented nowhere (Panopto/McKinsey). That is precisely the part needed for the questions that actually escalate.
The model will answer anyway, because that is what it is built for. The consequences are measured: 47 percent of enterprise AI users made at least one important business decision based on hallucinated content. And 4.3 hours per person per week go on re-checking AI answers, roughly 14,200 dollars a year. You then pay for acceleration and finance verification.
The upstream decision that determines everything else
Before you talk about models and interfaces, settle one question: is the bot meant to work on documents or to answer questions? Those are two different products, and they get confused regularly.
Document work means summarising, rephrasing, structuring, calculating in tables. The source is the document the person already has open. Risk is low, benefit is immediate, no separate project needed. An assistant inside your existing environment is enough.
Answering means: how do we proceed in this case? Now the source is your company knowledge, and its condition decides everything downstream. Which route holds when is set out in the comparison of Microsoft Copilot against custom enterprise AI.
Almost every failed project promised the second and built the first.
The building blocks of a solid setup
1 One use case with measurable benefit
Take an area where the same questions escalate regularly: support answers for a product line, decision support in an operational process, ramp-up in a key role. The use case determines which knowledge you need and bounds the effort. Covering the whole company at once sounds ambitious and produces the same mediocre quality everywhere.
2 A knowledge layer instead of a file list
This is the difference between a prototype and a rollout. Every knowledge object needs an owner, meaning a named accountable person, a validity date, a verification status and a record of provenance. That lets retrieval prioritise rather than guess, and lets the answer state its basis.
Only under that condition does the documented effect apply: 40 to 71 percent fewer hallucinations through RAG with validated context (meta-analysis 2024). The qualifier validated is the entire point. Why prototypes work and rollouts fail is set out in the piece on RAG architecture for enterprise.
3 Active capture for what is missing
A validation layer only checks what exists. The documented gap needs a different mechanism. AI-supported expert interviews get decision logic, exceptions and warning signals out of people's heads and introduce them as structured objects into the same layer. Consolidation runs along five dimensions: correctness, currency, relevance, freedom from conflict, completeness.
4 Connection to what already runs
The bot has to sit where the work happens. Via RAG API, REST API and MCP server the knowledge layer delivers context to your existing systems, including SharePoint delivery and Microsoft Copilot as a context consumer. A separate portal nobody opens is the second most common cause of an idle bot.
Adoption is not a communications topic
This part is missing from almost every project plan. Your chatbot does not compete with the previous system. It competes with a message to a colleague who gives a reliable answer in two minutes. Against that comparison, any system that is right 80 percent of the time loses, because 80 percent means the person has to check every answer. And then asking is faster.
What shifts this is less communication than product design. Every answer has to carry its source, not as a footnote but so that checking takes seconds. And the system has to be able to say it has no reliable basis. A bot that stays quiet when uncertain earns trust; one that always answers loses it. Both sound technically minor and they decide the outcome.
One risk is running regardless: over 50 percent of employees use private GenAI for work without oversight (Gartner, February 2026). As long as the internal route gives worse answers than the private tool, governance loses to convenience. A working internal bot is therefore also a security measure.
Two metrics, and one that misleads
The usual metric is the adoption rate. In the first weeks it is driven by curiosity and then falls, so it says nothing about quality.
What carries meaning is repeat use after eight weeks. Anyone still using the bot in week eight is using it because it works. The second value is the reduction in queries to experts, typically 20 to 40 percent where the knowledge base holds. That is an experience value, not a commitment.
And a warning signal: if the share of answers marked wrong or incomplete sits near zero, nobody is checking. That is not proof of quality, it is an indication that people are taking answers without looking at them.
What European organisations have to settle additionally
Two regulations are in force and belong in the architecture decision rather than in acceptance testing: EU AI Act Article 4 on AI literacy obligations since February 2025, and NIS-2 since December 2025. Both concern traceability and governance, not only hosting.
Relevant checkpoints: EU hosting on Azure with data centres in Germany, GDPR compliance, TISAX certification, no use of customer data for model training, no data transfer to the US, complete and demonstrable deletion, role-based access control with an audit trail.
For capture projects, employee representation belongs at the start. Voluntary participation, transparency about purpose and use, role-based access with an audit trail: that carries the conversation. The framing is decisive. This is about role knowledge, not the assessment of individuals.
The question log
Before you pick a model, you need a diagnosis. You can get one from material that already exists.
Pull the last 50 questions that escalated in your operation. From the ticket system, from the support inbox, from the messages sent to the two people where everything lands. Assign each question to one of four causes: the information was documented nowhere. It was stale. Two contradictory versions existed. Or it was there, correct and current, and nobody found it anyway.
Only the fourth category is a search problem, and only that one a chatbot solves on its own. The other three need work on the sources first. How your 50 questions distribute across those four buckets is therefore both your diagnosis and your project plan.
Most teams doing this for the first time find the smallest number in the fourth bucket. That is uncomfortable, but it is good news: it explains why the last attempt failed, and it shows where to start this time.
It also answers the question from the beginning. If your bot only gets two attempts, pick the area where you will win those two attempts reliably. The rest follows later.
