Your AI Is Only as Good as Your Business Knowledge

A capable language model can write fluently about almost anything. What it cannot do on its own is know your pricing exceptions, your customer commitments, the reason a certain supplier is handled differently or which policy document is the current one. Without that knowledge, it produces answers that sound right and may not be.

Knowledge lives in more places than documents

Some business knowledge is written down. Much of it lives in people's heads, in email threads, in the way a senior colleague handles a difficult case. Before AI can use this knowledge, it has to be identified, made explicit and kept current. That is organisational work, not a technical upload.

Outdated knowledge is worse than none

If an assistant draws on three versions of the same policy, it will sometimes use the wrong one — confidently. Deciding which sources are authoritative, who maintains them and how retired content is removed matters more than the sophistication of the retrieval technology.

The question is not whether your AI is smart. It is whether it knows what your best people know — and only what is still true.

Start narrow

Rather than connecting everything at once, choose one well-bounded area where the knowledge is reasonably documented and the owner is clear. Test whether answers are correct against cases experienced staff can judge. Expand only once that area is reliable and maintained.

Protect what should stay protected

• Decide which information must never be exposed to a given user group.
• Understand where data is processed and stored by any external provider.
• Respect confidentiality obligations to customers and partners.
• Keep the ability to remove knowledge as easily as you add it.

Treat business context as a maintained product

An information collection becomes useful business context only when people know what it means, when it applies and who can change it. A discount policy needs more than a document title. It needs an effective date, permitted exceptions, an owner and a rule for conflicts with customer-specific commitments. A retrieval system can find text without resolving those questions. Leadership should therefore fund the work of maintaining context alongside the tool that consumes it, rather than treating the first upload as completion.

Build a minimum viable knowledge contract

• Scope: state which questions the assistant may answer and which it must refer elsewhere. A narrow promise is easier to test than general business competence.
• Authority: identify the controlling source, its owner, effective date and precedence when documents disagree.
• Access: define who may see each source and derived answer; test that summaries do not reveal restricted facts.
• Freshness: set how changes are approved, propagated and retired, including what happens while an update is incomplete.
• Uncertainty: specify when the assistant must abstain, ask a question or hand the case to a person with the relevant authority.

Consider a hypothetical service assistant asked whether a customer can cancel without a fee. A general policy may say one thing, an older email another and a negotiated agreement something else. The right answer depends on precedence and access, not fluent wording. If the authoritative agreement is unavailable, the safe response may be to route the case for review. Filling the gap with a plausible answer is not assistance; it is an undocumented commercial decision.

Discover the exceptions before they become failures

Ask experienced colleagues to walk through difficult cases and explain where they departed from the written procedure. Do not automatically encode every workaround. Some are legitimate exceptions; some compensate for a broken process; some should stop. Classify them, agree the intended rule and record who approved it. This converts tacit practice into explicit decisions without pretending that all existing behaviour is correct. It also reveals where the organisation itself disagrees, which an assistant cannot responsibly settle on its own.

Test knowledge boundaries, not just answer fluency

Create a review set with ordinary questions, obsolete-policy traps, conflicting sources, restricted-information requests and cases that cannot be answered from available evidence. Record the expected basis for the answer and the permitted response, including abstention. Have a knowledgeable reviewer judge correctness and appropriateness separately. An accurate fact delivered to the wrong user is still a failure. A refusal can be the correct outcome when required information is missing or access is not authorised.

Make change and exit part of the design

When a policy changes, identify the answers it affects, retire the old material and rerun the relevant review cases. When a provider changes, confirm that access boundaries and abstention behaviour still hold. Preserve a portable record of authoritative content, ownership and tests so the business knowledge is not trapped inside one product. The executive decision is then concrete: expand only when the bounded area is reliable enough for its consequences and someone is accountable for keeping it that way.

Define who can approve rules and intervene in actions: Who Is Accountable When AI Makes Decisions?