Who Is Accountable When AI Makes Decisions?

When an AI-assisted process approves a claim, prioritises a request or drafts a reply to a customer, a decision has been made. If it is wrong, someone will ask who decided. "The system did" is not an answer customers, regulators or boards accept — nor should they.

Decision, recommendation or draft?

The first design choice is what the system is allowed to do. Producing a draft that a person approves is very different from acting automatically. Many organisations blur this line over time: a tool introduced as an assistant gradually becomes the decision-maker because reviewing every output feels slow. That drift should be a conscious choice, not an accident.

Name the owner

Every automated decision type should have a named business owner — a person or role who understands the rules, can explain an outcome and can change or stop the process. Technical teams operate the system; they should not be left owning business judgements by default.

Keep the evidence

Accountability depends on being able to reconstruct what happened: which inputs were used, which rules or model version applied, whether a person reviewed it and what they decided. Without that record, even a correct decision is hard to defend, and an incorrect one is hard to learn from.

If you cannot explain a decision afterwards, you did not really control it beforehand.

Regulation is moving in this direction

European and national rules increasingly expect organisations to understand and document how automated systems affect people, particularly in sensitive areas. The specific obligations depend on the use case and jurisdiction and should be assessed with qualified advice. The practical principle is stable regardless: design for explanation, oversight and correction from the start.

A simple accountability map

• What decisions does the system influence, and how much autonomy does it have?
• Who owns each decision type, and who can override it?
• What is recorded, for how long, and who can access it?
• How does an affected customer or employee raise a concern — and get a human answer?

Distinguish identity, permission and business authority

Knowing which agent made a request is not the same as knowing whether it was allowed to make it. Identity identifies the actor. Permission controls access to a system or resource. Business authority defines the purpose and limits of an action: whose instruction it serves, which commitments it may make and when that authority expires. An assistant able to read a customer record should not inherit permission to change a contract simply because both actions are technically available.

For each action, write a short authority record: the responsible business owner, the permitted purpose, the allowed systems, the approval boundary and the revocation path. In an illustrative procurement workflow, suggesting a supplier, preparing a purchase order and releasing a payment are separate authorities. Treating them as one broad capability makes an accidental escalation easier and a later explanation harder. This is a design example, not a claim about a deployed client system.

Put intervention where it can still change the outcome

Human oversight is weak if the reviewer sees an outcome only after it has become irreversible. Place review before a consequential action, or provide a genuinely usable pause, reversal or containment mechanism. Specify what causes escalation: missing evidence, conflicting rules, unusual commitments or an action outside the permitted purpose. A vague instruction to use judgement is not enough for an automated process, and a nominal approver without time or context is not effective control.

• Before action: check purpose, authority, required evidence and any mandatory approval.
• During action: limit reachable systems and transaction scope; surface failures rather than silently retrying a consequential step.
• After action: retain an appropriate record, review selected cases and route affected people to correction.
• On incident: name who can stop the process, contain effects, notify the right internal owner and authorise restart.

Make the record sufficient, not indiscriminate

An audit trail should answer a defined question. Recording everything forever can expose confidential information without improving accountability. Decide which inputs, source versions, approvals and action references are needed to reconstruct a decision, then set access and retention appropriate to the use case. Sensitive raw prompts may not belong in a broadly accessible log. Work with qualified privacy and legal advisers when the process affects individuals or regulated activity; this framework is not a determination of statutory obligations.

Use one control map for overlapping obligations

Different obligations may require similar evidence of access control, change management and review. Map the operational control once, identify the owner and record which requirements it supports. This is not a claim that one checklist satisfies every law. It is a way to avoid separate teams collecting inconsistent accounts of the same process. Test the map with a walkthrough: pick a disputed outcome and ask whether someone can explain it, correct it and show who authorised the relevant action.

Authority also depends on knowing which rule is current: Your AI Is Only as Good as Your Business Knowledge.