The principle: the assistant suggests, a person decides
In payments operations, the question is rarely whether a model can produce a good answer. It is whether you can show who approved an action, on what evidence, and what stops the system from acting on its own.
The design that answers those questions is simple to state. Separate what the assistant may read from what it may change. Reading a ledger entry, a settlement line or a procedure is low risk. Posting an entry, issuing a refund or releasing funds is high-risk. The simplest control is that the assistant has no tool that performs a high-risk action at all. It can prepare one, explain one and route one for approval. A person executes it, in the system that already has the right controls.
This guide describes the pattern we use for exception handling, disputes, merchant support and back-office investigations: the classification of actions, the building blocks, the evidence that makes it auditable and a rollout that earns trust step by step.
Classify every action first
Before any prompt is written, list the actions the use case involves and place each one in a tier. The tier decides what the assistant may do, not how confident the model is.
| Tier | Examples | What the assistant may do |
|---|---|---|
| Read | Look up a transaction, a settlement line, a merchant profile, a procedure | Call narrow, read-only tools with scoped permissions |
| Draft | Write a case note, a reply to a merchant, a summary for a supervisor | Produce the draft; a person edits and sends it |
| Recommend | Classify an exception, propose a resolution, suggest a chargeback response | Recommend with evidence; a person accepts or rejects |
| Act | Post a ledger entry, issue a refund, release a hold, change a limit, close a case | Nothing directly; the action stays in the source system behind its existing approval |
Most of the value sits in the first three tiers. Analysts spend much of their time gathering context and writing up what they found; an assistant that does that well saves hours without ever touching money.
Five building blocks
- Read tools and write tools are different things. Expose lookups as narrow, read-only tools, for example AWS Lambda functions behind an agent or a tool-calling application, each with its own IAM role that can read only what that tool needs. Keep write actions in your existing systems, behind their existing approvals.
- An explicit approval gate. When a recommendation needs action, the workflow pauses for a named person. AWS Step Functions supports this with a task token: the state machine waits until an approval callback returns the decision, with no polling and no timeout you did not choose.
- Evidence with every suggestion. Log the records consulted, the procedure retrieved, the model and prompt versions, the suggestion and the human decision. That log is what an auditor or a bank partner will ask to see.
- An evaluation set before launch. Build it from historical cases with known resolutions. Measure how often the suggestion matches what an experienced analyst decided, by case type, and decide in advance what level is acceptable for each type.
- Cost and safety limits. Set budgets per request and per day, cap the number of tool calls per request, and apply content filters such as Amazon Bedrock Guardrails where they fit the use case. A model's own confidence is not a control.
How a request flows
For an exception-handling assistant, a typical request runs like this:
- An analyst opens an exception. The application sends its ID, not its contents, to the assistant.
- The assistant calls read-only tools to fetch the exception, the related ledger, settlement and bank records, and retrieves the relevant procedure from an approved knowledge base.
- The model produces a structured result: likely cause, recommended action, the evidence it relied on and what it could not verify.
- The result is logged and shown to the analyst next to the source records, so the analyst checks the evidence rather than trusting the summary.
- If the analyst accepts a high-risk action, the request goes to the source system or to a Step Functions approval step, where a second person approves it when the amount or the action type requires it.
- The decision, and any edit the analyst made, are logged against the original suggestion.
The structured output in step 3 matters. A free-text answer is hard to evaluate and easy to over-trust. Fields such as cause, action, evidence and open questions can be measured, compared and displayed consistently.
The evidence log
Store one record per suggestion, in an append-only store such as an S3 bucket with Object Lock or a database table that the application can insert into but not update.
| Field | Why it is there |
|---|---|
| Case ID, timestamp, user | Who asked, when, about what |
| Records and documents retrieved | The evidence the suggestion was based on, by ID and version |
| Model ID, prompt version, tool versions | To reproduce the behavior and to compare versions |
| Structured suggestion | Cause, action, evidence, open questions |
| Guardrail and policy results | Whether any filter intervened and why |
| Human decision and edits | Accepted, changed or rejected, by whom, with a reason |
| Tokens and cost | To manage the budget per case type |
Avoid copying full card numbers or other sensitive values into the log. Reference records by ID; the source systems already hold the data under the right controls.
Building the evaluation set
An evaluation set turns "the demo looked good" into a number you can defend. Build it before launch and keep it up to date.
- Start from history. Take closed cases with known, correct resolutions, a few hundred if you have them, spread across the case types you plan to support.
- Include the hard ones. Ambiguous cases, cases where the procedure changed, and cases where the right answer was "escalate".
- Measure by case type. An overall accuracy hides the type where the assistant is consistently wrong.
- Score the evidence, not only the answer. A right answer for the wrong reason will fail on the next case.
- Re-run on every change of model, prompt, tool or procedure, and compare with the previous version before releasing.
| Measure | What it tells you |
|---|---|
| Agreement with expert resolution | How often the recommendation matches what an experienced analyst decided |
| Confident and wrong | The share of wrong answers presented without any stated doubt; the most dangerous kind |
| Correct escalation | Whether the assistant says "I cannot tell" when it should |
| Evidence accuracy | Whether the cited records actually support the conclusion |
| Time and cost per case | Whether the assistant is worth running for this case type |
Risks specific to generative AI
- Instructions hidden in data. Merchant messages, dispute documents and free-text fields can contain text that looks like instructions. Treat everything retrieved as data, never as instructions, and keep tools read-only so a manipulated answer cannot act.
- Stale or conflicting procedures. If the knowledge base holds three versions of a procedure, the assistant may cite the wrong one. Keep one approved version per procedure, with an owner and a review date.
- Over-trust. When suggestions are usually right, people stop checking. Show the evidence next to the suggestion, and sample accepted suggestions for review.
- Silent drift. A model update, a new procedure or a change in case mix can change behavior. Monitor agreement and override rates weekly, by case type.
- Cost creep. Long contexts and repeated tool calls add up. Track cost per case and set limits; our generative AI cost calculator helps estimate it.
The NIST AI Risk Management Framework and its Generative AI Profile are useful checklists for these risks, and many partners and auditors recognize them.
A staged rollout with exit criteria
Each stage has a condition for moving on, agreed before the stage starts.
| Stage | What happens | Move on when |
|---|---|---|
| 1. Shadow | The assistant produces suggestions that analysts do not see. You compare them with real decisions. | Agreement meets the target for a case type, and confident-and-wrong answers are rare and understood |
| 2. Assisted | Analysts see the suggestion and the evidence, and decide. You measure time saved and disagreement. | Analysts use it by choice, time per case falls, and override reasons are reviewed |
| 3. Selective automation | Only for low-risk, non-monetary steps with a strong track record, such as classifying and routing | Sampled review shows no rise in errors; switching it off is one setting |
Monetary actions stay with people. Some companies later automate very small, well-understood adjustments, but that is a separate decision with its own controls, not a natural next step of the rollout.
Ownership in production
An assistant in production needs an owner like any other system: someone accountable for its behavior, its cost, its evaluation results and its incidents. In practice that means a business owner for the use case, a technical owner for the platform, and a monthly review of agreement, overrides, cost and incidents. Without an owner, the assistant slowly drifts away from the procedures it was built on.
Choosing where to start
Not every operations task is a good first candidate. The best ones share four traits.
| Trait | Good first candidate | Poor first candidate |
|---|---|---|
| Volume | Hundreds or thousands of similar cases a month | A few complex cases a quarter |
| Context gathering | Analysts spend most of their time collecting records from several systems | The decision is quick once the case is opened |
| Known outcomes | History of resolved cases with clear resolutions | No record of how past cases were decided |
| Consequence | The assistant's output is reviewed before anything happens | The output would reach customers or move money directly |
Reconciliation exceptions, dispute preparation and merchant onboarding checks often fit all four. Customer-facing chat rarely does as a first project.
Questions to answer before you build
- Which actions are high-risk, and who may approve each one?
- Which data may the assistant read, and under which access controls?
- What does "correct" mean for each case type, and how will you measure it?
- What will you show the analyst besides the suggestion?
- Who owns the assistant in production, including its cost?
Answer these for one use case and you have a path to production. Skip them and you have a demo.
Questions we hear
Can we let the assistant close simple exceptions on its own?
Classification and routing can be automated once the track record supports it. Closing an exception that involves money should stay with a person, or with deterministic rules you can test, rather than with a model.
Do we need an agent framework?
Not necessarily. Many useful assistants are a fixed sequence of lookups followed by one model call. Use an agent only when the next lookup really depends on what the previous one found, and keep its tools read-only either way.
How big does the evaluation set need to be?
Large enough that each case type you support has dozens of examples, including difficult ones. A few hundred cases is a common starting point; more matters less than coverage of the hard cases.
What will a bank partner or auditor want to see?
The classification of actions, the approval steps, the evidence log, the evaluation results and who owns the system. If those exist, the conversation is short.
Should data leave our AWS account?
Using models through Amazon Bedrock keeps requests within AWS under your account's controls. Check the data terms of whichever model provider you use, and keep sensitive values out of prompts where an ID will do.