Your AI Approval Log Needs More Than “Human Reviewed”

Consider one accounts-payable workflow. A model flags an invoice as a probable duplicate, the payment system places it on hold, and an employee must either keep the hold or release the payment. The reviewer opens the approval screen, notices that the later invoice replaces a canceled invoice and clicks “approve.”

A log containing the employee’s name, the time and the word “approved” proves that someone clicked a button. It does not show whether the employee saw both invoices, noticed the cancellation, checked the relevant policy or changed anything the model extracted. During an incident, the organization cannot distinguish an informed override from a habitual click.

The fix is not a longer comment box. The approval screen and its underlying record need to capture the decision as a sequence: what the system recommended, which evidence it presented, what the reviewer corrected, which rule controlled the decision and what action the downstream system took.

Start with a requirement the system can enforce

“Human review required” is too vague for engineers and too weak for auditors. It names a participant but says nothing about the required behavior or evidence.

For the invoice hold, an internal control could say:

Before a model-assisted payment hold is released, the reviewer must record the evidence displayed, any correction made, the policy rule applied and the reason for an exception. The record must bind those entries to the model output, policy version and final action.

That is proposed control language, not a quotation from a law or standard. Its value is testability. A product team can prevent release when required fields are absent, while an auditor can sample approvals and check whether each record contains the specified evidence.

External requirements may set a broader constraint. Article 14 of the EU AI Act says certain high-risk systems must let human overseers “correctly interpret the high-risk AI system’s output” and, in a particular situation, “disregard, override or reverse its output.” The Act does not make an approval comment box sufficient by itself, and its obligations apply according to the law’s classification and implementation schedule. Whether a specific invoice tool falls within that regime requires a separate legal assessment.

The engineering lesson is narrower: when a policy promises interpretation and override, the interface must expose enough evidence to support both, and the log must show that the controls were available and used. A written policy cannot establish that after the fact.

Preserve the evidence that appeared on screen

The reviewer in the invoice case might see supplier identity, invoice number, purchase order, amount, payment account, extracted line items, the earlier invoice and the model’s duplicate score. Recording that these fields existed somewhere in a database is insufficient. Values can change after approval, and a later investigator needs the version presented at decision time.

Create an evidence snapshot, meaning a stored representation of the relevant values as displayed for that review. It should carry stable references to the source documents, extraction version, model output and applicable policy. If retaining full documents would duplicate sensitive data, store protected references plus the displayed fields and integrity information. A cryptographic hash, a value used to detect later alteration, can show that bytes changed; it cannot prove that an invoice was genuine or that the reviewer read it.

The interface should also record evidence availability. A link hidden behind an unopened tab is not equivalent to a document displayed beside the recommendation. Capture whether the reviewer opened the earlier invoice, viewed the cancellation notice or expanded the model’s explanation. Do not turn every mouse movement into compliance theater.

Record the interactions that correspond to evidence named in the control.

This creates storage and privacy costs. Snapshots may retain personal or financial information longer than operational systems otherwise would, so access controls and retention periods should follow the sensitivity of the decision. Logging everything indefinitely is not stronger governance.

Make corrections first-class events

Suppose the model extracted the replacement invoice number incorrectly, causing the duplicate flag. If the reviewer edits the number and releases payment, a final record that shows only the corrected value hides the model’s failure. The audit trail should preserve the original extraction, the replacement value, who changed it and whether the correction was sent back to the source system.

Use an append-only event history, a log where new events are added without overwriting earlier ones. The sequence for this case could show the model recommendation, the evidence snapshot, the reviewer’s field correction, the selected policy rule, the override decision and the payment system’s response. Corrections to the record should appear as later events rather than silent edits.

That sequence supports more than blame assignment. Teams can find recurring extraction errors, compare reviewers and detect whether one supplier format repeatedly triggers false duplicate flags. If the system stores only final approvals, those operational defects disappear into apparently successful human oversight.

Do not force reviewers to retype information the system already knows. The interface can generate the event identifiers, model reference and policy version automatically. Human effort belongs on the parts automation cannot establish: confirming the relevant evidence, correcting facts and explaining departures from the normal rule.

Separate an override from an exception

An override changes the model’s recommendation. An exception permits an action that the governing policy would normally block. They are not interchangeable.

In the invoice workflow, releasing a payment after determining that the duplicate flag was wrong is an override. Releasing it even though required supporting documentation is absent is an exception. The second decision needs an authorized exception category, an explanation tied to the missing requirement and, where policy requires it, approval from someone with separate authority.

A generic free-text field blurs these paths. Reviewers copy old language, write “business need” or describe the result without naming the rule they departed from. A better screen asks the reviewer to select the controlling policy rule, then opens an exception path only if the final action conflicts with that rule. Free text remains useful for facts that structured fields cannot express, but it should supplement the decision structure rather than replace it.

The system must enforce authority at the moment of action. If only a payments manager may accept a missing-document exception, the workflow should route the case to that role instead of letting any reviewer click through and relying on a later audit to discover the breach. The record should show the authority checked, the person who exercised it and the resulting system action.

Bind approval to the action that followed

Approval records often stop too early. They say the reviewer released the hold, while the payment service later rejected the command, retried it or sent a different amount after another system updated the invoice.

Bind the review decision to a transaction identifier used by the downstream service. The log should record the requested action and the confirmed result without pretending they are the same event. If the evidence changes after approval, invalidate the approval or require a new review according to a defined materiality rule. A changed payment account should trigger another review; a corrected display label may not.

This binding is what makes the invoice record reconstructable. An investigator can move from the payment back to the approval, then to the exact recommendation and evidence snapshot. Without those links, teams search several systems by approximate timestamps and hope that records align.

Test for rubber stamping

More fields can produce longer rubber stamps. A reviewer may select the first reason code, paste a standard sentence and approve every case without examining the documents.

Test behavior against the control. Sample records where reviewers overrode the model and verify that the cited evidence supports the correction. Look for repeated rationales across materially different cases, approvals completed without opening required evidence and accounts that approve nearly every exception presented to them. These patterns are indicators for investigation, not proof of misconduct.

Run controlled review exercises as well. Seed a test case with a visible conflict between the model output and source document, then check whether the reviewer identifies it and whether the record captures the correction. Use training or evaluation environments rather than inserting deceptive transactions into live payment queues.

There is a real throughput cost. Evidence review takes longer than a confirmation click, exception routing delays payment, and maintaining linked records adds engineering and storage work. Reserve the strongest controls for decisions with meaningful financial, safety, legal or access consequences. A detailed attestation for low-impact text formatting is usually not worth the interruption.

Questions people ask

Is a reviewer’s name and timestamp enough for an audit?

Usually not if the control promises substantive human oversight. Those fields establish identity and timing, but they do not show the model output, evidence presented, correction made, policy applied or action authorized. The required record depends on the decision and applicable rules, but it must support reconstruction of the claimed review.

Should reviewers be required to write a justification every time?

Require human explanation where judgment matters, especially for overrides and policy exceptions. Routine confirmations can use structured reason codes when the system already preserves the supporting evidence and rule. Mandatory prose on every approval often produces copied language without improving the decision record.

Can interface telemetry prove that someone reviewed the evidence?

It can show that a reviewer opened a document or expanded a panel, not that the person understood it. Treat telemetry as corroborating evidence. Pair it with the reviewer’s correction, selected rule, authority and final action rather than using clicks as a proxy for comprehension.

What should happen if evidence changes after approval?

The workflow should compare the changed field with a predefined materiality rule. A new bank account or altered amount should invalidate the invoice approval and start another review, while a nonmaterial display correction may remain attached to the existing decision as a later event.