An AI Denial Notice Needs More Than One Top Feature

Take an ordinary online auto-loan application. The lender collects income, employment, requested loan amount and information from a consumer report. Software transforms those fields, a model produces a risk score, and a policy engine compares that score with approval thresholds. A separate rule may stop the application because income could not be verified.

The applicant receives a denial notice naming high revolving-credit utilization as the principal reason. That phrase came from a feature-importance tool, which ranked the inputs that most affected the model’s score. It sounds precise. It may still describe the wrong part of the workflow.

If the failed income-verification rule controlled the outcome, reducing card balances would not have changed this decision. The useful record is the verification result, including the income value submitted, the source consulted, the matching rule and any error response. The model’s top feature belongs in the technical log, but it does not answer the applicant’s immediate problem.

That mismatch is the governance issue. Explanation software usually describes a calculation. An adverse-action notice needs to describe why the organization took the action.

The legally relevant reason is not always the model reason

For US credit decisions, the Equal Credit Opportunity Act and Regulation B impose a concrete requirement. Regulation B says:

A statement of reasons for adverse action shall be sufficient if it is specific and indicates the principal reason(s) for the adverse action.

The same provision says that references to internal standards or a failure to achieve a qualifying score are insufficient. That is an enforceable requirement, not a proposed transparency principle.

The Consumer Financial Protection Bureau has also stated through circulars that creditors do not receive an exception because an algorithm is complex or opaque. Its circulars communicate the agency’s interpretation and enforcement position; they do not replace the statute or regulation. The operational message is still direct: a creditor that cannot produce accurate, specific reasons has chosen a system it may be unable to use compliantly.

A different federal requirement can appear on the same notice. When adverse action is based on information from a consumer reporting agency, the Fair Credit Reporting Act generally requires information about that agency, a statement that the agency did not make the decision, and notice of rights to obtain and dispute the report. When a credit score is used, additional score and key-factor disclosures may apply.

Those key factors are easy to confuse with Regulation B’s principal reasons. They are not interchangeable. A credit score factor can explain what depressed a bureau score, while the creditor’s adverse-action reason must identify why the creditor denied this application. In the auto-loan workflow, utilization might be a valid score factor even though failed income verification was the controlling reason.

Feature importance answers a narrower question

Feature importance assigns influence to model inputs. A local explanation method, meaning one calculated for a single prediction, may report that revolving utilization pushed this applicant’s risk score farther toward denial than other model fields.

That result depends on the explanation method and its reference point. SHAP, a common method based on comparisons among possible feature combinations, measures how inputs contribute relative to a baseline. Change the background population, data transformation or grouping of correlated variables, and the ordering can change even when the model prediction does not.

Importance also does not establish causation. Two fields may encode overlapping information, leaving the tool to divide influence between them in a way that looks unstable to a reader. A derived feature such as recent credit-seeking intensity may combine several underlying records, while the notice exposes only the derived label. The person cannot tell which inquiry, date or source produced it.

Direction matters too. A field can rank highly because it strongly supported approval, not because it caused denial, unless the explanation pipeline preserves the sign and translates it correctly. Global importance, which summarizes behavior across many cases, is even less suitable for an individual notice. It describes the model population, not this application.

Then there is the rest of the decision stack. Eligibility rules may run before scoring. Pricing and fraud systems may run afterward. A human reviewer may override a recommendation, or the application may fall into a manual queue that closes after required documentation never arrives.

Reporting only the model’s top feature erases whichever component held final authority.

The auto-loan notice therefore needs a controlling-reason check: if the income-verification rule would have produced denial regardless of the model score, that rule outranks utilization in the explanation of the action.

Build the notice from the decision record

A defensible notice starts with a decision record, an event-level account of the inputs, transformations, model result, policy rules and human actions used for one outcome. This is more than a generic model card or a copy of the application.

First, preserve the facts as the system saw them. The record should identify the submitted income, the consumer-report fields returned, the source and retrieval time, and whether any values were missing, imputed or replaced with defaults. Raw input alone is insufficient when the model consumed a transformed value such as utilization calculated from balance and limit fields.

Next, capture the execution path. Store the model and policy versions, the score produced, the threshold applied, each rule result and the final decision owner. If a verification service timed out and the policy treated no response as failure, the log must distinguish that event from a confirmed mismatch. Otherwise the notice may convert a technical failure into an assertion about the applicant.

Finally, record how the notice reason was selected. A reason-code mapper translates technical outputs into language suitable for a notice, but it should choose only among factors that affected the real outcome. The system should test counterfactual relevance: if this item changed while the controlling rule remained fixed, could the result change? That test will not solve every explanation problem, yet it catches the obvious error of citing a model feature when a separate gate made the model irrelevant.

The applicant does not necessarily need source code, model weights or every internal fraud signal. The operator does need enough retained evidence to reconstruct the decision and investigate a dispute. A usable notice can then expose the pertinent record in recognizable terms: which income figure could not be verified, what type of source failed to confirm it, and how to submit corrected information where a review channel exists.

Better records cost more than generated prose

Feature-importance output is attractive because many model-development platforms can produce it after scoring. Generating an explanation may add computation, especially for methods that evaluate many feature combinations, but it can usually run after the decision rather than delaying the application screen.

The expensive work sits elsewhere. Teams must preserve versioned data, connect policy-engine events to model logs, test reason mappings and prevent later database updates from overwriting the evidence used at decision time. Human overrides need structured reason fields rather than free-form notes that cannot be audited consistently. Vendors must return stable identifiers and error states, not only a pass-or-fail result.

There is an accuracy tradeoff as well. A highly complex model may improve prediction while making reliable reason mapping harder, particularly when engineered variables are difficult to translate into source facts. If the organization cannot connect those variables to understandable records, the model’s marginal performance may not be worth the compliance and dispute-handling burden. More fluent notice text does not repair that gap.

A practical predeployment test uses complete application replays. Run a set of historical or synthetic cases through the model, verification services and policy engine, then ask reviewers to reproduce every notice reason from retained evidence. For the auto-loan case, a reviewer should be able to see that income verification controlled the denial and that utilization affected only a score that never became decisive.

The same test travels, but the rules do not

Insurers, landlords and employers also combine models with eligibility rules, external reports and human review. The technical requirement for traceability travels well: identify the decision component that controlled the outcome, preserve its inputs and link the explanation to records that can be corrected.

The legal notice duty does not travel unchanged. Federal consumer-reporting requirements can apply to employment, housing or insurance decisions that rely on consumer reports, while other explanation, disclosure and appeal obligations depend on the jurisdiction and activity. A lending notice designed around Regulation B should not be copied into hiring software and labeled compliant.

Governance teams can still apply one usefulness test before parsing sector-specific duties. Give the notice and available records to a reviewer who was absent from the original decision. If that reviewer cannot identify the controlling rule, verify the underlying fact and determine where a correction would enter the workflow, the explanation is documentation of the model, not an account of the denial.

Questions people ask

Does an

AI denial notice have to reveal the model’s source code?

Usually, a usable explanation does not require source code or every model weight. It requires specific reasons tied to the actual decision and supporting records that can be checked. The exact disclosure obligation depends on the sector, jurisdiction and data source, so source-code access should not be treated as the general standard.

Is

SHAP unsuitable for adverse-action notices?

No. SHAP can help identify inputs that moved a particular model score relative to a baseline. It becomes misleading when the organization presents that ranking as the reason for the final action without checking policy gates, data transformations, direction of influence and whether another system controlled the outcome.

What should an organization retain for each AI-assisted decision?

Retain the input snapshot and provenance, transformed values, model and policy versions, score, thresholds, rule results, external-service errors, human overrides and the mapping used to generate notice language. The record should be immutable enough to replay the decision after source systems or vendor models change.

Can a person challenge a denial by disputing the top feature?

They can challenge an underlying fact when a review or dispute process applies, but the top feature may not identify the controlling fact. The notice and supporting process should direct attention to the record that drove the outcome, such as an incorrect report field or failed verification, rather than implying that changing any prominent feature guarantees approval.