A Credit AI Must Show Why It Said No

The artifact that matters is the adverse-action packet assembled after a model helps reject an application: the input values used, the model and policy versions, the decision threshold, any human override, and the reason codes sent to the applicant. If that packet contains only a score and a generic label such as “did not meet lending standards,” the lender has an explanation problem regardless of how sophisticated the model’s internal documentation may be.

The controlling requirement is not new and does not depend on whether software is marketed as artificial intelligence. Under Regulation B, which implements the Equal Credit Opportunity Act, a statement of reasons for adverse action “shall be specific and indicate the principal reason(s) for the adverse action.” The regulation adds that saying an applicant failed to achieve a qualifying score, or failed to meet the creditor’s internal standards, is insufficient.

That is an enforceable obligation. The Consumer Financial Protection Bureau’s circulars on complex lending models did not create a proposed AI-specific disclosure regime; they stated the bureau’s view of how existing law applies. In its 2022 circular, the CFPB said creditors cannot use the opacity of a complex algorithm as a defense for failing to provide specific reasons. A 2023 circular further emphasized that the reasons disclosed must accurately describe the factors considered, even when a model uses factors that do not fit neatly into the sample adverse-action forms.

The engineering consequence is direct: a lender needs to reconstruct why this application crossed the unfavorable-decision boundary, not merely describe how the model behaves across thousands of applications.

A model explanation is not automatically an applicant reason

Many machine-learning teams use feature attribution, a technique that estimates how much each input contributed to a model output. SHAP, one common method, compares a prediction with a baseline and assigns portions of the difference to input features. It can help engineers spot drift, unexpected dependencies, or a model that relies heavily on a field the team thought was unimportant.

That output is useful. It may still be wrong for an adverse-action notice.

Suppose an underwriting model consumes reported income, existing monthly obligations, credit-history variables, and account activity. An attribution report might say several transformed variables pushed the score downward, including a logarithmic income feature, a binned utilization measure, and an interaction between recent inquiries and credit-file age. Those are model components. An applicant needs an accurate reason tied to information they can recognize, such as high revolving-credit utilization or insufficient income relative to requested credit, if those were among the principal factors that produced the decision.

Translation cannot be cosmetic. A lender should not map every negative income-related contribution to “insufficient income” if the decisive rule was an excessive debt-to-income ratio, because the latter depends on obligations as well as income. Nor should it name “limited credit history” merely because that phrase is available on a standard form when the model relied on a different factor. The CFPB’s 2023 circular made this point concrete: the sample reasons in Regulation B are illustrative, and checking the closest available box does not cure an inaccurate explanation.

Return to the adverse-action packet. It must preserve enough information to move from transformed model features back to the source fields and then to a reason phrase that accurately represents what the system considered. If the pipeline discards intermediate values after scoring, the compliance team may be left trying to infer a reason from a final probability. That reconstruction is weaker and can fail when correlated features point in different directions.

The decision pipeline needs receipts

A workable pipeline starts before model selection. The lender defines which model inputs may support applicant-facing reasons, how derived variables map to understandable concepts, and how it will rank principal factors for one decision. The model then scores an application, while the surrounding decision engine applies eligibility rules, pricing thresholds, fraud controls, or manual-review policies.

Each layer can cause the unfavorable result. Logging only the model’s top negative features misses a hard policy rule, while logging only the final rule misses the model contribution that sent the application below a threshold. The packet should identify the operative layer and retain the values that layer used.

For a model-driven denial, that generally means preserving the application snapshot as evaluated, including data later corrected or refreshed; the model identifier and feature-processing code; the raw score and relevant cutoff; the local explanation generated for that application; and the mapping from technical features to the reasons placed on the notice. If a loan officer overrides a recommendation, the record needs the override and its reason rather than an explanation for the model result that no longer controlled the outcome.

This has storage and operational costs. Versioned feature pipelines take engineering work, explanation methods add compute, and retaining input snapshots expands the amount of sensitive financial data under access controls and retention policies. The alternative is cheaper only until a complaint, audit, or model change requires the lender to reproduce a decision and the necessary state no longer exists.

Accuracy also has a cost. Some explanation methods approximate a model rather than expose its exact calculation, and their rankings can change with the choice of baseline or with small differences among correlated inputs. A lender using an approximation should test whether it identifies the factors that truly moved decisions around the relevant cutoff. Global importance charts, which summarize behavior across a portfolio, cannot perform that test for an individual applicant.

A simpler model may therefore be worth lower predictive performance if it produces stable, faithful reasons and is easier to monitor. That tradeoff is not automatic. A well-instrumented complex model can be more explainable in practice than a nominally transparent scorecard surrounded by undocumented policy rules and manual overrides.

What the vendor must return

A lender does not transfer the disclosure obligation by buying a score through an application programming interface. The lender’s adverse-action packet still needs the receipts, which makes vendor capability a procurement constraint rather than a feature to examine after deployment.

A vendor should be able to return application-level factors alongside the score, document how those factors are calculated, and identify the model version used for a historical request. It should also disclose whether its reason codes come from the scoring model, from a separate explanation model, or from a fixed lookup table. Those designs have different failure modes: a separate explainer can diverge from the production model, while a lookup table may collapse distinct causes into a generic phrase.

Contracts and technical evaluations should address what happens when the vendor changes feature engineering or retrains the model. Historical reproducibility matters here. If yesterday’s model cannot be queried and its feature definitions were overwritten, a lender may possess the notice it sent but lack evidence that the listed reasons matched the decision.

The strongest test uses controlled application records. Change one relevant input while holding the others fixed, rerun the decision, and inspect whether the score, outcome, and principal reasons move coherently. Then test correlated inputs and cases near the cutoff, where explanation rankings may be unstable. This is a form of counterfactual testing, meaning the team changes selected facts to observe whether the decision responds as expected; it is a diagnostic, not necessarily the explanation delivered to the applicant.

A vendor that returns only a score, a confidence value, and portfolio-level feature importance is not supplying enough for this workflow. The lender can build an explanation layer, but doing so without access to model behavior and version history may produce polished language that is not faithful to the decision. At that point, the model may not be worth adopting for a use that triggers adverse-action duties.

Audit the notice, not just the model

Model validation often concentrates on discrimination testing, predictive performance, stability, and data quality. Those reviews matter, but they do not establish that a particular notice states the principal reasons for a particular outcome.

The adverse-action packet closes that gap. An auditor can start with the notice, trace each stated reason to the decision record, verify that the cited factor had a principal negative role, and check that a more important factor was not omitted. Sampling should cover different products, decision pathways, model versions, overrides, and applicants near thresholds, because a reason-code system can perform well on ordinary cases while breaking on precisely the cases where small feature changes reverse the result.

Teams should also compare the applicant-facing wording with the underlying data. “High credit utilization” has a recognizable source and can be disputed if the data are wrong. “Risk profile does not meet requirements” does not identify a factor. More detail is not always better, however; dumping transformed variable names or a long ranked list can obscure the principal reasons and expose internal signals without helping the applicant understand the action.

This analysis is not legal or financial advice. It is an engineering reading of a concrete requirement: the notice must give specific principal reasons, so the production system must retain a faithful path from decision to explanation.

Questions people ask

Can a lender say the AI model rejected my application?

That alone is not a specific reason. Regulation B says that failing to achieve a qualifying score or meet internal standards is insufficient; the notice must identify the principal factors behind the adverse action, whether the lender used a scorecard, a machine-learning model, policy rules, or human review.

Is a

SHAP report enough for an adverse-action notice?

Not by itself. SHAP can estimate which technical features pushed a prediction, but the lender must verify that the method is faithful for the individual decision and translate the principal factors into accurate, recognizable reasons without replacing them with generic form language.

What should a lender require from an AI vendor?

The vendor should return application-level reason information, document feature and reason-code mappings, preserve model versions, and support historical reconstruction. If it supplies only a score and portfolio-wide importance chart, the lender may have to build an explanation system without enough access to prove that its reasons match the decision.

Does using a simpler model solve the problem?

It can make tracing decisions easier, but simplicity does not fix missing logs, undocumented rules, or inaccurate reason mappings. A complex model with versioned inputs and tested local explanations may produce better records than a scorecard embedded in an opaque decision workflow.