A Generative Candidate Summary Can Trigger NYC’s Hiring-AI Law

Take a recruiting workflow with one new button: “Generate candidate brief.” A generative model, meaning software that produces new text from supplied material, reads a resume and interview notes, then writes a short summary and assigns a fit label. The recruiter sees the brief first and advances the people labeled “strong fit.”

That button may look like an administrative aid. Under New York City’s Local Law 144, however, its status depends less on whether the vendor calls it a copilot, assistant, or writing feature than on what it outputs and how the employer uses that output. A prose summary that merely condenses documents presents a different case from a summary that recommends candidates, ranks them, or becomes the recruiter’s main basis for deciding whom to interview.

Local Law 144 has been enforced since 2023. It restricts employers and employment agencies from using a covered automated employment decision tool, or AEDT, unless the tool has received a recent independent bias audit and the employer has published required information. It also requires advance notice to covered candidates or employees. This explainer maps the operational questions; it is not legal advice.

The output and its influence set the boundary

The law defines an AEDT through both its technical mechanism and its role in a decision:

“Any computational process, derived from machine learning, statistical modeling, data analytics, or artificial intelligence, that issues simplified output, including a score, classification, or recommendation, that is used to substantially assist or replace discretionary decision making for making employment decisions that impact natural persons.”

A generated “strong fit” label is an obvious classification or recommendation. An ordered candidate list is similarly close to the definition. Free-form prose is harder to classify: a summary could reproduce job history without evaluating the applicant, or it could state that the person lacks required experience and should not proceed. Calling both outputs summaries hides the material difference.

The city’s rules give “substantially assist or replace” a narrower operational meaning. The threshold is met when an employer relies only on the simplified output, gives it more weight than any other criterion, or uses it to overrule conclusions drawn from other factors, including human judgment. Ordinary influence is not phrased as enough. The rule looks for specified forms of decisive weight.

Return to the candidate brief. If a recruiter reads the original resume, applies independently documented criteria, and treats the generated text as a convenience that can be corrected, the argument for AEDT coverage is weaker. If the recruiter reviews only briefs for the first cut, treats “strong fit” as the leading criterion, or advances a candidate because the model overrules the interview panel, the workflow tracks the city’s definition much more closely.

The interface alone cannot settle this. A vendor may state that humans make every final decision, while the employer’s dashboard places the generated label at the top, sorts by it, and hides source documents behind another click. A nominal human reviewer does not answer how much weight the output receives. Logs, recruiter instructions, default sorting, and observed use supply better evidence.

Local Law 144 also concerns screening for hiring or promotion, not every employment-related automation. A tool that drafts a job advertisement or schedules interviews does not become an AEDT merely because it uses a language model. Coverage becomes more plausible when the output evaluates a person for a particular employment decision involving a covered New York City position.

The bias audit tests outcomes, not prose quality

Before using a covered AEDT, the employer or employment agency must ensure that an independent bias audit was conducted no more than one year before the tool’s use. The required calculations examine selection or scoring outcomes across sex and race or ethnicity categories, including intersectional groups, using selection rates and impact ratios specified by the rules. A public summary must be available before deployment.

This requirement creates a practical problem for the “Generate candidate brief” feature. A vendor may have audited a ranking product while the employer enables an additional prompt, supplies different candidate data, or asks the model to produce a pass recommendation from interview notes. Those changes can alter the output being used and the population on which it operates. A generic vendor report should therefore be matched to the deployed tool, version, configuration, use case, and relevant data rather than accepted from a procurement folder without review.

The law permits audits based on historical data and addresses the use of test data when adequate historical data is unavailable. It also allows an independent auditor to evaluate an AEDT used by multiple employers under specified conditions. Vendor coordination can reduce duplicated work, but it does not erase the employer’s duty to ensure that the audit requirement has been met for its use.

An audit under Local Law 144 is not a general certificate that the generated prose is accurate, fair, or safe. The prescribed calculations will not, by themselves, reveal that the model omitted a license from a resume, inferred seniority from writing style, or turned an interviewer’s ambiguous note into a confident rejection. They also do not test whether substantially similar source records receive inconsistent summaries after repeated model calls.

Those failures need separate evaluation. A useful test set would preserve the source resume and notes, capture every generated brief, record the resulting recommendation, and have reviewers check unsupported claims and consequential omissions. When the model’s answer cannot be reconciled with the source, the fallback should be direct review of the underlying documents, with the generated recommendation removed from the decision rather than quietly edited after it has shaped the recruiter’s view.

Notice comes before the tool is used

For covered uses, the law requires notice at least 10 business days before the AEDT is used. The notice must state that an automated employment decision tool will be used and identify the job qualifications and characteristics it will assess. It must also tell the candidate or employee that an alternative selection process or accommodation may be requested.

That last statement is easy to misread. Local Law 144 requires a way to request an alternative process or accommodation; the city’s guidance says the law itself does not require an employer to provide an alternative selection process. Other disability, employment, or human-rights requirements may still apply, but they are separate legal questions.

The notice should describe the deployed workflow rather than hide behind “AI may be used.” If the candidate brief assesses relevant experience and converts the result into a fit classification, those are the operative characteristics and output. Employers also have disclosure duties concerning the type and source of data collected and the applicable retention policy, subject to the law’s procedures and exceptions.

Notice timing creates an engineering constraint. A recruiting team cannot switch on a generated ranking midway through an active pipeline and assume that a general privacy statement covers it. The system needs a deployment gate that checks whether affected candidates received the required notice early enough, while the recruiting team needs a non-AEDT path for people already in the pipeline if the timing condition has not been met.

Build the compliance record around one decision

The fastest internal review starts with a single candidate moving through the actual interface. Record which materials enter the model, the prompt or configured instruction, the complete output, what the recruiter sees first, and whether the software sorts or filters candidates from that output. Then identify the precise employment decision: invitation, advancement, rejection, or promotion screening.

Next, trace weight rather than intent. Written policy might tell recruiters to verify every brief, yet interaction logs could show that they open source resumes only after the shortlist is set. Conversely, a visible fit label may be routinely ignored while reviewers use a structured rubric based on the original application. Neither a vendor disclaimer nor the presence of a human is a substitute for this evidence.

If the workflow appears covered, deployment should stop until the employer can connect the exact use to a qualifying bias audit, publish the required summary, and deliver timely notices. If coverage remains uncertain, the lower-risk product choice is often to remove the evaluative output: let the model extract cited facts for human verification, do not rank candidates, and prevent the generated text from overriding the documented assessment. That fallback costs recruiter time because source records must be read, but it also reduces the chance that fluent prose becomes an unexamined decision rule.

Keep the receipts. Preserve the audit and its scope, notices and delivery dates, public disclosures, model and prompt configuration, reviewer instructions, change approvals, output logs where lawful, and records showing how recommendations affected decisions. Local Law 144 does not prescribe a complete observability architecture, but without these records an employer may struggle to show whether the candidate-brief workflow stayed within the use that was reviewed.

What the city enforces

The current obligation is not a proposed federal framework or a voluntary AI principle. New York City can pursue civil penalties for unlawful use and notice failures, with each day of noncompliant use potentially counted separately and notice violations assessed for each affected candidate or employee. The law does not create a city preapproval process in which an agency certifies a product before sale.

Enforcement scope also matters. Local Law 144 has defined limits around covered tools, covered decisions, and covered positions; broader discrimination laws can apply even when a system falls outside its AEDT definition. Passing the city’s required bias audit therefore answers one compliance question. It does not validate every generated candidate brief or establish that the underlying decision was nondiscriminatory.

Questions people ask

Does every

AI-generated candidate summary count as an AEDT?

No. Coverage depends on whether the system produces a score, classification, recommendation, or comparable simplified output and whether the employer uses it in one of the city’s specified ways to substantially assist or replace discretion. A factual digest checked against the resume presents a different case from a fit summary that determines the shortlist.

Does keeping a human recruiter in the loop avoid the law?

Not necessarily. The city’s rules look at the weight given to the automated output. A workflow may qualify when the recruiter gives that output more weight than any other criterion or lets it overrule other conclusions, even though a person still clicks the final advance or reject button.

Must an employer offer a non-AI screening option?

Local Law 144 requires notice that a candidate or employee may request an alternative selection process or accommodation, but city guidance says the law itself does not require the employer to provide an alternative process. Separate accommodation and anti-discrimination obligations may affect the response, so that decision requires its own review.

Is a vendor’s bias-audit report enough?

Only if it satisfies the law’s requirements and maps to the tool and use the employer deploys. An employer should check the auditor’s independence, audit date, covered configuration, data basis, required calculations, and public summary. A report for a different ranking setup may not address a newly added generative recommendation.