NYC’s Hiring-AI Law Depends on How You Use the Score

Start with the screen a recruiter sees after 200 applications arrive. A recruiting platform parses each résumé, predicts job fit, assigns a score from zero to 100, and ranks the applicants. The recruiter opens the top group first; candidates below a configured threshold are automatically rejected.

That composite workflow is a useful test case for New York City’s Local Law 144 because it exposes the two facts that matter: what the software produces and how people use that output. Calling the platform a copilot, chatbot, résumé assistant, or artificial intelligence system settles neither point.

Local Law 144 restricts an employer or employment agency from using an automated employment decision tool, or AEDT, unless the tool has undergone a recent bias audit, a summary of the results is publicly available, and required notices have been given. The New York City Department of Consumer and Worker Protection began enforcing the law in July 2023. This explainer offers a fact-gathering worksheet, not legal advice.

The law tests the workflow twice

The statutory definition starts with technology. An AEDT is a computational process derived from machine learning, statistical modeling, data analytics, or artificial intelligence that issues a simplified output, such as a score, classification, or recommendation.

The city’s rules narrow that technical language. In broad terms, the covered techniques generate a prediction or classification and use a computer to identify at least some inputs, their relative importance, or other model parameters. A large language model that summarizes interview notes may clear that technical threshold. A fixed form that rejects anyone who answers “no” to a required-license question may not, even though its result is decisive, because a person wrote the rule rather than having a computer identify the relevant weighting or parameters.

The second test is operational. The output must be “used to substantially assist or replace discretionary decision making” for an employment decision. Under the rules, substantial assistance exists when an employer relies solely on the output, weights it more heavily than any other criterion, or uses it to overrule a conclusion reached from other factors, including human judgment.

Return to the ranked résumé screen. Automatic rejection below 70 is the clearest case because the score replaces discretion for those applicants. The same tool may still qualify if recruiters can override it but ordinarily review only the top-ranked group, especially when the score carries more weight than experience, work samples, or an interview. A score displayed alongside other information is not automatically covered; counsel needs evidence of its real weight.

A worksheet for the ranked résumé screen

Do not begin with the vendor’s legal conclusion. Ask the recruiting, HR operations, procurement, and technical teams to reconstruct one application from submission to disposition, then record these fields.

1. Identify the decision and the people affected. Write down the specific job or promotion, where it is associated, and whether the system screens candidates for employment or employees for promotion. Local Law 144 does not regulate every workplace use of automation.

Tools used for payroll, scheduling, training, or workforce analytics may raise other obligations, but they do not become AEDTs merely because an employer bought them.

Agency guidance says the law applies when a job is located in New York City at least part time, or when a fully remote position is associated with a New York City office. Capture the posting, assigned office, remote-work designation, and promotion location rather than relying on the applicant’s home address.

2. Record every machine-generated output. Take screenshots or export field names showing scores, tiers, rankings, flags, recommendations, generated summaries, and pass-or-fail results. For the résumé tool, the relevant artifacts include the zero-to-100 score, the ordered candidate list, and the rejection flag.

A chatbot that only schedules interviews or answers benefit questions usually lacks an employment-decision output; one that labels candidates “recommended” after interpreting their answers deserves closer review.

3. Document how the output is produced. Obtain the vendor’s technical description, model card if one exists, configuration guide, and answers about whether training data or automated optimization determines inputs or their weighting. Product marketing that says “AI-powered” proves little.

The reverse is also true: software sold as matching, analytics, or assessment technology may fit the technical definition without using AI in its name.

This is where procurement often stalls. Vendors may disclose broad input categories while withholding model weights, training data, or proprietary logic. Counsel does not necessarily need source code to map the law, but the employer needs enough information to distinguish a learned prediction from a fixed rule and to identify the qualifications or characteristics assessed for candidate notice.

4. Trace configuration and thresholds. Record who set the rejection threshold, whether the vendor supplied a default, and what happens when a candidate falls below it. Include knockout settings, minimum-score filters, interview cutoffs, and ranking limits.

A recruiter’s ability to click “restore candidate” does not show that the human exercises meaningful discretion if nobody reviews rejected applications.

5. Measure the output’s weight. Ask hiring managers to describe the last several decisions, then compare that account with system logs. Did anyone advance a low-scoring candidate?

Did the ranking determine review order? Could a work sample outweigh the score, or did the score overrule it? Written policy matters, but click histories, disposition codes, and approval records show the implemented workflow.

For the anchored résumé screen, “humans make the final decision” is incomplete. If the software removes 140 applicants before a person sees them, the consequential employment decision has already occurred for that group. Human involvement later in the funnel does not undo the earlier screen.

6. Separate sourcing from screening. Record whether the tool searches a résumé database, recommends people who have not applied, or evaluates applicants for a specific position. The law’s employment-decision definition focuses on screening a candidate for employment or an employee for promotion.

Moving from talent discovery into scored applicant review can change the analysis even when both functions sit inside one platform.

7. Preserve the population and outcome data. Bias audits need the selection or scoring outcomes produced by the tool and demographic categories required by the rules. Determine whether the employer has usable historical data, whether the vendor plans to pool data across employers, and how records map to the particular AEDT configuration.

Missing demographic information, small groups, and changed thresholds can limit what the published ratios reveal.

8. Assign an owner for changes. Model updates are not the only concern. A recruiter can turn an informational score into a decisive screen by adding a cutoff, while a business unit can use the same output differently from another unit.

Procurement should trigger a new review when the vendor changes the model, HR changes the configuration, or managers change the weight given to its output.

The completed worksheet should leave counsel with a data-flow diagram, screenshots, configuration records, sample logs, vendor documentation, job-location facts, and the written hiring policy. It should also identify gaps. An unsupported assurance that the product is “Local Law 144 compliant” is not a workflow record.

What coverage requires before use

If counsel concludes that the ranked résumé workflow uses an AEDT, the employer or employment agency must check three operational controls.

First, an independent auditor must conduct a bias audit no more than one year before the tool is used. An independent auditor must exercise objective and impartial judgment and must not have disqualifying involvement in, or financial interests connected to, the AEDT. The audit generally calculates selection or scoring rates and impact ratios, which compare a group’s rate with the rate of the most favored comparison group, across the demographic categories specified in the rules.

The audit is a measurement requirement, not a government certification that the tool is fair, accurate, or lawful under every anti-discrimination rule. A ratio can reveal an outcome disparity without explaining whether résumé content, model design, recruiting channels, job requirements, or later human choices produced it. Small samples can also make results unstable or lead to omitted calculations.

Second, the employer or agency must publish a summary of the audit results and the tool’s distribution date on a public website before use. The rules specify information that the summary must contain and require it to remain available for a period after the employer stops using the tool. Ask for the exact URL and archive a copy; “the vendor handled it” is not evidence that the relevant configuration and use are represented.

Third, candidates or employees must receive notice at least 10 business days before use. The notice must say that an AEDT will be used and identify the job qualifications and characteristics it will assess. It must also allow a person to request an alternative selection process or accommodation, although agency guidance explains that Local Law 144 itself does not require the employer to grant an alternative process. Other disability and employment laws may still apply.

The employer must also make information about the tool’s data sources, the type of data collected, and its data-retention policy available on its website, or provide it after a written request within the period established by the law, subject to stated exceptions. That obligation is easy to miss when privacy disclosures and recruiting notices are owned by different teams.

The boundary is not a safety finding

Suppose the résumé platform removes its score and displays only an extractive summary of each applicant’s experience. If recruiters treat the summary as a convenience and verify qualifications against the résumé, the workflow may fall outside the substantial-assistance routes. If the system instead classifies applicants as qualified and that label controls review, changing the interface has not changed the function.

A conclusion that Local Law 144 does not cover a tool is also not permission to ignore discrimination, disability accommodation, privacy, or recordkeeping duties. The city law has a defined technical and operational scope. Other laws can reach ordinary software, human decisions, or practices that produce unlawful effects without meeting the AEDT definition.

For the ranked résumé screen, the durable control is therefore a use register tied to logs and configuration. It records which jobs used the ranking, which threshold applied, who could override it, and whether anyone did. That record gives an auditor something better than a product brochure and gives counsel a way to revisit coverage when the workflow changes.

Questions people ask

Does

Local Law 144 cover every recruiting chatbot?

No. A bot that schedules interviews or answers routine questions may not issue a score, classification, or recommendation used for an employment decision. A bot that interprets candidate responses, labels applicants, or decides who advances could qualify, depending on how its model works and how heavily the employer relies on its output.

Is a human review enough to keep a tool outside the law?

Not by itself. The rules cover workflows where the employer weighs the simplified output more than any other criterion or lets it overrule other conclusions, as well as fully automated decisions. Review logs and overrides matter more than a policy stating that a person remains responsible.

Can an employer rely on the vendor’s bias audit?

A vendor may commission an audit or support publication, but the employer should verify that the audit is recent, independent, and relevant to the tool and data permitted by the rules. It should also confirm that the required summary is public and that its own notices describe the qualifications or characteristics assessed.

Does passing a bias audit prove the hiring tool is fair?

No. The audit reports required selection or scoring comparisons for available demographic groups; it does not certify accuracy, explain every disparity, or resolve compliance with other employment laws. Keep validation studies, configuration history, outcome data, and human-review records alongside the published audit summary.