NYC’s Hiring AI Law Requires a Narrow Bias Test

Start with a resume-ranking workflow. Candidates upload resumes, software extracts skills and employment history, and a model assigns each person a score. A recruiter reviews only applicants above a threshold or opens the ranked list from the top. That sequence, particularly the weight given to the score, is the concrete case that makes New York City’s automated employment decision tool law relevant.

The law, commonly called Local Law 144, does not regulate every use of artificial intelligence in a hiring department. It restricts an employer or employment agency from using a covered automated employment decision tool, or AEDT, unless the tool has received a bias audit no more than one year before its use, a summary is publicly available, and required notices have been given. The city began enforcing the rules in July 2023.

That sounds like a product classification. It is better treated as a workflow test.

The output and its weight set the boundary

An AEDT is not merely software with an AI label. Under the city’s definition, it is a computational process derived from machine learning, statistical modeling, data analytics, or artificial intelligence that produces a simplified output, such as a score, classification, or recommendation, and uses that output to “substantially assist or replace discretionary decision making” in an employment decision.

Machine learning means software that derives patterns from data rather than following only instructions written in advance. The definition is still broad enough that a vendor’s branding does little compliance work. A conventional product can qualify, while a feature marketed as generative AI may sit outside the rule if it never evaluates a candidate or employee.

The implementing rules make “substantially assist” more concrete. The condition is met when an employer relies solely on the simplified output, gives it more weight than any other criterion, or uses it to overrule conclusions drawn from other factors, including a human decision.

Return to the resume ranker. If the score determines who reaches a phone screen, the output is doing more than organizing information. If a recruiter nominally reviews every resume but normally follows the ranking, the written claim that a human remains involved does not settle the matter. The relevant evidence includes the configured cutoff, recruiter instructions, screen views, override records, and what happens to candidates below the threshold.

A different setup may land elsewhere. Software that schedules interviews, removes duplicate applications, transcribes a call, or helps draft a job description does not automatically produce a recommendation about whom to hire or promote. A recruiter’s writing assistant may use a large language model, but if its output never scores a person and does not drive an employment decision, it does not fit the same mechanism as the ranker.

Some cases stay difficult. A chatbot may appear to collect availability while also classifying answers, or an interview platform may generate a summary whose labels become the recruiter’s main screening criterion. A knockout question may execute a fixed policy rather than a learned model, yet its surrounding system could still use statistical or AI-derived classifications. Those are points to preserve technical documentation and seek legal review, especially when product materials do not explain how the output is produced.

Hiring and promotion are the covered decisions

The city’s rules focus on screening candidates for employment and employees for promotion. They are not a general code for every algorithm used at work.

A model used to recommend termination, set compensation, assign shifts, monitor productivity, or identify employees for training may raise obligations under other laws and workplace policies, but Local Law 144’s audit requirement does not expand merely because the system affects a worker. The decision type matters alongside the technology and the output’s weight.

Location matters too. The Department of Consumer and Worker Protection, or DCWP, has explained the rule through its guidance for positions in New York City, while the law’s notice language refers to candidates and employees who reside in the city. Remote jobs, transfers, multi-office roles, and hiring pipelines that mix city and non-city positions can therefore create facts that deserve review rather than a broad assumption that a company is either wholly covered or wholly exempt.

For the resume-ranking workflow, record which requisitions use the model and where those positions are located. Applying one global configuration to every opening may be operationally convenient, but it makes it harder to prove which uses were audited, which people received notice, and whether a vendor changed the model during the audit period.

The required audit tests outcomes, not the whole model

The bias audit is an independent evaluation of disparate results. It generally calculates selection or scoring rates across specified sex and race or ethnicity categories, including intersectional groupings, then reports impact ratios comparing those results with the category receiving the highest rate or score.

For a ranker that decides which applicants advance, the auditor examines selection rates. For a system that issues scores, the rules call for scoring-rate calculations. The audit rules also address categories with small representation and people whose demographic category is unknown, because missing data can distort a comparison even when the arithmetic is correct.

This is a narrow test. It does not require the auditor to prove that resume parsing is accurate, that the model predicts job performance, or that each input is job-related. It does not by itself examine disability discrimination, age discrimination, privacy, accessibility, hallucinated interview summaries, or whether the vendor trained on material it had permission to use.

The law also does not specify a passing impact ratio that makes a tool lawful. An impact ratio is evidence about relative outcomes, not a certification that discrimination is absent. The federal four-fifths rule sometimes used in employment analysis is a separate framework, and treating it as an automatic Local Law 144 pass mark would overstate what the city requires.

That distinction changes what an employer is buying. A compliant audit can reveal that one group advances at a lower rate while saying nothing about why, and a result with limited apparent disparity can coexist with a model that reads dates incorrectly, penalizes unusual formatting, or relies on a weak proxy for job performance. Accuracy testing and job-validity analysis remain separate work.

Independence and data cannot be delegated away

The audit must be conducted by an independent auditor. DCWP’s rules limit financial and operational ties that could compromise that independence, including involvement in developing, using, or distributing the AEDT. A vendor cannot make an assessment independent merely by assigning it to another internal team.

Employers may use an audit arranged by a vendor when it covers the tool they deploy, but that creates a configuration problem. The resume ranker in the audit must correspond to the version, settings, output, and decision workflow in production. A generic vendor report becomes less informative when one customer applies a cutoff score, another reviews the top 20 candidates, and a third combines the score with a separate assessment.

Historical data from actual use is generally the preferred basis. Where sufficient historical data is unavailable, the rules permit test data under specified conditions and require disclosure of why it was used. Synthetic or pooled test records can make an initial audit possible, but they may not reproduce the applicant mix, resume formats, language patterns, or recruiter overrides found in one employer’s pipeline.

The employer must publish a summary of the audit results and the distribution date of the AEDT before use, then keep the information available for the required period after last use. That publication is a receipt, but a limited one. Readers can inspect the reported rates and ratios; they usually cannot reconstruct the model, verify every data-cleaning choice, or infer how a particular candidate was scored.

The direct cost includes auditor fees and staff time spent exporting demographic and outcome data, reconciling job stages, documenting exclusions, and checking that vendor identifiers match the employer’s records. The slower cost appears when a model update breaks comparability with the audited version, forcing compliance, recruiting, and procurement teams to decide whether use should pause.

Notice is required, but consent is not

Covered candidates and employees must receive notice at least 10 business days before the AEDT is used. The notice must say that an automated tool will be used and identify the job qualifications or characteristics it will assess. DCWP’s rules allow notice through routes such as a job posting, the employment section of a website, or direct mail or email, depending on the context.

The law also addresses information about the type and source of data collected and the employer’s data-retention policy. That information may be posted or provided after a written request within the applicable period, subject to limits involving other law or protected information.

Notice is not consent. The city does not require a candidate to approve the assessment before it runs, and DCWP’s published guidance says Local Law 144 does not require an employer to offer an alternative selection process merely because one is requested. Instructions for seeking a reasonable accommodation still matter, and disability and employment laws may create separate duties.

In the resume workflow, a sentence buried in a privacy policy is therefore a poor control. The hiring team needs a timed notice connected to the requisition, a description that matches the characteristics the ranker evaluates, and a fallback that prevents scoring until the notice period has elapsed. Otherwise, the model may be audited while the actual use still violates the procedural requirement.

A workable internal review follows the decision

Teams should map the candidate’s path before debating whether a product is AI. Record the input, the generated output, who sees it, how much weight it receives, which employment decision follows, and the New York City connection. Then attach evidence: vendor documentation, model and configuration identifiers, audit dates, public disclosures, notice templates, delivery records, and change logs.

That file will not answer every legal question. It will expose the practical ones. If nobody can say whether recruiters may override a low score, whether the model changed after the audit, or whether a generated interview summary includes an evaluative label, the organization does not yet understand the system well enough to classify its obligations.

Questions people ask

Does every

AI tool used by a recruiter need a bias audit?

No. The city’s rule targets systems that produce a score, classification, or recommendation and substantially assist or replace discretion in hiring or promotion. Scheduling, transcription, drafting, and administrative tools may fall outside that definition unless their outputs evaluate people or materially drive the covered decision.

Does passing the audit prove the tool is fair?

No. The required audit reports selection or scoring rates and impact ratios for specified demographic categories. It does not certify predictive accuracy, job relevance, accessibility, privacy compliance, or freedom from every form of discrimination, and the law does not provide a universal passing ratio.

Can a human reviewer keep the system outside the law?

Human involvement alone is not enough. A tool can qualify when its output receives more weight than any other criterion or overrides a human conclusion, even if a recruiter clicks the final button. Actual instructions, thresholds, override behavior, and decision logs matter more than a “human in the loop” label.

Must an employer offer a nonautomated alternative?

Local Law 144 requires advance notice and a way to request an alternative process or accommodation, but DCWP guidance says the law itself does not require the employer to provide an alternative selection process. Other disability, employment, or accommodation requirements may still apply, which is a reason to route such requests for review.