NYC’s Hiring-AI Rule Turns on How Recruiters Use the Score

Take a representative hiring workflow. An applicant tracking system parses each resume, rejects anyone who answered “no” to a work-authorization question, assigns the remaining applicants a score from zero to 100, and puts people scoring at least 80 into a recruiter’s review queue. The recruiter can retrieve lower-scoring applicants, but usually starts with the queue.

Call it the 80-point queue.

New York City’s Local Law 144 does not turn on whether the vendor markets that queue as “AI-powered.” The coverage analysis instead runs through the tool’s computational method, its output, how that output affects a screening decision, and the connection between the job and New York City. A human reviewer’s presence does not settle the issue, particularly when the system decides whose file the human sees first.

This is an operational explainer, not legal advice. Its purpose is to identify the records and vendor answers that let counsel assess the workflow instead of relying on a product description.

Start with the output, not the AI label

The law regulates certain automated employment decision tools, usually shortened to AEDTs. Its definition centers on a computational process derived from machine learning, statistical modeling, data analytics, or artificial intelligence that produces a simplified output, including “a score, classification, or recommendation,” and uses that output to substantially assist or replace discretionary decision-making in an employment decision.

The city’s implementing rules make “substantially assist or replace” more concrete. The threshold can be met when an employer relies only on the output, gives it more weight than any other criterion, or uses it to overrule a conclusion reached from other factors, including human judgment. Those are enforced rules, not a proposed standard for future systems.

In the 80-point queue, the numerical score is plainly a simplified output. The harder factual issue is weight. If recruiters cannot see applicants below 80, the output may function as the screening decision even though a recruiter later evaluates everyone who passed. If recruiters see every applicant and use the number as one modest signal among several, the assessment changes.

A policy saying recruiters retain discretion proves little when the interface, workload, or manager instructions make the ranked queue the practical default.

Ask the vendor to describe how the score is generated without stopping at “proprietary model.” Counsel needs to know whether the software uses a trained model, a statistical formula, configurable rules, keyword matching, or some combination; which inputs affect the output; whether the employer sets thresholds; and whether a software update can change scoring behavior without a new configuration record.

A fixed eligibility question may sit differently from a learned ranking model. A calculator, database, or ordinary data-processing feature is not automatically an AEDT, although an excluded component can feed a covered decision system. The work-authorization knockout in the example therefore needs its own analysis rather than inheriting the ranking model’s status.

Reconstruct what the recruiter could do

A useful coverage file shows the recruiter’s actual screen. Preserve screenshots or recordings of the default queue, displayed score, explanation panel, filters, sorting controls, override path, and any warnings that appear when someone advances a lower-ranked applicant. Product documentation alone may describe capabilities that the employer never enabled, while leaving out tenant-specific settings that determine who reaches review.

Then collect the operating evidence: standard procedures, training material, manager instructions, score cutoffs, requisition templates, access permissions, and system logs. Logs should establish which version ran, which candidates received an output, who viewed it, whether recruiters changed the sort order, and which candidates advanced after an override. Aggregate override counts can help, but candidate-level event records are more revealing because they show whether the human acted before or after seeing the recommendation.

Return to the 80-point queue. Suppose a recruiter may search below the threshold, yet the interface loads only the passing group and the team must fill a role under a short deadline. That design imposes a time cost on independent review, so the formal availability of an override does not establish that the score carried little weight. Conversely, logs showing routine review across the full applicant pool would support a different factual account.

The same distinction separates several common setups. A resume parser that extracts education and employment dates for display may not issue a decision-oriented score or recommendation. A model that ranks candidates from strongest to weakest does. A chatbot that schedules interviews after a recruiter selects the candidates is performing administration, while a chatbot that recommends whom to interview may enter the covered workflow.

The feature name is weak evidence; the output and its use are stronger.

Tie the workflow to a covered employment decision

Local Law 144 addresses screening candidates for employment and employees for promotion. A tool used only for workforce scheduling, payroll calculations, or interview transcription does not become covered merely because hiring staff can access it. If a transcript model grades interview answers or recommends advancement, however, the grading step deserves review.

Geography also needs a written finding. The law’s notice language addresses candidates and employees who reside in New York City, while the Department of Consumer and Worker Protection’s published guidance frames application around jobs located at a city office, at least part time, or fully remote roles associated with a city office. Distributed hiring makes those facts easy to lose inside an applicant tracking system.

For each requisition touched by the 80-point queue, retain the listed work location, office association, remote-work designation, hiring entity, posting text, applicant residence information collected at the time, and dates when the tool operated. Counsel can then address the statute, rules, and agency guidance against a stable record rather than reconstructing location from an employee’s later work arrangement.

This city rule should not be treated as the full discrimination analysis. Federal, state, and local employment laws can apply even when a workflow falls outside Local Law 144’s definition or geographic reach. Guidance from the US Equal Employment Opportunity Commission is also distinct from the city’s enforceable audit and notice requirements.

An audit receipt must match the deployed tool

Before using a covered AEDT, an employer or employment agency must obtain a bias audit conducted by an independent auditor, ensure the audit is no more than one year old, and publish the required summary. The rules define independence through the auditor’s relationship to the developer and user, including involvement with the tool and certain financial interests. A vendor’s internal fairness review is not interchangeable with that requirement.

The audit examines selection rates and impact ratios across specified sex and race or ethnicity categories, including intersectional categories. A selection rate is the proportion of a group receiving the favorable outcome; an impact ratio compares that rate with the rate for the most selected group. Scoring systems require related calculations based on score distributions under the rules.

That audit has limits. It does not certify that the model is unbiased, accurate, accessible, or lawful under every employment statute, and an impact ratio does not explain why a disparity appeared. The result also may not describe the employer’s deployment if the vendor tested another model version, used pooled customer data, assumed a different cutoff, or audited a recommendation that the employer converts into a stricter pass-fail gate.

For the 80-point queue, request the audit report and public summary, auditor identity and independence basis, audit period, data source, model and configuration identifiers, excluded records, category counts, handling of unknown demographic data, calculated rates, distribution date, and publication URL. Reconcile those records with deployment logs. If the audit names a model family but the production system records a different version or threshold, the mismatch needs an answer before anyone treats the document as a receipt for this workflow.

Historical data are central to the rules, with provisions for using other employers’ historical data or test data in specified circumstances when adequate employer data are unavailable. The vendor should state which path it used and whose outcomes shaped the calculations. “Audited annually” is not enough.

Notice is a deployment dependency

The rule also creates notice and publication work that product teams cannot leave until launch day. Covered candidates or employees must receive notice at least 10 business days before use, including notice that an AEDT will be used and the job qualifications and characteristics it will assess. The framework also addresses requests for an alternative selection process or accommodation, if one is available, and requests for information about data collected, its source, and the employer’s retention policy.

The employer should retain the notice text, delivery channel, recipient population, send time, posting history, accommodation route, response records, and the public webpage carrying the audit summary and required distribution information. A current webpage does not prove what candidates could see when the 80-point queue first ran, so dated snapshots matter.

Vendor contracts should allocate the underlying work without pretending responsibility disappears. The employer may need the vendor to preserve model identifiers, produce audit inputs, notify customers before scoring changes, support demographic calculations, explain data retention, and provide enough lead time for notices. A clause promising “compliance with all applicable laws” supplies none of those artifacts.

The practical decision is narrower than “Does this product use AI?” Map one output from generation to display to employment action. If the score controls the 80-point queue, the records should show who was screened out, what the recruiter could override, which model produced the result, and whether the audit and notices matched that deployment.

Questions people ask

Does a human recruiter keep a resume-ranking tool outside the rule?

Not by itself. The city’s rules look at whether the employer relies solely on the output, weights it more heavily than any other criterion, or uses it to overrule another conclusion. Interface defaults, cutoff settings, instructions, and usage logs can show more than a policy that says a human makes the final decision.

Is a keyword search in an applicant tracking system an AEDT?

It depends on the computational method, output, and use. A basic search or data-retrieval function may fall outside the definition, while a system that converts keyword patterns into a candidate score, classification, or recommendation and gives that output substantial decision weight presents a stronger coverage case.

Can an employer rely on the vendor’s bias audit?

Only after checking that the audit satisfies the city’s requirements and corresponds to the deployed model, configuration, data context, and decision output. A fairness report, internal vendor test, expired audit, or audit of another version should not be treated as equivalent evidence without further analysis.

What should an employer collect before asking counsel for an assessment?

Collect the data-flow diagram, model and configuration identifiers, screenshots, thresholds, recruiter instructions, event and override logs, affected requisitions, location records, vendor contract, audit materials, public summary, and candidate notices. For the 80-point queue, those records reveal whether the score merely informed review or determined who entered it.