How to Read NYC’s Hiring-AI Bias Audit Before You Apply

The concrete object to find is the employer’s intersectional impact-ratio table, usually a PDF or webpage linked from a careers site. That table compares outcomes for groups defined by both sex and race or ethnicity, and it is the best place to see what the audit measured, what it combined, and what disappeared from the calculation.

Local Law 144 makes it unlawful for a covered employer or employment agency to use an automated employment decision tool unless, in the law’s words, “no more than one year prior to the use of the tool, the tool has been the subject of a bias audit.” A summary of the latest results and the tool’s distribution data must be public before use.

That is a publication and testing requirement. It is not a government certification that the tool is fair, accurate, job-related, or lawful under every applicable civil-rights rule.

Start with the employer, not the audit vendor

Open the employer’s careers site and look for links labeled “bias audit,” “automated employment decision tool,” “AEDT,” “Local Law 144,” or “candidate notice.” An AEDT is a computational system that produces a score, classification, or recommendation and substantially assists or replaces a hiring or promotion decision.

The employer’s notice matters because it should connect the audit to a live hiring workflow. New York City’s rules require notice before a covered tool is used, including the job qualifications or characteristics the tool will assess. The employer must also explain how a candidate can request an alternative selection process or accommodation, although the law does not require the employer to provide an alternative in every case.

Save the notice and audit summary together. Record the employer, the tool or vendor name, the audit date, the distribution date listed for the tool, and the page where the employer says it uses the system. If the notice names a video-interview assessment while the linked audit covers a résumé-ranking product, the documents do not establish that the system evaluating you was audited.

Names alone may not resolve the match. A vendor can sell several assessments under one platform name, and employers can configure scoring thresholds, job competencies, knockout questions, or workflow stages differently. The public-summary rule does not guarantee a model version, configuration identifier, or technical change log, so an audit that names only a broad platform may leave the tested system ambiguous.

That ambiguity is a finding. Do not repair it by assumption.

Trace the exact decision point

Next, locate the sentence describing what the tool does. The meaningful distinction is not whether marketing material calls it AI. It is whether the system ranks applicants, assigns scores, sorts people into classifications, or recommends who moves forward, and whether the employer relies on that output in one of the ways covered by the city’s rules.

The rules treat a tool as substantially assisting a decision when its output is the sole basis for the decision, receives more weight than any other criterion, or overrules conclusions drawn from other factors. Software that only stores applications may fall outside that definition. A résumé score used to decide who receives an interview can fall inside it.

Write the workflow in order: application submitted, résumé parsed, score assigned, threshold applied, recruiter review. Then mark the step covered by the audit. If the audit measures who passed the score threshold but the employer later rejects candidates through a separate automated interview assessment, the first audit says nothing about the second system.

Return to the intersectional table. Its title or accompanying methods note should identify whether the results concern screening, scoring, classification, hiring, or promotion. If the document never names the decision point, it cannot show which stage produced the reported disparities.

Check which jobs were pooled

Find the audit’s scope statement, data description, appendix, or list of job categories. Look for recognizable groups such as sales roles, hourly operations positions, technical jobs, or all applicants assessed during a stated period. The important question is whether the job you want belongs to the population behind the table.

A pooled result can be mathematically correct while saying little about one position. Suppose a tool is used across jobs with different applicant pools, selection thresholds, and hiring volumes. Combining those records can make one large category dominate the calculation, while a smaller occupation with a different outcome pattern becomes difficult to see.

Local Law 144 requires published rates and ratios for the audited data, but a public summary does not necessarily give an applicant a separate table for every requisition or model configuration. If the audit says “all positions” without listing job families, locations, thresholds, or assessment types, mark the job-level result as untested in public, not as favorable.

Also inspect geography. An employer may publish a nationwide dataset for a tool used in New York City. That can increase the sample available to the auditor, but it does not establish that city applicants, a particular borough, or a specific recruiting channel experienced the same outcomes.

Recalculate one row

For a tool that selects people to advance, the selection rate is the number selected in a demographic category divided by the number assessed in that category. The impact ratio compares that category’s selection rate with the rate for the category selected most often.

If the table provides counts, calculate one row yourself. Divide selected candidates by assessed candidates, then divide that selection rate by the highest rate shown in the relevant comparison. Small differences caused by displayed rounding are ordinary. A larger mismatch may mean the auditor used unrounded figures, excluded records, or applied a definition that the summary does not explain.

For systems that assign scores or classifications rather than directly selecting candidates, the rules call for average-score calculations and ratios based on the highest average score. Do not read an average-score ratio as though it were a hiring rate. A narrow difference in average scores can still matter when an employer sets a cutoff near where many applicants cluster, while a larger average difference may have little effect if recruiters ignore the score.

The frequently cited four-fifths measure, an impact ratio below 0.80, comes from federal employment guidance as a practical indicator for examining adverse impact. Local Law 144 does not turn that number into a passing grade, and the city does not require employers to stop using a tool because an audit reports a low ratio.

The intersectional table is more revealing than separate sex and race tables because aggregation can hide a disparity affecting, for example, one sex-and-race category. Even then, the result describes association in the audited records. It does not explain whether the tool caused the difference, whether the underlying job requirements were valid, or whether later human decisions changed the outcome.

Account for people missing from the denominator

Now look beneath the table for exclusions. The city’s rules require the summary to report how many people were not included because they fell into an unknown demographic category. That count matters because demographic information is often voluntary, unavailable, or stored separately from recruiting records.

An audit with a large unknown group describes the subset whose categories could be matched, not every applicant the tool assessed. If disclosure rates differ among applicant populations, the observed ratios may not represent the missing records. The audit cannot recover that uncertainty by labeling the remaining dataset representative.

An independent auditor may also exclude a category that contains less than 2 percent of the data used for the audit. The public summary should give the justification and the number of applicants in an excluded category. This rule limits unstable calculations and disclosure risks in very small groups, but it also means the people most likely to vanish from the impact-ratio table may be members of the least represented categories.

The mandated categories do not cover every protected or relevant characteristic. The audit framework centers on specified sex and federal race or ethnicity categories, including intersectional combinations. It does not by itself test differences by age, disability, religion, sexual orientation, gender identity beyond the reported categories, socioeconomic status, or caregiving status.

Absence is not a neutral result. Write down each unknown, excluded, uncollected, or unreported group beside the intersectional table, because the table’s clean ratios apply only to the records left inside it.

Separate the audit from the claims around it

A Local Law 144 audit demonstrates that an independent auditor calculated the required outcome measures on a described dataset, assuming the summary supplies enough detail to connect those measures to the deployed tool. It can expose disparities worth investigating and give applicants a concrete basis for asking which assessment affected them.

It does not test predictive accuracy unless the auditor separately performed that work. It does not prove that the traits measured are necessary for the job, inspect accessibility for disabled applicants, establish why demographic rates differ, or show how often recruiters override the system. A vendor may commission broader validation, but that evidence should appear as a separate study with its own methods.

Independence has a defined role here: the auditor must be capable of exercising objective and impartial judgment and cannot have prohibited involvement or financial interests specified by the rules. Independence reduces one conflict. It does not make sparse data complete or turn a pooled analysis into a job-specific one.

The final applicant record should fit on one page: the named tool, decision point, covered jobs, data source, audit period, selection or scoring calculation, unknown count, small-group exclusions, and any missing version or configuration. Keep the employer notice beside it. If those two documents do not connect, the published audit has not shown that its intersectional table describes the system used on your application.

Questions people ask

Does a published

NYC bias audit mean the hiring tool passed?

No. Local Law 144 requires a recent independent audit and public summary, but it does not establish a passing impact ratio or require the city to approve the tool before use. The table reports measured differences in the audited dataset; it is not a certificate of accuracy, fairness, or compliance with every employment law.

What should I do if my job category is not listed?

Check the methods note for pooled positions, job families, locations, and assessment types. If your role cannot be connected to the audited population, treat the public evidence as incomplete for that job and preserve the employer’s AEDT notice, which should identify the qualifications or characteristics assessed in your hiring process.

Why are some demographic categories missing from the table?

Applicants may not have supplied demographic data, records may have been categorized as unknown, or an auditor may have excluded a category representing less than 2 percent of the audit data. The summary should disclose unknown counts and justify qualifying small-category exclusions, but those records do not contribute to the published ratio.

Can

I compare impact ratios from two employers directly?

Only with caution. Confirm that both audits cover the same type of decision, comparable job populations, similar data periods, and the same selection or scoring calculation. Ratios produced from different thresholds, occupations, applicant sources, or rates of missing demographic data can look comparable while describing different hiring systems.