Your Model Provider Cannot Clear Your Hiring Assistant

Take a hiring assistant with one narrow job. It reads a résumé and job description, calls a hosted language model, then returns a match score and a short explanation to a recruiter. Scores below a configured threshold go to a rejection queue. The application log records `model=latest`.

That last field creates a governance problem. The developer knows which commercial service handled the request, but not necessarily which model snapshot produced the score, what changed since the previous snapshot, which evaluations covered that release, or whether the provider now restricts this use. If a rejected candidate challenges the result, the team may be unable to reconstruct the decision.

A general-purpose AI model, or GPAI model, can perform a wide range of tasks and can be integrated into many downstream systems. Its provider can document the model and test recurring model-level hazards. The hiring application’s builder controls the résumé parser, prompt, scoring rule, recruiter interface, logs, and fallback. Governance depends on keeping those responsibilities separate.

The provider’s paper trail has to identify the model

The first procurement request should concern identity, not benchmark performance. Ask for a stable model name or snapshot, its release status, supported input and output formats, context limits, integration requirements, geographic availability, and the conditions under which the provider may route a request to another model. A model card, meaning a structured account of intended uses, tests, and limitations, is useful only when it maps to the version called in production.

The documentation should also explain data handling. The hiring-app team needs to know whether prompts and outputs are retained, whether they can be used for provider training, where processing occurs, which subprocessors may receive them, and what account setting or contract controls each behavior. Those facts determine whether résumés can enter the service at all. Marketing language about enterprise privacy does not replace a retention period or a description of abuse-monitoring access.

For developers serving the European Union, this request now has a regulatory basis. Article 53 of the EU AI Act requires GPAI providers to “draw up and keep up-to-date the technical documentation of the model, including its training and testing process and the results of its evaluation.

” It also requires them to make information and documentation available to downstream providers that intend to integrate the model.

Annex XII makes that downstream file more concrete. It covers the model’s tasks, acceptable-use policies, release information, supported modalities, technical integration requirements, input and output design, relevant training information, and known or foreseeable circumstances that could create risks. The rule does not promise downstream developers a raw training corpus, model weights, or every internal research artifact. Confidential information and intellectual property remain protected.

As of March 2026, the GPAI duties apply to models placed on the EU market after the relevant August 2025 application point. The European Commission’s fining powers for GPAI providers begin later, in August 2026, while models placed on the market earlier have a longer transition period. The Commission’s GPAI Code of Practice is voluntary; providers may use it to demonstrate compliance, but signing the code is not the statutory requirement itself.

Usage limits belong in the system design

An acceptable-use policy can change the product architecture. If a provider permits employment-related assistance only with qualified human review, the application cannot send low scores directly into an automated rejection action. The recruiter needs enough information and authority to disagree, while the system needs to record whether that review occurred.

Ask the provider to identify prohibited uses, restricted high-impact uses, required safeguards, input-content restrictions, and the consequences of a policy breach. Those terms should appear in the contract or another controlled document with a revision history. A web page that can change without notice is hard to translate into an engineering control.

The hiring team should then encode the applicable limit. It can disable automatic disposition, require a recruiter acknowledgment before moving a candidate, and block unsupported file types before an API call. These controls add interface steps and review time. They also prevent the organization from operating a workflow that its model provider never agreed to support.

Evaluation evidence needs methods, not badges

A provider’s evaluation report should name the tested model snapshot, test set, prompt conditions, sampling settings, scoring method, and known exclusions. Results from safety testing, adversarial testing, multilingual tasks, hallucination checks, and tool-use evaluations can show where the base model tends to fail. Confidence intervals or repeated runs matter when outputs vary between calls.

Public system cards from companies such as OpenAI, Anthropic, and Google illustrate the type of evidence available, although coverage and detail differ by model. A useful report lets the buyer connect a claim to a test. “Improved instruction following” does not reveal whether the comparison used the same prompt, evaluator, or release configuration.

Provider evidence still cannot validate the résumé workflow. The provider did not choose the match-score prompt, the rejection threshold, or the candidate pool. It also does not know whether the parser drops employment dates from image-based résumés, whether names influence scores, or whether recruiters defer to a confident explanation despite contradictory experience listed in the source document.

The deployer must run an evaluation set drawn from the intended use, with lawful access and appropriate privacy controls. Reviewers should compare model output with job-related criteria, examine error rates across relevant groups where such testing is permitted, repeat variable calls, and test the entire path from file upload to recruiter action. This costs API calls and staff time. A smaller, representative test with documented limits is more defensible than a large public benchmark that never exercises the application.

Adversarial testing also belongs downstream. A résumé can contain hidden or visible text telling the model to ignore its scoring instructions. The provider can test its base model for prompt injection, which is an instruction embedded in untrusted input, but only the application builder can test whether its parser preserves that text, whether the system prompt resists it, and whether the output can trigger a consequential action.

Change notices keep old decisions reproducible

Return to `model=latest`. If the provider silently redirects that alias after a model update, two identical résumés may receive different scores even though the hiring team changed no code. The new model may follow instructions better overall while assigning weight differently to gaps in employment. A general release note will not reconstruct the earlier call.

Ask for pinned snapshots, a deprecation schedule, advance notice of material behavior or policy changes, and a rollback path. Each production record should capture the provider, exact model identifier, application version, prompt version, relevant settings, and output. Sensitive inputs require access controls and a retention rule; governance does not justify keeping every résumé indefinitely.

Before adopting a replacement snapshot, the team should replay an approved evaluation set in a staging environment and compare outcomes with the production model. A canary release, which sends a limited share of traffic to the candidate version, can expose operational failures before full migration, although parallel calls increase cost and retaining the old snapshot may not be available on every service tier.

A provider that offers no stable identifier, no material-change notice, and no usable evaluation record may still be adequate for drafting low-stakes internal text. It is a poor dependency for the hiring assistant. The application builder would be accepting behavior changes it cannot attribute and decisions it may not be able to reproduce.

Put the responsibility split in writing

The procurement record should state what the provider supplies and what the deployer verifies. Provider deliverables include versioned documentation, applicable usage rules, model-level evaluation evidence, security and data-handling terms, incident channels, and change notices. Deployer work covers workflow testing, population-specific performance, legal classification, human-review design, monitoring, and response to affected people.

That record should name an owner for each item and define what blocks release. For the hiring assistant, missing snapshot identification should stop deployment before the first résumé is scored. A provider questionnaire without a release gate is only paperwork.

Questions people ask

Can a provider’s model card satisfy an application audit?

Usually not by itself. A model card can establish what the provider tested and which limitations it disclosed, but an application audit also needs the deployed prompt, model snapshot, data flow, decision rule, logs, human-review evidence, and tests conducted on the intended population.

Is using a model alias such as `latest` always unsafe?

No. It can be reasonable for low-stakes drafting where behavior drift is tolerable and outputs receive ordinary review. For a consequential workflow, an alias weakens reproducibility unless the provider exposes the resolved snapshot in every response and gives the developer enough notice to test changes.

Does human review make AI-assisted hiring compliant?

Human review can satisfy a provider restriction or reduce automation risk, but the label alone proves little. The reviewer needs relevant evidence, time to inspect it, authority to override the score, and a recorded action. The deployer must test whether people meaningfully review outputs rather than approving them by default.

What should a builder do if the provider will not disclose training data?

Ask for the information it can supply: data categories and sources, collection periods, curation methods, copyright controls, known gaps, and evaluation results tied to the model version. If those disclosures cannot support the application’s risk assessment, narrow the use or choose a provider whose documentation can clear the release gate.