Why HR tech is the highest-risk AI category
Every enterprise AI deployment carries reputational, regulatory and operational risk. HR tech concentrates all three. The decisions are consequential to individuals (hiring, promotion, performance management, compensation, termination). The training data is heavily correlated with protected attributes (race, sex, age, disability). The regulatory environment is hostile to opacity. And the vendor pool is dominated by mid-market SaaS firms whose default contractual language was written for performance-management software, not for systems making decisions about humans.
Across 28 HR tech AI procurement engagements over the last 18 months, we have not yet seen a vendor first draft that met our minimum clause standard. The good news is that the gap is consistently negotiable. The bad news is that the buyer must require it explicitly from the first redline; otherwise the contract carries forward default language that will fail on first audit or first regulatory enquiry.
The regulatory frame: EU AI Act, GDPR, state laws
The EU AI Act classifies most HR tech AI uses as "high-risk" under Annex III, which triggers a substantial compliance load: conformity assessment, risk management system, data governance, technical documentation, record-keeping, transparency and human oversight obligations. For a US-headquartered enterprise with even a single EU subsidiary or EU-resident employee, the obligations apply to the AI provider, the deployer, the distributor and the importer in the supply chain. Contracts must allocate these obligations or leave the buyer exposed.
GDPR Article 22 governs automated decisions producing legal or similarly significant effects. HR tech systems making hiring decisions almost certainly meet this threshold. Article 22 requires meaningful human review, the right to obtain human intervention, the right to express a point of view, and the right to contest the decision. The vendor contract should explicitly support these rights operationally, not merely acknowledge them.
US state laws (notably the New York City AEDT law, Illinois' AIVIA Act, Colorado's AI Act and California's emerging frameworks) add further obligations: bias-audit publication, candidate notification, AI-system inventory and impact assessments. The contractual implication is that AI deployers need defensible records they can produce on regulatory request, and the vendor needs to deliver them.
Watch for "Customer is solely responsible for ensuring compliance with applicable laws" as the entirety of the regulatory allocation. This shifts the entire compliance burden to the customer while leaving the vendor in control of the system. The clause should be replaced with a specific allocation: vendor provides documentation, bias audit results and conformity assessment evidence; customer provides operational deployment governance and human-review processes.
Data protection clauses
Four data-protection clauses are non-negotiable for HR tech AI procurement.
First, training-data warranties. Customer data (CVs, performance reviews, interview transcripts, compensation data) must not be used to train the vendor's foundation models or to improve other customers' systems, except where the customer explicitly opts in to a clearly-scoped programme. The default vendor language often permits "improvement of the service", which is too broad. Tighten to "improvement of the customer-specific instance only, with no propagation to other tenants or to foundation models".
Second, data residency. EU and UK data must rest and be processed in EU and UK regions, with explicit prohibition on transfer to non-adequate jurisdictions absent appropriate safeguards (Standard Contractual Clauses with documented Transfer Impact Assessment). Major HR tech vendors will agree to region-locking when asked; few volunteer it.
Third, deletion rights. GDPR Article 17 right to erasure applies to training contributions, not only to operational data. The contract should provide for verifiable erasure of an individual's data from training sets within a defined window (typically 30 days), with a vendor obligation to confirm execution.
Fourth, sub-processor transparency. The vendor must list all sub-processors, provide notice of changes, and grant the customer reasonable objection rights. For HR tech AI vendors using third-party foundation models (OpenAI, Anthropic, Google), the foundation-model provider is a sub-processor and must be named.
Bias-audit rights
Bias-audit clauses separate compliant HR tech vendors from non-compliant ones. The defensible specification has six elements.
Auditor selection: the customer selects the auditor from a panel of pre-approved firms, with the vendor's right of reasonable objection limited to conflict-of-interest concerns rather than general preference. The vendor must not be the auditor and the auditor must not be paid by the vendor.
Audit scope: model performance disaggregated by protected characteristic (race, ethnicity, sex, age, disability status where collectable). The disaggregation should cover false-positive rate, false-negative rate, calibration and rank-order accuracy. Aggregate metrics alone do not detect bias.
Audit frequency: annual minimum, plus on any material model change (defined as a change to training data, architecture or fine-tuning approach). Quarterly audits are appropriate for high-volume decision systems.
Remediation obligations: the vendor must remediate findings within a timeline that reflects severity. Material findings (impact > 4/5 rule by EEOC guidance) require remediation within 60 days; minor findings within 180 days.
Disclosure rights: the customer must have the right to disclose audit results internally, to regulators on request, and to candidates where applicable law requires. The vendor should not have veto over disclosure.
Costs: bias-audit costs should be shared 50/50 between vendor and customer, or borne by the vendor where the audit results from a contractual trigger such as a customer regulatory enquiry.
Model documentation requirements
Model documentation should be a contractual deliverable, refreshed at each material model change, covering five areas.
Model card: model name, version, architecture family, training data sources at category level (e.g. "publicly available CVs from open repositories", "customer-supplied training data for fine-tuning"), known limitations, intended use, out-of-scope use.
Training data summary: high-level description of training data composition, including demographic distribution where available, time period covered, and any data-cleaning or rebalancing applied. The vendor will resist disclosing proprietary training data details but should commit to a category-level summary.
Evaluation results: performance metrics on held-out test sets, disaggregated by protected characteristic where available, with documentation of evaluation methodology.
Conformity assessment evidence: where the EU AI Act applies, the vendor must provide the conformity assessment documentation. For high-risk systems this includes the technical documentation under Annex IV of the Act.
Change log: every material model change documented with date, scope and impact assessment. This becomes the basis for re-evaluation triggers and bias-audit re-tests.
Performance SLAs and accuracy thresholds
Most HR tech AI contracts specify SLAs as system availability (99.9% uptime). For decision-support systems, availability is the least important SLA. Accuracy thresholds, latency commitments and degradation policies are more material.
Accuracy thresholds should specify the minimum acceptable performance level on a representative test set, with consequences for breach. For a CV-screening model, this might be "minimum 0.78 AUROC on the customer's industry-representative held-out test set, measured monthly, with model-revert obligation on three consecutive monthly failures".
Latency commitments matter for candidate-facing experiences. End-to-end latency above 3 seconds materially degrades candidate experience and conversion.
Degradation policies cover what happens when the model's performance drifts. The contract should require monitoring, customer notification on drift, and remediation obligations on the vendor.
IP indemnity and training-data warranties
IP indemnity is the most contested clause in any AI vendor contract. For HR tech, the indemnity must cover three risk categories.
Training-data IP: the vendor must warrant that training data was lawfully obtained and that use does not infringe third-party rights. The vendor should indemnify the customer for any claim arising from the vendor's training data. The standard carve-out for "publicly available data" is too broad; tighten to "data lawfully obtained with appropriate licences for AI training use".
Output IP: the customer should have ownership of model output. The vendor should indemnify the customer for any third-party IP claim arising from output, subject to standard exclusions (customer modifications, combination with other content, use outside scope).
Prompt injection and adversarial use: where the customer's use of the system exposes the vendor's model to adversarial input (e.g. malicious CVs designed to manipulate the model), the contract should allocate liability based on whether the vendor had reasonable defences in place.
Major HR tech AI vendors privately maintain a "tier 2" indemnity language available only on request, materially stronger than their published terms. We have seen this in 11 of 14 engagements where we asked specifically for "the enterprise indemnity language used for regulated customers". The published terms are a starting position, not the ceiling.
Five default vendor failures to fix
Across the 28 HR tech AI procurement engagements referenced above, the same five default failures recur in vendor first drafts.
Training-data warranties scoped only to "Customer prompts and inputs" rather than "Customer prompts, inputs, and any documents or data the Customer makes available to the service". Tighten the scope.
Bias-audit clauses that name the vendor as auditor, specify only vendor-led assessment, or omit independent third-party rights entirely. Require an independent auditor with customer selection.
IP indemnity carve-outs that exclude "training data" or "model output" from coverage. Negotiate explicit coverage of both, with reasonable exclusions.
Performance SLAs that measure only availability, not accuracy or latency. Add accuracy thresholds and degradation policies tied to monthly measurement.
Data residency commitments described as "best efforts" rather than contractually binding. Require region-locked guarantees with audit rights and remedies.
For a fuller treatment of AI procurement strategy, see our AI procurement advisory practice page or the AI Vendor Contract Red Flags white paper. For HR tech vendors specifically, our Workday negotiation experts page covers the Workday-specific commercial mechanics.
Strategic advisory — not legal advice. Specific clause language should be reviewed with qualified counsel in your jurisdiction.