Third-Party AI Risk: What to Evaluate Before You Buy or Build

Why Third-Party AI Risk Is Central

Most organizations consume far more AI than they build. Foundational models, embedded features in SaaS, and API-based tools bring third-party AI into the enterprise through paths that are often invisible to procurement. This makes third-party AI risk the single largest surface of AI risk for most companies.

The special difficulty is that a third-party AI tool is a black box with rights over your data and behavior you cannot fully observe. You could not run the vendor's risk assessment yourself, and you cannot easily verify what the model does with your inputs. Third-party AI risk management is therefore about asking the right questions, obtaining the right documentation, and designing the right controls before deployment.

The Risk Dimensions to Evaluate

• Data handling — key question: how is our data processed, stored, transmitted? — evidence: data flow diagram, encryption documentation.

• Access controls — who can access our data, what auth is required? — access control policy, SOC 2 report.

• Data retention — how long is data retained, can we request deletion? — data retention policy, deletion procedures.

• Subprocessors — does the vendor use subprocessors, who? — subprocessor list, DPA.

• Incident response — what happens in a breach? — incident response plan, breach notification procedures.

Model Behavior and Verification

For an AI tool, "does it work" is a risk question, not just a quality question. Evaluate:

• Model cards and documentation. Established vendors publish model cards describing training data, intended uses, and limitations. Treat their absence as a red flag.

• Evaluation results. Ask for accuracy, safety, and fairness benchmark results relevant to your use case.

• Red-teaming and safety testing. For higher-risk tools, ask whether the vendor conducted adversarial safety testing and how it handled findings.

• Known failure modes. Understand the specific failure class that matters for your use — confabulation for a chatbot, bias for a screening tool, drift for a forecasting model (NIST, 2024).

Security and Access Controls

Third-party AI adds a new channel for your data. Evaluate:

• Data in transit and at rest. Confirm encryption is documented.

• Access and permissions. How does the tool authenticate? Can you scope which employees and data it can reach?

• Supply chain. Does the vendor's own model come from a provider you'd accept? The AI supply chain now includes model providers, hosting, and inference infrastructure.

• Incident handling. Does the vendor have an incident-response and breach-notification process that reaches you on time?

Governance and Contractual Controls

Ask for the vendor's own AI governance. A vendor that cannot articulate its own risk-management practices (mapped to a framework like the NIST AI RMF) is less likely to manage yours responsibly (NIST, 2023).

Contract for the future, not just today. Include clauses for: transparency about material changes, a right to audit, data-deletion on termination, and flow-down of sub-processor obligations. As the EU AI Act's GPAI and transparency obligations mature, contract language should track them (European Commission, 2024).

A Practical Evaluation Workflow

• Classify the tool by data sensitivity and autonomy before evaluating.

• Questionnaire — run a standard vendor-AI questionnaire covering the dimensions above.

• Evidence review — check certifications, model cards, and DPAs rather than taking claims at face value.

• Contract — encode the controls and rights you need.

• Monitor — schedule a review of the risk each year or when the vendor materially changes.

This workflow is the subject of our companion procurement-checklist guide, which turns it into a usable checklist.

Sources

NIST. (2023). *Artificial Intelligence Risk Management Framework (AI RMF 1.0)*. National Institute of Standards and Technology. https://doi.org/10.6028/NIST.AI.100-1
NIST. (2024). *Generative Artificial Intelligence Profile (NIST AI 600-1)*. National Institute of Standards and Technology.
European Commission. (2016). Regulation (EU) 2016/679 — General Data Protection Regulation. *Official Journal of the European Union*.
European Commission. (2024). Regulation (EU) 2024/1689 of the European Parliament and of the Council. *Official Journal of the European Union*. https://eur-lex.europa.eu/eli/reg/2024/1689