Plain language · Practitioner-maintained

Glossary

61 terms for AI governance, risk management, and compliance — written for practitioners, not lawyers.

A

AI Risk · Risk & Safety: The potential for negative consequences arising from the development, deployment, or use of AI systems. AI risks span technical failures, misuse, unintended consequences, and systemic impacts. Governa…
AI Regulatory Sandbox · Regulation: A controlled environment established by a regulator allowing innovators to test AI systems in real-world conditions under regulatory supervision, often with relaxed requirements. Required in each EU m…
Algorithmic Impact Assessment · Governance: A structured evaluation of the potential impacts of an AI or algorithmic system on individuals, groups, and society before and during deployment. Similar to privacy impact assessments but broader in s…
Autonomous System · AI Systems: An AI system capable of performing tasks and making decisions with minimal or no human intervention. Autonomy in AI systems exists on a spectrum from decision-support to fully autonomous operation. Go…
AI Safety · Risk & Safety: The field concerned with ensuring AI systems behave as intended and do not cause unintended harms. Encompasses both near-term safety (preventing AI failures in deployed systems) and long-term safety (…
Artificial Intelligence (AI) · AI Systems: A machine-based system that can, for a given set of objectives, make predictions, recommendations, or decisions influencing real or virtual environments. AI systems are designed to operate with varyin…
AI Procurement · Organizational: The process by which organisations acquire AI systems or services from external vendors, including due diligence, contracting, and ongoing vendor management. Responsible AI procurement requires assess…
Algorithmic Decision-Making · AI Systems: The use of automated systems or algorithms to make or significantly influence decisions affecting individuals or organisations. Encompasses everything from simple rule-based systems to complex machine…
Adversarial Robustness · Technical: The ability of an AI system to maintain correct performance when exposed to adversarial inputs designed to cause errors or unexpected behaviour. Adversarial examples are inputs crafted to fool AI syst…
AI Audit · Governance: A systematic examination of an AI system to assess its compliance with requirements, standards, or ethical principles. Can be conducted internally or by independent third parties. The EU AI Act requir…
Alignment · Risk & Safety: The challenge of ensuring that AI systems pursue goals and exhibit behaviours consistent with human values and intentions. Misalignment occurs when an AI system optimises for a proxy goal that diverge…
AI Governance · Governance: The systems, structures, policies, and processes by which AI development and deployment are directed, managed, and controlled. Encompasses organisational governance (internal policies, oversight board…
AI Maturity Model · Standards: A framework for assessing and improving an organisation's AI governance capabilities across defined maturity levels. Typically covers dimensions such as strategy, culture, data, processes, and technol…
AI Ethics Board · Organizational: An internal or external governance body responsible for overseeing an organisation's AI development and deployment from an ethical perspective. Provides guidance on ethical issues, reviews high-risk A…
Algorithmic Accountability · Governance: The principle that organisations and individuals responsible for AI systems should be answerable for their impacts. Requires mechanisms for identifying, assessing, and addressing harms caused by algor…
Algorithmic Fairness · Ethics & Fairness: The property of AI systems that do not produce systematically biased outcomes for individuals or groups based on protected characteristics. Multiple competing mathematical definitions of fairness exis…

B

Bias · Ethics & Fairness: Systematic errors in AI system outputs that produce unfair outcomes for certain groups. Can originate in training data (historical biases, representation gaps), model design (inappropriate features), …
Black Box · Ethics & Fairness: An AI system whose internal workings are opaque and difficult to understand, even for its developers. Deep learning models are typically considered black boxes due to their complexity. Black box syste…

C

Conformity Assessment · Regulation: The process of verifying that a high-risk AI system meets the requirements of the EU AI Act before it is placed on the market. Can be conducted through self-assessment or third-party audits depending …
Catastrophic Risk · Risk & Safety: The risk of AI causing large-scale irreversible harm, potentially affecting entire populations or humanity. Includes scenarios involving misuse of AI for weapons of mass destruction, loss of human con…

D

Deep Learning · AI Systems: A type of machine learning using artificial neural networks with many layers to learn representations of data. Underpins most modern AI capabilities including image recognition, natural language proce…
Dual-Use AI · Risk & Safety: AI systems or capabilities with legitimate civilian applications that can also be repurposed for harmful ends. Examples include AI-generated synthetic media (deepfakes), AI-assisted cybersecurity tool…
Differential Privacy · Technical: A mathematical framework for adding calibrated noise to datasets or model outputs to protect individual privacy while preserving statistical utility. Provides formal, provable privacy guarantees unlik…
Demographic Parity · Ethics & Fairness: A fairness criterion requiring that an AI system's positive outcome rate be equal across demographic groups. Also known as statistical parity. While intuitive, demographic parity can conflict with oth…
Data Poisoning · Technical: An attack in which malicious data is injected into an AI system's training dataset to manipulate its behaviour. Can cause a model to learn incorrect patterns, develop backdoors, or produce targeted er…
Disparate Impact · Ethics & Fairness: A legal and statistical concept describing when an AI system or policy produces significantly different outcomes for groups defined by protected characteristics, regardless of intent. A disparate impa…

E

Emergent Capabilities · Risk & Safety: Abilities that arise unexpectedly in AI models as they scale in size or training data, not present in smaller versions of the same model. Examples include chain-of-thought reasoning and multi-step pro…
Explainability · Ethics & Fairness: The ability to provide understandable reasons for an AI system's outputs or decisions to relevant stakeholders. Distinct from interpretability: explainability focuses on providing useful post-hoc expl…

F

Foundation Model · AI Systems: A large AI model trained on broad data at scale that can be adapted to a wide range of downstream tasks. Examples include GPT-4, Claude, and Gemini. The EU AI Act addresses foundation models under the…
Federated Learning · Technical: A machine learning approach in which model training is distributed across multiple devices or organisations without centralising raw data. Participants share model updates rather than raw data, preser…
Fine-tuning · Technical: The process of further training a pre-trained foundation model on a smaller, task-specific dataset to adapt it for a particular application. Fine-tuning allows organisations to customise general-purpo…
Fundamental Rights Impact Assessment (FRIA) · Regulation: An assessment required under the EU AI Act for deployers of high-risk AI systems in certain public contexts, evaluating the system's potential impact on fundamental rights. Must be conducted before de…

G

General-Purpose AI (GPAI) · AI Systems: An AI model trained on broad data at scale that is capable of competently performing a wide range of distinct tasks. The EU AI Act defines GPAI models and imposes specific obligations on providers inc…
Generative AI · AI Systems: AI systems capable of generating novel content including text, images, audio, video, and code. Generative AI models learn patterns from training data and produce new outputs that resemble but are not …

H

High-Risk AI System · Regulation: Under the EU AI Act, an AI system that poses significant risks to health, safety, or fundamental rights. Includes AI used in biometric identification, critical infrastructure, education, employment, e…
Human-in-the-Loop · Organizational: A design pattern in which a human is actively involved in the decision-making process of an AI system, typically reviewing and approving AI-generated outputs before they take effect. Required for cert…
Human Oversight · Organizational: The capacity for humans to monitor, evaluate, and intervene in AI system operations to prevent or correct harmful outcomes. A core requirement of the EU AI Act for high-risk AI systems and a principle…
Hallucination · Risk & Safety: The tendency of AI language models to generate factually incorrect, fabricated, or nonsensical outputs with apparent confidence. Hallucinations arise from models completing patterns in training data r…

I

Interpretability · Ethics & Fairness: The degree to which a human can understand the internal mechanisms of an AI model and predict its behaviour. More interpretable models (e.g., decision trees, linear regression) are inherently easier t…
ISO 42001 · Standards: The first international standard for AI management systems (ISO/IEC 42001:2023), specifying requirements for establishing, implementing, maintaining, and continually improving an AI management system.…

J

Jailbreak · Risk & Safety: A technique used to bypass an AI system's safety constraints or content policies through carefully crafted prompts or inputs. Differs from prompt injection in that jailbreaks typically involve the mod…

L

Large Language Model (LLM) · AI Systems: A type of foundation model trained on large corpora of text data capable of generating, summarising, translating, and reasoning about natural language. LLMs are the basis for most modern generative AI…

M

Meaningful Human Control · Organizational: The principle that humans should maintain sufficient understanding and ability to intervene in AI-assisted decisions to bear moral and legal responsibility for outcomes. Goes beyond nominal human revi…
Model Drift · Technical: The degradation of an AI model's performance over time as the real-world data distribution diverges from the training data distribution. Can be caused by changes in user behaviour, external events, or…
Model Card · Governance: A short document accompanying an AI model that communicates its intended use, performance characteristics, limitations, and ethical considerations. Proposed by Mitchell et al. (2019) and now widely ad…
Machine Learning · AI Systems: A subset of AI in which systems learn from data to improve their performance on specific tasks without being explicitly programmed. Common approaches include supervised learning, unsupervised learning…

N

NIST AI RMF · Standards: The NIST Artificial Intelligence Risk Management Framework, a voluntary framework for managing AI risks organised around four functions: Govern, Map, Measure, and Manage. Provides a common language an…
Notified Body · Regulation: An independent organisation designated by an EU member state to conduct third-party conformity assessments for certain high-risk AI systems under the EU AI Act. Required for high-risk AI systems in bi…
Natural Language Processing (NLP) · AI Systems: A field of AI focused on enabling computers to understand, interpret, and generate human language. Underpins applications including chatbots, translation, sentiment analysis, and large language models…

P

Prohibited AI Practice · Regulation: Under the EU AI Act, AI applications deemed to pose unacceptable risks that are banned entirely. Includes AI systems using subliminal or manipulative techniques, exploiting vulnerabilities of specific…
Post-Market Monitoring · Regulation: The ongoing process of collecting and analysing data on AI system performance after deployment to identify and address issues. Required for high-risk AI systems under the EU AI Act, with providers obl…
Prompt Injection · Risk & Safety: An attack technique in which malicious instructions are embedded in input to an AI system to override its original instructions or extract sensitive information. Can occur in direct attacks (user mani…

R

Reinforcement Learning from Human Feedback (RLHF) · Technical: A training technique in which AI models are fine-tuned using feedback from human evaluators to align model outputs with human preferences. Used to train modern LLMs including ChatGPT, Claude, and Gemi…
Red-Teaming · Technical: A structured adversarial testing process in which a team attempts to find failures, vulnerabilities, and harmful outputs in an AI system. Borrowed from cybersecurity practice and adapted for AI safety…
Responsible AI · Organizational: An approach to AI development and deployment that considers the ethical, social, and governance implications throughout the AI lifecycle. Encompasses principles of fairness, transparency, accountabili…
Risk Tier · Standards: A classification of AI systems based on their potential for harm, used to determine applicable governance requirements. The EU AI Act uses four tiers: unacceptable risk (prohibited), high risk (strict…

S

System Card · Governance: An extended transparency document for AI systems describing their capabilities, limitations, safety evaluations, and deployment context. Broader than a model card as it covers the full deployed system…
Systemic Risk · Risk & Safety: Under the EU AI Act, the risk that GPAI models with very high training compute may pose risks at the level of the EU due to their reach, capabilities, or potential for misuse. Providers of GPAI models…

T

Transparency · Governance: The quality of AI systems and their governance processes being open, understandable, and explainable to relevant stakeholders. Encompasses technical transparency (how the system works), process transp…
Trustworthy AI · Organizational: AI systems that are lawful, ethical, and robust throughout their lifecycle. The EU's High-Level Expert Group on AI identified seven key requirements: human agency, technical robustness, privacy, trans…
Training Data · Technical: The dataset used to train an AI model, from which the model learns patterns and representations. The quality, diversity, and representativeness of training data fundamentally shapes model behaviour an…