AI concepts
Bias in AI (algorithmic bias)
Systematic deviations in an AI model's predictions that harm specific groups of people or cases. Bias can come from training data (historical inequalities), from model design, from the data labelling process, or from prompt engineering. AI models without a bias audit categorically fall under the documentation obligations of the AI Act from August 2026.
Primary source: NIST AI Risk Management Framework 1.0, Stanford HAI Algorithmic Justice Initiative 2024, MIT Sloan Hidden Bias in AI 2025
The history of ML has recorded dozens of high-profile bias cases: Amazon's recruitment system (2018) downgrading women's CVs, COMPAS (2016) estimating a higher recidivism risk for African Americans, the Apple Card (2019) offering women lower credit limits. The common denominator: the training data contained historical inequalities, and the model amplified them.
Four sources of bias
NIST AI Risk Management Framework 1.0 classifies them:
Historical bias. The training data reflects historical social inequalities. The model learns that certain groups received credit more often, so it recommends credit to those groups. The most common class.
Representation bias. Some groups are underrepresented in the training data. The model performs worse on them because it has seen fewer examples.
Measurement bias. The way a variable is measured differs between groups. A medical diagnosis is made differently for different genders, and a model trained on such data replicates the difference.
Aggregation bias. The model treats the whole population as homogeneous, while in fact it contains heterogeneous subpopulations that require different approaches.
Measuring bias
The Stanford HAI Algorithmic Justice Initiative from 2024 standardises four metrics. Demographic parity (the proportion of positive decisions is equal across groups). Equal opportunity (the true positive rate is equal across groups). Predictive parity (precision is equal across groups). Calibration (the predicted probability genuinely matches the empirical frequency).
These metrics are often mutually contradictory. Optimising one damages another. The choice of which to minimise is an ethical decision, not a technical one.
Polish regulatory context
The AI Act from February 2026 requires a documented bias assessment for high-risk systems (recruitment, credit scoring, access to education, healthcare). Without this documentation the deployment is illegal, with penalties of up to 35 million euro or 7 percent of annual turnover.
In addition, PUODO interprets the fair processing requirement from Article 5 of the GDPR as covering algorithms. A Polish firm can be held liable for bias in AI tools used in customer service.
Mitigants
MIT Sloan Hidden Bias in AI from 2025 describes five practices. A data audit before training (checking group representation, identifying historical bias). Diverse training data (deliberate expansion of the corpus). Fairness constraints in training (the model is penalised for unequal predictions). Continuous monitoring (bias drift over time). Human-in-the-loop for high-stakes decisions.
A bias audit for planned deployments is part of the AI Readiness Audit plus, optionally, Cybersecurity & AI Policy.