Red lines · Fundamental rights
Prohibited AI practices
A short list of AI uses is banned in the EU outright — not because the technology is exotic, but because the practice itself is judged incompatible with fundamental rights. This page explains the logic with educational examples.
01What "prohibited" means here
For most risky AI the Act's answer is obligations. For prohibited practices the answer is no: no amount of documentation, oversight or consent forms makes them acceptable. That absoluteness is rare in the regulation and worth dwelling on in class: what kind of harm justifies a red line?
02Educational examples
- Generalized social scoring — aggregating unrelated behaviour (payments, associations, protests) into a score that gates access to services in unrelated contexts. The canonical example, and the game's opening case.
- Harmful manipulation — systems that exploit vulnerabilities (age, disability, situation) to materially distort behaviour and cause harm.
- Emotion recognition in schools and workplaces — inferring feelings of students or employees, with narrow exceptions.
- Untargeted facial-image scraping — bulk harvesting of faces to build recognition databases.
These summaries are deliberately compressed; each prohibition in the actual text has precise conditions and exceptions.
03Why these, and not others
The common thread is structural harm to dignity and autonomy: being scored as a person rather than assessed for a service; being manipulated below the threshold of awareness; being emotionally X-rayed by your school or employer. These are harms to standing, not just outcomes — which is why safeguards don't cure them.
The practices in this list also sit in the highest penalty tier: how much, and from when, on AI Act penalties.
04The boundaries are the lesson
Not every score is social scoring: credit scoring for credit, with relevant data and contestability, is a different animal. The game's mirror case ("Civic credit") exists precisely to make students argue the boundary. Boundary-drawing — not memorising the list — is the durable skill.
05Educational caution
Continue with high-risk systems — the level where most real classroom debates end up — or test the red lines in the game.
06Boundary cases, side by side
A prohibition almost never covers an entire technology: it covers a use, in a context, with certain effects. That is why systems that look identical land on opposite sides of the line.
| Use that looks prohibited | Use that normally is not | What makes the difference |
|---|---|---|
| A score that follows a citizen across school, healthcare and housing | Creditworthiness assessment for a single loan | Generalisation: when a penalty migrates into contexts unrelated to where it originated, you are in social-scoring territory. |
| Detecting students' emotions in class | Detecting driver drowsiness for safety | Context and purpose: school and work are settings where the regulation draws a sharp line, because of the power imbalance between the parties. |
| Predicting who will commit a crime from a profile alone | Analysing where thefts have already clustered | The object of the prediction: predicting a person is not the same as describing a phenomenon that already happened. |
| Live biometric identification in a square for policing purposes | Unlocking your phone with your face | The perimeter: public space, real time and law-enforcement purpose are the three elements that trigger the prohibition (with narrow, authorised exceptions). |
Notice the recurring structure across the three columns: the difference is almost never how advanced the system is, but who bears the effect and how free they are to walk away. It is the same question that governs the boundary with high risk.
07Exercise: move the boundary
A short activity that works well right after playing a case. Take an allowed use and change one element at a time, until the class agrees it has crossed the line.
- Step 1 — start from something uncontroversial: an app suggesting revision exercises based on mistakes made.
- Step 2 — change who decides: the result now determines class assignment. What changes?
- Step 3 — change the data: the system now also reads facial expression while studying.
- Step 4 — change persistence: the profile follows the student through their whole school career and is shared with other bodies.
- Closing — at which step did the class feel uncomfortable, and why? Discomfort usually arrives before the formal prohibition does: it is a good didactic indicator.
Ready-made classroom material, with timings and a discussion rubric, is in the classroom activities and the classroom lab.