AI-generated caricatures, avatars, and chatbots are now a common part of the online world. However, few people realize that uploading photos or entering data into AI tools may involve the processing of biometric and personal data under the GDPR. Read on to find out what to watch out for when using artificial intelligence.
Do you already have a caricature of yourself on your profile, as seen by artificial intelligence (AI)?
Thousands of people have shared their AI caricatures in recent weeks. It’s a cute picture. But did you know that in doing so, you likely provided your biometric data to an American company, agreed to the use of your face to train an AI model, and if you also uploaded a photo of a child or colleague to the app, you may have violated data privacy laws?
Caricatures are just one example. Artificial intelligence has become part of our everyday lives—chatbots, text generators, AI assistants at work. However, few people think about what uploading photos or entering personal information into AI tools actually means from a data protection perspective.
Caricature, avatar, Ghibli filter, and your biometric data
When you upload a photo to an AI app that turns it into a caricature, an animated avatar, or an artistic portrait, the app does more than just “change the filter.” Technically, it performs facial recognition, extracts geometric parameters—eye distance, jaw shape, proportions—and processes them using an AI model. These parameters constitute biometric data under Article 9 of the GDPR1: a special category with the highest level of protection.
What typically happens to your data:
- Most free apps include a clause in their terms of service allowing them to use your photos to train their own AI model.
- Apps like Lensa require 10–20 selfies—thereby creating a mini-model directly from your biometric data.
- The viral ChatGPT Ghibli trend processes photos on OpenAI servers in the U.S.—this constitutes a transfer of personal data outside the EU, the conditions for which are governed by Chapter V of the GDPR.
- If you uploaded a photo of a child or a colleague: these individuals did not give their consent. This is a violation of the GDPR regardless of how “innocent” the result may seem.
Personal data is not just a birth number
A facial photograph is personal data. If it allows for the identification of a specific person—and in most cases it does—this constitutes the processing of personal data under the GDPR. Furthermore, people often do not realize what they are inputting into AI tools:
- face photos (including those of children)—uploaded to portrait generators, face-swap apps, and image editing tools,
- voice recordings—AI can infer health status, emotions, or identity from a voice,
- texts containing information about third parties—colleagues, clients, family members,
- scans of documents, contracts, invoices, client databases,
- health information, financial data, or internal company documents.
Where does your photo end up?
Most popular AI tools are operated on servers in the US or other countries outside the European Economic Area (EEA). The transfer of personal data to these countries is permitted only under the conditions set forth in Chapter V of the GDPR, e.g., based on Standard Contractual Clauses (SCCs) or an EU Commission adequacy decision.
Slovak Act No. 18/2018 Coll. on the Protection of Personal Data incorporates these obligations. The Slovak Data Protection Authority (ÚOOÚ SR) actively enforces them. Before using any AI platform, therefore, always verify whether the provider guarantees adequate safeguards under the GDPR.
Not only the transfer but also the retention period of data is of fundamental importance. Many AI tools reserve the right in their terms and conditions to use input data to improve services, train models, or develop new products. Users may thus lose control over their own content—including photos they uploaded in accordance with the terms of service they didn’t read during registration.
Would it send a chill down your spine if that got out?
Before you enter anything into an AI tool, ask yourself a simple question:
“What would happen if the content of my AI conversations became public?” If that thought sends a chill down your spine, it’s time to rethink how you use it.
Many people don’t realize that AI accounts with conversation histories are attractive targets for attacks. If someone gains access to your AI account (for example, through phishing), they can access your entire history of entered data—including internal plans, personal information, contracts, and financial data. The internet has one unpleasant characteristic: once something gets out, completely erasing it is often practically impossible.
Without proper security measures, you should not enter the following into public AI tools:
- sensitive personal data (health data, social security numbers, identification details),
- internal company documents and strategies, trade secrets,
- complete contracts, invoices, and client databases,
- passwords and login credentials,
- photos of other people without their knowledge or consent.
Who is responsible and for what?
When you use an AI tool for personal purposes, the controller (responsible entity) is usually the service provider. However, if you enter data about other people—colleagues, clients, partners—into the AI, you are the one responsible for protecting that data.
This applies doubly in a corporate environment. If a company uses ChatGPT, Microsoft Copilot, or another AI tool and employees enter clients’ personal data into it, the company acts as the controller and has the obligation to:
- have a legal basis for such processing,
- enter into a contractual relationship with the AI provider as a processor (processing agreement under Article 28 of the GDPR),
- inform the data subjects about this processing.
Failure to do so constitutes a violation of the GDPR, even if done in good faith. The result of employees’ errors—where they (even if unknowingly) enter sensitive company data or internal know-how into an AI tool—is not only a violation of internal guidelines but also damage to the company’s reputation, financial losses, or regulatory issues.
What the AI Act Brings—the European Regulation on Artificial Intelligence
As of August 2024, Regulation EU 2024/1689—the AI Act—is in effect, marking the first comprehensive legal framework regulating artificial intelligence. Its key provisions will take effect gradually:
- February 2025: prohibitions on prohibited practices (including real-time biometric identification in public spaces and social scoring systems) are fully enforceable.
- August 2025: rules for general-purpose models (e.g., ChatGPT, Gemini)—transparency and documentation obligations.
- August 2026: most obligations for high-risk AI systems (e.g., in healthcare, education, HR, transportation).
- August 2027: extended transition period for high-risk AI systems embedded in regulated products (e.g., medical devices, machinery).
The AI Act and GDPR complement each other. If an AI system processes personal data—which caricature apps and chatbots almost always do—the requirements of both regulations must be met simultaneously.
Practical advice – dos and don’ts
For individuals:
- Never upload photos of other people (including children) without their knowledge and consent.
- Before using an AI app, read the Privacy Policy, not just the Terms of Service.
- Do not enter sensitive personal data (social security numbers, health information, documents) into chatbots.
- Verify where the AI provider is based and where your data is stored.
- Check whether the AI tool allows you to delete your data (right to erasure under Article 17 of the GDPR).
- Secure your AI account: strong password + multi-factor authentication (MFA).
For companies and employers:
- Develop an internal AI policy specifying what employees may and may not enter into AI tools and how they should proceed in the event of an incident.
- Consider whether a Data Protection Impact Assessment (DPIA) under Article 35 of the GDPR is necessary when implementing AI tools.
- Enter into a data processing agreement with the AI provider (in accordance with Article 28 of the GDPR).
- Prohibit the entry of clients’ personal data and other “sensitive data” into public AI tools.
- Ensure regular employee training on the secure use of AI.
- Implement multi-factor authentication (MFA) on corporate AI accounts and conduct regular audits of their use.
- Seek advice from experts specializing in AI to ensure your processes comply with the law.
A cartoon instead of a profile photo is cute. But just as with emails, cloud storage, or social media, the same rule applies: technology is only as secure as the way we use it. The GDPR and the AI Act clearly establish these rules. Our role—whether we are ordinary users, companies, or public institutions—is to know them and follow them. Because data protection isn’t bureaucracy. It’s respect for human dignity in the digital age.
