By George Curta · Last updated 2026-04-07
Penyelidikan Serangan Privasi LLM
12 makalah penyelidikan yang dikaji oleh rakan menunjukkan mengapa pseudonimitas gagal terhadap AI.
Deanonimisasi, ekstraksi PII, inferens keahlian, serangan suntikan cepat — dan cara melindungi diri.
Kategori Serangan Privasi
Deanonimisasi
LLM memadankan siaran anonim dengan identiti sebenar menggunakan gaya penulisan, fakta dan corak temporal. Ketepatan 68% dengan harga $1-$4/profil.
Inferens Atribut
LLM membuat kesimpulan atribut peribadi (lokasi, pendapatan, umur) daripada teks walaupun tidak dinyatakan. GPT-4 mencapai ketepatan teratas-1 sebesar 85%.
Ekstraksi PII
Mengekstrak maklumat peribadi daripada data latihan atau cepat. Ketepatan ekstraksi e-mel 100% dengan GPT-4. Peningkatan 5x dengan serangan lanjutan.
Suntikan Cepat
Memanipulasi ejen LLM untuk bocor data peribadi semasa pelaksanaan tugas. Kadar kejayaan serangan ~20% dalam senario perbankan.
Large-scale online deanonymization with LLMs
Simon Lermen (MATS), Daniel Paleka (ETH Zurich), Joshua Swanson (ETH Zurich), Michael Aerni (ETH Zurich), Nicholas Carlini (Anthropic), Florian Tramèr (ETH Zurich)
Published: February 18, 2026
Penemuan Utama
68% recall at 90% precision for deanonymization using ESRC framework
Metodologi
Designed attacks for closed-world setting with scalable attack pipeline using LLMs to: (1) extract identity-relevant features, (2) search for candidate matches via semantic embeddings, (3) reason over top candidates to verify matches and reduce false positives.
Rangka Kerja ESRC
LLM mengekstrak fakta pengenalan daripada siaran anonim
Gunakan fakta untuk membuat pertanyaan pangkalan data awam (LinkedIn, dll.)
LLM memberikan alasan tentang padanan calon
Pemarkahan keyakinan untuk meminimalkan positif palsu
Hasil Eksperimental
| Set Data | Ingat Kembali @ Ketepatan 90% | Nota |
|---|---|---|
| Hacker News → LinkedIn | 68% | vs near 0% for classical methods |
| Reddit cross-community | 8.5% | Multiple subreddits |
| Reddit temporal split | 67% | Same user over time |
| Internet-scale (extrapolated) | 35% | At 1M candidates |
Implikasi
Practical obscurity protecting pseudonymous users online no longer holds. Classical methods achieve near 0% recall under same conditions.
Semua Makalah Penyelidikan
11 kajian tambahan yang dikaji oleh rakan tentang serangan privasi LLM
Beyond Memorization: Violating Privacy via Inference with Large Language Models
Robin Staab, Mark Vero, Mislav Balunović, et al. (ETH Zurich)
85% top-1 accuracy inferring personal attributes from Reddit posts
First comprehensive study on LLM capabilities to infer personal attributes from text. GPT-4 achieved highest accuracy among 9 tested models.
Penemuan Utama
- •85% top-1 accuracy, 95% top-3 accuracy at inferring personal attributes
- •100× cheaper and 240× faster than human annotators
- •Tested 9 state-of-the-art LLMs including GPT-4, Claude 2, Llama 2
- •Infers location, income, age, sex, profession from subtle text cues
AutoProfiler: Automated Profile Inference with Language Model Agents
Yuntao Du, Zitao Li, Bolin Ding, et al. (Virginia Tech, Alibaba, Purdue University)
85-92% accuracy for automated profiling at scale using four specialized LLM agents
Framework using specialized LLM agents (Strategist, Extractor, Retriever, Summarizer) for automated profile inference from pseudonymous platforms.
Penemuan Utama
- •Four specialized agents: Strategist, Extractor, Retriever, Summarizer
- •Iterative workflow enables sequential scraping, analysis, and inference
- •Outperforms baseline FTI across all attributes and LLM backbones
- •Short-term memory for Extractor/Retriever, long-term memory for Strategist/Summarizer
Large Language Models are Advanced Anonymizers
Robin Staab, Mark Vero, Mislav Balunović, et al. (ETH Zurich SRI Lab)
Adversarial anonymization reduces attribute inference from 66.3% to 45.3% after 3 iterations
LLMs can be used defensively in adversarial framework to anonymize text. Outperforms commercial anonymizers in both privacy and utility.
Penemuan Utama
- •Adversarial feedback enables anonymization of significantly finer details
- •Attribute inference accuracy drops from 66.3% to 45.3% after 3 iterations
- •Evaluated 13 LLMs on real-world and synthetic online texts
- •Human study (n=50) showed strong preference for LLM-anonymized texts
AgentDAM: Privacy Leakage Evaluation for Autonomous Web Agents
Arman Zharmagambetov, Chuan Guo, Ivan Evtimov, et al. (Meta AI, CMU)
GPT-4, Llama-3, and Claude web agents are prone to inadvertent use of unnecessary sensitive information
Benchmark measuring if AI web agents follow data minimization principle. Simulates realistic web interactions across GitLab, Shopping, and Reddit.
Penemuan Utama
- •Evaluates GPT-4, Llama-3, Claude-powered web navigation agents
- •Measures data minimization compliance: use PII only if 'necessary' for task
- •Agents often leak sensitive information when unnecessary
- •Three test environments: GitLab, Shopping, Reddit web apps
SoK: The Privacy Paradox in Large Language Models
Various researchers
Systematization of 5 distinct privacy incident categories beyond memorization
Comprehensive survey categorizing privacy risks: training data leakage, chat leakage, context leakage, attribute inference, and attribute aggregation.
Penemuan Utama
- •Five privacy incident categories identified:
- •1. Training data leakage via regurgitation
- •2. Direct chat leakage through provider breaches
- •3. Indirect context leakage via agents and prompt injection
PII-Scope: A Comprehensive Study on Training Data PII Extraction Attacks in LLMs
Krishna Kanth Nakka, Ahmed Frikha, Ricardo Mendes, et al. (Various)
PII extraction rates increase up to 5× with sophisticated adversarial capabilities and limited query budget
Comprehensive benchmark for PII extraction attacks. Reveals notable underestimation of PII leakage in existing single-query attacks.
Penemuan Utama
- •PII extraction rates can increase up to 5× with sophisticated attacks
- •Existing single-query attacks notably underestimate PII leakage
- •Taxonomy: Black-box (True-prefix, ICL, PII Compass) and White-box (SPT) attacks
- •Hyperparameters like demonstration selection crucial to attack effectiveness
Evaluating LLM-based Personal Information Extraction and Countermeasures
Yupei Liu, Yuqi Jia, Jinyuan Jia, et al. (Penn State, Duke University)
GPT-4 achieves 100% accuracy extracting emails and 98% for phone numbers from synthetic profiles
Systematic measurement study benchmarking LLM-based personal information extraction (PIE). Proposes prompt injection as novel defense.
Penemuan Utama
- •GPT-4: 100% email extraction, 98% phone number extraction on synthetic data
- •Larger LLMs more successful: vicuna-7b achieves 65%/95% vs GPT-4's 100%/98%
- •LLMs better at: emails, phone numbers, addresses, names
- •LLMs worse at: work experience, education, affiliation, occupation
Preserving Privacy in Large Language Models: A Survey on Current Threats and Solutions
Michele Miranda, Elena Sofia Ruzzetti, Andrea Santilli, et al. (Various)
Comprehensive taxonomy of privacy attacks: training data extraction, membership inference, model inversion
Survey examining privacy threats from LLM memorization. Proposes solutions from dataset anonymization to differential privacy and machine unlearning.
Penemuan Utama
- •Privacy attacks covered: Training data extraction, Membership inference, Model inversion
- •Training data extraction: non-adversarial and adversarial prompting
- •Membership inference: shadow models and threshold-based approaches
- •Model inversion: output inversion and gradient inversion
Beyond Data Privacy: New Privacy Risks for Large Language Models
Various researchers
LLM autonomous capabilities create new vulnerabilities for inadvertent data leakage and malicious exfiltration
Explores privacy vulnerabilities from LLM integration into applications and weaponization of autonomous abilities.
Penemuan Utama
- •LLM integration creates new privacy vulnerabilities beyond traditional risks
- •Opportunities for both inadvertent leakage and malicious exfiltration
- •Adversaries can exploit systems for sophisticated large-scale privacy attacks
- •Autonomous LLM abilities can be weaponized for data exfiltration
Simple Prompt Injection Attacks Can Leak Personal Data Observed by LLM Agents
Various researchers
15-50% utility drop under attack with ~20% average attack success rate for personal data leakage
Examines prompt injection causing tool-calling agents to leak personal data during task execution. Uses fictitious banking agent scenario.
Penemuan Utama
- •16 user tasks from AgentDojo benchmark evaluated
- •15-50 percentage point drop in LLM utility under attack
- •~20% average attack success rate across LLMs
- •Most LLMs avoid leaking passwords due to safety alignments
Membership Inference Attacks on Large-Scale Models: A Survey
Various researchers
First comprehensive review of MIAs targeting LLMs and LMMs across pre-training, fine-tuning, alignment, and RAG stages
Survey analyzing membership inference attacks by model type, adversarial knowledge, strategy, and pipeline stage.
Penemuan Utama
- •Analyzes MIAs across: pre-training, fine-tuning, alignment, RAG stages
- •Strong MIAs require training multiple reference models (computationally expensive)
- •Weaker attacks often perform no better than random guessing
- •Tokenizers identified as new attack vector for membership inference
Strategi Pertahanan daripada Penyelidikan
Apa Yang Tidak Berfungsi
- ✗Pseudonimitas — LLM mengalahkan nama pengguna, pemegang, nama paparan
- ✗Penukaran teks kepada imej — Hanya penurunan kecil terhadap LLM multimodal
- ✗Penyelarasan model sahaja — Pada masa ini tidak berkesan dalam mencegah inferens
- ✗Anonimisasi teks mudah — Tidak mencukupi terhadap penalaran LLM
Apa Yang Berfungsi
- ✓Anonimisasi adversarial — Kurangkan inferens 66.3% → 45.3%
- ✓Privasi pembezaan — Kurangkan ketepatan PII 33.86% → 9.37%
- ✓Pertahanan suntikan cepat — Paling berkesan terhadap PIE berasaskan LLM
- ✓Penyingkiran/penggantian PII sebenar — Singkirkan isyarat yang digunakan LLM
Mengapa Penyelidikan Ini Penting
12 makalah penyelidikan ini menunjukkan perubahan asas dalam ancaman privasi. Pendekatan anonimisasi tradisional seperti nama samaran, nama pengguna dan perubahan pemegang tidak lagi perlindungan yang mencukupi terhadap musuh yang bertekad dengan akses kepada LLM.
Metrik Ancaman Utama
- Ketepatan deanonimisasi 68% pada ketepatan 90% (Hacker News → LinkedIn)
- Ketepatan inferens atribut 85% untuk lokasi, pendapatan, umur, pekerjaan
- Ekstraksi e-mel 100% dan ekstraksi nombor telefon 98% (GPT-4)
- Peningkatan 5 kali ganda dalam kebocoran PII dengan serangan berbilang kueri lanjutan
- Kos $1-$4 setiap profil menjadikan serangan berskala besar boleh dilakukan dari segi ekonomi
Siapa Yang Berisiko
- Pemberi maklumat & aktivis: Siaran anonim boleh dikaitkan dengan identiti sebenar
- Profesional: Aktiviti Reddit dikaitkan dengan profil LinkedIn
- Pesakit penjagaan kesihatan: Inferens keahlian mendedahkan sama ada data dalam latihan
- Sesiapa yang mempunyai siaran sejarah: Bertahun-tahun data boleh deanonimisasi secara surut
Bagaimana anonym.legal Mengatasi Ancaman Ini
anonym.legal menyediakan anonimisasi sebenar yang menghilangkan isyarat yang digunakan LLM:
- 285+ Jenis Entiti: Nama, lokasi, tarikh, penanda masa, pengecam
- Gangguan Corak Penulisan: Gantikan teks yang mendedahkan cap jari stylometric
- Enkripsi Boleh Balik: AES-256-GCM untuk kes yang memerlukan akses yang dibenarkan
- Pelbagai Operator: Ganti, Redaksi, Hash, Enkripsi, Topeng, Tersuai
Soalan Yang Kerap Ditanya
Apakah deanonimisasi berasaskan LLM?
Deanonimisasi berasaskan LLM menggunakan model bahasa besar untuk mengenalpasti individu sebenar daripada siaran dalam talian yang anonim atau pseudonim. Tidak seperti kaedah tradisional yang gagal pada skala besar, LLM boleh menggabungkan analisis gaya penulisan (stylometry), fakta yang dinyatakan, corak temporal dan penalaran kontekstual untuk memadankan profil anonim dengan identiti sebenar. Penyelidikan menunjukkan ketepatan sehingga 68% pada ketepatan 90%, berbanding hampir 0% untuk kaedah klasik.
Seberapa tepat deanonimisasi LLM?
Penyelidikan menunjukkan tahap ketepatan yang membimbangkan: ingat kembali 68% pada ketepatan 90% untuk pemadanan Hacker News kepada LinkedIn, 67% untuk analisis temporal Reddit (pengguna yang sama dari masa ke masa), 35% pada skala internet (1 juta + calon). Untuk inferens atribut, GPT-4 mencapai ketepatan teratas-1 sebesar 85% membuat kesimpulan atribut peribadi seperti lokasi, pendapatan, umur dan pekerjaan daripada siaran Reddit sahaja.
Apakah rangka kerja ESRC?
ESRC (Ekstrak-Cari-Alasan-Kalibir) ialah rangka kerja deanonimisasi LLM empat langkah: (1) Ekstrak - LLM mengekstrak fakta pengenalan daripada siaran anonim menggunakan NLP, (2) Cari - soal pangkalan data awam seperti LinkedIn menggunakan fakta yang diekstrak dan pembenaman semantik, (3) Alasan - LLM memberikan alasan tentang padanan calon menganalisis ketekalan, (4) Kalibir - pemarkahan keyakinan untuk meminimalkan positif palsu sambil memaksimalkan padanan sebenar.
Berapakah kos deanonimisasi LLM?
Penyelidikan menunjukkan deanonimisasi berasaskan LLM berharga $1-$4 setiap profil, menjadikan deanonimisasi berskala besar boleh dilakukan dari segi ekonomi. Untuk anonimisasi pertahanan, kos adalah di bawah $0.035 setiap ulasan menggunakan GPT-4. Kos rendah ini membolehkan aktor negara, perbadanan, pengikut dan individu berniat jahat melakukan serangan privasi berskala besar.
Apakah jenis PII yang boleh diekstrak oleh LLM daripada teks?
LLM cemerlang dalam mengekstrak: alamat e-mel (ketepatan 100% dengan GPT-4), nombor telefon (98%), alamat surat dan nama. Mereka juga boleh membuat kesimpulan PII bukan eksplisit: lokasi, tahap pendapatan, umur, jantina, pekerjaan, pendidikan, status hubungan dan tempat kelahiran daripada isyarat teks halus dan corak penulisan.
Apakah serangan inferens keahlian (MIA)?
Serangan inferens keahlian menentukan sama ada data tertentu digunakan untuk melatih model AI. Untuk LLM, ini mendedahkan sama ada maklumat peribadi anda berada dalam set data latihan. Penyelidikan menunjukkan alamat e-mel dan nombor telefon sangat terdedah. Vektor serangan baru termasuk inferens berasaskan tokenizer dan analisis isyarat perhatian (AttenMIA).
Bagaimana serangan suntikan cepat bocor data peribadi?
Suntikan cepat memanipulasi ejen LLM untuk bocor data peribadi yang diperhatikan semasa pelaksanaan tugas. Dalam senario ejen perbankan, serangan mencapai kadar kejayaan ~20% pada data peribadi eksfiltrasi, dengan kemerosotan utiliti 15-50% di bawah serangan. Walaupun penyelarasan keselamatan mencegah kebocoran kata laluan, data peribadi lain kekal terdedah.
Bagaimana anonym.legal dapat membantu melindungi daripada serangan privasi LLM?
anonym.legal menyediakan anonimisasi sebenar melalui: (1) Pengesanan PII - 285+ jenis entiti termasuk nama, lokasi, tarikh, corak penulisan, (2) Penggantian - gantikan PII sebenar dengan alternatif yang sah format, (3) Redaksi - sepenuhnya singkirkan maklumat sensitif, (4) Enkripsi Boleh Balik - AES-256-GCM untuk akses yang dibenarkan. Tidak seperti pseudonimitas yang dikalahkan LLM, anonimisasi sebenar menghilangkan isyarat yang digunakan LLM untuk deanonimisasi.
Lindungi Daripada Serangan Privasi LLM
Jangan bergantung pada pseudonimitas. Gunakan anonimisasi sebenar untuk melindungi dokumen sensitif, data pengguna dan komunikasi daripada serangan pengenalan yang dikuasai AI.
About this page
We update this page when our platform or the law changes.
Read our founder note for how we work.
Each change shows up in the timestamp at the top.
Related reading
- Common questions
- Glossary
- How tokens work
- Security posture
- Where we comply
- What we detect
- Case studies
- Release notes
We follow these rules
- GDPR (EU 2016/679).
- ISO/IEC 27001:2022.
- NIS2 (EU 2022/2555).
- HIPAA safe harbor under 45 CFR § 164.514(b)(2).
Our promise
We do not sell your data.
We do not train models on your text.
We store your files in Germany.
You can delete your account at any time.
You own your work.
Where we run
Our servers live in Falkenstein, Germany.
We use Hetzner. They hold ISO 27001 certification.
All data stays in the EU.
Backups run every day.
Need help?
Email support@anonym.legal.
We reply within one business day.
How we test
We run a full check suite on every release.
Each surface gets its own sweep script and report.
Human reviewers spot-check the output each week.
We track recall and precision on a labelled set.
Bad runs block the deploy.
What we never do
- We never sell your information to third parties.
- We never train models on what you upload.
- We never keep your work after you delete it.
- We never share keys with any outside firm.
- We never run ads inside the product.
Plans in plain words
We sell credits, not seats.
One credit covers one short job.
Long jobs use a few credits each.
You can top up at any time.
Unused credits roll over each month.
Read the plans page for current rates.
Who built this
A small team of engineers and lawyers built this.
We ship from Europe and work in the open.
Our founder note spells out why we started.
Where to start
- Open the web app and try a sample file.
- Learn how credits get counted.
- See current plans and limits.
- Meet the team behind the product.
How the parts fit
A browser add-on cleans text inside Chrome.
A Word plug-in handles drafts in Office.
A small desktop tool works on whole folders.
An agent protocol link feeds large models safely.
All four share one core engine and one rule set.
Words from our team
We started this work after a lunch about cookies.
One friend kept getting odd ads on her phone.
We asked why a court file leaked through a draft.
We sketched the first build on a napkin that week.
By month three we had a tiny demo for a friend.
She used it on her first case the next day.
Common questions we hear
Can the tool read scanned PDFs? Yes, with OCR.
Does it work on long files? Yes, in small chunks.
Can I roll my own rule set? Yes, save it as a preset.
Does it run offline? The desktop build runs offline.
Do you keep my files? No, the cloud build wipes after each run.
Will it learn from my work? No, we never train on inputs.
A short tour of the workflow
Upload a file or paste a snippet of prose.
Pick the entities you want gone from the draft.
Choose a method: replace, mask, hash, encrypt, or redact.
Press run and watch the side panel show each hit.
Skim the result and tweak any rule that misfired.
Save the cleaned file or send it to a teammate.