Model-agnostic PII detection with LLMs

Fine-tuning a model on real-world text creates a personally identifiable information (PII) detection problem. Training corpora are full of PII: names, home addresses, email and phone numbers, national-ID and social-security numbers, bank accounts, dates of birth.

What changed

A model trained on uncleaned text can memorize that data and later reproduce it, leaking a real person’s details through a prompt that was never meant to surface them. In this post, we describe a configurable, model-agnostic detector built on large language models (LLMs), walk through its implementation, benchmark it against an off-the-shelf tool, and show how to run it on your own data.

Price and bundle value

Sample code: The detector described in this post ships as the pii-detector package, available in the sample-llm-pii-detection repository . Every code snippet that follows is drawn from that package, and the Running the detector end to end section walks through installing and running it on your own data. For people paying for AI tools, the difference only matters when it removes real steps from writing, research, meetings, coding, or operations rather than adding another feature label. The readers who should look most closely are usually freelancers, content teams, product teams, and smaller businesses deciding which paid AI layer is actually worth it.

AI features that change the value

Large language models reframe the problem. An LLM reads its instructions at inference time, so the entities to detect, the output format, and the deployment backend all become configuration rather than code. One detector can target a new entity type by editing a prompt instead of retraining, run on a managed API or inside your own virtual private cloud (VPC), and reason about context across eight languages without a translation step. The rest of this post describes such a detector, walks through the engineering behind it, and shows how it measures up against existing tools.

Who should pay attention

The detector treats the language model as a configurable, swappable component. You wrap the input text in instructions that define the PII entities to detect and the expected output. The model then returns a structured list of detected entities. Two design choices make it model-agnostic:. Even once the story is verified, the useful follow-up is which company keeps practical value alive after the launch-day noise fades. That is why the useful reading move is not to stop at the headline, but to compare the promise, the workflow change, and the likely cost before deciding anything.

Patrick Tech Media take

Customization comes from two independent components. The first is the model, which sets accuracy, latency, and cost: you choose a frontier Amazon Bedrock model or a small open model on a single GPU. The second is the entity set, which defines what counts as PII. To extend it, you add a domain-specific identifier or drop one that you don’t need. Changing the entity set is a one-line edit to the instructions, with no retraining and no redeployment.

Source notes

AWS ML Blog