Skip to content

Project Documentation

Welcome to the technical developer documentation for PDF Anonymizer.

This section covers everything you need to install, run, configure, and extend the codebase. PDF Anonymizer is built as a modular Python monorepo, designed to keep developer tooling lightweight and separate from core business logic.

Monorepo Project Structure

The project separates the command-line application from the underlying core library using a monorepo setup:

anonymizer/
├── .github/workflows/          # CI/CD & Deploy workflows
├── data/                       # Local data directory for sample files
├── docs/                       # MkDocs documentation source files
├── packages/
│   ├── pdf-anonymizer-core/    # Core SDK package (logic, providers, prompts)
│   ├── pdf-anonymizer-cli/     # CLI executable wrapper using Typer
│   └── pdf-anonymizer-api/     # Optional HTTP service (core only)
├── tests/                      # Global pytest test suite
├── Makefile                    # Developer shortcut commands
├── pyproject.toml              # Workspace and dev dependencies config
└── mkdocs.yml                  # MkDocs site configuration

The Packages

The project contains three decoupled Python packages inside packages/:

pdf-anonymizer-core

Contains all the core engines, including:

  • Text extraction from PDF, Markdown, and plain text formats.
  • Table loaders for CSV (stdlib) and Excel (.xlsx, [excel] extra): per-cell regex, row-addressed LLM batches, per-cell apply.
  • Word loader for .docx ([docx] extra): per-paragraph regex, part-wise flatten, native .docx write-back.
  • Hybrid detection: RE2 regex (with checksums / TYPE_LIKE) plus LLM NER.
  • LLM router and adapters for various providers (Ollama, Gemini, OpenAI, etc.).
  • Prompt templates (simple, detailed, hipaa) and identity-clue detection.
  • Per-type operators (replace, mask, hash, generalize, shift, fake).
  • Span-based replacement, keep/deny gazetteers, optional encrypted maps.
  • Residual leftover scan, linkage-risk report, TAB-style eval helpers.
  • Streaming chunk utility to process large text files.
  • Mapping and restoration engine for deanonymization.

pdf-anonymizer-api

A FastAPI process that calls pdf-anonymizer-core only. It does not import the CLI. Install with pip install pdf-anonymizer-api.

pdf-anonymizer-cli

A CLI tool built on top of pdf-anonymizer-core that:

  • Exposes a command-line interface using Typer (supporting autocompletion, clean logs, and command help).
  • Handles loading local .env files automatically.
  • Manages output file paths for anonymized logs and mapping tables.

Getting Started

To dive deeper into the technical details, navigate through the following guides:

  • Terminology: One list of product words (pseudonymization, leftover, gold corpus, mention vs entity). Other pages should link here.
  • Installation & Setup: Learn how to set up the development environment using uv, manage packages, and define environment variables.
  • CLI Reference: Explore the command-line arguments, options (including --config-profile), custom model strings, and usage examples.
  • SDK & API Usage: Learn how to import PDF Anonymizer as a Python library in your own applications.
  • HTTP service and Docker: POST /anonymize and the official image. No authentication.
  • API Reference (auto): Living signature reference generated from source docstrings.
  • Recipes & Common Workflows: Practical end-to-end examples — local Ollama, locked maps, operators, HIPAA aid, keep/deny lists, leftover checks, CSV/Excel rosters, eval harness, batching, caching, and more.
  • Gold corpus & eval: Why we measure leftover risk, what is in the gold corpus, and the download / benchmark / CI workflows.
  • Mapping encryption: Argon2id + AES-256-GCM envelopes, AAD, 0600 writes, ephemeral maps, and the tests that lock those in.
  • Troubleshooting: Common errors (auth, rate limits, LLM parsing, empty results, large files) and solutions.
  • Architecture Design: Understand the data flow, prompt styling, LLM adapters, and file splitting mechanisms.