Open-source building blocks
for LLM applications

PII tokenization, evaluation, runtime guardrails and document extraction. Python and TypeScript, MIT licence, installed from PyPI and npm.

# pip install ag-pii-vault && python -m spacy download en_core_web_lg
from ag_pii_vault import Detector, LocalVault, Tokenizer

detector = Detector()
tokenizer = Tokenizer(vault=LocalVault(path=":memory:"), key="my-hmac-secret")

text = "Contact John Smith at john@acme.com"
safe = tokenizer.tokenize(text, detector.analyze(text))
print(safe)                        # Contact PERSON_353fcd26 at EMAIL_9abc0eb1
print(tokenizer.dehydrate(safe))   # Contact John Smith at john@acme.com

Libraries

ag-pii-vault

Detects personal data, replaces it with deterministic HMAC tokens before text reaches an LLM, and restores it in the response. SafeOpenAI and SafeAnthropic wrap the official clients.

pip install ag-pii-vault · npm install ag-pii-vault

GitHubPyPInpm

ag-evalforge

Scores model outputs against a dataset with exact match, regex, ROUGE-L, semantic similarity, LLM-as-judge or custom scorers, and compares two models. Command line ag-evalforge.

pip install ag-evalforge · npm install ag-evalforge

GitHubPyPInpm

ag-guardrail

Scans prompts and responses for prompt injection, jailbreaks, PII in output, harmful content and off-topic requests. Policies in YAML, flagged calls written to an audit log.

pip install ag-guardrail · npm install ag-guardrail

GitHubPyPInpm

ag-docparse

Extracts typed fields from PDF or text documents with an LLM, using built-in schemas (invoice, loan application, W-2, NDA, contract) or a custom JSON schema. Command line ag-docparse.

pip install ag-docparse · npm install ag-docparse

GitHubPyPInpm

Usage: the README of each library. LLM features use your own OpenAI or Anthropic key.

EU AI Act (Regulation 2024/1689)

Art. 9

Risk management: ag-evalforge, ag-guardrail

Repeatable evaluation runs give documented performance baselines. Runtime checks block unsafe inputs and outputs.

Art. 10

Data governance: ag-pii-vault

Personal data is tokenized before it reaches a third-party LLM provider, and restored only on your side.

Art. 12

Record-keeping: ag-guardrail

AuditLog records each flagged call with input, output, triggered rules and timestamp.

Art. 13

Transparency: ag-docparse

Extraction into typed, schema-defined fields makes AI-processed document data inspectable.

Using these libraries is not a conformity assessment.

Benchmarks

Scripts, public datasets and result files with bootstrap 95 % confidence intervals: ag-benchmarks. The results were measured in May 2026 against a hosted deployment of the libraries, including its LLM-based modes.