pyveil manual

Local PII and secret redaction for Python LLM apps, AI agents, tools, MCP, memory, logs, and traces.

pyveil is dependency-free redaction middleware. You put it on the boundary between raw context and anything an agent touches — a model prompt, a tool call, an MCP resource, a memory store, a trace exporter, or a log sink. The core is standard-library only: deterministic HMAC placeholders, channel-aware policy, and high-precision detectors.

Installation

pyveil supports Python 3.8 through 3.14 and has no required runtime dependencies.

From PyPI (recommended)

pip install pyveil

OpenAI and Anthropic integrations

Install only the provider SDK you use. The current official SDKs require Python 3.9 or newer; pyveil core and both keyless dry-runs still support Python 3.8.

pip install "pyveil[openai]"
pip install "pyveil[anthropic]"

Ollama local-model integration

Install the official Ollama client and optional env/YAML loaders while keeping the core dependency-free:

pip install "pyveil[ollama]"
ollama pull qwen3.5:4b

Azure OpenAI integration

Keep the zero-dependency core, and install provider/config packages only when needed:

pip install "pyveil[azure-openai]"

From source / Git

Install the latest main directly from GitHub:

pip install "git+https://github.com/hyeonsangjeon/pyveil.git"

Pin to a released tag for reproducible builds:

pip install "git+https://github.com/hyeonsangjeon/pyveil.git@v0.2.4"

For development (editable clone)

git clone https://github.com/hyeonsangjeon/pyveil.git
cd pyveil
pip install -e ".[dev]"     # ruff, mypy, pytest, build, twine
# or just the test extras:
pip install -e ".[test]"

Verify the install

python -c "import pyveil; print(pyveil.__version__)"
pyveil --help

Installing the package also registers the pyveil command line tool. See Command line.

Quickstart

Create one redactor, then call redact_text for strings or redact_data for dict/list/JSON.

30-second example

from pyveil import Veil

veil = Veil.high(secret=b"tenant-or-run-secret", scope="tenant/session")

result = veil.redact_text(
    "Email alice@example.com and call 010-1234-5678",
    channel="prompt.input",
)

print(result.text)
# Email [EMAIL:a3d2b564c825] and call [PHONE:cbbfb8059b58]

One-off scripts

When you do not want to hold a Veil instance, use the module-level helpers. They read the secret from the secret= argument or the PYVEIL_SECRET environment variable.

from pyveil import redact_text

safe = redact_text(
    "email alice@example.com",
    secret=b"tenant-or-run-secret",
    scope="tenant/session",
)
print(safe.text)
# email [EMAIL:a3d2b564c825]

How it works

Every redaction call runs the same three stages:

1. Detect — high-precision detectors find sensitive spans in text, and sensitive key names in structured data.
2. Decide — the policy looks at the channel and entity type and returns an action: REDACT PASS or BLOCK.
3. Replace — redacted values become a deterministic HMAC placeholder (HIGH) or a shape-preserving mask (LOW). Blocked values raise BlockedSensitiveData.

The result is a RedactionResult carrying the redacted payload, a tuple of Finding records (metadata only — never the raw value), and small stats.

Channels

A channel names the boundary the data is about to cross. Channels are first-class so a policy can treat the same value differently depending on where it is going. Pass a channel as a string or a Channel enum member.

Defaults: redact_text uses prompt.input and redact_data uses tool.call.arguments.

Levels: HIGH vs LOW

The level controls how a redacted value looks. Use HIGH by default for anything an agent or external system sees. Use LOW only for human-facing diagnostics where a little shape helps a person recognize a value.

from pyveil import Veil

high = Veil.high(secret=b"docs-secret", scope="docs/reference")
low  = Veil.low(secret=b"docs-secret", scope="docs/reference")

print(high.redact_text("call 010-1234-5678").text)
# call [PHONE:3b2b3e0e6c51]
print(low.redact_text("call 010-1234-5678").text)
# call 010-****-5678

Policy & actions

A Policy maps (channel, entity) pairs to one of three actions:

• REDACT — replace the value with a placeholder or mask (the default).
• PASS — leave the value untouched (still reported as a finding).
• BLOCK — refuse the whole payload by raising BlockedSensitiveData.

The default policy redacts every supported finding, but blocks credential-like material in tool.call.arguments, because model-controlled tool arguments should never carry raw credentials:

Customizing a policy

Start from Policy.default_high() (or default_low()) and layer immutable override(...) calls. Each override returns a new policy.

from pyveil import Action, Channel, Entity, Policy, Veil

policy = (
    Policy.default_high()
    .override(Channel.PROMPT_INPUT, Entity.EMAIL, Action.PASS)          # allow emails into prompts
    .override(Channel.PROMPT_OUTPUT, Entity.PHONE, Action.BLOCK)        # never emit phones
)

veil = Veil.high(secret=b"tenant-secret", policy=policy)

Placeholders & scope

At HIGH level a detected value becomes [TYPE:digest], where digest is a truncated HMAC-SHA256 over the value, keyed by your secret and namespaced by scope and entity type.

[EMAIL:a3d2b564c825]
   |         |
 entity   12 hex chars of HMAC-SHA256(secret, scope + type + value)

Two properties follow from this:

• Deterministic: the same secret, scope, entity, and value always yield the same placeholder — so an agent can correlate "the same email" across a run without ever seeing it.
• Scoped: changing scope changes every placeholder, which breaks cross-context linkability. Use distinct scopes (for example tenant-123/session-456) when you do not want values linkable across tenants, sessions, or runs.

Findings & results

Both redaction methods return a RedactionResult. It never stores raw sensitive values.

from pyveil import Veil

veil = Veil.high(secret=b"tenant-secret", scope="tenant/session")
result = veil.redact_text("Authorization: Bearer synthetic-token-value")

result.text                       # redacted string (redact_text only)
result.data                       # redacted payload (str, dict, list, ...)
result.findings                   # tuple[Finding, ...]
result.stats.total_findings       # e.g. 1
result.stats.counts_by_type       # e.g. {"AUTH_HEADER": 1}

f = result.findings[0]
f.type, f.detector, f.rule_id     # "AUTH_HEADER", "auth_header", "authorization_header"
f.placeholder, f.fingerprint      # stable placeholder + digest
f.start, f.end, f.path            # span offsets (text) / JSON pointer (structured)
f.raw                             # always None by design

Text redaction

Use redact_text for free-form strings: prompts, retrieved passages, model output, or log lines.

from pyveil import Channel, Veil

veil = Veil.high(
    secret=b"tenant-secret",
    scope="tenant_123/session_456",
    max_input_chars=1_000_000,
)

safe = veil.redact_text(
    "Authorization: Bearer synthetic-token-value",
    channel=Channel.PROMPT_INPUT,
)

print(safe.text)               # [AUTH_HEADER:...]
print(safe.findings[0].type)   # AUTH_HEADER

max_input_chars caps input size before detection runs; oversized input raises ValueError. Set it to None to disable the cap.

Structured redaction

Use redact_data for dictionaries, lists, tuples, JSON strings, tool arguments, MCP payloads, trace attributes, and memory records. The payload keeps its shape; only string values change.

from pyveil import Veil

payload = {
    "user": "alice@example.com",
    "headers": {"Authorization": "Bearer synthetic-token-value"},
    "args": {"phone": "+82 10-1234-5678"},
    "debug": True,
}

veil = Veil.high(secret=b"tenant-secret", scope="tenant/session")
safe = veil.redact_data(payload, channel="tool.call.result")

print(safe.data)

{
  "user": "[EMAIL:4ad330c24162]",
  "headers": {"Authorization": "[AUTH_HEADER:c042667cf238]"},
  "args": {"phone": "[PHONE:39759ae5a2f8]"},
  "debug": true
}

Two structured-specific behaviors:

• Sensitive key names. If a key looks sensitive (for example password, api_key, token, authorization), pyveil treats its value as sensitive even when the value alone would not match a token pattern.
• Non-string scalars are preserved. {"api_key": 12345} or {"password": True} keep their original type instead of becoming a string. Notice debug stays a real boolean above.

If you pass a JSON string, pyveil parses it, redacts, and returns a compact JSON string in result.data.

Custom rules

The dependency-free core does not guess arbitrary names or company-specific identifiers. Add values your application already knows are sensitive, plus narrow trusted regexes from your own domain.

from pyveil import CustomRule, Veil

rules = [
    CustomRule.exact("PERSON", ["Alice Kim", "Hong Gildong"]),
    CustomRule("CUSTOMER_ID", r"\bCUS-[A-Z0-9]{8}\b", rule_id="customer_id"),
]

veil = Veil.high(
    secret=b"tenant-secret",
    scope="tenant/session",
    rules=rules,
)

safe = veil.redact_text("Alice Kim owns CUS-A1B2C3D4.")
print(safe.text)
# [PERSON:...] owns [CUSTOMER_ID:...].

CustomRule.exact escapes values, prefers longer matches, and applies word boundaries by default. Custom regexes are trusted application code: keep them narrow, test positive and negative samples, and keep max_input_chars enabled.

Recipes

Copy-paste patterns for the boundaries agents actually leak through. The rule is always the same: raw context → pyveil → model / tool / memory / log / trace.

1. Redact before any LLM client

Keep pyveil one layer above the provider SDK. OpenAI, Azure OpenAI, Anthropic, Gemini, LiteLLM, and internal gateways all use the same boundary shape — only call_llm changes.

from typing import Callable, Dict, List, cast
from pyveil import Channel, Veil

Message = Dict[str, str]
LLMCall = Callable[[List[Message]], str]

veil = Veil.high(secret=b"tenant-secret", scope="tenant-123/session-456")

def call_with_redaction(messages: List[Message], call_llm: LLMCall) -> str:
    safe = veil.redact_data(messages, channel=Channel.PROMPT_INPUT)
    return call_llm(cast(List[Message], safe.data))

2. OpenAI Responses API, keyless first

The installable OpenAI helper redacts a prompt immediately before the official SDK serializes a client.responses.create(...) request. The returned redacted_input is the exact value allowed to cross that boundary.

from pyveil.integrations.openai import ask_openai, load_settings

settings = load_settings()
result = ask_openai(
    "Write a follow-up for alice@example.com or 010-1234-5678.",
    settings,
)

print(result.redacted_input)  # exact Responses API input
print(result.output_text)
print(result.input_tokens, result.output_tokens, result.total_tokens)

Prove the boundary without an API key

PYVEIL_SECRET=docs-demo-secret \
OPENAI_MODEL=gpt-5.6-luna \
  python -m pyveil.integrations.openai --dry-run

mode: dry-run
model: gpt-5.6-luna
sent-to-openai: Write a one-sentence support follow-up for [EMAIL:17c25f8a4fe3] or [PHONE:3f6dc5a3c9f3].
findings: EMAIL=1, PHONE=1
openai-response: skipped (--dry-run)

No provider key or network request is used here. The repository also sends the same synthetic prompt through the real OpenAI SDK into a local mock HTTP transport and asserts that its serialized /v1/responses body contains placeholders but no raw email or phone. This verifies the SDK contract without claiming a paid live request.

For live use, set OPENAI_API_KEY, OPENAI_MODEL, and PYVEIL_SECRET. Process environment overrides optional .env and non-secret YAML settings. Model access changes over time, so the model remains configuration rather than a pyveil runtime default. Historical model IDs are not a free testing tier and may be retired. See the full OpenAI guide.

3. Anthropic Messages API, keyless first

The Anthropic helper applies the same boundary immediately before the official SDK serializes client.messages.create(...). It extracts text response blocks and returns safe request metadata.

from pyveil.integrations.anthropic import ask_anthropic, load_settings

settings = load_settings()
result = ask_anthropic(
    "Write a follow-up for alice@example.com or 010-1234-5678.",
    settings,
)

print(result.redacted_input)  # exact Messages API user content
print(result.output_text)
print(result.input_tokens, result.output_tokens)

Prove the boundary without an API key

PYVEIL_SECRET=docs-demo-secret \
ANTHROPIC_MODEL=claude-haiku-4-5 \
  python -m pyveil.integrations.anthropic --dry-run

mode: dry-run
model: claude-haiku-4-5
sent-to-anthropic: Write a one-sentence support follow-up for [EMAIL:0b77abd1b26b] or [PHONE:ec56e2456ba2].
findings: EMAIL=1, PHONE=1
anthropic-response: skipped (--dry-run)

The offline contract test uses the real Anthropic SDK with a local mock HTTP transport and checks the serialized /v1/messages body. No API key, network request, or Claude credit is consumed.

For live use, set ANTHROPIC_API_KEY, ANTHROPIC_MODEL, and PYVEIL_SECRET. Older Claude models are not free and may be retired, so keep the model ID configurable. See the full Claude guide.

4. Ollama, local and memory-aware

The installable Ollama helper redacts the prompt immediately before the official Client.chat(...) call. It returns the exact model input, response text, timing, and token counts. The default qwen3.5:4b profile limits context to 4096 tokens and unloads after each response.

from pyveil.integrations.ollama import ask_ollama, load_settings

settings = load_settings()
result = ask_ollama(
    "Write a follow-up for alice@example.com or 010-1234-5678.",
    settings,
)

print(result.redacted_input)    # exact prompt sent to Ollama
print(result.output_text)       # local model response
print(result.total_duration_ms) # optional server metric

Prove the boundary without loading a model

PYVEIL_SECRET=docs-demo-secret \
  python -m pyveil.integrations.ollama --dry-run

mode: dry-run
model: qwen3.5:4b
host: http://127.0.0.1:11434
sent-to-ollama: Write a one-sentence support follow-up for [EMAIL:71c6727a7fa2] or [PHONE:b4b889df07ce].
findings: EMAIL=1, PHONE=1
ollama-response: skipped (--dry-run)

Environment, .env, or YAML

# One-shot live call; the default keep_alive=0 releases memory
export PYVEIL_SECRET="a-long-random-hmac-secret"
python -m pyveil.integrations.ollama

# Or .env combined with non-secret YAML
python -m pyveil.integrations.ollama \
  --config examples/ollama.example.yaml --env-file .env

# Keep the model warm for repeated agent turns
export OLLAMA_KEEP_ALIVE=5m

5. Azure OpenAI, end to end

The installable Azure helper loads environment, .env, or YAML settings, redacts the prompt, then calls the Azure OpenAI v1 Responses API. redacted_input is the exact value that crossed the provider boundary.

from pyveil.integrations.azure_openai import (
    ask_azure_openai,
    load_settings,
)

settings = load_settings()
result = ask_azure_openai(
    "Write a follow-up for alice@example.com or 010-1234-5678.",
    settings,
)

print(result.redacted_input)  # exact text sent to Azure
print(result.output_text)     # model response

Prove the boundary without calling Azure

PYVEIL_SECRET=docs-demo-secret \
  python -m pyveil.integrations.azure_openai --dry-run

mode: dry-run
deployment: not configured
sent-to-azure: Write a one-sentence support follow-up for [EMAIL:347ab11285a3] or [PHONE:548017338f6f].
findings: EMAIL=1, PHONE=1
azure-response: skipped (--dry-run)

Environment, .env, or YAML

# Process environment only
export AZURE_OPENAI_ENDPOINT="https://YOUR-RESOURCE-NAME.openai.azure.com"
export AZURE_OPENAI_DEPLOYMENT="YOUR_DEPLOYMENT_NAME"
export AZURE_OPENAI_API_KEY="..."
export PYVEIL_SECRET="a-long-random-hmac-secret"
python -m pyveil.integrations.azure_openai

# Or .env, optionally combined with non-secret YAML
python -m pyveil.integrations.azure_openai --env-file .env
python -m pyveil.integrations.azure_openai \
  --config examples/azure_openai.example.yaml --env-file .env

Priority is process environment, .env, YAML, then defaults. YAML may store endpoint, deployment, scope, and environment variable names. Plaintext api_key and pyveil secret fields are rejected. See the repository .env template and YAML template.

6. Block credentials before model-controlled tool calls

The default policy blocks credential-like material in tool.call.arguments. Catch the block and refuse to run the tool.

from pyveil import BlockedSensitiveData, Channel, Veil

veil = Veil.high(secret=b"tenant-secret", scope="tenant-123/run-789")

tool_args = {
    "url": "https://api.example.test/accounts",
    "headers": {"Authorization": "Bearer synthetic-token-value"},
}

try:
    safe_args = veil.redact_data(tool_args, channel=Channel.TOOL_CALL_ARGUMENTS).data
except BlockedSensitiveData as exc:
    print(exc)   # Blocked sensitive data on channel tool.call.arguments (AUTH_HEADER=1)
else:
    run_tool(**safe_args)

7. Redact tool results before returning them to the model

Tool results often carry customer records, URLs, headers, or hidden tokens.

veil = Veil.high(secret=b"tenant-secret", scope="tenant-123/tool-result")

raw_result = {
    "owner": "alice@example.com",
    "callback": "https://example.test/callback?access_token=synthetic-token&state=ok",
}
safe_result = veil.redact_data(raw_result, channel="tool.call.result").data

8. Redact MCP resource content

veil = Veil.high(secret=b"tenant-secret", scope="tenant-123/mcp")

def read_resource_for_agent(resource_id: str) -> object:
    raw_resource = read_resource(resource_id)
    return veil.redact_data(raw_resource, channel="mcp.resource.content").data

9. Redact memory before embedding or persistence

veil = Veil.high(secret=b"tenant-secret", scope="tenant-123/memory")

def write_memory(text: str) -> None:
    safe = veil.redact_text(text, channel="memory.write")
    embedding = embed(safe.text)
    memory_store.save(text=safe.text, embedding=embedding)

10. Redact logs before export

import logging
from pyveil import Veil
from pyveil.integrations import PyVeilLogFilter

logger = logging.getLogger("app")
logger.addFilter(PyVeilLogFilter(Veil.high(secret=b"log-secret", scope="prod/logs")))

logger.warning("failed request for alice@example.com")
# handlers see: failed request for [EMAIL:...]

11. CLI preflight in CI or local scripts

export PYVEIL_SECRET="tenant-or-run-secret"
export PYVEIL_SCOPE="tenant/session"

pyveil demo
printf 'Email alice@example.com' | pyveil redact -
pyveil scan prompt.txt --format json
pyveil redact prompt.txt --channel prompt.input > prompt.safe.txt
pyveil redact request.json --channel tool.call.result --format json > request.safe.json

Command line

Installing pyveil registers the pyveil command. It reads the secret from --secret or PYVEIL_SECRET, and the scope from --scope or PYVEIL_SCOPE (default default). JSON-shaped input is treated as structured data.

# set credentials once
export PYVEIL_SECRET="tenant-or-run-secret"
export PYVEIL_SCOPE="tenant/session"

# immediate synthetic demonstration
pyveil demo

# redact a prompt file to stdout
echo "call 010-1234-5678" | pyveil redact - --channel prompt.input
# call [PHONE:cbbfb8059b58]

# scan for findings as JSON
echo "key sk-proj-abcdefghijklmnopqrstuvwxyz123456" | pyveil scan --format json
# {"findings":[{"type":"API_KEY","placeholder":"[API_KEY:38ded98a17e7]", ...}],"stats":{"total_findings":1, ...}}

# redact structured JSON (use --format json for JSON input)
echo '{"user":"alice@example.com"}' | pyveil redact --channel tool.call.result --format json
# {"data":{"user":"[EMAIL:a3d2b564c825]"},"stats":{"total_findings":1, ...}}

A blocked payload exits non-zero and prints a summary (counts by type, no raw values) to stderr.

Integrations

Thin helpers under pyveil.integrations wrap common boundaries. They wrap a Veil you build, so secret, scope, and policy stay in your control.

LLM provider boundaries

The OpenAI, Anthropic, Ollama, and Azure OpenAI helpers provide runnable client boundaries with dry-run inspection, env/.env/YAML loading, and the exact redacted input returned to the caller. OpenAI and Anthropic add offline tests against the real SDK serialization contracts. See OpenAI, Claude, Ollama, and Azure OpenAI.

Standard logging

PyVeilLogFilter is a logging.Filter that redacts each record's message before handlers export it (channel log.record).

import logging
from pyveil import Veil
from pyveil.integrations import PyVeilLogFilter

veil = Veil.high(secret=b"log-secret", scope="prod/logs")
handler = logging.StreamHandler()
handler.addFilter(PyVeilLogFilter(veil))

logger = logging.getLogger("app")
logger.addHandler(handler)
logger.warning("failed request for alice@example.com")

MCP-style wrappers

Redact tool results and resource payloads before they enter agent context.

from pyveil import Veil
from pyveil.integrations.mcp import redact_tool_result, redact_resource

veil = Veil.high(secret=b"tenant-secret", scope="tenant/mcp")

safe_result   = redact_tool_result(raw_tool_result, veil)   # channel tool.call.result
safe_resource = redact_resource(raw_resource, veil)         # channel mcp.resource.content

See the repository examples/ directory for FastAPI middleware, a LiteLLM proxy filter, an OpenAI Agents guardrail, and more.

API reference

Everything below is importable from the top-level pyveil package.

The Veil class

Module-level helpers

Configuration types

Result types

Detectors & masking board

Core detectors favor high precision over broad coverage. Each entity is masked as shown below. Examples use secret=b"docs-secret", scope="docs/reference"; the 12 hex characters differ for other secrets or scopes.

Free-text authorization headers recognize the common Bearer, Basic, Token, and ApiKey schemes. Phone detection covers common Korean, separated international, and compact E.164 shapes; it is not a country-aware phone-number parser.

See the reproducible synthetic detector evaluation for the public corpus, exact CI pass criteria, current supported-shape results, and limitations.

What is intentionally not detected

Core does not broadly discover personal names, postal addresses, or arbitrary domain-specific identifiers (account, billing, telecom, membership, device IDs). Broad semantic detection has higher false-positive and false-negative risk. Add known values and narrow domain IDs with CustomRule, or layer a dedicated NER system upstream when unknown semantic entities must be discovered.

Safety model & non-goals

pyveil reduces sensitive-context exposure at agent boundaries. It works alongside — not instead of — access control, logging discipline, secret scanning, and provider-side controls.

• No raw sensitive value is stored in a Finding by default.
• Placeholders use HMAC-SHA256 with your secret; scope separates contexts.
• max_input_chars caps input before detection.
• The core has no network calls and no required third-party runtime dependency.

FAQ

Is pyveil a DLP product?

No. It is lightweight redaction middleware for agent context boundaries. It does not replace enterprise DLP, access control, secret scanning, or provider-side safety controls.

Does pyveil guarantee compliance?

No. It can reduce sensitive-context exposure, but it does not prove GDPR, HIPAA, PCI, or any other regime.

Why are placeholders stable?

Stable HMAC placeholders let an agent preserve references across a run without seeing the raw value. Use different scope values when cross-context linkability is not desired.

Why not detect names and addresses by default?

Broad semantic detection has higher false-positive and false-negative risk than the high-precision core detectors. Use CustomRule.exact(...) for values the application already knows, narrow CustomRule regexes for domain IDs, or a dedicated NER layer for unknown names and addresses.

Can I unmask values later?

No. pyveil has no reversible vault and no unmasking API. This is intentional: the default design avoids storing raw sensitive values.

Does pyveil call external services?

No. The core uses the Python standard library and makes no network calls.