Why LLM Output Filtering Isn't Enough — A Defender's Stack

Most teams adopting LLMs reach for output filtering first — a regex or another LLM that scans the response for "bad" content before it reaches the user. It is necessary. It is also nowhere near sufficient.
What output filtering actually does
An output filter is a last-mile check. It catches profanity, obvious PII, the words you told it to look for. It cannot tell you whether the model just made the wrong tool call, leaked data into a downstream API, or gave the right answer for the wrong reasons.
The four-layer defender's stack
Real defense is staged and redundant. Each layer assumes the previous one might fail.
- Layer 1 — Input governance. Validate, sanitize, and structurally separate user input from system context. Drop zero-width and bidi characters; bound size; type-check.
- Layer 2 — Capability constraints. Restrict what the model is even allowed to attempt. Tool-use schemas, allowlisted endpoints, parameter type-checking.
- Layer 3 — Policy gating. A deterministic policy engine between the model's intent and any side-effecting action. "Yes you may send an email — but only to addresses on this user's contact list."
- Layer 4 — Output filtering and monitoring. The familiar last-mile check, plus anomaly detection on what the model is trying to do over time.
Where teams skip steps
The mistake we see most often is skipping Layer 3. Without a deterministic policy gate between the model and your tools, you are trusting the model to enforce its own boundaries — and prompt injection exists precisely because that trust does not hold.
Telemetry that actually helps
Log the prompt, the tool calls attempted, the tool calls executed, the policy decisions, and the final output — for every request. Without those five fields per turn, you cannot reconstruct an incident. Most LLM observability tooling captures the first and last; the middle three are where the answers live.
Start where the blast radius is biggest
Map your LLM applications by the worst thing they can do if compromised. Send wire transfers? Modify customer records? Draft an email? Order them by impact and harden top-down. Our cybersecurity team helps SMBs run LLM threat-models that produce a concrete prioritized backlog instead of a 60-page report.

Further reading
Frequently asked questions
What does an LLM output filter do?
An output filter is a last-mile check. It catches profanity, obvious PII, the words you told it to look for. It cannot tell you whether the model just made the wrong tool call, leaked data into a downstream API, or gave the right answer for the wrong reasons.
What should you log for LLM security?
Log the prompt, the tool calls attempted, the tool calls executed, the policy decisions, and the final output - for every request. Without those five fields per turn, you cannot reconstruct an incident.
Where should LLM hardening start?
Map your LLM applications by the worst thing they can do if compromised. Send wire transfers? Modify customer records? Draft an email? Order them by impact and harden top-down.
