Guardrails for AI Agents: How to Stop Your LLM from Going Rogue in Production

Why System Prompts Are Not Enough

Every developer's first guardrail instinct is correct: add a system prompt. "Do not reveal customer data." "Only answer questions about our products." "Never provide financial advice." These instructions work — until they don't.

The fundamental problem with system-prompt-only safety is that it relies on the LLM following instructions under adversarial conditions. Research published in late 2025 showed that 94% of GPT-4-class models can be prompted to ignore system prompt instructions given sufficiently crafted adversarial inputs. Direct jailbreaks. Indirect prompt injection via documents. Role-playing scenarios that shift the model's frame. The LLM is a reasoning engine, not a policy enforcer. You cannot build your security model on reasoning-based compliance alone.

Code-level guardrails are different. They run outside the LLM. They are deterministic (or near-deterministic). They cannot be bypassed by injecting text into the conversation. When an output guardrail scans for account number patterns and blocks the response, the attacker cannot convince it otherwise by telling it "ignore previous instructions."

Guardrails are your last line of enforcement. System prompts are your first line of guidance. You need both.

The Three Guardrail Layers

Layer 1: Input Guardrails

Input guardrails intercept user requests before they reach the LLM. They reject, sanitise, or flag inputs that are:

• Malicious: prompt injection attempts, jailbreak patterns, role-override instructions
• Out-of-scope: requests that fall outside the agent's configured domain (a customer service bot asked to write code)
• Policy-violating: requests that violate business rules (asking a trading bot to place orders above a risk threshold)
• Resource-exhausting: inputs designed to trigger long, expensive LLM completions (token DoS)

Layer 2: Output Guardrails

Output guardrails scan the LLM's response before it is returned to the user or fed to the next agent in a pipeline. They check for:

• PII leakage: SSNs, account numbers, email addresses, phone numbers, passport numbers
• Hallucination signals: fabricated citations, non-existent product names, invented statistics
• Toxicity and harmful content: offensive language, dangerous instructions
• Format violations: invalid JSON when JSON was required, truncated structured output
• Off-topic drift: responses that wander outside the permitted domain

Layer 3: Action Guardrails

Action guardrails control what tools the agent can call and under what conditions. This is the layer most teams skip — and the one that causes the most catastrophic incidents. Action guardrails enforce:

• Blast-radius limits: agent cannot modify more than 5 records per operation without human confirmation
• Tool allowlists: certain tools (delete, send_email_external, execute_sql_write) require explicit human approval tokens in agent state
• Rate limits: agent cannot invoke the same tool more than N times per workflow to prevent runaway loops
• Scope enforcement: read-only agents cannot be coerced into write operations regardless of instructions

The following Python implementation wraps all three layers around a LangChain LLM call:

import re
from dataclasses import dataclass, field
from typing import Optional, Callable
from langchain_openai import ChatOpenAI
from langchain_core.messages import HumanMessage, SystemMessage

@dataclass
class GuardrailViolation:
    layer: str        # "input", "output", or "action"
    rule: str         # which rule triggered
    severity: str     # "block", "warn", "redact"
    detail: str       # human-readable description

class MultiLayerGuardrail:
    """
    Three-layer guardrail wrapper around any LLM call.

    WHY separate layers: input, output, and action risks are distinct
    and require different detection strategies. Combining them into one
    monolithic check creates false positives and performance bottlenecks.
    """

    # PII patterns — WHY comprehensive: the account number incident shows
    # that regex-only SSN checks miss other sensitive identifiers.
    # Pattern library covers the formats most common in enterprise data.
    PII_PATTERNS = {
        "ssn":            r"\b\d{3}-\d{2}-\d{4}\b",
        "account_number": r"\b\d{10,18}\b",          # bank account numbers
        "credit_card":    r"\b(?:\d{4}[- ]?){3}\d{4}\b",
        "passport":       r"\b[A-Z]{1,2}\d{6,9}\b",
        "email_bulk":     None,                       # handled separately (count-based)
        "api_key":        r"(?i)(api[_-]?key|secret)[_-]?\w*\s*[:=]\s*\S{10,}",
    }

    INPUT_INJECTION_PATTERNS = [
        r"ignore\s+(previous|prior|all)\s+instructions?",
        r"you\s+are\s+now\s+a?\s+",
        r"forget\s+(everything|your\s+instructions?)",
        r"<\s*system\s*>",
        r"\[INST\].*?\[/INST\]",
        r"###\s*(system|instruction)",
    ]

    def __init__(self, llm: ChatOpenAI, system_prompt: str,
                 max_input_chars: int = 16000,
                 on_violation: Optional[Callable] = None):
        self.llm = llm
        self.system_prompt = system_prompt
        self.max_input_chars = max_input_chars
        self.on_violation = on_violation or (lambda v: None)  # logging hook

    def check_input(self, user_input: str) -> list[GuardrailViolation]:
        violations = []

        # Length check — prevents token-flooding DoS
        if len(user_input) > self.max_input_chars:
            violations.append(GuardrailViolation(
                layer="input", rule="max_length",
                severity="block",
                detail=f"Input {len(user_input)} chars exceeds {self.max_input_chars} limit"
            ))

        # Injection pattern check
        for pattern in self.INPUT_INJECTION_PATTERNS:
            if re.search(pattern, user_input, re.IGNORECASE):
                violations.append(GuardrailViolation(
                    layer="input", rule="injection_pattern",
                    severity="block",
                    detail=f"Injection pattern detected: {pattern[:40]}"
                ))
                break  # one injection flag is enough to block

        return violations

    def check_output(self, response_text: str) -> list[GuardrailViolation]:
        violations = []

        # Structured PII pattern scan
        for pii_type, pattern in self.PII_PATTERNS.items():
            if pii_type == "email_bulk":
                # WHY count-based: a single email in a response may be legitimate
                # (e.g., "contact support@example.com"). 5+ emails = bulk extraction.
                emails = re.findall(
                    r"[a-zA-Z0-9._%+-]+@[a-zA-Z0-9.-]+\.[a-zA-Z]{2,}",
                    response_text
                )
                if len(emails) >= 5:
                    violations.append(GuardrailViolation(
                        layer="output", rule="pii_bulk_email",
                        severity="block",
                        detail=f"Response contains {len(emails)} email addresses — possible bulk extraction"
                    ))
            elif pattern and re.search(pattern, response_text):
                violations.append(GuardrailViolation(
                    layer="output", rule=f"pii_{pii_type}",
                    severity="redact",
                    detail=f"PII type '{pii_type}' detected in response"
                ))

        return violations

    def redact_output(self, response_text: str) -> str:
        """Redact detected PII patterns from response text."""
        redacted = response_text
        for pii_type, pattern in self.PII_PATTERNS.items():
            if pattern:
                redacted = re.sub(pattern, f"[REDACTED-{pii_type.upper()}]", redacted)
        return redacted

    def invoke(self, user_input: str) -> dict:
        """
        Full guardrail pipeline: check input → call LLM → check output → return.
        Returns dict with 'response', 'violations', and 'blocked' keys.
        """
        all_violations = []

        # --- Layer 1: Input Guardrails ---
        input_violations = self.check_input(user_input)
        all_violations.extend(input_violations)
        for v in input_violations:
            self.on_violation(v)

        if any(v.severity == "block" for v in input_violations):
            return {
                "response": "I'm unable to process that request.",
                "violations": all_violations,
                "blocked": True
            }

        # --- LLM Call ---
        messages = [
            SystemMessage(content=self.system_prompt),
            HumanMessage(content=user_input)
        ]
        raw_response = self.llm.invoke(messages).content

        # --- Layer 2: Output Guardrails ---
        output_violations = self.check_output(raw_response)
        all_violations.extend(output_violations)
        for v in output_violations:
            self.on_violation(v)

        # Redact rather than block — user gets a response, PII is masked
        final_response = raw_response
        if any(v.severity == "redact" for v in output_violations):
            final_response = self.redact_output(raw_response)
        if any(v.severity == "block" for v in output_violations):
            final_response = "I found relevant information but cannot display it due to data policy restrictions. Please contact your administrator."

        return {
            "response": final_response,
            "violations": all_violations,
            "blocked": False
        }

# --- Usage ---
llm = ChatOpenAI(model="gpt-4o", temperature=0)
guardrail = MultiLayerGuardrail(
    llm=llm,
    system_prompt="You are a helpful financial analyst assistant. Only discuss market trends and portfolio strategy.",
    on_violation=lambda v: print(f"GUARDRAIL [{v.layer}] {v.rule}: {v.detail}")
)

result = guardrail.invoke("What are the high-value client accounts this quarter?")
print(result["response"])

NeMo Guardrails: Declarative Safety Policies

The Python wrapper above is excellent for team-specific rules, but it has a limitation: safety logic lives in application code. Every rule change requires a code deployment. In regulated environments — financial services, healthcare, government — that means a change management ticket, a review cycle, and a deployment window. For fast-moving AI systems, this is too slow.

NVIDIA's NeMo Guardrails solves this by externalising safety policies into declarative Colang configuration files. Policies are separate from code, version-controlled in their own repository, and hot-reloadable without service restarts. An audit trail of policy changes is built in.

The following Colang config implements a production-grade policy for a financial AI assistant — blocking off-topic requests, preventing harmful advice, and enforcing response factuality:

# nemo-config/config.co
# NeMo Guardrails Colang configuration for a financial AI assistant
# WHY Colang: policies are readable by compliance teams, not just engineers.
# This file can be reviewed and signed off by a Chief Risk Officer.

# === DEFINE ALLOWED TOPICS ===
define user ask about market analysis
  "market analysis"
  "portfolio strategy"
  "investment trends"
  "economic indicators"
  "sector performance"

define user ask about customer data
  "show me client accounts"
  "list customer data"
  "what are the account numbers"
  "give me client details"
  "show me client portfolios"

define user ask harmful financial advice
  "how to manipulate stock prices"
  "insider trading"
  "how to hide assets"
  "tax evasion strategies"

# === INPUT RAILS ===

# Block requests for customer PII — Layer 1
define flow block customer data requests
  user ask about customer data
  bot refuse customer data request

define bot refuse customer data request
  "I can't provide individual customer account details. For client-specific queries, please use the authorised CRM system with appropriate access controls."

# Block harmful financial advice — Layer 1
define flow block harmful advice
  user ask harmful financial advice
  bot refuse harmful advice

define bot refuse harmful advice
  "I'm not able to provide guidance on that topic. I'm configured to assist with legitimate market analysis and portfolio strategy only."

# === OUTPUT RAILS ===

# Factual grounding — prevent hallucinated statistics
define flow check factual output
  bot respond with statistics
  $accuracy = execute check_statistics_accuracy(response=$last_bot_message)
  if $accuracy < 0.8
    bot add disclaimer

define bot add disclaimer
  "Note: Some figures in this response are estimates. Please verify against authorised data sources before use in client communications."

# Topic enforcement — keep responses on topic
define flow enforce topic
  bot ...*
  $on_topic = execute check_topic_relevance(response=$last_bot_message, allowed_topics=["market analysis", "portfolio strategy", "economic indicators"])
  if not $on_topic
    bot redirect to topic

define bot redirect to topic
  "I notice my response drifted off topic. Let me focus on what I can help with: market analysis, portfolio strategy, and economic indicators."

# === JAILBREAK RAIL ===
define flow jailbreak detection
  user ask ...
  $is_jailbreak = execute detect_jailbreak(input=$last_user_message)
  if $is_jailbreak
    bot refuse jailbreak

define bot refuse jailbreak
  "I'm not able to modify my operating guidelines. I'm here to help with financial market analysis."

Wire NeMo Guardrails into a LangChain agent with three lines:

from nemoguardrails import RailsConfig, LLMRails

# Load policy from config directory — hot-reloadable without code changes
config = RailsConfig.from_path("./nemo-config")
rails = LLMRails(config)

# Drop-in replacement for direct LLM calls
response = await rails.generate_async(
    messages=[{"role": "user", "content": user_input}]
)

Production Failure Mode: The Account Number Incident

Let me return to the wealth management incident I opened with and walk through exactly why the guardrail failed and precisely what the fix looks like.

What happened: The firm's output guardrail used this pattern:

# The broken guardrail — SSN-only check
# WHY this fails: account numbers (10-18 digits) are not SSNs (###-##-####).
# A regex that matches SSN format passes account numbers entirely.
import re

SSN_PATTERN = r"\b\d{3}-\d{2}-\d{4}\b"  # Only catches ###-##-#### format

def check_pii_simple(text: str) -> bool:
    return bool(re.search(SSN_PATTERN, text))

# This returns False for "Account: 4821930471029" — the number passes through

Why the regex-only approach is fundamentally limited: PII takes hundreds of forms. Account numbers vary by institution (8 to 18 digits, sometimes with dashes or spaces). UK sort codes. IBAN numbers. Indian PAN numbers. Employee IDs. Building individual regex patterns for every format is an arms race you will lose. Every new data type requires a new pattern. Attackers can introduce deliberate spacing (e.g., "4821 9304 7102 9") to evade fixed-format patterns.

The fix: semantic similarity scoring against a PII pattern library. Rather than pattern-matching specific formats, embed the response text and measure cosine similarity against a library of known PII templates. Anything above 0.75 cosine similarity to a PII template gets flagged, regardless of specific format:

from sentence_transformers import SentenceTransformer
import numpy as np

# WHY sentence-transformers: semantic similarity catches format variations
# that regex misses. "Account: 4821930471029" and "Acct # 4821-930-471-029"
# both embed near a PII template, despite different surface forms.

model = SentenceTransformer("all-MiniLM-L6-v2")

# PII template library — canonical examples of sensitive output patterns
PII_TEMPLATES = [
    "The account number is 1234567890123",
    "Customer SSN: 123-45-6789",
    "Card number: 4111 1111 1111 1111",
    "Account balance: $125,000. Account: 987654321098",
    "Client ID 456789 with balance of",
    "Here are the top accounts: Account 123, Account 456",
]

template_embeddings = model.encode(PII_TEMPLATES)

def semantic_pii_check(response_text: str, threshold: float = 0.72) -> bool:
    """
    Returns True if response resembles a PII disclosure pattern.
    Threshold 0.72 balances recall (catching leaks) vs precision (false positives).
    """
    # Split into sentences — check each sentence independently
    sentences = [s.strip() for s in response_text.split('.') if len(s.strip()) > 10]

    for sentence in sentences:
        sent_embedding = model.encode([sentence])
        # Cosine similarity against all PII templates
        similarities = np.dot(sent_embedding, template_embeddings.T) / (
            np.linalg.norm(sent_embedding) * np.linalg.norm(template_embeddings, axis=1)
        )
        if np.max(similarities) > threshold:
            return True  # Semantic match to a PII pattern

    return False

After implementing semantic PII detection alongside the regex layer, the firm ran a retroactive test against 500 historical agent responses. Semantic detection caught 34 additional potential leaks that the regex layer had passed. Seventeen of those contained account numbers.

Guardrail Decision Tree: Which Layer for Which Risk

Use this decision framework when designing your guardrail architecture:

Implementation Priority

If you are just getting started, implement in this order:

1. Output PII guardrail — this is the highest-frequency production incident type. Implement today.
2. Action blast-radius limits — critical for any agent with write access to production systems.
3. Input injection detection — important for user-facing agents; can be implemented with existing libraries like Garak.
4. NeMo Guardrails policy layer — adds declarative, auditable controls; implement once your basic guardrails are stable.

Conclusion: Build the Safety Layer Before You Need It

Every enterprise AI deployment I have reviewed that suffered a significant incident — data leakage, runaway tool calls, regulatory violations — had one thing in common: the safety architecture was planned "for the next sprint" and never quite arrived. The demo worked. The guardrails were "good enough." And then they weren't.

The three-layer guardrail architecture — input, output, action — is not complex to implement. The Python multi-layer wrapper I've shown here takes a day to build and test. The NeMo Guardrails Colang config takes another day. The semantic PII detector using sentence-transformers takes an afternoon. What takes months is recovering from a production incident that those three days of work would have prevented.

In Day 5 of the Agentic AI Workshop, we build and stress-test guardrail architectures in live labs — injecting known attack patterns, measuring what gets through, and iterating on defences. By end of day, every participant has a production-grade safety layer around their agent that they built themselves and understand deeply. That hands-on muscle memory is what makes the difference between a guardrail that gets deployed and one that stays in the backlog.