Intro to LLM Security - OWASP Top 10 for Large Language Models (LLMs)

Large language model security is increasingly about catching risky behavior before it reaches users—and doing it continuously once models go live. A practical framework from OWASP’s “Top 10 for Large Language Models” highlights recurring failure modes such as prompt injections (often framed as jailbreaks), insecure handling of malicious outputs, training-data poisoning, denial-of-service via excessive resource use, supply-chain and permission gaps, data leakage, and insecure plugin integrations. The core takeaway is that these risks aren’t one-off bugs; they require detection, monitoring, and guardrails that evolve as prompts, models, and usage patterns change.

The session walks through the OWASP Top 10 themes and then connects them to measurable signals that can be extracted from prompts and model responses. Prompt injection risk can be approximated with “jailbreak similarity” style metrics—if a prompt resembles known jailbreak patterns, systems can block the prompt or withhold the model’s response. Insecure output handling can be monitored by looking for response-side indicators such as toxicity, jailbreak-related similarity, or pattern matches for sensitive content. Data leakage is treated as both directions of flow: sensitive information leaking out of the model (e.g., PII like phone numbers, Social Security numbers, or credit cards) and sensitive information being sent into a hosted model unintentionally. The talk also emphasizes “excessive agency” risk—when models are given too much control—along with the need for authorization and permission tracking, especially when plugins connect LLMs to external tools and accept free-form inputs.

To make these ideas operational, the workshop introduces WhyLabs’ open-source Linkit and its integration with YLabs for AI observability. Linkit profiles prompt/response pairs and produces privacy-preserving data profiles rather than storing raw text. Those profiles include a mix of security-relevant metrics (pattern detection for PII, toxicity, jailbreak similarity, sentiment, and other similarity/relevancy signals) and general quality signals (readability and distribution statistics). Once metrics are written into YLabs, teams can visualize trends over time, set monitors, and trigger alerts or workflows when metrics drift or spike—turning security monitoring into something closer to standard ML/data observability.

A hands-on portion uses Google Colab with Hugging Face’s gpt2 to generate responses, then profiles multiple prompt/response examples and uploads the resulting metrics into YLabs. The dashboard demonstrates time-series behavior: metrics like sentiment can shift across simulated “days,” and the platform supports monitors such as data drift detection using algorithms like Hellinger distance. For security use cases, the workflow is to (1) profile inputs/outputs, (2) set guardrails based on thresholds (e.g., block responses when toxicity exceeds a cutoff or when jailbreak similarity indicates risk), and (3) continuously monitor for changes that might indicate new attack patterns or regressions.

The session closes by showing how to implement guardrails locally using extracted metrics and simple conditional logic, and how to extend Linkit with custom metrics (user-defined functions) for organization-specific signals—such as similarity checks against a vector database of known jailbreaks. Overall, the message is clear: LLM security scales when detection and evaluation become part of the production pipeline, not an after-the-fact review.