Intro to LLM Security - OWASP Top 10 for Large Language Models (LLMs)

LLM security hinges on treating every prompt-and-response cycle as potentially hostile—then building monitoring and guardrails that catch failures early. The workshop frames OWASP’s “Top 10 for Large Language Models” as a practical checklist for teams shipping LLM-powered features, especially when models are accessed through APIs, wrapped in plugins, or connected to downstream systems like databases and automation tools.

A key starting point is the lack of control teams have over model internals. When using popular LLMs via APIs (or even open-source models), developers often don’t know what data the model was trained on, what post-processing happens behind the scenes, or whether training data poisoning and other supply-chain issues occurred. Even “internal” deployments don’t eliminate risk because LLMs can be manipulated through natural-language attacks. That uncertainty is why the OWASP list matters: it turns a squishy, fast-moving threat landscape into concrete categories teams can design against and measure.

The first and most emphasized threat is prompt injection. Malicious users can craft instructions that override prior system or developer guidance, potentially leading the model to behave toxically or to take actions it otherwise shouldn’t. The mitigation guidance is blunt: assume LLM applications are malicious, apply least-privilege design so the model can’t control the whole user experience or upstream/downstream systems, monitor prompts and responses for unusual patterns (including known injection signatures), and use a human-in-the-loop for high-risk flows when feasible.

From there, the workshop walks through additional OWASP categories with the same theme: validate outputs, isolate risky components, and instrument everything. Insecure output handling is illustrated with an LLM generating SQL—if outputs aren’t validated, an attacker could steer the model toward destructive queries. Training data poisoning is treated as especially hard when the model is an API black box; teams should assume undesirable training data may exist and isolate the model’s impact, while those fine-tuning should validate, clean, anonymize, and test their datasets. Denial of service is addressed through input validation, monitoring resource utilization, and enforcing rate limits and resource caps.

Supply-chain vulnerabilities broaden the lens to the entire lifecycle: training data, libraries, plugins, and even vendor terms that might allow data reuse or retraining. Sensitive information disclosure (PII leakage) is handled through prevention (block prompts containing sensitive data), careful dataset hygiene for fine-tuning, and monitoring for leaks in responses. Plugin risks and “excessive agency” are treated as closely related: plugins should run with least privilege, be vetted, and require user confirmation for sensitive actions; open-ended actions like arbitrary URL opening are hard to secure and should be avoided.

The later items—overreliance and model theft—focus on operational and business risk. Overreliance happens when users treat fluent but incorrect outputs as authoritative; mitigation includes quality monitoring, interface nudges that force review, and cross-verification for high-stakes decisions. Model theft is addressed with strong access control to model assets and monitoring for suspicious activity.

To make the guidance actionable, the workshop introduces WhyLabs’ open-source LLM telemetry library “Lanit” and a hands-on monitoring demo. The approach extracts security-relevant signals (e.g., jailbreak similarity, toxicity, refusal likelihood, and PII patterns) from prompts and responses, then sends only derived scores—not raw text—to a dashboard for visibility and alerting. In the demo, the dashboards flag PII leakage in both prompts and responses and show how spikes in jailbreak similarity and toxicity can correlate with increased PII leakage, enabling targeted investigation even without storing sensitive raw content.