What is Large Language Model Security (LLM security) (LLM security)?
Large Language Model Security (LLM security) is the practice of protecting LLM-based applications from attacks and failures such as prompt injection, jailbreaks, sensitive data leakage, and unsafe model actions. It covers the full lifecycle from training data through runtime inference and any downstream actions an LLM takes on behalf of users.
What it is
LLM security addresses the unique risks that arise when applications are built on top of large language models. Unlike traditional software, LLMs accept free-text input, generate free-text output, and can be manipulated through the content they process rather than through code exploits alone.
Core threat areas include:
- Prompt injection: Malicious instructions hidden in user input or retrieved documents that redirect the model's behavior.
- Jailbreaks: Inputs crafted to bypass safety guidelines baked into the model.
- Data leakage: The model repeating training data, system prompts, or user PII in its responses.
- Insecure output: Generated content that contains harmful instructions, malware, or false information acted on downstream.
- Unsafe agent actions: An LLM agent that calls APIs, writes files, or executes code in ways the developer did not intend.
Why it matters
LLM applications sit at the boundary between natural language and real system actions. A successful prompt injection can cause an agent to exfiltrate data, delete records, or send messages as the victim user. Standard input validation does not stop these attacks because the attack surface is semantic, not syntactic. Regulatory pressure around AI output accuracy and data handling adds compliance risk on top of technical risk.
How tools address it
LLM Guardrails products sit between the application and the model, inspecting prompts before they reach the model and responses before they reach the user. Common controls include: