Promptfoo LLM Vulnerability Scanner

Promptfoo LLM Vulnerability Scanner is an open-source tool for identifying and reporting security vulnerabilities in large language model (LLM) applications. It is developed by Promptfoo, which has since joined OpenAI. The scanner performs automated adversarial testing (red teaming) against LLM-based systems including standalone models, fine-tuned models, retrieval-augmented generation (RAG) pipelines, and AI agents. Vulnerability categories covered: - PII Leaks: Detection of personally identifiable information exposure - Prompt Injection: Identifying susceptibility to unauthorized prompt manipulations - Jailbreaking: Testing whether users can bypass model restrictions - Excessive Agency: Assessing whether the model takes unwanted or unauthorized actions - Hallucination: Detecting generation of false or misleading information - Hijacking: Testing for unauthorized or off-topic use - Overreliance: Checking for incorrect input assumption dependencies - Competitor Endorsements: Identifying cases where the model suggests competitor products - Unintended Contracts: Detecting output that could constitute unauthorized commitments Key capabilities: - Automatically generates custom adversarial probes targeted at the specific application and use case - Produces detailed vulnerability reports with analysis and remediation guidance - Provides continuous monitoring of an LLM application's security posture across models, prompts, and applications - Supports compliance and privacy performance verification The tool is fully open source and available on GitHub.