TrojAI Detect

TrojAI Detect is an AI security testing platform that performs automated red teaming and penetration testing on AI, ML, and GenAI models during the build phase. The tool identifies security vulnerabilities and behavioral flaws before model deployment through comprehensive testing methodologies. The platform tests for multiple attack vectors including prompt injection, jailbreaking, unbounded model consumption, sensitive information disclosure, toxic content generation, improper output handling, data and model poisoning, system prompt leakage, vector and embedding weaknesses, and misinformation. It also evaluates model robustness, explainability, bias, drift, and performance. TrojAI Detect employs three red teaming methodologies: static testing using benchmark datasets, manipulated inputs created algorithmically, and dynamic testing where LLMs attack models while other LLMs judge attack success. The platform provides over 150 built-in security and safety tests with options for custom test creation. The tool supports testing across tabular, NLP, and LLM models from commercial, open source, and custom sources. Results are prioritized by severity and presented through automated reports that map to AI security standards including OWASP, MITRE, and NIST frameworks. Testing policies are customizable and content-specific to address organizational requirements.