Arize AI Observability is a platform designed for AI and agent engineering that combines development, observability, and evaluation capabilities. The platform provides tracing functionality based on OpenTelemetry (OTEL) standards to monitor AI agents and applications in production. The development tools include a prompt playground for debugging and optimization, prompt serving and management capabilities, and automated prompt optimization using evaluations and annotations. The platform supports CI/CD experiments to detect prompt and agent regressions before deployment. For evaluation, the platform offers LLM-as-a-Judge functionality to automatically evaluate prompts and agent actions at scale. It includes human annotation capabilities with labeling queues for production annotations and golden dataset creation. Online evaluations run in real-time to identify issues as they occur. The observability features provide distributed tracing for AI agents and frameworks, real-time monitoring dashboards, and analytics for AI system performance. The platform processes traces at scale and supports both cloud-hosted and self-hosted open source deployment options. The platform includes dataset management, experiment tracking, and annotation workflows. It is designed to integrate development and production environments, enabling teams to use production data to improve development processes while maintaining alignment between production observability and evaluation metrics.