Ecosystem overview for everything related to llm-eval.
Promptfoo is a CLI and library designed for evaluating and red-teaming Large Language Model (LLM) applications. It helps developers build secure and reliable AI apps, moving beyond trial-and-error. It offers automated evaluations, vulnerability scanning, side-by-side model comparison, and CI/CD integration. A key feature is its ability to run evaluations 100% locally, ensuring privacy. It is developer-first, flexible, battle-tested, and provides data-driven insights.