ragas

About

Ragas, an open-source Python library developed by vibrantlabsai, is a comprehensive toolkit for evaluating and optimizing Large Language Model (LLM) applications. It aims to replace subjective and time-consuming assessments with data-driven, efficient evaluation workflows by offering objective metrics (both LLM-based and traditional), intelligent test data generation, and actionable insights. Ragas can automatically create diverse test datasets, seamlessly integrates with popular LLM frameworks like LangChain and major observability tools, and facilitates building feedback loops to continuously improve LLM apps using production data.

Features

• Objective LLM-based and traditional evaluation metrics
• Automated comprehensive test data generation
• Seamless integration with popular LLM frameworks & observability tools
• Supports production data-driven feedback loops
• Extensible custom evaluators

Supported Platforms