What's New 🔥
- Custom prompt template overriding for all RAG metrics. This was introduced for folks using weaker models for evaluation, or just models in general that don't fit too well with OpenAI's prompt formatting, which is what most of
deepeval's metrics are built around. You can still use your favorite metrics and algorithms, but now with a custom template if required. Example here: https://docs.confident-ai.com/docs/metrics-answer-relevancy#customize-your-template - Fixes to our model providers. Now more stable and usable.
- Including
save_as()for datasets to save test cases as well: https://docs.confident-ai.com/docs/evaluation-datasets#save-your-dataset - Bug fixes for
Synthesizer - Improvements to prompt templates of
DAGMetric: https://docs.confident-ai.com/docs/metrics-dag