New to deepeval? Get started here.
This release also collects the changes from 4.1.2 through 4.1.10, which shipped to PyPI without a GitHub release.
✨ New Feature
- Features/voice (#3036 by @penguine-ip)
4.1.10 - support user credentials for Vertex AI (#3017 by @ykocaogullar)
4.1.8 - add support for mcp servers (#3046 by @tanayvaswani)
4.1.8 - Add SDK support for golden update and delete (#2945 by @A-Vamshi)
4.1.6 - Revamp DAG metrics to be constructed in a top-down approach (#2912 by @A-Vamshi)
4.1.3 - support server-driven login questionnaires (#2938 by @penguine-ip)
4.1.3
⚡ Improvement
- update changelogs for 2026 till August's latest release (#3106 by @A-Vamshi)
- drop retired model ids from the model pages (#3074 by @chiruu12)
4.1.10 - typescript docs (#2996 by @kritinv)
4.1.6 - . (#2990 by @penguine-ip)
4.1.5 - document StylingConfig and ConversationalStylingConfig for Synthesizer (#2932 by @jsaimanoj)
4.1.3 - tag llms.txt Confident AI links with UTMs (#2920 by @tonycueva)
4.1.2
🐛 Bug Fix
- Refactor metrics to standardize score. (Higher is better) (#3102 by @A-Vamshi)
- remove coercive and PII prompt content that trips provider guardrails (#3071 by @Patwaji)
4.1.10 - record zero token counts and per-token rates on LLM spans (#3069 by @Anai-Guo)
4.1.10 - honor include_reason and align sync/async reason generation (#3061 by @biztex)
4.1.10 - use final score token for logprob weighting (#3060 by @tunglambk)
4.1.10 - align CSV context delimiters with save_as output (#3049 by @Koustav-github)
4.1.10 - respect CONFIDENT_REGION setting in backend routing (#3041 by @lorenzozanee)
4.1.10 - reject a root node that is reachable from another root (#3027 by @jhochenbaum)
4.1.8 - count mutually exclusive verdict edges as one incoming edge (#3024 by @jhochenbaum)
4.1.8 - fix tool call type serialization globally for any pydantic serialization (#2979 by @A-Vamshi)
4.1.5 - stamp utm_content first so GA4's 100-char link_url keeps the label (#2934 by @tonycueva)
4.1.3 - correct silent answer mis-scoring in MathQA, DROP, and BigBenchHard (#2916 by @hassaanch23)
4.1.3 - fix typo in tool name on metrics step efficiency doc (#2933 by @huisman)
4.1.3 - Add new DAG API e2e tests and fix docs (#2937 by @A-Vamshi)
4.1.3 - serialize tool call enums in JSON mode (#2926 by @nightcityblade)
4.1.3 - serialize ToolCall enums safely in save_as (#2928 by @Solaris-star)
4.1.2
Contributors to these changes: @A-Vamshi, @Anai-Guo, @biztex, @chiruu12, @hassaanch23, @huisman, @jhochenbaum, @jsaimanoj, @Koustav-github, @kritinv, @lorenzozanee, @nightcityblade, @Patwaji, @penguine-ip, @Solaris-star, @tanayvaswani, @tonycueva, @tunglambk, @ykocaogullar
A huge thank you to everyone who contributed to this release ❤️