What's Changed
This release adds support for multimodal input and output rails, allowing users to check for the safety of both text and image prompts and vision-language model outputs. We introduce new support for models with reasoning traces, such as Deepseek-R1, and we have added support for the NemoGuard JailbreakDetect NIM.
This release adds a new integration with Fiddler Guardrails, providing access to Fiddler’s low-latency hosted guardrail models for content safety, hallucination detection, and more. We also added support for Python 3.12 .
🚀 Features
- Support models with reasoning traces (#996) by @trebedea
- Add SHA-256 hashing option (#988) by @mdambski
- Add Fiddler Guardrails integration (#964, #1043) by @copperstick6
- Add generation metadata to streaming chunks (#1011) by @Pouyanpi
- Improve alpha to beta bot migration (#878) by @schuellc-nvidia
- Support multimodal input and output rails (#1033) by @Pouyanpi
- Add support for NemoGuard JailbreakDetect NIM. (#1038) by @erickgalinkin
- Set default start and end reasoning tokens (#1050) by @Pouyanpi
- Improve output rails error handling for SSE format (#1058) by @Pouyanpi
🐛 Bug Fixes
- Ensure parse_task_output is called after all llm_call invocations (#1047) by @Pouyanpi
- Handle exceptions in generate_events to propagate errors in streaming (#1012) by @Pouyanpi
- Ensure output rails streaming is enabled explicitly (#1045) by @Pouyanpi
- Improve multimodal prompt length calculation for base64 images (#1053) by @Pouyanpi
🚜 Refactor
- Move startup and shutdown logic to lifespan in server (#999) by @xiaobo8204
📚 Documentation
- Add multimodal rails documentation (#1061) by @mikemckiernan
- Add content safety tutorial (#1042) by @tgasser-nv
- Revise reasoning model info (#1062) by @mikemckiernan
- Consider new GS experience (#1005) by @mikemckiernan
- Restore deleted configuration files (#963) by @Pouyanpi
⚙️ Miscellaneous Tasks
New Contributors
- @mdambski made their first contribution in #988
- @copperstick6 made their first contribution in #964
- @xiaobo8204 made their first contribution in #999
- @tgasser-nv made their first contribution in #1042
Full Changelog: v0.12.0...v0.13.0