docker/docker-agent v1.130.0

on GitHub

This release expands the safety classifier coverage, improves skills handling, and adds new evaluation capabilities including assertions, verify scripts, and pass@k metrics.

What's New

• Adds assertions schema and verify field to EvalCriteria for code-based grading checks against agent output
• Adds a verify script runner that executes a shell script via docker exec on the eval container after the agent completes
• Upgrades the relevance judge prompt with a rubric, chain-of-thought guidance, and anti-bias rules for more consistent evaluation judgments
• Exposes pass@k and pass^k metrics when using --repeat runs to quantify answer consistency across repetitions
• Adds PowerShell Remove-Item and Clear-Content patterns to the safety classifier
• Adds destructive SQL, app-CLI, and key-value-store patterns to the safety classifier
• Safely gates embedded skill commands, asking user permission before running commands embedded in skills
• Expands embedded commands in inline skills so they go through the normal approval path

Bug Fixes

• Fixes flag-order variants for docker system prune --volumes and compose down -v in the safety classifier
• Fixes shell-metachar patterns to anchor on whitespace instead of word boundaries in the safety classifier
• Fixes safety classifier to use wildcards for flag-order and quoted-path gaps, and narrows SQL patterns
• Fixes inline skills to override discovered skills when the same name exists in both
• Fixes inline skill names to reject whitespace, preventing unreachable slash commands
• Stabilizes slash command description alignment in the TUI completion list

Technical Changes

• Refreshes the embedded models.dev catalog snapshot
• Updates CI to run checks on pull requests targeting any base branch, not just main

What's Changed

• docs: update CHANGELOG.md for v1.129.0 by @docker-read-write[bot] in #4079
• chore: refresh embedded models.dev snapshot by @github-actions[bot] in #4078
• chore: bump direct Go dependencies by @dgageot in #4080
• safety: close four classifier coverage gaps by @trungutt in #4081
• fix(tui): stabilize slash command description alignment by @rumpl in #4090
• ci: run ci and codeql checks on pull requests to any base branch by @aheritier in #4075
• fix(skills): let inline skills override discovered ones by @dgageot in #4094
• fix(skills): reject whitespace in inline skill names by @dgageot in #4095
• feat: safely gate embedded skill commands by @dgageot in #4093
• feat(skills): expand embedded commands in inline skills by @dgageot in #4096
• feat: add assertions schema to EvalCriteria for code-based grading by @melmennaoui in #4088
• feat: add verify script runner for post-agent outcome verification by @melmennaoui in #4089
• feat: upgrade relevance judge prompt with rubric, CoT, and anti-bias rules by @melmennaoui in #4091
• feat: expose pass@k and pass^k metrics for --repeat runs by @melmennaoui in #4092

Full Changelog: v1.129.0...v1.130.0