What's Changed
- Print braintrust link at end of evals by @aantn in #708
- Dont require defining evals twice by @aantn in #707
- Updating HolmesGPT readme by @pavangudiwada in #705
- Update eval tags by @aantn in #710
- Update README.md by @aantn in #712
- add skip tag for broken evals by @aantn in #714
- evals framework: further improvements by @aantn in #716
- Improve evals by @aantn in #709
- Fix nitpick on version string by @aantn in #703
- Fix more evals by @aantn in #718
- Changed the position of GIF and some copy changes by @pavangudiwada in #720
- Autocomplete for /show command by @pavangudiwada in #722
- WIP: Fix more evals2 by @aantn in #724
- Fixing k9s Plugin text by @pavangudiwada in #726
- Eval fixes3 by @aantn in #728
- Eval fixes4 by @aantn in #732
- ask for multiple tool calls for reasoning models by @Sheeproid in #734
- chore: don't hardcode agent name by @mainred in #730
- fix setups on logging evals by @aantn in #737
- Improve HolmesGPT accuracy on questions about itself, configuring tools, and using runbooks by @aantn in #729
- Further runbook improvements by @aantn in #740
- Dont run version check on dev versions by @aantn in #738
- Print braintrust links at beginning of evals run (in addition to end) by @aantn in #735
- fix a bug where holmes was fetching runbooks from slab + promote 3 evals to easy by @aantn in #741
- Updated docs build instructions by @pavangudiwada in #733
- Fixed double exit crash in /show by @pavangudiwada in #745
- Share more logic for system prompt templating by @aantn in #727
- [ROB-1732] Holmes able to unzip logs from findings by @Avi-Robusta in #731
- fix log evals by @aantn in #742
- speed improvements to
holmes askstartup via lazy imports by @aantn in #691 - /show yaml syntax highlighting by @pavangudiwada in #725
- ROB-1078: add cluster name to system prompt by @nherment in #630
- speed up pytest by @aantn in #749
- toggle version check by @mainred in #748
- Update confluence tool description so it doesn't mention runbooks by @aantn in #746
- Update permissions.md by @aantn in #751
- ROB-1778 openshift prom by @RoiGlinik in #736
- add missing namespace cleanup to evals by @aantn in #752
- Removed toolset refresh note by @pavangudiwada in #753
- Slack file upload fix by @pavangudiwada in #754
- add line count to tool output by @aantn in #757
- make config file dir configurable by @mainred in #750
- small eval fixes by @aantn in #755
- ROB-1797 fix bug of losing temp arg by @RoiGlinik in #698
- Add interactive mode guide by @aantn in #719
- Updated CICD guide by @pavangudiwada in #760
- Update CLAUDE.md by @aantn in #762
- bump azure-mgmt-sql to the latest 4.0.0b21 by @mainred in #747
- config mcp server in config file by @mainred in #761
- Fix logs toolsets prompts by @moshemorad in #767
- [ROB-1752] - CVE patchs by @Avi-Robusta in #764
- [ROB-1752] MCP cve- patch by @Avi-Robusta in #768
- make agent name in welcome banner configurable by @mainred in #766
- better braintrust experiment_id by @aantn in #771
- Update llm instructions for argo by @arikalon1 in #770
- Remove vmss run command by @aritraghosh in #756
- Rob 1868 fix issue when user asks for logs holmes isnt fetching logs but spans by @moshemorad in #773
- DataDog metrics toolset improvement by @arikalon1 in #758
- refactor: unified test cases iterations creation logic by @Sheeproid in #781
Full Changelog: 0.12.2...0.12.4