HolmesGPT/holmesgpt 0.12.3-alpha.1 on GitHub

What's Changed

fix(doc): don't split OpenAI in Azure OpenAI by @mainred in #487
Update ADOPTERS.md with our organization by @aritraghosh in #496
Update ADOPTERS.md by @pavangudiwada in #499
marking non 100 Percent test as flaky by @Sheeproid in #501
Update README.md by @pavangudiwada in #498
ROB-1356: holmes is not following DNS troubleshooting instructions by @nherment in #436
fix(test): check classifier model pattern for Azure OpenAI by @mainred in #500
feat: cache toolset status and add toolset management tool command by @mainred in #459
add logs showing the classifier model by @aantn in #505
Check promethues toolset health also on auto discovery by @moshemorad in #507
On exception close eval by @moshemorad in #508
chore(toolset): aks toolset relies on the azure account setup by @mainred in #509
ROB-1413 CVE packages updates by @RoiGlinik in #504
Updating workflows to run only on code changes by @pavangudiwada in #511
fix(toolset): fix toolset override in config by @mainred in #512
OpenSearchLogsToolset uses the correct docs link by @itisallgood in #514
Revert Path Ignore by @moshemorad in #516
Update llm-evaluation.yaml by @moshemorad in #517
Update ADOPTERS.md | adding innovaccer by @chitenderkumar in #518
Add Qingchuan Hao as maintainer to HolmesGPT by @aritraghosh in #515
Disable streaming output when calling robusta AI (tools streaming sti… by @arikalon1 in #503
update eval docs to use poetry by @aantn in #486
Fix server loading tools by @moshemorad in #534
improve truncation by @aantn in #539
ROB-1457: issue with holmes not returning instructions by @nherment in #548
Fix version warning from pipx by @aantn in #544
If using Gemini - disable structured output by @arikalon1 in #551
Load sentry only on official releases by @moshemorad in #553
add coverage check by @moshemorad in #554
ROB-1441 global date context by @RoiGlinik in #552
Fix interactive mode so it can answer without follow up questions by @aantn in #546
MCP toolset validation fixes by @vrutkovs in #556
ROB-1441: add ability to mock datetime prompt by @nherment in #555
feat(runbook): introduce runbook toolset to fetch internal runbooks by @mainred in #547
Adding toolsets for Azure Network Security Groups (NSGs) by @julia-yin in #538
make llm aware of tool truncation by @aantn in #545
ROB-1267: Unified Holmes logging by @nherment in #408
ROB-1515: RCA eval for DNS resolution by @nherment in #549
Rob 1573 fix evals mocks by @nherment in #562
Rename fetch_pod_logs_ERROR_ask-holmes-namespace-51_my-app-51-59d94fd… by @moshemorad in #564
ROB-1573 fix evals mocks 2/2 by @nherment in #563
make interactive mode the default by @aantn in #568
More tool logging improvements by @aantn in #543
feat: Enrich AKS node health tools by @nilo19 in #531
ROB-1441 atlas mongodb toolset by @RoiGlinik in #557
Tweak toolset cache logging by @aantn in #541
Add initial user ask to history by @aantn in #567
fail evals earlier when llm-as-judge not available by @aantn in #575
remove unused prompt by @aantn in #569
speed up cli startup by @aantn in #542
improve cli output by @aantn in #572
fix(toolset): fix toolset config is not initialized when check prerequisites is not called by @mainred in #535
minor improvements to user-facing tool descriptions by @aantn in #582
Fix bug: LLM-as-judge expected in the output 2% of the time by @Sheeproid in #586
add test case by @Sheeproid in #580
fix: use correct prerequisites to check az cli login status by @mainred in #584
fix: runbook instruction should list all items clearly by @mainred in #587
further pytest improvements to improve DX by @aantn in #578
fix mocks ask 45 due to failures - one mock was missing by @Sheeproid in #598
add tags to failing tests by @Sheeproid in #597
remove duplicate tool_span, already in FallbackToolWrapper and MockToolWrapper by @Sheeproid in #602
ROB-1646: use kubectl command instead of kubernetes api by @nherment in #596
fix: braintrust group by input breaks because system input is dynamic by @Sheeproid in #603
test: stabilise port forward eval by @nherment in #608
ROB-1516: add kafka lag eval by @nherment in #588
ROB-1554 azure sql toolset by @nherment in #574
Don't log performance when env var is disabled by @aantn in #606
Pass azure env vars to eval gh workflow by @nherment in #610
ROB-1600: make opensearch env var and config description more generic by @nherment in #595
ROB-1470 notion toolset precheck by @RoiGlinik in #585
Update .gitignore to ignore .claude-trace/ by @aantn in #613
Ignore .AUTOGENERATED in .pre-commit-config.yaml by @aantn in #612
ROB-1554: azure sql toolset test by @nherment in #609
Create runbook for troubleshooting AKS upgrade failures by @julia-yin in #611
Improve tools-output and add slash commands for it by @aantn in #605
ROB-1646: add tool call safeguards by @nherment in #614
ROB-1611 servicenow experimental toolset by @RoiGlinik in #599
Adding fix to encode utf8 by @aritraghosh in #624
Add HTTP API documentation by @kristeey in #592
isolate evals in separate namespaces so cleanup/teardown for one does… by @aantn in #600
evals should not fail when braintrust is disable by @aantn in #618
[ROB-1641] Bump protobuf from 6.30.2 to 6.31.1 by @dependabot[bot] in #625
Fix bug in opensearch cluster health command by @arikalon1 in #521
Update azure sql install.md by @nherment in #617
fix: fix several check error introduced by old code by @mainred in #565
Enable holmes to count kubernetes resources by @aantn in #607
Possible fix for release process by @aantn in #628
Dont mark failing evals as passed by @aantn in #619
Fix bug with bash toolset and kubectl --sort-by by @aantn in #627
HolmesGPT standalone docs by @pavangudiwada in #497
fix(toolset): access model fields via class by @emmanuel-ferdman in #540
Added docs for PyPi by @pavangudiwada in #647
make pytest output more concise + skip grafana test in more conditions by @aantn in #577
ROB-1723: datadog logs toolset by @nherment in #636
ROB-1740: datadog metrics toolset by @nherment in #645
ROB-1739: datadog/traces toolset by @nherment in #648
add github action for publishing to pypi + a script to do so manually by @aantn in #646
feat: add --refresh-toolsets flag in ask command to refresh toolsets status by @mrchocha in #652
Interactive mode: add /run, /clear, /show, and /context commands + other improvements by @aantn in #644
More interactive mode improvements by @aantn in #655
chore: check toolset prerequisites in parallel by @mainred in #583
Create CLAUDE.md by @aantn in #657
Fix eval 28_ by @aantn in #654
Create cncf-self-assesment.md by @aantn in #637
fix parallel tool calling by @aantn in #661
Try load robusta ai by @moshemorad in #650
ROB-1546 health check json by @RoiGlinik in #635
add tracing to holmes cli by @aantn in #660
Support piped input by @aantn in #656
Added the Prometheus alert investigation workflow by @pavangudiwada in #663
Fixes a keyerror when URL parameter is missing in the internet toolset by @pavangudiwada in #665
Updated K9s plugin docs and fixed to work with contexts by @pavangudiwada in #666
Improve interactive mode autocompletion by @aantn in #658
Fix 'G' key behavior in /show command to go to beginning of last line by @pavangudiwada in #673
Print logger name on logs tool calls by @arikalon1 in #670
show evals summary at end of pytest run by @aantn in #620
chore: move utils functions out of main.py by @mainred in #672
Fixes broken Slack link and page edit buttons by @pavangudiwada in #681
Fixing docs to use Poetry by @pavangudiwada in #674
chore: use zlib to compress/decompress data by @mainred in #671
Move sql import to init due to dependency issue by @moshemorad in #685
Improve evals mock generation by @aantn in #687
fix: use dummy tracer by @mainred in #683
Fixed Some test cases during a POC for Claude models by @Sheeproid in #682
Update LLM Version and Add Exception for Max Step Exceeded by @Sheeproid in #694
new evals for logging and more by @aantn in #601
Use RichHandler and remove existing log handler when init log by @mainred in #693
Update CLAUDE.md by @aantn in #692
check if there is anything to read() so pycharm debugger works by @Sheeproid in #699
use --refresh-toolsets instead of toolset refresh in the log by @mainred in #700
Interactive mode: fix /clear by @aantn in #696
Do version check in background to avoid slowing down startup by @aantn in #659
Improvements to evals framework by @aantn in #702
Convert old mocks by @aantn in #689
ROB-1778 support openshift token for prometheus toolset by @RoiGlinik in #701

New Contributors

@chitenderkumar made their first contribution in #518
@vrutkovs made their first contribution in #556
@julia-yin made their first contribution in #538
@nilo19 made their first contribution in #531
@kristeey made their first contribution in #592
@dependabot[bot] made their first contribution in #625
@emmanuel-ferdman made their first contribution in #540
@mrchocha made their first contribution in #652

Full Changelog: 0.11.5...0.12.3