Highlights
- Major improvements to the NewRelic toolset
- New Cilium and Hubble toolset (thank you @matmerr!)
- Improved truncation and handling of long conversations
- Better Gemini support
- Dozens of small improvements and bugfixes
What's Changed
- Update docs: datadog and modelList by @aantn in #942
- Fix docs for github toolset by @aantn in #979
- ROB-1933 tool approval workflow by @nherment in #919
- better datadog error messages by @aantn in #980
- ROB-2034: Limit evidence size by @nherment in #978
- Freeform newrelic by @Sheeproid in #969
- Addding llm_instructions reference to docs by @pavangudiwada in #901
- Fixes to
holmes versionby @aantn in #893 - Newrelic traces improvement by @Avi-Robusta in #985
- New Evals for newrelic by @Avi-Robusta in #983
- Fix the httpx dependency to make it compatible with http_proxy by @feiskyer in #991
- mute unncessary info log for console by @mainred in #994
- feat: add initial implementation of feedback command by @mainred in #989
- Support embeds for NR transactions by @mershal in #996
- ROB-2111: limit max output tokens by @nherment in #1005
- log any loaded model by @mainred in #1001
- Fix MCP toolset prerequisites check by @feiskyer in #990
- Publish model benchmarks by @aantn in #897
- make newrelic formatting result is off by default by @Sheeproid in #1008
- ROB-2189 add empty check for str case by @RoiGlinik in #1000
- ROB-2197 Allow passing custom sys prompt param in Holmes chat endpoint by @mershal in #1006
- fix alignment issue in docs banner by @Sheeproid in #1012
- ROB-2196 Add NewRelic instructions regarding traces time filters by @mershal in #1011
- ROB-2141 new relay args test by @RoiGlinik in #982
- Add benchmarks for Claude 4.5 by @aantn in #1017
- Skip test from github action which requires Prometheus by @aantn in #1010
- refactoring newrelic formatting by @Avi-Robusta in #1013
- Updated community meeting details. by @pavangudiwada in #988
- updated new relic docs to match revised toolset by @Sheeproid in #1021
- add extended benchmark by @aantn in #1020
- feat(feedback): add privacy notice by @mainred in #1002
- chore(toolset): add Cilium and Hubble to toolsets by @matmerr in #769
- Skip importing the Prometheus toolset by @arikalon1 in #992
- remove the POST version of tempo as it is not supported by @Sheeproid in #1023
- ROB-2136 link tool limit to global truncation limit by @nherment in #987
- ROB-2136: increase prometheus tool token limit to 5% by @nherment in #1026
- ROB-2136: remove prometheus tool token limit and fallback on global limit by @nherment in #1027
- fixes for gemini by @aantn in #999
- Improve cli experience by making output less spammy by @aantn in #981
- Make benchmarks more prominent in docs by @aantn in #1030
- remove experimental tag from 3 toolsets by @aantn in #1031
- Fix instruction broken into two lines by @Munken in #1032
- no trying bash tools with no env vars by @Sheeproid in #1024
- evals: exclude throttled responses from latency calculation by @aantn in #1033
- Update default models by @aantn in #864
- change default fallback context window size by @aantn in #1034
- Fix issue with env vars like CLASSIFIER_MODEL in benchmarks script by @aantn in #1038
- ROB-2227 avoid env exception when using bedrock with model list by @RoiGlinik in #1022
- Evals: create k8s object in test namespace and use local manifest by @Sheeproid in #1040
- ROB-1933: issues with SaaS approval flow by @nherment in #1037
- fix: allow the user to config api key in config file by @mainred in #1036
- add benchmarks with on-prem models by @aantn in #1042
- ROB-2233: model attributes returned by the new relay models api by @nherment in #1028
- improve ai provider docs by @aantn in #1043
- ROB-2117 add docs and option to use 1m sonnet with cli by @RoiGlinik in #1041
- limit max tool size to hard coded 25K regardless of model size by @aantn in #1045
- Build docker images on every commit by @aantn in #964
- ROB-2269 improve token counting by @nherment in #1046
- log details on context window by @aantn in #1047
- Add google-cloud-aiplatform dependency to allow Vertex AI Claude models by @fahlmant in #1029
- feat: Add experimental AG-UI supported chat endpoint and PPL query assist by @kylehounslow in #1035
- chore: add prerequisites for OpenSearchQueryAssistToolset. by @kylehounslow in #1052
- Recommend models in installation docs by @aantn in #1051
- benchmark with qwen (not optimally configured though) by @Sheeproid in #1053
New Contributors
- @feiskyer made their first contribution in #991
- @mershal made their first contribution in #996
- @matmerr made their first contribution in #769
- @Munken made their first contribution in #1032
- @fahlmant made their first contribution in #1029
- @kylehounslow made their first contribution in #1035
Full Changelog: 0.14.2...0.15.0