Breaking Changes
Heretic 2.0 is a major release that breaks backwards compatibility:
- Configuration files made for Heretic 1.x are not compatible with Heretic 2.0! See the migration guide for what you need to change in order to make your old configurations work.
- Research features have been removed. The intended replacement is Lophius.
Changes
- fix: correct total trial count when adding additional trials by @kali113 in #385
- feat: let the optimizer disable MLP ablation via a 0 max_weight floor by @rocker-zhang in #387
- fix: resolve UnicodeEncodeError on Windows during model evaluation by @umran666 in #389
- feat: headless operation + end-to-end tests by @p-e-w in #392
- fix: use
W_orgmatrix only where needed by @Vinay-Umrethe in #398 - feat: generic plugin system by @red40maxxer in #53
- fix: validate scorer instance names in config by @Bortlesboat in #407
- feat: Add alternate CI hash in the valid list (for qwen2.5) by @Vinay-Umrethe in #412
- docs: quote pip extras install examples by @nyxst4ck in #395
- feat: add
reproducibleproperty to plugins by @red40maxxer in #414 - docs: update model count in README by @Vinay-Umrethe in #424
- chore: remove Gemini style guide by @p-e-w in #427
- chore: bump version to 2.0.0.dev0 by @p-e-w in #428
- feat: Show a summary at end of tests by @Vinay-Umrethe in #425
- feat: add benchmark scorer by @p-e-w in #444
- feat: support specifying a dataset's specific config/subset by @Vinay-Umrethe in #445
- feat: A better logic for detecting the thinking prefix in the response. by @Vinay-Umrethe in #423
- feat: Add base model information in the README.md model card by @Vinay-Umrethe in #459
- feat: modifier plugins + ARA by @p-e-w in #446 (contributions to ARA by @kabachuha, @joninco, and @coder3101)
- chore: update dependencies by @p-e-w in #469
- chore: add new Windows test hashes by @Vinay-Umrethe in #470
- fix: stop
lora_Afrom carrying over between trials by @benthecarman in #476 - fix: prevent empty layer range and too many neighbors in ARA by @benthecarman in #478
- fix: escape prompts and responses printed with
print_responsesby @jayhemnani9910 in #482 - fix: silence warning spam from PyTorch's C++ logging by @p-e-w in #485
- fix: skip non-metrics in benchmark output by @p-e-w in #486
- fix: report a save or upload before re-ablating the model by @HamedShams in #488
- chore: add pull request template by @p-e-w in #502
- fix: say which check found the Chain-of-Thought prefix by @HamedShams in #489
- fix: restore stored plugin settings as they are on reproduce and resume by @jayhemnani9910 in #484
- fix: support models whose chat template rejects the system role by @varunbln in #498
- feat: make upload tags configurable by @officialayush5839-arch in #506
- ci: add Python 3.14 and 3.15 to CI matrix by @p-e-w in #511
- fix: reject blank refusal keyword markers by @jovial-liu in #512
- Support models with Hyper-Connections by @benthecarman in #519
- fix: Improve response prefix logic, handle default behaviors, and clarify documentation by @Vinay-Umrethe in #523
- fix: support flattened hyper-connection residual streams by @clee in #525
- perf(ara): compute closure outputs via low-rank update instead of dense weight by @varunbln in #499
- feat: add config.nosyco.toml by @UnstableLlama in #349
- fix: rename reproduce checksum file to SHA256SUMS.txt by @mvanhorn in #479
- feat: make adjective describing the type of model produced configurable by @p-e-w in #539
- feat: Generic BERT Classifier scorer plugin by @Vinay-Umrethe in #500
- docs: prepare for 2.0 release by @p-e-w in #531
New Contributors
- @kali113 made their first contribution in #385
- @Bortlesboat made their first contribution in #407
- @nyxst4ck made their first contribution in #395
- @benthecarman made their first contribution in #476
- @jayhemnani9910 made their first contribution in #482
- @HamedShams made their first contribution in #488
- @varunbln made their first contribution in #498
- @officialayush5839-arch made their first contribution in #506
- @jovial-liu made their first contribution in #512
- @clee made their first contribution in #525
- @mvanhorn made their first contribution in #479
Full Changelog: v1.4.0...v2.0.0