github TecharoHQ/anubis v1.28.0
v1.28.0: Wuk Lamat
on GitHub

4 hours ago

This release adds WebAssembly based proof of work checks to Anubis. They are documented in the Proof of Work (WebAssembly) page. This uses Rust code compiled to WebAssembly to run proof of work code. When browsers support SIMD, the WASM will use hardware acceleration.

When clients are configured to disable WebAssembly, Anubis falls back to a pure JavaScript implementation of the WebAssembly-based check logic. As clients that disable WebAssembly usually disable the JavaScript JIT (the thing that makes JavaScript fast), this makes checks slower.

Additionally due to complicated facts and circumstances involving it being complicated to pass the messages from the wasm2js world back to JavaScript, the progress bar will not update while a wasm2js check is running. This is a known issue and will be fixed in a later release.

The difficulty values for the WebAssembly based checks are going to be much greater than the equivalent difficulty values for the JavaScript based checks. Generally these count the number of leading bits that much match instead of the number of leading nibbles that must match. Here's a rough translation table:

fast difficulty argon2id difficulty sha256 difficulty hashx difficulty
4 4 16 15
2 2 8 7
6 6 24 20

As I'm not certain this will work fine out of the box for a few edge cases, I have disabled these new challenge methods by default. I will enable the new methods by default in a future version after they have been sufficiently tested.

  • Use node.js for running wasm in CI.
  • Fix GHSA-5gwm-367w-7fgj, which allowed attackers that control the PTR records for their IP addresses to bypass Anubis via setting multiple PTR records pointing to various search engines.
  • Replace Thoth with GeoIP databases. GeoIP databases are configured in the geoip block.
  • Thoth functionality has been removed. THOTH_URL, THOTH_TOKEN, and THOTH_INSECURE are now deprecated environment variables that will log a warning upon startup.
  • ASN description values now use the Maxmind values instead of the values that Thoth returned. This may impact your fail2ban rules.
  • Native packages now create an anubis group and assign all Anubis instances permissions to /var/lib/anubis.
  • Fix the original-referer cookie not being affected by the COOKIE_PREFIX setting (#1977)
  • Default to the simplified explanation to avoid people misinterpreting words. USE_SIMPLIFIED_EXPLANATION is now deprecated.
  • Add Portuguese (Portugal) (pt-PT) localization.
  • Add documentation for the Headless Browser Detection extension and the Soteria challenge methods exclusive to BotStopper.
  • Add DYNAMIC_COOKIE_SUFFIX setting for toggling the dynamically generated cookie suffix #1992
  • Retry a missing challenge verification cookie once per client before reporting that cookies are disabled, allowing browsers affected by transient cookie loss during navigation to recover without creating an infinite challenge loop (#1916).
  • Use a bundled version of wasm2js in order to make the WebAssembly proof of work checks run in non-wasm environments.
  • Make the bundled wasm2js/wasm-opt WebAssembly modules build reproducibly and fix the build on arm64.
  • Add the concept of Challenge Extensions and add the sample css-load extension.
  • Fix npm run test:integration so the Playwright suite can connect to browsers and Firefox can reach the test server again.
  • Add weighing rule for Cloudflare Kitesurf. Kitesurf doesn't currently support Cookies, but it might in the future.
  • Improved Norwegian Nynorsk localization.
  • Stop clearing the authorization cookie when a challenged request did not send one. A subresource request that starts before the challenge is passed but finishes after it no longer deletes the cookie that pass-challenge just issued (#1314).
  • Fix proof of work worker spawning fallback logic to properly detect Content-Security-Policy failures and fall back to the older logic that fans out to one request per hardware core (#1864).
  • Content-Security-Policy advice has been added to the documentation.
  • Passthru Host header as X-Forwarded-Host in Open Graph requests so backends can dispatch based on host.
  • Detect and block trivial attempts at domain fronting as bots have been starting to use that to try and turn web applications or HTTP servers into open proxies.
  • Add HyperAgent to the headless browsers block rule.
  • Fix Dutch localization typo in the "go home" link.
  • WASM challenges now share watchdog, CSP, and defer fixes from v1.27.0.
  • WASM challenge workers now report errors to the challenge page instead of failing silently.
  • Add prebuilt binaries for NetBSD on amd64 and arm64.
  • Fix WASM build scripts on macOS, which has ancient coreutils that lacks features present on GNU+Linux.
  • Consolidate the purejs and webcrypto workers into one sha256 browser worker to avoid code duplication.
  • Rename X-Real-Ip to X-Real-IP in challenge metadata.
  • Fix client-supplied X-Anubis-* header spoofing
  • Log "challenge accepted" at INFO level when challenge is accepted, providing challenge lifecycle observability at quieter log levels than DEBUG.
  • Fix npm run test:integration and the Playwright CI step, which were pinned to playwright@1.61.1 while go.mod's mxschmitt/playwright-go had already been bumped to a client expecting protocol 1.62.x, causing every Playwright-driven test to fail with a version mismatch.
  • Restore the original Referer header on the request forwarded to the target after a challenge is passed, so server-side logs and analytics no longer see the internal challenge page as the referrer. Add an opt-in --preserve-referer-query-param flag that also appends utm_source/utm_medium query parameters to the post-challenge redirect for client-side analytics tools (e.g. Plausible) that read query parameters instead of document.referrer, which cannot be corrected from the server side (#1596).
  • Clarify getChallenge failure response and cite related log entry.
  • Share redirect validation between challenge completion and subrequest authentication. Reject ambiguous URL forms before checking allowed domains.
  • Respond with the configured DENY status code instead of HTTP 500 when a challenged client is rejected for not advertising gzip support, and log that rejection at INFO instead of ERROR. The rejection is deliberate, so it no longer shows up in 5xx rates or as a server fault. The log message text is unchanged, but fail2ban filters that match on the ERROR level need updating (#1009).
  • Anubis now can use dynamic IP lists from providers like OpenAI or Google instead of static IP lists.

What's Changed

  • test(internal/test): make the Playwright suite runnable again by @CybotTM in #1830
  • feat(blog): add v1.27.0 release blogpost by @Xe in #1841
  • docs(admin/faq): fix typo by @Xe in #1843
  • docs(test): document tales of woe by @Xe in #1837
  • feat(data/bots): block Cloudflare Kitesurf by @Xe in #1839
  • fix(lib): don't clear the auth cookie when the request sent none by @CybotTM in #1828
  • docs: rework developer docs tree by @Xe in #1848
  • chore: npm run format by @Xe in #1849
  • chore(localization): improved norwegian nynorsk localization by @ysccsyysc in #1844
  • chore: bump grpc to v1.82.1 by @Xe in #1862
  • fix(web/js): properly spawn workers with strict CSP settings by @Xe in #1874
  • feat(default-config): add default rule for domain fronting by @Xe in #1882
  • fix(localization): correct Dutch go_home typo by @fogrye in #1892
  • feat(data/bots): add HyperAgent by @Xe in #1894
  • feat(lib): add AI agent honeypot method by @Xe in #1895
  • feat(utils): add wazero-exec fallback for running wasi modules on any host by @Xe in #1898
  • ci: add rust to all steps of the CI flow by @Xe in #1899
  • feat(utils): add precompiled wasm-opt and wasm2js binaries by @Xe in #1900
  • feat: add rust code for proof of work by @Xe in #1901
  • feat: add wasm go package by @Xe in #1902
  • feat(web): add frontend JS for WASM patch by @Xe in #1903
  • feat(lib/challenge)!: add wasm challenge by @Xe in #1904
  • docs: update for wasm changes by @Xe in #1905
  • feat(yeetfile): build binaries for netbsd by @Xe in #1911
  • feat(anubis): log when challenges are accepted by @poespas in #1879
  • fix(wasm): adopt resilience fixes, actually test this with CI by @Xe in #1912
  • build(deps): bump the npm group across 1 directory with 10 updates by @dependabot[bot] in #1909
  • build(deps): bump the github-actions group across 1 directory with 14 updates by @dependabot[bot] in #1910
  • revert(lib): remove AI agent honeypot method by @Xe in #1913
  • chore(js/worker): consolidate webcrypto and purejs workers by @Xe in #1767
  • chore: fix wasm build scripts on macOS by @Xe in #1914
  • build(deps): bump the gomod group across 1 directory with 16 updates by @dependabot[bot] in #1907
  • fix(ci): update golangci lint to not die by @JasonLovesDoggo in #1915
  • fix(ogtags): OG fetch omits X-Forwarded-Host, silently breaking passthrough when --target-host is set by @tsueri in #1881
  • fix: remove client-supplied X-Anubis-* header spoofing vector by @JasonLovesDoggo in #1919
  • fix(lib): Don't log challenge twice by @dwhitemv25 in #1922
  • fix(ssh-ci): minor fixes to appease CI by @Xe in #1923
  • docs(blog): add post about WASM in Anubis by @Xe in #1941
  • fix(lib): Clarify getChallenge error log by @dwhitemv25 in #1942
  • build(deps): bump github/codeql-action/upload-sarif from 4.37.8 to 4.37.9 in the github-actions group by @dependabot[bot] in #1945
  • build(deps-dev): bump the npm group with 4 updates by @dependabot[bot] in #1943
  • docs: fix typo by @Coobyk in #1928
  • fix: restore original Referer across the challenge redirect by @fabianpeusser in #1926
  • fix(lib): validate subrequest authentication redirects consistently by @nijel in #1947
  • build(deps): bump the gomod group across 1 directory with 8 updates by @dependabot[bot] in #1944
  • feat: challenge extensions by @Xe in #1962
  • fix(lib): use the configured deny status for non-gzip challenge clients by @bhcopeland in #1958
  • docs: add 's3api' to list of storage backends by @bibliotechy in #1975
  • fix(lib): original-referer cookie not being affected by cookie-prefix setting by @Earl0fPudding in #1977
  • chore: use the simplified explanation by default by @Xe in #1983
  • feat(geoip)!: rip out Thoth in favour of local Maxmind GeoIP databases by @Xe in #1986
  • docs: update enterprise features by @Xe in #1987
  • build(deps-dev): bump the npm group across 1 directory with 8 updates by @dependabot[bot] in #1978
  • build(deps): bump the github-actions group across 1 directory with 7 updates by @dependabot[bot] in #1979
  • docs: replace removed DIFFICULTY env with bot policy mount in Kubernetes guide by @MeGaurav4 in #1984
  • feat: Support for dynamic ip lists by @SniperCZE in #1965
  • fix: integrate security findings and hardening by @Xe in #1989
  • docs: add synthient diamond sponsor banner by @Xe in #1990
  • build(deps): bump github/codeql-action/upload-sarif from 4.38.1 to 4.38.2 in the github-actions group by @dependabot[bot] in #2001
  • build(deps-dev): bump the npm group with 5 updates by @dependabot[bot] in #1999
  • chore: Add Valve's SteamOS gitlab to known-instances.md by @lbellomo in #1964
  • build(deps): bump the gomod group across 1 directory with 14 updates by @dependabot[bot] in #2000
  • fix(lib): retry missing verification cookie once by @GustavFredrikson in #2004
  • feat(localization): add Portuguese (Portugal) locale by @Blackspirits in #1993
  • feat(lib): add DYNAMIC_COOKIE_SUFFIX setting for toggling the dynamically generated cookie suffix by @Earl0fPudding in #1992
  • fix(wasm): run wasm-opt / wasm2js via nodejs in CI by @Xe in #2009
  • fix(lib/store/bbolt): stop syncing the freelist on every commit by @dominik-matic in #2013

New Contributors

Full Changelog: v1.27.0...v1.28.0

Check out latest releases or
releases around TecharoHQ/anubis v1.28.0

Don't miss a new anubis release

NewReleases is sending notifications on new releases.

Get notifications