blacklanternsecurity/bbot v3.1.0

3.1.0

on GitHub

Breaking changes

• Python 3.10 dropped, 3.15 added (#3474). requires-python is now >=3.11,<3.16; CI tests 3.11 through 3.15. 3.10 reaches EOL this month.
• portfilter module removed (#3534). CDN/WAF port filtering now happens in core and is on by default; opt out with --allow-cdn-ports.
• Custom module loading hardened (#3593). config.module_dirs is deprecated and ignored; module_dirs is honored only as a top-level preset key. BBOT asks before loading each custom module folder (-y answers yes, -S no longer implies -y), refuses world-writable folders and files, and refuses custom modules that reuse an existing module name. Preset conditions are read-only, and version / home are validated.
• lightfuzz probes WAF-tagged hosts instead of skipping them (#3336). modules.lightfuzz.avoid_wafs defaults to try_bypasses, which fuzzes a WAF-tagged host only if a nowafpls probe gets past the WAF, and the lightfuzz presets enable the nowafpls module. The top-level avoid_wafs is no longer read. For the 3.0 behavior, set modules.lightfuzz.avoid_wafs: always and pass -em nowafpls.
• dnsdumpster requires an API key (#3458), waf_bypass targets scan targets only by default (#3460), CPE-derived technology names (#3535) and no affiliate tag on in-scope events (#3524) can affect existing presets and output consumers.

Highlights

DNS (#3457)

• massdns removed. DNS brute-forcing now runs through blastdns 2.0, and the massdns dependency is gone entirely.
• Brute-force gets its own retry budget, a short per-query timeout, and persistent sockets (one per resolver rather than one per in-flight query).
• New --custom-nameservers flag (dns.nameservers), replacing the system resolvers for normal resolution. An invalid value aborts the scan during prep instead of producing a "completed" scan with zero results.
• Resolver diagnostics reported when brute-force delivery drops.
• New dns.brute_max_resolvers (default 600, 0 for the whole list) (#3612). Each scan draws that many brute-force resolvers at random, once. Dead ones are dropped at startup, and every brute-force run in the scan shares the rest, so pacing learned in one run carries into the next. Also raises the blastdns floor to 2.0.3, which paces attempts when per-attempt loss shows a shared bottleneck.
• Ordinary DNS concurrency raised and made explicit (#3606): dns.threads 10 to 50, new dns.max_concurrency (256), and the dnsresolve worker count derived from the client's total slots. On a large wildcard subdomain-enum scan this halved wall time compared with 3.0.2, with the same subdomains found. Lower dns.threads for a fragile resolver. The subdomain-enum preset no longer sets dns.threads: 25, so it gets the new default too (#3613).
• Already-resolved events skip ahead in dnsresolve's queue (#3533).
• Fix nested wildcard detection in is_wildcard (#3536).

baddns 2.5.0 (#3457)

• New DELEGATION submodule, enabled by default and in the baddns / baddns-heavy presets. Checks _acme-challenge, _dmarc and 15 common DKIM selectors for dangling delegations, with wildcard and generic-finding filtering to keep volume down.
• NS gained a lame-delegation finding (ceiling raised to HIGH/CONFIRMED).
• baddns's YARA-backed WordMatcher is passed through, matching all signatures in a single pass.
• blastdns capped at <2.1 to match baddns's own ceiling.

New modules

• favicon: technology detection from favicon hashes (#3532).
• js_unpacker: detects and unpacks obfuscated JavaScript (#3335).
• nowafpls: WAF bypass detection via body padding (#3336). lightfuzz, ajaxpro and generic_ssrf pad their payloads on hosts where the bypass is confirmed.
• otilabs: subdomain enumeration (#3496).
• scanmalware: subdomains from ScanMalware's public scan archive (#3528). Keyless and opt-in (-m scanmalware); not in subdomain-enum.

Modules

• telerik overhaul with native RAU and DialogHandler crypto (#3333).
• nuclei: expose newer template-selection options (#3479). Results are parented to the URL, or the host and port, that nuclei actually scanned, so findings on sibling subdomains or on several ports of one host are no longer merged or mislabeled (#3629).
• dnsdumpster converted to the authenticated API (#3458).
• wayback verifies hosts are alive before emitting URLs (#3359).
• waf_bypass only attempts bypasses against scan targets by default (#3460).
• gitdumper: origin check and depth limit on directory listing (#3466).
• crt: request an explicit / path, which crt.sh requires (#3428).
• SQL output modules request sqlalchemy's asyncio extra (#3505).
• lightfuzz: require positive confirmation for code-change findings (#3357), tighten serial error resolution (#3427), fix unbounded queue growth from parameter re-discovery (#3405), count baseline URLs against spider distance (#3523), read the raw Cookie header in the sqli cookie test (#3498), correct the WAF-gating comments and test the GET-side skip (#3608).

excavate

• Long redirection response detection (#3481).
• Relative URLs resolved against <base href> (#3522).
• Hostnames are no longer extracted from partway through a hyphenated label (#3590).

Core

• Overhauled api_request rate limiting, plus config deprecation support via Field(deprecated=True) (#3461). API calls made through api_request now send a BBOT/<version> User-Agent instead of the scan's browser User-Agent.
• CPE-derived technologies are named by product, with the CPE retained (#3535).
• In-scope events are no longer tagged affiliate (#3524).
• FINDING text output shows the host when it isn't otherwise present.
• FINDING has an optional port for findings that aren't on a URL, such as a TLS or TCP service. It's part of the event's identity and shown as host:port in text output, and legba and medusa findings now keep their port (#3629).
• HTTP_RESPONSE events carry url_extension (#3406).
• ClosestHostEvent keeps its inherited port, and the port is restored from JSON (#3462).
• download() respects cache_hrs in its freshness check (#3563).
• Reject symlinks in unarchive output after extraction (#3560).
• Fix hostless sort sentinel type mismatch in Target.add() (#3391).
• Version output includes the git commit hash for git checkouts (#3530).

Performance

• Selectively replace DeepDiff (#3340), then remove the dependency entirely (#3604).
• Faster compute_simhash, with corrected executor placements (#3331).
• response_to_json removed in favor of response_to_event_dict (#3473).

Dependencies

• blastdns 2.0.4 (#3612, #3631), blasthttp 0.10.1, baddns 2.5.0, badsecrets 1.3.0 (#3526), cloudcheck 11.2.0 (#3559), radixtarget 4.3.1 (#3558, #3617), pycryptodome 4.3.0 (#3542, #3597, #3620).
• deepdiff removed, along with cachebox and orderly-set (#3604). ansible-core resolves to 2.22 on Python 3.13+, which is what makes 3.15 installable.
• dnspython removed (#3610). Nothing imported it, and its old pin held every install below 2.9.0, the fix for GHSA-45c3-73f7-pv4m. The baddns modules still install it, uncapped, through baddns.
• pyjwt, lxml and socksio removed, and xmltojson replaced by xmltodict (#3627).
• trufflehog 3.97.0 to 3.99.8 (#3605).
• Security dependency updates (#3557).

Maintenance

• CI: test-matrix status check with every job named (#3356), faster test suite via teardown, polling and dep-install fixes (#3404), benchmark base-branch comparison restored and pytest-benchmark 5.3.1 for pytest 9.2 (#3611), CLA workflow permissions (#3570), version updater workflow name (#3415), pinned runner image (#3456, #3483), weekly grouped dependabot updates with shared auto-merge for minor and patch (#3487).
• Docs: 3.1 breaking changes guide (#3594), docs updated for 3.1 (#3613), AGENTS.md as an index of the org standards (#3486), mkdocstrings 1.0.8 (#3585), module setup troubleshooting notes (#3228), AGENTS.md options section and module examples refreshed (#3420), 3.0.0-3.0.2 release notes (#3408).
• Project metadata lists Black Lantern Security as the author (#3628).
• Merged stable into dev to resolve release conflicts (#3592).
• Dependabot bumps across the uv and github-actions groups, and automated docs regeneration.