mongodb/kingfisher v2.14.0

Kingfisher v2.14.0

on GitHub

[v2.14.0]

• Cache GitHub App installation tokens across workers, coordinate refresh before expiry, and retry rate-limited exchanges with shared failure cooldowns. #556

• Improved rules check to verify negative_examples and apply scan-time capture selection, entropy, safe-list, pattern requirements, checksums, and rule filters to both positive and negative examples. Positive examples rejected only by ignore_if_contains remain warnings (errors with --warnings-as-errors) for compatibility with runtime filtering; independent entropy, safe-list, checksum, and rule-filter failures remain errors. #555

• Fixed Rust/Python scanning to enforce YAML pattern requirements for raw and Base64 candidates; added offline luhn_valid and iban_valid Liquid checksum filters. #554

• Aligned SDK rule execution with the CLI: added typed URI syntax checks, compact full-match component context, shared bounded candidate verification and per-batch validation request grouping. Python preserves finding locations and serialization; Rust exposes additive context APIs. No-options context scans isolate raw and decoded-buffer components; explicit detection options retain their selected matching policy.

• Rust source API migration only — existing rule files require no changes: prepared kingfisher-rules 2.0.0 and kingfisher-scanner 2.0.0 for the exhaustive HttpValidation.auth field and public rules types/re-exports. OAuth2 service authentication is opt-in; existing Kingfisher YAML and Betterleaks TOML rules continue to load without an auth block or OAuth2 credentials. Only Rust code constructing HttpValidation struct literals needs auth: None, or Some(Box::new(HttpValidationAuth { oauth2: config })) with OAuth2Config/OAuth2PasswordCredentials; Rust scanner consumers must use rules 2.x types. Core remains 1.0.3, CLI 2.14.0 and Python SDK 1.4.0; there is no Python API major break. See the Rust migration guide.

• Memoize only definitive active/inactive validation results within a batch, keyed by rule and resolved inputs; inconclusive failures and cancellation remain retryable, and shrinking candidate-attempt budgets do not duplicate successful checks.

• Sped up large Git history scans with candidate-first metadata, subtree pruning and exact ancestor summaries, preserving commit/path attribution without persisting scan data between runs.

• Reduced memory and decoding overhead with compact indexes, copy-on-write ancestry bitmaps, per-worker Git caches and SIMD Unicode checks.

• Losslessly packed the tokenizer vocabulary and prioritized scan throughput with release opt-level = 3 (previously "s"); cold-crate size-focused overrides remain.

• Corrected pairing-budget diagnostics when dependency candidate lists are capped.

• Added rsa_sha256 Liquid signing for RS256 JWT validation with PKCS#1/PKCS#8 PEM keys. #552

• Added Azure Blob Storage scanning with SAS, account-key, Entra-token, and anonymous authentication.

• Added shared path/extension selection across files, Git sources, and archive members; S3, GCS, Azure Blob, and Hugging Face storage scans filter before downloading and support resumable listings and incremental caching with complete reports.

• Made storage page checkpoints append-only with end-of-scan compaction; retry changed Azure/GCS objects with current version identities and skip deleted objects with warnings.

• Enforced S3/GCS/Azure Blob download caps while streaming and honored Azure timeout controls; improved per-object failure handling without marking failed pages complete. Isolated cache entries by endpoint/selection policy and removed stale, orphaned and abandoned temporary objects safely.

• Preserved literal ! in source/archive-member paths, rejected empty path prefixes and retained nonempty slash-suffixed storage objects. Skip malformed Hugging Face entries without losing valid files, reject unsafe/repeated pagination links and bound Confluence pagination without visible progress.

• Preserved repository links, commit IDs and file paths with --commit-metadata=false while hiding author, committer and date details.

• Added OAuth2 password-grant authentication for custom HTTP validators, with environment-backed credentials, token reuse/refresh, failure cooldowns, and fixed-origin delivery. HTTPS token/verifier requests retain strict TLS and environment proxy/NO_PROXY handling; explicitly allowed loopback plain-HTTP token/verifier requests connect directly without proxies. Redirects stay disabled for both; other validators' TLS/proxy policies are unchanged.

• Added Jira access checks and project exclusions; hardened Jira/Confluence rate-limit handling, pagination, and API context paths. Expanded target/authentication examples in the README and guides.

• Jira migration: API requests no longer follow redirects, including same-origin redirects. Reverse-proxy installations must pass the final canonical --url, including any context path such as /jira.

• Retry transient GitHub API failures and rate limits with bounded backoff and server-provided cooldowns. #551

• Behavior change: --no-limits lifts content caps and preserves timeouts; add --no-timeouts to disable scan operation deadlines. Bound SMTP cleanup and SQLite lock waits. #524