Bitcoin Stamps Indexer v1.9.1

dev → main · Requires Counterparty v11.0.1+ (CP V2 API endpoint fix)

v1.9.1 is the 1.9 feature release — a consensus-validated feature + hardening line: SRC-101 P2WSH/OLGA activation, a new market-data and webhook layer, a Rust-accelerated parse path, and a full genesis→tip consensus re-validation. All consensus-affecting changes are activation-gated or flag-gated and were proven hash-identical to the production ledger (txlist + ledger) via a from-genesis reparse validated against the prod-derived reference (on Python 3.12).

This release supersedes an internal 1.9.0 tag that was withdrawn. 1.9.1 is functionally identical to that build plus repository hygiene (compose consolidation, removal of an unused private-tools submodule and dead scripts) — no functional or consensus change between them.

Highlights

• SRC-101 P2WSH / OLGA encoding activates at block 940,000 — SRC-101 supported via both MULTISIG and P2WSH after activation.
• CP-skip on empty blocks — skips the Counterparty API fetch on blocks with no CP data, cutting round-trips across pre-CP history (validated hash-identical).
• Supported interpreters: Python 3.10–3.12 — consensus proven byte-identical across 3.10/3.11/3.12. Do not run on Python 3.13: it tightened base64/binascii decoding and diverges from consensus at block 783,775 (stamp mis-classification). 3.13 remains in the unit CI matrix only; full 3.13 support is tracked in #871.
• Market-data & sales-history infrastructure — multi-source cache, holder caches, and a sales-history pipeline.
• Real-time webhook notifications for new blocks and reorgs, with SSRF protection.
• RPC resilience — connect/read timeouts, session reset + pool rebuild, retry on transient network errors.
• Fresh bootstrap snapshot to block 956,000 published to S3.

Consensus & Protocol

Activation heights

SRC-101 P2WSH/OLGA

• Enables destination_nvalue extraction from vout[0] for P2WSH SRC-101.
• Before activation: SRC-101 via MULTISIG only; after: MULTISIG and P2WSH.
• Shared stamp: prefix pipeline — protocol differentiation at the JSON level ("p":"src-101").

SRC protocol improvements

• SRC-721R (v:r0) deploy detection for P2WSH transactions.
• Recursive SRC-721 description-field parsing; SRC-721 ident flag for OLGA mints.
• Safe dual processing for cursed stamps.
• decode_checkmultisig fix — length-prefix-bounded extraction prevents data corruption.

Consensus validation

• All consensus-affecting changes are activation-gated or flag-gated (default-off) and proven output-neutral by the Tier-3 Reparse Consensus Validation CI gate (stratified blocks, Python 3.10–3.12).
• Final certification: a from-genesis → tip reparse on Python 3.12 with CP_SKIP=true, validated hash-identical to the production ledger — txlist + ledger hashes matched across all 176,328 blocks (compare_tables + per-block reference check). messages_hash and STAMP→SRC-721 IDENT refinements are non-consensus-critical and expected.

Performance

• CP-skip (CP_SKIP_NO_COUNTERPARTY_BLOCKS) avoids Counterparty API calls on blocks with no CP data.
• Rust-accelerated parse path wired into the main block loop (batch_parse_transactions).
• Hash-computation levers and validation-mode controls (VALIDATION_MODE file-based reference validation).
• Batch-optimized input fetching for fee-rate calculation.

New capabilities

• Fee rate: fee_rate_sat_vb (sats/vByte) on the transactions table with proper SegWit virtual-size calculation.
• Market data: multi-source cache (KuCoin, StampScan) with confidence-weighted averaging; activity-tiered updates (hot/warm/cool/cold/dormant); circuit-breaker coordinator.
• Sales history: dispensers, atomic swaps, OTC, and auctions tracked in stamp_sales_history.
• Async ops: background SRC-20 holder-count updates; SQLite-based validation/reprocessing queues with exponential backoff.
• Webhooks: configurable endpoints, SSRF protection, timeouts + retries; non-blocking (never impacts indexing).

Stability & reliability

• Blocks no longer permanently stuck when the CP API fails (fallback mode near tip only).
• Sales-history processor race condition eliminated; worker-thread self-join prevention during fallback rollback.
• Critical-failure handler for robust shutdown; async holder-updater timeout raised (2 → 5 min).

Security & dependencies

• cryptography → 48.0.1 (GHSA-537c-gmf6-5ccf, OpenSSL-in-wheels) — reparse-gated.
• aiohttp 3.14.1, requests 2.34.2, ecdsa 0.19.2, urllib3 2.7.0, idna 3.18, msgpack 1.2.1, pyasn1 0.6.3 (python-jose 3.5.0).
• Rust parser: rand → 0.8.6.
• Tooling: mypy → 2.1.0; maturin → 1.11.5. All consensus-adjacent bumps passed the Tier-3 reparse gate.

Database schema

• New tables: stamp_market_data, stamp_holder_cache, market_data_sources, src20_market_data, collection_market_data, stamp_sales_history, src20_token_stats.
• New views: v_stamp_market_overview, v_trending_stamps.
• New column: fee_rate_sat_vb on transactions; new performance indexes.

CI/CD

• Python 3.13 added to the unit CI matrix (not consensus — see the 3.13 caveat above); Integration Tests in the pipeline; Dockerfile runtime stage parameterized by PYTHON_VERSION.
• Reparse Consensus Validation + Freeze Drift Check gates.
• docker-auto-publish ships btcstamps/indexer:1.9.0 + :latest on release.

Upgrade notes

• Requires Counterparty v11.0.1+.
• Apply the schema changes above (new tables/views/column/indexes).
• A fresh bootstrap snapshot to block 956,000 is available on S3 for fast sync.
• The published Docker image (btcstamps/indexer:1.9.1) is built on Python 3.12 — the certified consensus runtime. If you build your own image, use Python 3.10–3.12; not 3.13 (see the consensus caveat above).

Repository hygiene (1.9.1)

• Consolidated Docker Compose to a canonical root base + override set; removed the superseded per-indexer compose files and 2.7yr-old orphaned env samples.
• Removed an unused private-tools git submodule and a dead run helper script.
• No functional or consensus change — the running indexer is byte-for-byte the validated 1.9 build.

Versioning

Cut via bump-version.yml: strip canary (1.9.0+canary.N → 1.9.0) → apply [patch] → 1.9.1, tag 1.9.1 (no v prefix), publish the GitHub Release + Docker :1.9.1/:latest, and auto-open the version-only sync-back PR that restores dev to 1.9.1+canary.1.