Breaking changes
- Set of environment variables for OP-stack is changed. The list has been reduced for simplicity of configuration. The new set is updated in the documentation. Deprecated
INDEXER_OPTIMISM_...variables have been added to this table. - Docker image, starting from 6.8.0, is rootless. Please make sure to update security context in your setup.
Experimental feature highlight
The release contains optional experimental feature, which allows reducing the size of internal_transactions table by removing of output field data and reducing input field size to 4 bytes signature. Our preliminary results from staging environment give estimation of up to 50% reducing of table data + indexes size for mentioned table. In order to enable this feature you should set SHRINK_INTERNAL_TRANSACTIONS_ENABLED=true and use 6.8.0-shrink-internal-txs Docker image tag for all-in-one setup and 6.8.0-shrink-internal-txs-api and 6.8.0-shrink-internal-txs-indexer for splitted setup.
New environment variables
💡List of new environment variables
| Variable | Description | Parameters |
|---|---|---|
ETHEREUM_JSONRPC_FALLBACK_WS_URL
| The fallback WebSockets RPC endpoint used to subscribe to the newHeads subscription alerting the indexer to fetch new blocks. Implemented in #10407.
| Version: v6.8.0+ |
ETHEREUM_JSONRPC_WS_RETRY_INTERVAL
| The interval between retries of connecting to WebSocket RPC endpoint after the previous attempt is failed. Implemented in #10407. | Version: v6.8.0+ |
DATABASE_EVENT_URL
| Variable to define the Postgres Database endpoint that will be used by event listener process. Applicable for separate indexer and API setup. More info in related PR. Implemented in #10164. | Version: v6.8.0+ |
PUBLIC_METRICS_ENABLED
| Variable to enable running queries at /public-metrics endpoint. Implemented in #10469. | Version: v6.8.0+ |
PUBLIC_METRICS_UPDATE_PERIOD_HOURS
| Public metrics update period in hours at /public-metrics endpoint. Implemented in #10469. | Version: v6.8.0+ |
SHRINK_INTERNAL_TRANSACTIONS_ENABLED
| Variable to enable internal transactions shrinking logic. Implemented in #10567. | Version: v6.8.0+ |
SHRINK_INTERNAL_TRANSACTIONS_BATCH_SIZE
| Batch size of the shrink internal transactions migration. Implemented in #10567. | Version: v6.8.0+ |
SHRINK_INTERNAL_TRANSACTIONS_CONCURRENCY
| Concurrency of the shrink internal transactions migration. Implemented in #10567. | Version: v6.8.0+ |
IPFS_PUBLIC_GATEWAY_URL
| IPFS public gateway url which is used by frontend to display IPFS resources such as token instance image. | Version: v6.8.0+ |
INDEXER_TOKEN_INSTANCE_RETRY_MAX_REFETCH_INTERVAL
| Maximum interval between attempts to fetch token instance metadata. Time format. Implemented in #10027. | Version: v6.8.0+ |
INDEXER_TOKEN_INSTANCE_RETRY_EXPONENTIAL_TIMEOUT_BASE
| Base to calculate exponential timeout. Implemented in #10027. | Version: v6.8.0+ |
INDEXER_TOKEN_INSTANCE_RETRY_EXPONENTIAL_TIMEOUT_COEFF
| Coefficient to calculate exponential timeout. Implemented in #10027. | Version: v6.8.0+ |
MISSING_BALANCE_OF_TOKENS_WINDOW_SIZE
| Minimal blocks count until the next token balance request will be executed for tokens that doesn't implement balanceOf function. Implemented in #10142
| Version: v6.8.0+ |
ETHEREUM_JSONRPC_GETH_ALLOW_EMPTY_TRACES
| Allow transactions to not have internal transactions. Implemented in #10200 | Version: v6.8.0+ |
INDEXER_DISABLE_REPLACED_TRANSACTION_FETCHER
| If true, Indexer.Fetcher.ReplacedTransaction fetcher doesn't run
| Version: v6.8.0+ |
SANITIZE_INCORRECT_WETH_BATCH_SIZE
| Number of token transfers to sanitize in the batch. Implemented in #10134 | Version: v6.8.0+ |
SANITIZE_INCORRECT_WETH_CONCURRENCY
| Number of parallel sanitizing token transfer batches processing. Implemented in #10134 | Version: v6.8.0+ |
MIGRATION_RESTORE_OMITTED_WETH_TOKEN_TRANSFERS_BATCH_SIZE
| Number of logs to process in the batch. Implemented in #10466 | Version: v6.8.0+ |
MIGRATION_RESTORE_OMITTED_WETH_TOKEN_TRANSFERS_CONCURRENCY
| Number of parallel logs batches processing. Implemented in #10466 | Version: v6.8.0+ |
MIGRATION_RESTORE_OMITTED_WETH_TOKEN_TRANSFERS_TIMEOUT
| Time interval between checks if queue are not empty. The same timeout multiplied by 2 used between checks if qeueue are not full. Implemented in #10466 | Version: v6.8.0+ |
EXCHANGE_RATES_SOURCE
| Source for native coin and tokens price fetching. Possible values are: coin_gecko, coin_market_cap or mobula.
| Version: v6.8.0+ |
EXCHANGE_RATES_SECONDARY_COIN_SOURCE
| Source for secondary coin fetching. Possible values are: coin_gecko, coin_market_cap or mobula.
| Version: v6.8.0+ |
TOKEN_EXCHANGE_RATES_SOURCE
| Sets the source for tokens price fetching. Available values are coin_gecko, cryptorank. Implemented in #10550.
| Version: v6.8.0+ |
EXCHANGE_RATES_CRYPTORANK_SECONDARY_COIN_ID
| Sets Cryptorank coin ID for secondary coin market chart. Implemented in #10550. | Version: v6.8.0+ |
EXCHANGE_RATES_CRYPTORANK_PLATFORM_ID
| Sets Cryptorank platform ID. Implemented in #10550. | Version: v6.8.0+ |
EXCHANGE_RATES_CRYPTORANK_BASE_URL
| If set, overrides the Cryptorank API url. Implemented in #10550. | Version: v6.8.0+ |
EXCHANGE_RATES_CRYPTORANK_API_KEY
| Cryptorank API key. Current implementation uses dedicated beta Cryptorank API endpoint. If you want to integrate Cryptorank price fetching you should contact Cryptorank to receive an API key. Implemented in #10550. | Version: v6.8.0+ |
EXCHANGE_RATES_CRYPTORANK_COIN_ID
| Sets Cryptorank coin ID. Implemented in #10550. | Version: v6.8.0+ |
EXCHANGE_RATES_CRYPTORANK_LIMIT
| Sets the maximum number of token prices returned in a single request. Implemented in #10550. | Version: v6.8.0+ |
WHITELISTED_WETH_CONTRACTS
| Comma-separated list of smart-contract address hashes of WETH-like tokens which deposit and withdrawal events you'd like to index. Implemented in #10134 | Version: v6.8.0+ |
API_NO_RATE_LIMIT_API_KEY
| API key with no rate limit. Implemented in #10515 | Version: v6.8.0+ |
🚀 Features
- Detect Diamond proxy pattern on unverified proxy smart-contract (#10665)
- Support smart-contract verification in zkSync (#10500)
- Add icon for secondary coin (#10241)
- Integrate Cryptorank API (#10550)
- Enhance /api/v2/smart-contracts/:hash API endpoint (#10558)
- Add method name to transactions CSV export (#10579)
- Add /api/v2/proxy/metadata/addresses endpoint (#10585)
- More descriptive status for Arbitrum message for the transaction view (#10593)
- Add internal_transactions to Tx interpreter request (#10347)
- Add token decimals to token transfers CSV export (#10589)
- Add DELETE /api/v2/import/token-info method (#10580)
- Add block number to token transfer object in API v2 endpoint (#10591)
- L1 tx associated with Arbitrum message in /api/v2/transactions/{txHash} (#10590)
- No rate limit API key (#10515)
- Support for
:celochain type (#10564) - Public IPFS gateway URL (#10511)
- Add CSV_EXPORT_LIMIT env (#10497)
- Backfiller for omitted WETH transfers (#10466)
- Add INDEXER_DISABLE_REPLACED_TRANSACTION_FETCHER env (#10485)
- Revisited approach to catchup missed Arbitrum messages (#10374)
- Missing Arbitrum batches re-discovery (#10446)
- Add memory metrics for OnDemand fetchers (#10425)
- Add Celestia blobs support to Optimism batches fetcher (#10199)
- AnyTrust and Celestia support as DA for Arbitrum batches (#10144)
- Broadcast updates about new Arbitrum batches and L1-L2 messages through WebSocket (#10272)
🐛 Bug Fixes
- Logs list serialization (#10565)
- nil in OrderedCache (#10647)
- Fix for metadata detection at ipfs protocol(#10646)
- Change default shrink internal_transactions table migration params (#10644)
- Fix bug in update_replaced_transactions query (#10634)
- Fix mode dependent processes starting (#10641)
- Better detection IPFS links in NFT metadata fetcher (#10638)
- Change mode env name (#10636)
- Proper default value of gas used for dropped Arbitrum transactions (#10619)
- Fix fetch_first_trace tests (#10618)
- Add SHRINK_INTERNAL_TRANSACTIONS_ENABLED arg to Dockerfile (#10616)
- Fix raw-trace test (#10606)
- Fix internal transaction validation (#10443)
- Fix internal transactions runner test for zetachain (#10576)
- Filter out incorrect L1-to-L2 Arbitrum messages (#10570)
- Fetch contract methods decoding candidates sorted by inserted_at (#10529)
- Sanitize topic value before making db query (#10481)
- Fix :checkout_timeout error on NFT fetching (#10429)
- Proper handling confirmations for Arbitrum rollup block in the middle of a batch (#10482)
- Sanitize contractURI response (#10479)
- Use token_type from tt instead of token (#10555)
- Non-consensus logs in JSON RPC and ETH RPC APIs (#10545)
- Fix address_to_logs consensus filtering (#10528)
- Error on internal transactions CSV export (#10495)
- Extend block search range for
getblocknobytimemethod (#10475) - Move recon dep to explorer mix.exs (#10487)
- Add missing condition to fetch_min_missing_block_cache (#10478)
- Mud api format fixes (#10362)
- Add no overlapping constraint to missing_block_ranges (#10449)
- Avoid infinite loop during batch block range binary search (#10436)
- Fix "key :bytes not found in: nil" issue (#10435)
- Missing clauses in MetadataPreloader functions (#10439)
- Code compiler test (#10454)
- Include internal transactions in state change (#10210)
- Race condition in cache tests (#10441)
- Fix on-demand fetchers metrics (#10431)
- Transactions and token transfers block_consensus (#10285)
- Allow fetching image from properties -> image prop in token instance metadata (#10380)
- Filter out internal transactions belonging to reorg (#10330)
- Fix logs sorting in API v1 (#10405)
- Fix flickering transaction_estimated_count/1 test (#10403)
- Fix flickering "updates cache if initial value is zero" tests (#10402)
- /addresses empty list flickering test fix (#10400)
- Fix missing expectation in mock_beacon_storage_pointer_request (#10399)
- Fix /stats/charts/market test (#10392)
- Alternative way to detect blocks range for ArbitrumOne batches (#10295)
- Fix exchange rate flickering test (#10383)
- Fix gas price oracle flickering test (#10381)
- Fix address controller flickering test (#10382)
- Empty revert reasons in geth variant (#10243)
- Proper handling for re-discovered Arbitrum batches (#10326)
- Proper lookup of confirmed Arbitrum cross-chain messages (#10322)
- Indexer first block usage to halt Arbitrum missed messages discovery (#10280)
📚 Documentation
- Refine PR template
- Move note in README.md higher for visibility (#10450)
⚡ Performance
- Speed up worlds list query (#10556)
- Reduce LookUpSmartContractSourcesOnDemand fetcher footprint (#10457)
⚙️ Miscellaneous Tasks
- Make Dockerfile use specified user with uid/gid (#10070)
- Run shrink internal transactions migration for indexer instance only (#10631)
- Shrink internal transactions (#10567)
- Upgrade WS client (#10407)
- Add API endpoint for OP batch blocks (#10566)
- Public metrics config API endpoint (#10568)
- Add workflow to generate separate pre-release indexer/API images for Arbitrum
- Fix some comments (#10519)
- Set Geth as default JSON RPC Variant (#10509)
- Return ex_abi core lib dependency (#10470)
- Add recon dependency (#10486)
- Manage Solidityscan platform id via runtime variable (#10473)
- Add test for broadcasting fetched_bytecode message (#10244)
- Disable public metrics by default, set 1 day as default period of update (#10469)
- Move eth_bytecode_db_lookup_started event to smart contract related event handler (#10462)
- Token transfers broadcast optimization (#10465)
- Remove catchup sequence logic (#10415)
- Remove single implementation name, address from API v2 response (#10390)
- Refactor init functions to use continue if needed (#10300)
- Update buildkit builders (#10377)
What's Changed
- fix: indexer first block usage to halt Arbitrum missed messages discovery by @akolotov in #10280
- fix: proper lookup of confirmed Arbitrum cross-chain messages by @akolotov in #10322
- Update buildkit builders by @alik-agaev in #10377
- fix: proper handling for re-discovered Arbitrum batches by @akolotov in #10326
- feat: broadcast updates about new Arbitrum batches and L1-L2 messages through WebSocket by @akolotov in #10272
- fix: empty revert reasons in geth variant by @k1rill-fedoseev in #10243
- chore: Refactor init functions to use continue if needed by @Qwerty5Uiop in #10300
- fix: Fix address controller flickering test by @vbaranov in #10382
- fix: Fix gas price oracle flickering test by @vbaranov in #10381
- fix: Fix exchange rate flickering test by @vbaranov in #10383
- feat: AnyTrust and Celestia support as DA for Arbitrum batches by @akolotov in #10144
- chore: Remove single implementation name, address from API v2 response by @vbaranov in #10390
- chore(deps-dev): bump ex_doc from 0.34.1 to 0.34.2 by @dependabot in #10394
- fix: alternative way to detect blocks range for ArbitrumOne batches by @akolotov in #10295
- fix: Fix flickering /stats/charts/market test by @vbaranov in #10392
- Fix missing expectation in mock_beacon_storage_pointer_request by @vbaranov in #10399
- fix: /addresses empty list flickering test fix by @vbaranov in #10400
- fix: Fix flickering "updates cache if initial value is zero" test by @vbaranov in #10402
- fix: Fix flickering transaction_estimated_count/1 test by @vbaranov in #10403
- fix: Fix logs sorting in API v1 by @nikitosing in #10405
- chore(deps): bump mime from 2.0.5 to 2.0.6 by @dependabot in #10395
- chore(deps): bump phoenix_ecto from 4.6.1 to 4.6.2 by @dependabot in #10396
- feat: Add Celestia blobs support to Optimism batches fetcher by @varasev in #10199
- Add Brotli decompression to Indexer.Fetcher.Optimism.TxnBatch by @varasev in #10410
- chore: Remove catchup sequence logic by @Qwerty5Uiop in #10415
- chore(deps): bump cldr_utils from 2.27.0 to 2.28.0 by @dependabot in #10421
- chore(deps): bump tesla from 1.11.1 to 1.11.2 by @dependabot in #10419
- fix: Filter out internal transactions belonging to reorg by @Qwerty5Uiop in #10330
- fix: allow fetching image from "properties" -> "image" prop in token instance metadata by @vbaranov in #10380
- fix: Transactions and token transfers block_consensus by @Qwerty5Uiop in #10285
- chore(deps): bump absinthe from 1.7.6 to 1.7.7 by @dependabot in #10420
- feat: Add memory metrics for OnDemand fetchers by @Qwerty5Uiop in #10425
- fix: Fix on-demand fetchers metrics by @Qwerty5Uiop in #10431
- fix: race condition in cache tests by @k1rill-fedoseev in #10441
- doc: Move note in README.md higher for visibility by @robriks in #10450
- fix: include internal transactions in state change by @sl1depengwyn in #10210
- fix: code compiler test by @k1rill-fedoseev in #10454
- fix: Missing clauses in MetadataPreloader functions by @vbaranov in #10439
- Fix "key :bytes not found in: nil" issue by @vbaranov in #10435
- fix: avoid infinite loop during batch block range binary search by @akolotov in #10436
- chore: Token transfers broadcast optimization by @Qwerty5Uiop in #10465
- perf: Reduce LookUpSmartContractSourcesOnDemand fetcher footprint by @vbaranov in #10457
- chore: Move eth_bytecode_db_lookup_started event to smart contract related event handler by @Qwerty5Uiop in #10462
- fix: Add no overlapping constraint to missing_block_ranges by @Qwerty5Uiop in #10449
- feat: missing Arbitrum batches re-discovery by @akolotov in #10446
- feat: revisited approach to catchup missed Arbitrum messages by @akolotov in #10374
- chore: disable public metrics by default, set 1 day as default update period by @vbaranov in #10469
- chore: Add test for broadcasting fetched_bytecode message by @vbaranov in #10244
- fix: mud api format fixes by @k1rill-fedoseev in #10362
- fix: Add missing condition to fetch_min_missing_block_cache by @Qwerty5Uiop in #10478
- chore: Manage Solidityscan platform id via runtime variable by @vbaranov in #10473
- chore: Add recon dependency by @Qwerty5Uiop in #10486
- feat: Add INDEXER_DISABLE_REPLACED_TRANSACTION_FETCHER env by @nikitosing in #10485
- Move recon to explorer by @Qwerty5Uiop in #10487
- chore: Return ex_abi core lib dependency by @vbaranov in #10470
- fix: extend block search range for
getblocknobytimemethod by @fedor-ivn in #10475 - feat: Backfiller for omitted WETH transfers by @nikitosing in #10466
- fix: Error on internal transactions CSV export by @nikitosing in #10495
- chore(deps): bump jason from 1.4.3 to 1.4.4 by @dependabot in #10504
- chore(deps): bump ex_cldr from 2.39.2 to 2.40.0 by @dependabot in #10503
- chore: Set Geth as default JSON RPC Variant by @vbaranov in #10509
- fix: Fix address_to_logs consensus filtering by @Qwerty5Uiop in #10528
- chore(deps): bump photoswipe from 5.4.3 to 5.4.4 in /apps/block_scout_web/assets by @dependabot in #10540
- chore(deps): bump mixpanel-browser from 2.53.0 to 2.54.1 in /apps/block_scout_web/assets by @dependabot in #10538
- chore(deps): bump @amplitude/analytics-browser from 2.8.1 to 2.9.3 in /apps/block_scout_web/assets by @dependabot in #10535
- chore(deps): bump highlight.js from 11.9.0 to 11.10.0 in /apps/block_scout_web/assets by @dependabot in #10533
- chore(deps): bump sweetalert2 from 11.12.1 to 11.12.4 in /apps/block_scout_web/assets by @dependabot in #10532
- chore(deps-dev): bump eslint-plugin-promise from 6.4.0 to 6.6.0 in /apps/block_scout_web/assets by @dependabot in #10536
- chore(deps-dev): bump sass from 1.72.0 to 1.77.8 in /apps/block_scout_web/assets by @dependabot in #10539
- chore(deps-dev): bump webpack from 5.92.1 to 5.93.0 in /apps/block_scout_web/assets by @dependabot in #10534
- chore: fix some comments by @jianghuyiyuan in #10519
- fix: non-consensus logs in JSON RPC and ETH RPC APIs by @k1rill-fedoseev in #10545
- chore(deps): bump absinthe from 1.7.7 to 1.7.8 by @dependabot in #10502
- fix: skip the batch 0 on Arbitrum chains by @akolotov in #10553
- fix: Use token_type from tt instead of token by @Qwerty5Uiop in #10555
- feat: Add CSV_EXPORT_LIMIT env by @nikitosing in #10497
- fix: Sanitize contractURI response by @vbaranov in #10479
- fix: limited usage of infinite timeout in Arbitrum related DB queries by @akolotov in #10551
- feat: Public IPFS gateway URL by @vbaranov in #10511
- chore(deps): bump ex_cldr_units from 3.17.0 to 3.17.1 by @dependabot in #10561
- chore(deps): bump postgrex from 0.18.0 to 0.19.0 by @dependabot in #10562
- chore(deps): bump gettext from 0.24.0 to 0.25.0 by @dependabot in #10560
- chore(deps): bump tesla from 1.11.2 to 1.12.1 by @dependabot in #10559
- fix: proper handling confirmations for Arbitrum rollup block in the middle of a batch by @akolotov in #10482
- feat: support for
:celochain type by @fedor-ivn in #10564 - fix: Fix :checkout_timeout error on NFT fetching by @nikitosing in #10429
- fix: Sanitize topic value before making db query by @vbaranov in #10481
- fix: Fetch contract methods decoding candidates sorted by inserted_at by @vbaranov in #10529
- feat: No rate limit API key by @vbaranov in #10515
- chore: Public metrics config API endpoint by @vbaranov in #10568
- perf: speed up worlds list query by @k1rill-fedoseev in #10556
- fix: filter out incorrect L1-to-L2 Arbitrum messages by @akolotov in #10570
- fix: Fix internal transactions runner test for zetachain by @Qwerty5Uiop in #10576
- chore: Add API endpoint for OP batch blocks by @varasev in #10566
- feat: L1 tx associated with Arbitrum message in /api/v2/transactions/{txHash} by @akolotov in #10590
- feat: Add block number to token transfer object in API v2 endpoint by @vbaranov in #10591
- feat: Add DELETE /api/v2/import/token-info method by @nikitosing in #10580
- feat: Add token decimals to token transfers CSV export by @vbaranov in #10589
- feat: Add internal_transactions to Tx interpreter request by @nikitosing in #10347
- fix: Fix internal transaction validation by @vbaranov in #10443
- feat: more descriptive status for Arbitrum message for the transaction view by @akolotov in #10593
- feat: Add /api/v2/proxy/metadata/addresses endpoint by @nikitosing in #10585
- feat: Add method name to transactions CSV export by @nikitosing in #10579
- feat: enhance /api/v2/smart-contracts/:hash API endpoint by @vbaranov in #10558
- feat: Integrate Cryptorank API by @nikitosing in #10550
- chore: Upgrade WS client by @Qwerty5Uiop in #10407
- feat: Add icon for secondary coin by @sl1depengwyn in #10241
- chore: Shrink internal transactions by @Qwerty5Uiop in #10567
- feat: Support smart-contract verification in zkSync by @vbaranov in #10500
- fix: Fix raw-trace test by @Qwerty5Uiop in #10606
- fix: Add SHRINK_INTERNAL_TRANSACTIONS_ENABLED arg to Dockerfile by @Qwerty5Uiop in #10616
- fix: Fix fetch_first_trace tests by @Qwerty5Uiop in #10618
- fix: proper default value of gas used for dropped Arbitrum transactions by @akolotov in #10619
- chore: Run shrink internal transactions migration for indexer instance only by @Qwerty5Uiop in #10631
- fix: Change mode env name by @Qwerty5Uiop in #10636
- fix: Better detection IPFS links in NFT metadata fetcher by @vbaranov in #10638
- fix: Fix mode dependent processes starting by @Qwerty5Uiop in #10641
- fix: Fix bug in update_replaced_transactions query by @vbaranov in #10634
- Make Dockerfile use specified user with uid/gid by @chainchad in #10070
- fix: Change default shrink internal_transactions table migration params by @vbaranov in #10644
- fix: Fix for metadata detection at ipfs protocol by @vbaranov in #10646
- fix:
nilinOrderedCacheby @sl1depengwyn in #10647 - fix: logs list serialization by @k1rill-fedoseev in #10565
- feature: Detect Diamond proxy pattern on unverified proxy smart-contract by @vbaranov in #10665
New Contributors
- @robriks made their first contribution in #10450
- @jianghuyiyuan made their first contribution in #10519
- @chainchad made their first contribution in #10070
Full Changelog: v6.7.2-beta...v6.8.0-beta