Smart Contracts
With Hedera's increased focus on Smart Contracts, we took the time to revamp the mirror node's smart contract support and lay the groundwork for future enhancements. As detailed in the design document, plans include new contract-specific REST APIs and Ethereum-compatible APIs in the future.
To prepare for that, the database schema and importer were updated to normalize and store all contract-related information, fixing long-standing bugs like not storing contract bytecode and child contracts. The contract table was split from the generic entity table and work was started on making all the entity tables maintain a history of all changes. The REST API now supports searching for and retrieving specific contracts. This API will be considered alpha for a few releases as we iterate on the design so it may change. Below is an example of retrieving a contract:
/api/v1/contracts/{id}
{
"admin_key": {
"_type": "ProtobufEncoded",
"key": "7b2233222c2233222c2233227d"
},
"auto_renew_period": 7776000,
"bytecode": "0xc896c66db6d98784cc03807640f3dfd41ac3a48c",
"contract_id": "0.0.10001",
"created_timestamp": "1633466229.96874612",
"deleted": false,
"expiration_timestamp": "1633466229.96874612",
"file_id": "0.0.1000",
"memo": "First contract",
"obtainer_id": "0.0.101",
"proxy_account_id": "0.0.100",
"solidity_address": "0x00000000000000000000000000000000000003E9",
"timestamp": {
"from": "1633466229.96874612",
"to": "1633466568.31556926"
}
}Data Architecture
Over the last few months, work has been underway to analyze possible PostgreSQL replacements as the need for handling an ever-increasing amount of data puts strain on the existing mirror node database. After agreeing upon the acceptance criteria, priority was placed on a PostgreSQL-compatible distributed database that can shard our time-series data across many nodes for scale-out reads and writes. That would ensure the quickest time to market and ease transition for Hedera and others using the open source mirror node software. The four distributed databases we chose for our proof of concept included CitusDB, CockroachDB, TimescaleDB, and YugabyteDB.
After a detailed analysis of each, we chose CitusDB for our next database due to its excellent PostgreSQL compatibility (it's a PostgreSQL extension) and its mature support for sharding time-series data. Its distributed query engine routes and parallelizes DDL, DML, and other operations on distributed tables across the cluster. And its columnar storage can compress data up to 8x, speeds up table scans, and supports fast projections. This release contains some foundational work to get our schema ready for partitioning. You can track our progress as we work towards integrating CitusDB into our codebase over the next few months. We plan on maintaining support for both databases for a period of time after the work is complete.
Performance Improvements
As is usually the case, we took the time to optimize various pieces of the system to work at scale. Our transactions REST API saw some performance improvements by rewriting them using Common Table Expressions (CTE). This will pay future dividends with CitusDB as it allows queries to be ran in parallel easier. An issue with /api/v1/topics/{id}/messages timing out for some topics was addressed and the realm and topic number columns were combined to reduce the table and index size. /api/v1/tokens/{id}/balances also saw some performance improvements that decreased its average response time. Configuration options for faster historical ingestion were documented so that mirror node operators can get historical data faster.
Enhancements
- Update smart contract parsing logic #2728
- Contract get and list REST apis #2717
- Combine realmNum and topicNum in TopicMessage ingestion and streaming #2712
- Improve transaction REST API query performance #2693
- Adopt CTEs for REST transactions sub queries #2676
- Add contract database schema #2674
- Add an entity_history table #2666
- Remove the side effect of v1.43.2 migration #2644
- Upgrade rosetta PostgreSQL #2643
- Add a smart contracts design document #2638
- Add recommended importer configuration for historical data ingestion #2637
- [Monitor, Acceptance Tests] Support HTS pause transaction #2635
- Contracts parsing #2632
- Contracts get and list REST APIs #2631
- Entity history #2630
- Contracts database schema #2629
- Update GCP Marketplace to latest #2624
- Upgrade PostgreSQL in Rosetta image #2576
- Design smart contract enhancements #2567
- Convert v1.43.2 add missing token account associations migration to a manual sql script #2511
- Combine realm_num and topic_num in topic_message #2456
- Citus POC #2273
Bug Fixes
- Fix accounts performance regression caused by account_contract view #2797
- Fix entity query in accounts and balances REST apis #2780
- Add missing TOKENPAUSE/UNPAUSE to OpenApi spec (0.43) #2764
- Update OpenAPI max_automatic_token_associations type to int32 (0.43) #2756
- Smart contract fixes #2745
- Improve topic messages by topic id query performance #2722
- Fix importer not persisting some account entity fields #2720
- max_automatic_token_associations is null in accounts REST api response #2719
- Improve token balances query performance #2695
- /topics/{id}/messages response time is high #2694
- /tokens/{id}/balances response time is high #2691
- Remove RepositoryEntityListener #2689
- Fix notes bug in helm template #2673
- Fill missing transaction data for mainnet #2672
- Fix entity timestamp #2645
- Show deleted nfts #2640
- /tokens/{tokenId}/nfts doesn't show nft instances when nft class is deleted #2577
- TokenAccount handling in RepositoryEntityListener is broken #2480
- Not persisting child contracts on contract create and contract call #1664
- Not persisting file entity for contract transactions #1237
Dependency Upgrades
- Bump aws-sdk from 2.1004.0 to 2.1010.0 in /hedera-mirror-rest #2714
- Bump axios from 0.22.0 to 0.23.0 in /hedera-mirror-rest #2710
- Bump boxen from 6.2.0 to 6.2.1 in /hedera-mirror-rest/check-state-proof #2709
- Bump jest-circus from 27.2.5 to 27.3.0 in /hedera-mirror-rest #2708
- Bump jest from 27.2.5 to 27.3.0 in /hedera-mirror-rest/check-state-proof #2707
- Bump mathjs from 9.5.0 to 9.5.1 in /hedera-mirror-rest #2706
- Bump eslint-plugin-jest from 25.0.1 to 25.2.2 in /hedera-mirror-rest #2705
- Bump jest from 27.2.5 to 27.3.0 in /hedera-mirror-rest #2703
- Bump jest from 27.2.5 to 27.3.0 in /hedera-mirror-rest/monitoring/monitor_apis #2702
- Bump hedera-protobuf-java-api from 0.19.0-alpha.2 to 0.19.0 #2701
- Bump parse-duration from 1.0.1 to 1.0.2 in /hedera-mirror-rest/monitoring/monitor_apis #2699
- Bump software.amazon.awssdk:bom from 2.17.57 to 2.17.61 #2698
- Bump mathjs from 9.5.0 to 9.5.1 in /hedera-mirror-rest/monitoring/monitor_apis #2697
- Bump software.amazon.awssdk:bom from 2.17.56 to 2.17.57 #2671
- Bump hibernate-types-52 from 2.12.1 to 2.13.0 #2669
- Bump vertx-pg-client from 4.1.4 to 4.1.5 #2665
- Bump protobuf-java from 3.18.0 to 3.18.1 #2664
- Bump github.com/spf13/viper from 1.8.1 to 1.9.0 in /hedera-mirror-rosetta #2663
- Bump gorm.io/gorm from 1.21.15 to 1.21.16 in /hedera-mirror-rosetta #2662
- Bump jest from 27.2.4 to 27.2.5 in /hedera-mirror-rest/check-state-proof #2661
- Bump boxen from 6.0.0 to 6.2.0 in /hedera-mirror-rest/check-state-proof #2660
- Bump jest-circus from 27.2.4 to 27.2.5 in /hedera-mirror-rest #2659
- Bump aws-sdk from 2.999.0 to 2.1004.0 in /hedera-mirror-rest #2658
- Bump eslint-plugin-jest from 24.5.2 to 25.0.1 in /hedera-mirror-rest #2657
- Bump jest-junit from 12.3.0 to 13.0.0 in /hedera-mirror-rest #2656
- Bump reporting-plugin from 5.2.1 to 7.0.0 #2655
- Bump jest from 27.2.4 to 27.2.5 in /hedera-mirror-rest #2654
- Bump cucumber.version from 6.11.0 to 7.0.0 #2653
- Bump parse-duration from 1.0.0 to 1.0.1 in /hedera-mirror-rest/monitoring/monitor_apis #2652
- Bump embedded.testcontainers.version from 2.0.15 to 2.0.16 #2651
- Bump software.amazon.awssdk:bom from 2.17.51 to 2.17.56 #2650
- Bump jest from 27.2.4 to 27.2.5 in /hedera-mirror-rest/monitoring/monitor_apis #2649
- Bump reactor-grpc-stub from 1.2.2 to 1.2.3 #2648
Contributors
We'd like to thank all the contributors who worked on this release!