This update brings a brand new DB architecture that cuts database size by several hundred gigabytes as well as exposing new configuration endpoints for users to tune their node to maximize performance based on their hardware.
New Performance Tuning Settings
Several new endpoints have been created as well as a new detailed documentation page on how to configure them. The default for these are targeted for 4 CPU cores and 8 GBs of RAM – better machines will benefit from updating them.
Notable mention is the --db-compression flag which will set the database compression level. Current compression is called snappy which has been left as default since it is safer to use with the current database layout.
If using the new database layout, zstd compression level is recommended because it enables much more deep compression, reducing DB size significantly at a fraction of the CPU cost. The zstd compression level will become the new default for Juno 0.16.0
Usage example:
--db-compression snappyRemember to take a look at the Performance Tuning section to get more details.
New Database Layout
A new database layout is now available which changes the way we store transaction headers and receipts. Previously, transaction headers and receipts were stored using a key composed of the block number and the transaction’s index within the block. Given the millions of transactions on Starknet Mainnet, this would allot for a total of 500GB of storage.
The new layout, combined with zstd compression, reduces this value from 500 GBs to only 200 GBs.
To apply this new database layout, please run the node with the following flags
<your command to run juno>
...
--transaction-combined-layout
--db-compression zstd⚠️ Note 1: once this new flag has been applied it won't be possible to revert back to the original database. Syncing from the beginning or from a snapshot is the only possible way for reverting this layout.
⚠️ Note 2: this flag will trigger a migration which we will take around 3 hours on a machine running the minimum specs. Validators beware.
ℹ️ Info: the
--db-compression zstdflag is optional but it is highly recommended since it will get the most benefit for database size reduction. Additionally, from now onward your database growth will be heavily reduced.
The reason this database update is optional through a flag is to make it available sooner and avoid the breaking change. Once Juno 0.16.0 releases this migration will become mandatory and nodes which haven't migrated yet will be required to.
Note that when enabling a new compression, the whole database won't be compressed at once but gradually, as new information is written while existing data will remain stored with the previous compression. However, when updating to this new database layout with zstd compression, storage savings will be immediately noticeable given that 80% of the database is being rewritten. As a result, while a full re-sync is required to achieve maximum reduction, most of the benefits will be seen after the migration.
A fully compressed snapshot will be made available by us in the near future.
Full Changelog: v0.15.17...v0.15.18