filecoin-project/boost v2.1.0-rc2 on GitHub

Overview

We're happy to announce the Boost v2.1.0-rc2 release. Boost v2 revamps the way Boost handles indices for user data, introducing the Local Index Directory (LID) and removing the DAG store. In addition, there are a number of new features including bug fixes, increased flexibility on LID setup, and more.

This release is compatible with the Lotus v1.24.0-rc1 and v1.25.0-rc1 release. See the Boost and Lotus compatibility matrix for up to date information to plan your upgrade.

👉 ⚠️ ❗ ❗ Before updating, please read the release notes and documentation carefully, as storage providers will need to setup a database service and go through a database migration process. Incorrect process can result in permanent data loss ❗ ❗ ⚠️

⭐ Highlights

Boost UI will allow SPs to check the payload CIDs for a pieceCID. These CID are useful for retrieval testing and debugging.

Index Provider

The index-provider has switched from Graphsync data transfer protocol to HTTP-libp2p protocol.
The Retrieval page in the UI has been split up so that only retrieval deal information is shown on the existing page, and all indexing information is shown on the Indexer page.
The indexer UI in Boost displays the lag between the latest local advertisement and the sync status of the network indexers.

🏗️ Improvements

The devnet build time has been reduced from 40+ minutes to <5 minutes.
The migrate-lid command now support parallelisation of index migration to speed up the over all migration time required for switching from Boost v1 to v2.
boostd init generates a commented out config file
The deal filter mechanism has been redesigned to run in parallel. This should speed up the deal acceptance time and reduce client timeouts.
Graphsync server has been updated to respond correctly for identity CID requests to reduce the overall retrieval failure rates.
The deal download has been redesigned to perform multi-part downloads to reduce the download time for online deals(like aria2c).
The pieceReader is closed correctly after generating index to fix the issue of dangling deleted files on Lotus AP workers.

What's Changed

fix(docker): no inline sed commands by @hannahhoward in #1658
Build and publish boost devnet containers via CI by @masih in #1661
Allow container build job to push images by @masih in #1663
feat: add InUnsealed utility to boostx by @LexLuthr in #1680
Disable flaky TestConcurrentTransfers by @dirkmc in #1682
fix: disable leveldb as LID default by @LexLuthr in #1681
fix: init with commented config by @LexLuthr in #1694
fix: add parallelisation to migrate-lid and improve logging by @LexLuthr in #1689
fix missing icons by @dirkmc in #1698
chore(deps): upgrade graphsync for identity CID fix by @rvagg in #1704
feat: update docker devnet build by @LexLuthr in #1702
fix config.toml for devnet by @nonsense in #1708
fix: close piece reader after generating indexes by @LexLuthr in #1707
fix: docker devnet entrypoint by @LexLuthr in #1712
recover from panics in go-routine in client-side io.Reader encoder by @nonsense in #1706
fix: GS query panic by @LexLuthr in #1714
fix: Rename deal data root cid in Boost UI by @LexLuthr in #1719
fix: remove payloadRootCid lookup by @LexLuthr in #1721
ci: explicitly switch macos build to silicon and upgrade xcode by @galargh in #1720
feat: show payload CIDs in UI by @LexLuthr in #1722
Ivan/http transport upgrades by @ischasny in #1732
feat: Ipnisync by @LexLuthr in #1731
ci: switch macos builds to apple silicon by @galargh in #1726
direct data onboarding - disabled by @nonsense in #1627
Fix boostd-data go module path and remove replace directives by @masih in #1741
Fix static check issues by @masih in #1743
fix LID web UI by @nonsense in #1745
use explicit repo location for boostd-data by @nonsense in #1748
add support for yugabytedb for devnet by @nonsense in #1756
feat: run deal filters in parallel by @LexLuthr in #1746

New Contributors

@galargh made their first contribution in #1720

Full Changelog: v2.1.0-rc1...v2.1.0-rc2