filecoin-project/lotus v1.27.0

on GitHub

This is an optional feature release of Lotus. Lotus v1.27.0 includes numerous improvements, bugfixes and enhancements for node operators, RPC- and ETH RPC-providers. This feature release also introduces Curio in a Beta release. Check out the Curio Beta release section for how you can get started with Curio.

☢️ Upgrade Warnings ☢️

• Event Database: Two sequential migrations will adjust indexes without altering data or columns, ensuring minimal invasiveness when upgrading to this release. However, these migrations may be time-consuming for nodes with extensive event databases.
• This feature release drops the Raft cluster code experiment from the codebase. This Raft cluster never graduated beyond an experiment, had poor UX (e.g. no way to manage a running cluster, so it didn't provide High Availability), and pulled in a lot of heavy dependencies. We keep the multi-node RPC feature, it is not perfect, but it is useful.

Indexers, RPC- and ETH RPC-providers improvements

This release includes a lot of improvements and fixes for indexers, RPC- and ETH RPC-providers. Specifically these PRs:

• Significant performance improvements of eth_getLog
• Return the correct block gas limit in the EthAP
• Accept input data in call arguments under field 'input'
• Length check the array sent to eth_feeHistory RPC
• ETH subscribe tipsets API should only return tipsets that have been executed
• Adjust indexes in event index db to match query patterns

⭐️ Curio Beta Release ⭐️

Curio, the next generation of Lotus-Miner, also referred to as MinerV2! This release officially transitions Curio into beta and introduces a suite of powerful features designed to enhance your storage operations.

Highlights

• Curio as MinerV2: Embrace the revolutionary upgrade from Lotus-Miner to Curio. This transition is not just a rebranding—it's an upgrade to a more robust, scalable, and user-friendly version.
• High Availability: Curio is designed for high availability. You can run multiple instances of Curio nodes to handle similar type of tasks. The distributed scheduler and greedy worker design will ensure that tasks are completed on time despite most partial outages. You can safely update one of your Curio machines without disrupting the operation of the others.
• Node Heartbeat: Each Curio node in a cluster must post a heartbeat message every 10 minutes in HarmonyDB updating its status. If a heartbeat is missed, the node is considered lost and all tasks can now be scheduled on remaining nodes.
• Task Retry: Each task in Curio has a limit on how many times it should be tried before being declared lost. This ensures that Curio does not keep retrying bad tasks indefinitely. This safeguards against lost computation time and storage.
• Polling: Curio avoids overloading nodes with a polling system. Nodes check for tasks they can handle, prioritizing idle nodes for even workload distribution.
• Simple Configuration Management: The configuration is stored in the database in the forms of layers. These layers can be stacked on top of each other to create a final configuration. Users can reuse these layers to control the behavior of multiple machines without needing to maintain the configuration of each node. Start the binary with the appropriate flags to connect with YugabyteDB and specify which configuration layers to use to get desired behaviour.

Checkout this Github Dicsussion thread for testing Curio in developer network: #11991

New features

• feat: exchange: change GetBlocks to always fetch the requested number of tipsets (filecoin-project/lotus#11565)
• feat: syncer: optimize syncFork for one-epoch forks (filecoin-project/lotus#11533)
• feat: api: improve the correctness of Eth's trace_block (#11609) (filecoin-project/lotus#11609)
• perf: api: add indexes to event topics and emitter addr (#11477) (filecoin-project/lotus#11477)
• feat: drand: refactor round verification (filecoin-project/lotus#11598)
• feat: add Forest bootstrap nodes (#11636) (filecoin-project/lotus#11636)
• feat: curio: add miner init (#11775) (filecoin-project/lotus#11775)
• feat: curio: sectors UI (#11869) (filecoin-project/lotus#11869)
• feat: curio: storage index gc task (#11884) (filecoin-project/lotus#11884)
• feat: curio: web based config edit (#11822) (filecoin-project/lotus#11822)
• feat: spcli: sectors extend improvements (#11798) (filecoin-project/lotus#11798)
• feat: curio: Add schemas for DDO deal support (#11805) (filecoin-project/lotus#11805)
• feat: curioweb: add favicon (#11804) (filecoin-project/lotus#11804)
• feat: lotus-provider: Fetch params on startup when needed (filecoin-project/lotus#11650)
• feat: mpool: Cache actors in lite mode (#11668) (filecoin-project/lotus#11668)
• feat: curio: simpler reservation release logic (#11900) (filecoin-project/lotus#11900)
• feat: curio: add StorageInit api (#11918) (filecoin-project/lotus#11918)
• feat: lotus-provider: SDR Sealing pipeline (filecoin-project/lotus#11534)
• feat: curioweb: Sector info page (#11846) (filecoin-project/lotus#11846)
• feat: curio web: node info page (#11745) (filecoin-project/lotus#11745)
• feat: fvm: optimize FVM lanes a bit (#11875) (filecoin-project/lotus#11875)
• feat: Gateway API: Add ETH -> FIL and FIL -> ETH address conversion APIs to the Gateway (#11979) (filecoin-project/lotus#11979)

Improvements

• fix: api: return the correct block gas limit in the EthAPI (#11747) (filecoin-project/lotus#11747)
• fix: exchange: explicitly cast the block message limit const (#11511) (filecoin-project/lotus#11511)
• fix: Eth API: accept input data in call arguments under field 'input' (#11505) (filecoin-project/lotus#11505)
• fix: api: Length check the array sent to eth_feeHistory RPC (#11696) (filecoin-project/lotus#11696)
• fix: api: fix EthSubscribe tipsets off by one (#11858) (filecoin-project/lotus#11858)
• fix: lotus-provider: Fix log output format in wdPostTaskCmd (filecoin-project/lotus#11504)
• fix: lmcli: make 'sectors list' DDO-aware (#11839) (filecoin-project/lotus#11839)
• fix: lpwinning: Fix MiningBase.afterPropDelay (filecoin-project/lotus#11654)
• fix: exchange: allow up to 10k messages per block (filecoin-project/lotus#11506)
• fix: harmony: Fix task reclaim on restart (filecoin-project/lotus#11498)
• fix: lotus-provider: Wait for the correct taskID (filecoin-project/lotus#11493)
• fix: lotus-provider: show addresses in log (filecoin-project/lotus#11490)
• fix: sql Scan cannot write to an object (filecoin-project/lotus#11485)
• fix: lotus-provider: Fix winning PoSt (filecoin-project/lotus#11482)
• fix: lotus-provider: lotus-provider msg sending (filecoin-project/lotus#11480)
• fix: chain: use latest go-state-types types for miner UI (filecoin-project/lotus#11566)
• fix: Dockerfile non-interactive snapshot import (#11579) (filecoin-project/lotus#11579)
• fix: daemon: avoid prompting to remove chain when noninteractive (#11582) (filecoin-project/lotus#11582)
• fix: (events): check for sync-in-progress (#11932) (filecoin-project/lotus#11932)
• fix: curio: common commands (#11879) (filecoin-project/lotus#11879)
• fix: curio: fix incorrect null check for varchar column (#11881) (filecoin-project/lotus#11881)
• fix: local storage reservations fixes (#11866) (filecoin-project/lotus#11866)
• fix: curio: Check deal start epoch passed in PrecommitSubmit (#11873) (filecoin-project/lotus#11873)
• fix: curio: base config by default (#11676) (filecoin-project/lotus#11676)
• fix: curio: Start BoostAdapters before blocking rpc serve (#11871) (filecoin-project/lotus#11871)
• fix: cli: json flag (#11868) (filecoin-project/lotus#11868)
• feat: curio/lmrpc: Ingest backpressure (#11865) (filecoin-project/lotus#11865)
• feat: curio: Cleanup data copies after seal ops (#11847) (filecoin-project/lotus#11847)
• fix: spcli: add reference to the terminate command (#11851) (filecoin-project/lotus#11851)
• fix: sealing: improve gasEstimate logging (#11840) (filecoin-project/lotus#11840)
• fix: harmony: Try other tasks when storage claim fails
• fix: test: TestForkPreMigration hanging when env-var is set (#11838) (filecoin-project/lotus#11838)
• fix: piece: Don't return StartEport in PieceDealInfo.EndEpoch (#11832) (filecoin-project/lotus#11832)
• fix: paths/local: Fix on-disk storage accounting in new reservations (#11825) (filecoin-project/lotus#11825)
• fix: sealing pipeline: Fix panic on padding pieces in WaitDeals (#11708) (filecoin-project/lotus#11708)
• feat: ipfs: remove IPFS client backend (#11661) (filecoin-project/lotus#11661)
• fix: docs: Modify generate-lotus-cli.py to ignoring aliases. (filecoin-project/lotus#11535)
• fix: eth: decode as actor creation iff "to" is the EAM (#11520) (filecoin-project/lotus#11520)
• fix(events): properly decorate events db errors (#11856) (filecoin-project/lotus#11856)
• fix: CLI: adjust TermMax for extend-claim used by a different client (#11764) (filecoin-project/lotus#11764)
• fix: copy Flags field from SectorOnChainInfo (#11963) (filecoin-project/lotus#11963)
• feat: libp2p: Lotus stream cleanup (#11993) (filecoin-project/lotus#11993)

Dependencies

• chore: update deps (#11819) (filecoin-project/lotus#11819)
• chore: mod: use upstream poseidon (filecoin-project/lotus#11557)
• deps: multiaddress (filecoin-project/lotus#11558)
• chore:libp2p: update libp2p deps in master (filecoin-project/lotus#11522)
• dep: go-multi-address (filecoin-project/lotus#11563)
• chore: update go-data-transfer and go-graphsync (#12000) (filecoin-project/lotus#12000)
• chore: update drand (#12021) (filecoin-project/lotus#12021)
• chore: libp2p: update to v0.34.1 (12027) (filecoin-project/lotus#12027)
• github.com/filecoin-project/go-amt-ipld/ (v4.2.0 -> v4.3.0)
• github.com/filecoin-project/go-state-types (v0.13.1 -> v0.13.3)
• github.com/libp2p/go-libp2p-pubsub (v0.10.0 -> v0.10.1)
• github.com/libp2p/go-libp2p (v0.33.2 -> v0.34.1)

Others

• ci: ci: create gh workflow that runs go checks (#11761) (filecoin-project/lotus#11761)
• ci: ci: create gh workflow that runs go build (#11760) (filecoin-project/lotus#11760)
• ci: cancel in progress runs on pull requests only (#11842) (filecoin-project/lotus#11842)
• ci: ci: list processes before calling apt-get to enable debugging (#11815) (filecoin-project/lotus#11815)
• ci: ci: allow master main sync to write to the repository (#11784) (filecoin-project/lotus#11784)
• ci: ci: create gh workflow that runs go tests (#11762) (filecoin-project/lotus#11762)
• ci: ci: deprecate circle ci in favour of github actions (#11786) (filecoin-project/lotus#11786)
• misc: Drop the raft-cluster experiment (filecoin-project/lotus#11468)
• chore: fix some typos in comments (#11892) (filecoin-project/lotus#11892)
• chore: fix typos (#11848) (filecoin-project/lotus#11848)
• chore: fix typo (#11697) (filecoin-project/lotus#11697)
• chore: fix 2 typo's (#11542) (filecoin-project/lotus#11542)
• chore: calibnet: Update bootstrap peer list (filecoin-project/lotus#11672)
• chore: build: Bump version in master (filecoin-project/lotus#11475)
• chore: releases: merge releases branch to master (filecoin-project/lotus#11578)
• chore: Add systemd memory note on install and in config (#11641) (filecoin-project/lotus#11641)
• chore: switch back to upstream ledger library (#11651) (filecoin-project/lotus#11651)
• chore: build: update minimum go version to 1.21.7 (#11652) (filecoin-project/lotus#11652)
• chore: docs: nv-skeleton documentation (#11065) (filecoin-project/lotus#11065)
• chore: Add v13 support to invariants-checker (#11931) (filecoin-project/lotus#11931)
• chore: remove unmaintained bootstrappers (#11983) (filecoin-project/lotus#11983)
• chore: go mod: revert go version change as it breaks Docker build (#12050) (filecoin-project/lotus#12050)

Contributors