filecoin-project/lotus v1.19.0

on GitHub

This is an optional feature release of Lotus. This feature release includes the SplitStore beta, the experimental Lotus node cluster feature, as well as numerous enhancements and bug fixes.

Highlights

🟢 SplitStore v2（Beta) 🟢

Splitstore aims to reduce the node performance impact that's caused by the Filecoin's very large, and continuously growing chain datastore by having a hot and cold blockstore. You can find more about the Splitstore implementation here.
Splitstore has three basic modes for node operators to configure according to your needs:

• discard: hotstore only, automatically archive out-of-scope objects that are beyond 4 finalities(3600 epochs).
• universal: stores all chain data that's beyond 4 finalities into coldstore.
• messages: only stores on-chain messages into coldstore.

The EnableColdStoreAutoPrune= configuration is being deprecated in this release, as there is only ever one compaction running. We welcome all node operators to try the new feature and let us know here if you have any feedback!
There are more configuration one may set, you can read the full documentation about the SplitStoreV2 here: https://lotus.filecoin.io/lotus/configure/splitstore/.

🧪 Node Cluster (EXPERIMENTAL.) 🧪

The Lotus HA node cluster feature allows you to run multiple Lotus daemons for the same lotus-miner increasing resiliency. We welcome all Lotus users to join the early testing for this feature and provide your feedback. Please note that this feature is targeted towards more enterprise users of Lotus and requires at least 3 lotus nodes to be set up in a cluster.
Check out the documentation here: https://lotus.filecoin.io/lotus/configure/clusters/

⭐️ SnapDeals Enhancements ⭐️

Numerous SnapDeals related improvements and fixes made it into this release before the code freeze. Some the highlights of the issues that has been fixed in this feature release are:

• Unable to snap-up a sector again if something went wrong. - This has now been fixed ✅
• Error messages on loop during an open deadline. - This has now been fixed ✅

New features

• feat:splitstore:single compaction that can handle prune aka two marksets one compaction (#9571) (filecoin-project/lotus#9571)
  • Introduces a new SplitStore-mode, messages, which will only store on-chain messages. Fixes previously issues with regards to AutoPrune not compacting the coldstore. Link to documentation
• feat: Raft consensus for lotus nodes in a cluster (filecoin-project/lotus#9294)
  • Adds the experimental node cluster feature.
• feat: storage: Force exit GenerateSingleVanillaProof on cancelled context (filecoin-project/lotus#9613)
  • GenerateSingleVanillaProof now respects context, which means that it will skip slow to read sectors 🐌 and return a context error. Instead of being blocked forever if storage reads where blocked (e.g disconnected NFS).
• feat: wdpost: Configurable pre-check timeouts (filecoin-project/lotus#9680)
  • Adds configuration knobs for setting custom amount of time a proving pre-check can take before a sector and partition will be skipped. Link to documentation
• feat: chain: future proof the from & to address protocols (filecoin-project/lotus#9515)
  • This lets us add new address protocols to go-address without implicitly accepting them in messages on the network.
• feat: Retrieval into remote blockstores (filecoin-project/lotus#9565)
  • Makes it possible to point retrievals at a network-backed blockstore.
• feat: Add node uptime rpc / output in info command (filecoin-project/lotus#9436)
  • Adds node uptime stats to the lotus-miner info and lotus info commands
• feat: wdpost: Add ability to only have single partition per msg for partitions with… (filecoin-project/lotus#9413)
  • Adds a configuration option to have a single partition per PoSt Message for partitions containing recovering sectors.
• feat: miner paramfetch: Don't fetch param files when not needed (filecoin-project/lotus#9391)
  • A Lotus-Miner processes that has disabled local PoSt / C2 / PR2 does not need the param-files. This makes node startup much faster, reducing downtime by a lot when restarts are needed.
• feat: client: Add retrieval deal ID and bytes transferred to retrieval output (filecoin-project/lotus#9398)
  • Appends retrieval deal ID and bytes transferred to the retrieval output.
• feat: dealpublisher: check for duplicate deals before adding (filecoin-project/lotus#9365)
• feat: Drop active retrieval check (#764) (filecoin-project/go-fil-markets#764)
• feat(retrievalmarkets): expose GetDynamicAsk (#748) (filecoin-project/go-fil-markets#748)
• feat: handle retrieval queries for unindexed identity payload CIDs (#747) (filecoin-project/go-fil-markets#747)
• feat: add a method for validating an address for a network version (#115) (filecoin-project/go-state-types#115)

Improvements

• fix: miner-cli: Fix lotus-miner proving check (filecoin-project/lotus#9643)
  • Fixes the issue where the lotus-miner proving check command always outputted Error: rg is nil
• fix: sealing pipeline: Clear CreationTime when starting sector upgrade (filecoin-project/lotus#9642)
  • Fixes the issue where an aborted SnapDeal upgrade could no longer be retried with SnapDeals.
• fix:sealing-fsm:wait mutable fsm state for immutable sector upgrade error (filecoin-project/lotus#9598)
  • Creating a new WaitMutable state - now if the deadline is open and the sectors are trying finalize they will wait on the worker until the deadline has closed. Important to note that they will not finalize as soon as the deadline closes, they will wait 1h before continuing. Fixes the previous issue where upgraded Snap-sectors for an open deadline cause a lot of error-messages and p_aux issues
• fix: cli: add beneficiary info to lotus-miner actor control list (filecoin-project/lotus#9632)
  • Adds the beneficiary address to the lotus-miner actor control list output.
• fix: sealing pipeine: Release assigned deals on snapdeals abort (filecoin-project/lotus#9601)
• fix: docker: make compatible with arm platform (filecoin-project/lotus#9363)
  • Makes the Dockerfile.lotus compatible with ARM-platforms (e.g Mac M1).
• fix: post worker sched: Don't check worker session in a busy loop (filecoin-project/lotus#9495)
  • Fixes a looping pattern which could result in a flood of requests between lotus-miner<->lotus-worker, potentially exhausting resources needed to make http requests, that lead to all sorts of random RPC-related issues.
• fix: miner: init miner's with 32GiB sectors by default (filecoin-project/lotus#9364)
  • Makes the lotus-miner init defualt to 32GiB sectors.
• fix: store identity CIDs in CARs for online deals (#749) (filecoin-project/go-fil-markets#749)
• fix: cliutil: Fix URL-based API endpoint parsing

Dependencies

• deps: upgrade go-merkledag to 0.8.1 (#9717)
• deps: Update go-fil-markets to v1.25.0 (filecoin-project/lotus#9633)
• deps: upgrade go-merkledag to 0.8.0 (filecoin-project/lotus#9455)

Others

• fix/build: Update Zondax/hid to 0.9.1
• refactor: sealing: minor refactor of FinalizeReplicaUpdate (filecoin-project/lotus#9614)
• update ffi to 280c4f8b94fd46dc (#9608) (filecoin-project/lotus#9608)
• fix: tvx: make it work with the FVM (filecoin-project/lotus#9604)
• fix: autobatch: remove potential deadlock when a block is missing (filecoin-project/lotus#9602)
• feat: shed: set control address: add dump bytes option (filecoin-project/lotus#9572)
• feat: shed: Online export-car (filecoin-project/lotus#9590)
• fix: chain: Update chain.go (filecoin-project/lotus#9373)
• fix: fvm: Allow setting local bundles for Debug FVM for av 9+ (filecoin-project/lotus#9509)
• fix: ux: Add outputs to wallet commands (filecoin-project/lotus#9416)
• fix: ux: specify arg in net ping cmd (filecoin-project/lotus#9459)
• fix: cli: renew --only-cc with sectorfile (filecoin-project/lotus#9402)
• fix: bstore: Handle codecs correctly in membstore Get (filecoin-project/lotus#9471)
• fix: not multiplied by the number of seconds (filecoin-project/lotus#9460)
• Makefile: Drop rarely used binaries from build-devnets (#9513) (filecoin-project/lotus#9513)
• ci Remove unneeded homebrew deps (filecoin-project/lotus#9559)
• ci: Have apt-get wait until it can get a lock in packer builds (filecoin-project/lotus#9534)
• ci Appimage go1.18.1 fix (filecoin-project/lotus#9496)
• ci: Fix failing Digital Ocean and Amazon Machine Image release process (filecoin-project/lotus#9425)
• ci: Don't publish new homebrew releases for RC builds (filecoin-project/lotus#9350)
• ci: Use golang 1.18.1 to build appimage (filecoin-project/lotus#9386)
• ci Refactor release pipeline to better support m1 builds (filecoin-project/lotus#9600)
• ci: Rely on local env varibale instead of context (filecoin-project/lotus#9740)
• feat: shed: FIP0036 post poll result processing (filecoin-project/lotus#9387)
• misc: github: Cleanup PR template (filecoin-project/lotus#9472)
• docs: release template: Mention codegen in release prep (filecoin-project/lotus#9430)
• chore: merge releases (v1.17.2) into master (filecoin-project/lotus#9433)
• chore: merge release/v1.18.0 into master (filecoin-project/lotus#9597)
• chore:shed: Teach shed/sim to understand --tipset=@NNN notation (filecoin-project/lotus#9434)
• chore: Upgrade hid (filecoin-project/lotus#9406)
• chore: release: Update release_issue_template (filecoin-project/lotus#9150)
• chore: update lotus version to 1.19.0-rc1
• chore: merge release into master (filecoin-project/lotus#9657)
• Backport: #9061 rpc errors (filecoin-project/lotus#9384)
• shed: util: get all msig (filecoin-project/lotus#9322)
• fix: test: simplify TestDeadlineToggling (filecoin-project/lotus#9356)
• fix: build: set PropagationDelaySecs correctly (filecoin-project/lotus#9358)
• fix: test: flaky TestDeadlineToggling around nulls (#9354) (filecoin-project/lotus#9354)
• fix: retrievals: price-per-byte calculation fix (filecoin-project/lotus#9353)
• fix: docs: update Go-badge in readme (filecoin-project/lotus#9347)
• docs: release template: clarify location of version.go (filecoin-project/lotus#9338)
• build: Bump version to v1.17.3-dev (filecoin-project/lotus#9337)
• github.com/filecoin-project/go-fil-markets (v1.24.0-v17 -> v1.25.0):
• Merge branch 'release/v1.24.3'
• Update ffi and update markets to v9 (#751) (#761) (filecoin-project/go-fil-markets#761)
• chore: extract Provider piece logic to separate file (#750) (filecoin-project/go-fil-markets#750)
• Merge branch 'release/v1.24.2'
• Feat/support custom metadata (#759) (filecoin-project/go-fil-markets#759)
• Revert "Update ffi and update markets to v9 (#751)" (#755) (filecoin-project/go-fil-markets#755)
• Update ffi and update markets to v9 (#751) (filecoin-project/go-fil-markets#751)
• release: v1.24.0 (filecoin-project/go-fil-markets#745)
• github.com/filecoin-project/go-state-types (v0.9.8 -> v0.9.9):

lotus-market EOL notice

As mentioned in lotus v1.17.0 release notes, markets related features, enhancements and fixes is now lower priority for Lotus. We recommend our users to migrate to other deal making focused software, like boost as soon as possible. That being said, the lotus maintainers will be:

• Lotus maintainers will stop supporting lotus-market subcomponent/storage deal making related issues or enhancements on Jan 31, 2023.
• In Q2 2023, we will be deprecating/removing lotus-market related code from this repository.

If you have any questions or concerns, please raise them in Lotus discussion!

Contributors