github filecoin-project/lotus v1.23.0-rc1
on GitHub

latest releases: v1.29.1, v1.28.3, miner/v1.28.2...
pre-release17 months ago

This is the first release candidate of the upcoming optional but highly recommended feature release of Lotus. It includes numerous improvements and enhancements for node operators, ETH RPC-providers and storage providers.

Note: this release is bumped from the v1.21.0-rcx plus all the changes from v1.22.0 that supports the upcoming nv19 upgrade, scheduled on Apr 27th. You can find more details about the upgrade here

☢️ Upgrade Warnings ☢️

Before upgrading to this feature release read carefully through these bullet points:

  • Starting from this release, the SplitStore feature is automatically activated on new nodes. However, for existing Lotus users, you need to explicitly configure SplitStore by uncommenting the EnableSplitstore option in your config.toml file. To enable SplitStore, set EnableSplitstore=true, and to disable it, set EnableSplitstore=false. It's important to note that your Lotus node will not start unless this configuration is properly set. Set it to false if you are running a full archival node!
  • This feature release requires a minimum Go version of v1.19.7 or higher to successfully build Lotus. Additionally, Go version v1.20 and higher is now also supported.
  • Storage Providers: The proofs libraries now have CUDA enabled by default, which requires you to install (CUDA)[https://lotus.filecoin.io/tutorials/lotus-miner/cuda/] if you haven't already done so. If you prefer to use OpenCL on your GPUs instead, you can use the FFI_USE_OPENCL=1 flag when building from source. On the other hand, if you want to disable GPUs altogether, you can use the FFI_NO_GPU=1 environment variable when building from source.
  • Storage Providers: The lotus-miner sectors extend command has been refactored to the functionality of lotus-miner sectors renew.
  • Exchanges/Node operators/RPC-providers:: Execution traces (returned from lotus state exec-trace, lotus state replay, etc.), has changed to account for changes introduced by the by the FVM. Please make sure to read the Execution trace format change section carefully, as these are interface breaking changes
  • Syncing issues: If you have been struggling with syncing issues in normal operations you can try to adjust the amount of threads used for more concurrent FMV execution through via the LOTUS_FVM_CONCURRENCY enviroment variable. It is set to 4 threads by default. Recommended formula for concurrency == YOUR_RAM/4 , but max is 128. If you are a Storage Provider and are pushing many messages within a short period of time, exporting LOTUS_SKIP_APPLY_TS_MESSAGE_CALL_WITH_GAS=1 will also help with keeping in sync.
  • Catching up from a Snapshot: Users have noticed that catching up sync from a snapshot is taking a lot longer these day. This is largely related to the built-in market actor consuming a lot of computational demand for block validation. A FIP for a short-term mitigation for this is currently in Last Call and will be included network version 19 upgrade if accepted. You can read the FIP here.

Highlights

Execution Trace Format Changes

Execution traces (returned from lotus state exec-trace, lotus state replay, etc.), has changed to account for changes introduced by the FVM. Specifically:

  • The Msg field no longer matches the Filecoin message format as many of the message fields didn't make sense in on-chain sub-calls. Instead, it now has the fields To, From, Value, Method, Params, and ParamsCodec where ParamsCodec is a new field indicating the IPLD codec of the parameters.
    • Importantly, the Msg.CID field has been removed. This field is still present in top-level invocation results, just not inside the execution trace itself.
  • The MsgRct field no longer includes a GasUsed field and now has a ReturnCodec field to indicating the IPLD codec of the return value.
  • The Error and Duration fields have been removed as these are not set by the FVM. The top-level message "invocation result" retains the Error and Duration fields, they've only been removed from the trace itself.
  • Gas Charges no longer include "virtual" gas fields (those starting with v...) or source location information (loc) as neither field is set by the FVM.

A note on "codecs": FVM parameters and return values are IPLD blocks where the "codec" specifies the data encoding. The codec will generally be one of:

  • 0x51, 0x71 - CBOR or DagCBOR. You should generally treat these as equivalent.
  • 0x55 - Raw bytes.
  • 0x00 - Nothing. If the codec is 0x00, the parameter and/or return value should be empty and should be treated as "void" (not specified).
Old ExecutionTrace: 
{
  "Msg": {
    "Version": 0,
    "To": "f01234",
    "From": "f04321",
    "Nonce": 1,
    "Value": "0",
    "GasLimit": 0,
    "GasFeeCap": "1234",
    "GasPremium": "1234",
    "Method": 42,
    "Params": "<base64-data-or-null>",
    "CID": {
        "/": "bafyxyz....."
    },
  },
  "MsgRct": {
    "ExitCode": 0,
    "Return": "<base64-data-or-null>",
    "GasUsed": 12345,
  },
  "Error": "",
  "Duration": 568191845,
  "GasCharges": [
    {
      "Name": "OnMethodInvocation",
      "loc": null,
      "tg": 23856,
      "cg": 23856,
      "sg": 0,
      "vtg": 0,
      "vcg": 0,
      "vsg": 0,
      "tt": 0
    },
    {
      "Name": "wasm_exec",
      "loc": null,
      "tg": 1764,
      "cg": 1764,
      "sg": 0,
      "vtg": 0,
      "vcg": 0,
      "vsg": 0,
      "tt": 0
    },
    {
      "Name": "OnSyscall",
      "loc": null,
      "tg": 14000,
      "cg": 14000,
      "sg": 0,
      "vtg": 0,
      "vcg": 0,
      "vsg": 0,
      "tt": 0
    },
  ],
  "Subcalls": [
    {
      "Msg": { },
      "MsgRct": { },
      "Error": "",
      "Duration": 1235,
      "GasCharges": [],
      "Subcalls": [],
    },
  ]
}
New ExecutionTrace: 
{
  "Msg": {
    "To": "f01234",
    "From": "f04321",
    "Value": "0",
    "Method": 42,
    "Params": "<base64-data-or-null>",
    "ParamsCodec": 81
  },
  "MsgRct": {
    "ExitCode": 0,
    "Return": "<base64-data-or-null>",
    "ReturnCodec": 81
  },
  "GasCharges": [
    {
      "Name": "OnMethodInvocation",
      "loc": null,
      "tg": 23856,
      "cg": 23856,
      "tt": 0
    },
    {
      "Name": "wasm_exec",
      "loc": null,
      "tg": 1764,
      "cg": 1764,
      "sg": 0,
      "tt": 0
    },
    {
      "Name": "OnSyscall",
      "loc": null,
      "tg": 14000,
      "cg": 14000,
      "sg": 0,
      "tt": 0
    },
  ],
  "Subcalls": [
    {
      "Msg": { },
      "MsgRct": { },
      "GasCharges": [],
      "Subcalls": [],
    },
  ]
}

SplitStore

This feature release introduces numerous improvements and fixes to tackle SplitStore related issues that has been reported. With this feature release SplitStore is automatically activated by default on new nodes. However, for existing Lotus users, you need to explicitly configure SplitStore by uncommenting the EnableSplitstore option in your config.toml file. To enable SplitStore, set EnableSplitstore=true, and to disable it, set EnableSplitstore=false. It's important to note that your Lotus node will not start unless this configuration is properly set. Set it to false if you are running a full archival node!

SplitStore also has some new configuration settings that you can set in your config.toml file:

  • HotstoreMaxSpaceTarget suggests the max allowed space (in bytes) the hotstore can take.
  • HotstoreMaxSpaceThreshold a moving GC will be triggered when total moving size exceeds this threshold (in bytes).
  • HotstoreMaxSpaceSafetyBuffer a safety buffer to prevent moving GC from an overflowing disk.

The SplitStore also has two new commands:

  • lotus chain prune hot is a much less resource-intensive GC and is best suited for situations where you don't have the spare disk space for a full GC.
  • lotus chain prune hot-moving will run a full moving garbage collection of the hotstore. This commands create a new hotstore before deleting the old one so you need working room in the hotstore directory. The current size of a fully GC'd hotstore is around 295 GiB so you need to make sure you have at least that available.

You can read more about the new SplitStore commands in the documentation.

RPC API improvements

This feature release includes all the RPC API improvements made in the Lotus v1.20.x patch releases. It includes an updated FFI that sets the FVM parallelism to 4 by default.

Node operators with higher memory specs can experiment with setting LOTUS_FVM_CONCURRENCY to higher values, up to 48, to allow for more concurrent FVM execution.

Experimental scheduler assigners

In this release there are four new expirmental scheduler assigners:

  • The experiment-spread-qcount - similar to the spread assigner but also takes into account task counts which are in running/preparing/queued states.
  • The experiment-spread-tasks - similar to the spread assigner, but counts running tasks on a per-task-type basis
  • The experiment-spread-tasks-qcount - similar to the spread assigner, but also takes into account task counts which are in running/preparing/queued states, as well as counting running tasks on a per-task-type basis. Check the results for this assigner on (storage-only lotus-workers here).
  • The experiment-random - In each schedule loop the assinger figures a set of all workers which can handle the task and then picks a random one. Check the results for this assigner on (storage-only lotus-workers here).

Graceful shutdown of lotus-workers
We have cleaned up some commands in the lotus-worker to make it less confusing how to gracefully shutting down a lotus-worker while there are incoming sealing tasks in the pipeline. To shut down a lotus-worker gracefully:

  1. lotus-worker tasks disable --all and wait for the worker to finish processing its current tasks.
  2. lotus-worker stop to detach it and do maintenance/upgrades.

CLI speedups

The lotus-miner sector list is now running in parallel - which should speed up the process from anywhere between 2x-10x+. You can tune it additionally with the check-parallelism option in the command. The Lotus-Miner info command also has a large speed improvement, as calls to the lotus legacy market has been removed.

New features

  • feat: splitstore: Pause compaction when out of sync (filecoin-project/lotus/#10641)
    • Pause the SplitStore compaction if the node is out of sync. Resumes the compation when its back in sync.
  • feat: splitstore: limit moving gc threads (#10621) (filecoin-project/lotus/#10621)
    • Makes moving gc less likely to cause node falling out of sync.
  • feat: splitstore: Update config default value (#10605) (filecoin-project/lotus/#10605)
    • Sets Splitstore HotStoreMaxSpaceTarget config to 650GB as default
  • feat: splitstore: Splitstore enabled by default (#10429) (filecoin-project/lotus#10429)
    • Enables SplitStore by default on new Lotus nodes. Existing Lotus users need to explicitly configure
  • feat: splitstore: Configure max space used by hotstore and GC makes best effort to respect (filecoin-project/lotus#10391)
    • Adds three new configs for setting the maximum allowed space the hotstore can take.
  • feat: splitstore: Badger GC of hotstore command (filecoin-project/lotus#10387)
    • Adds a lotus chain prune hot command, to run the garbage collection of the hotstore in a user driven way.
  • feat: sched: Assigner experiments (filecoin-project/lotus#10356)
    • Introduces experimental scheduler assigners that works better for setups that uses storage-only lotus-workers.
  • fix: wdpost: disabled post worker handling (filecoin-project/lotus#10394)
    • Improved scheduling logic for Proof-of-SpaceTime workers.
  • feat: cli: list claims and remove expired claims (filecoin-project/lotus#9875)
    • Adds a command to list claims made by a provider lotus filplus list-claims. And lotus filplus remove-expired-claims to remove expired claims.
  • feat: cli: make sectors list much faster (filecoin-project/lotus#10202)
    • Makes lotus-miner sector list checks run in parallel.
  • feat: cli: Add an EVM command to fetch a contract's bytecode (filecoin-project/lotus#10443)
    • Adds an lotus evm bytecode command to fetch a contract's bytecode.
  • feat: mempool: Reduce minimum replace fee from 1.25x to 1.1x (#10416) (filecoin-project/lotus#10416)
    • Reduces replacement message fee logic to help include update message replacements from developers using Ethereum tools like MetaMask.
  • feat: update renew-sectors with FIP-0045 logic (filecoin-project/lotus#10328)
    • Updates the lotus-miner sectors extend with FIP-0045 logic to include the ability to drop claims and set the maximum number of messages contained in a message.
  • feat: IPC: Abstract common consensus functions and consensus interface (filecoin-project/lotus#9481)
    • Add eudico's consensus interface to Lotus and implement EC behind that interface. This abstraction is the stepping-stone for Mir's integration.
  • fix: worker: add all tasks flag (filecoin-project/lotus#10232)
    • Adds an all flag for the lotus-worker tasks enable/disable cmds.
  • feat:shed:add cid to cbor serialization command (filecoin-project/lotus#10032)
    • Adds two lotus-shed commands, lotus-shed cid bytes and lotus-shed cid cbor to serialize cid to cbor and cid to bytes.
  • feat: add toolshed commands to inspect statetree size (filecoin-project/lotus#9982)
    • Adds two commands, lotus-shed stat-actor and lotus-shed stat-obj that work with an offline lotus repo to report dag size stats.
  • feat: shed: encode address to bytes (filecoin-project/lotus#10105)
    • Adds a lotus-shed address encode for encoding a filecoin address to hex bytes.
  • feat: chain: export-range (filecoin-project/lotus#10145)
    • Adds a lotus chain export-range command that can create archival-grade ranged exports of the chain as quickly as possible.
  • feat: stmgr: cache migrated stateroots (filecoin-project/lotus#10282)
    • Cache network migration results to avoid running migrations twice.
  • feat: shed: Add a tool to read data from sectors (filecoin-project/lotus#10169)
    • Adds a lotus-shed sectors read command that extract data from sectors from a running lotus-miner deployment.
  • feat: cli: Refactor renew and remove extend (filecoin-project/lotus#9920)
    • Refactors the lotus-miner sectors extend command to have the functionality of lotus-miner sectors renew. The lotus-miner sectors renew command has been deprecated.
  • feat: shed: Add beneficiary commands (filecoin-project/lotus#10037)
    • Adds the beneficiary address command to lotus-shed. You can now use lotus-shed actor propose-change-beneficiary and lotus-shed actor confirm-change-beneficiary to change beneficiary addresses.

Improvements

Dependencies

Others

Contributors

Contributor Commits Lines ± Files Changed
Hannah Howard 2 +2909/-6026 84
Łukasz Magiera 42 +2967/-1848 95
Steven Allen 20 +1703/-1345 88
Alfonso de la Rocha 17 +823/-1808 86
Peter Rabbitson 9 +1957/-219 34
Geoff Stuart 12 +818/-848 29
hannahhoward 5 +507/-718 36
Hector Sanjuan 6 +443/-726 35
Kevin Li 1 +1124/-14 22
zenground0 30 +791/-269 88
frrist 1 +992/-16 13
Travis Person 4 +837/-53 24
Phi 20 +622/-254 34
Ian Davis 7 +35/-729 20
Aayush 10 +378/-177 40
Raúl Kripalani 15 +207/-138 19
Arsenii Petrovich 7 +248/-94 30
ZenGround0 5 +238/-39 15
Neel Virdy 1 +109/-107 58
ychiao 1 +135/-39 3
Jorropo 2 +87/-82 67
Marten Seemann 8 +69/-64 17
Rod Vagg 1 +55/-16 3
Masih H. Derkani 3 +39/-27 12
raulk 2 +30/-29 5
dependabot[bot] 4 +37/-17 8
beck 2 +38/-2 2
Jennifer Wang 4 +20/-19 19
Richard Guan 3 +28/-8 5
omahs 7 +14/-14 7
dirkmc 2 +19/-7 6
David Choi 2 +16/-5 2
Mike Greenberg 1 +18/-1 1
Adin Schmahmann 1 +19/-0 2
Phi-rjan 5 +12/-4 5
Dirk McCormick 2 +6/-6 3
Aayush Rajasekaran 2 +9/-3 2
Jiaying Wang 5 +6/-4 5
Anjor Kanekar 1 +5/-5 1
vyzo 1 +3/-3 2
0x5459 1 +1/-1 1
Check out latest releases or
releases around filecoin-project/lotus v1.23.0-rc1

Don't miss a new lotus release

NewReleases is sending notifications on new releases.

Get notifications