This release contains the changes from polkadot-stable2412-3 to polkadot-stable2503-rc2.

⚠️ Warning: Pre-release Version

The currently published version is a pre-release and is not yet a final stable version. This release is provided for testing purposes only, and there is no guarantee that everything will work as expected. Errors and unusual behaviours of some features are to be expected.

Please use this version at your own risk and report any issues you encounter. We recommend waiting for the official release if you need a stable version.

Changelog

Changelog for `Node Dev`

ℹ️ These changes are relevant to: Those who build around the client side code. Alternative client builders, SMOLDOT, those who consume RPCs. These are people who are oblivious to the runtime changes. They only care about the meta-protocol, not the protocol itself.

[#6561]: slot-based-collator: Move spawning of the futures

Move spawning of the slot-based collator into the run function. Also the tasks are being spawned as blocking task and not just as normal tasks.

[#4880]: Collation fetching fairness in collator protocol

Implements collation fetching fairness in the validator side of the collator protocol. With core time in place if two (or more) parachains share a single core no fairness was guaranteed between them in terms of collation fetching. The current implementation was accepting up to max_candidate_depth + 1 seconded collations per relay parent and once this limit is reached no new collations are accepted. A misbehaving collator can abuse this fact and prevent other collators/parachains from advertising collations by advertising max_candidate_depth + 1collations of its own.
To address this issue two changes are made:

For each parachain id the validator accepts advertisements until the number of entries in the claim queue equals the number of seconded candidates.
When new collation should be fetched the validator inspects what was seconded so far, what's in the claim queue and picks the first slot which hasn't got a collation seconded and there is no candidate pending seconding for it. If there is an advertisement in the waiting queue for it it is fetched. Otherwise the next free slot is picked.
These two changes guarantee that:
Validator doesn't accept more collations than it can actually back.
Each parachain has got a fair share of core time based on its allocations in the claim queue.

[#6647]: `fatxpool`: proper handling of priorities when mempool is full

Higher-priority transactions can now replace lower-priority transactions even when the internal tx_mem_pool is full.

[#7554]: sc-informant: Print full hash when debug logging is enabled

When debugging stuff, it is useful to see the full hashes and not only the "short form". This makes it easier to read logs and follow blocks.

[#6703]: network/libp2p-backend: Suppress warning adding already reserved node as reserved

Fixes #6598.

[#7005]: Log peerset set ID -> protocol name mapping

To simplify debugging of peerset related issues like #6573 (comment).

[#7866]: Make litep2p the default backend in Kusama

A new trait IdentifyNetworkBackend is introduced for the polkadot-service. The purpose of the trait is to specify the default network backend for individual chains. For Kusama based chains, the default is now litep2p. For other chains, the default remains unchanged to libp2p.
The network backend field of the network configuration is made optional to accomodate for this change.

[#6636]: Optimize initialization of networking protocol benchmarks

These changes should enhance the quality of benchmark results by excluding worker initialization time from the measurements and reducing the overall duration of the benchmarks.

[#6979]: Update prometheus binding failure logging format

Using {:#?} for the error details is a bit annoying, this change makes a more consistent formatting style for error messages.

[#6262]: Size limits implemented for fork aware transaction pool

Size limits are now obeyed in fork aware transaction pool

[#7368]: Add chain properties to chain-spec-builder

Adds support for chain properties to chain-spec-builder.

[#7640]: Bring the latest compatibility fixes via litep2p v0.9.1

This release enhances compatibility between litep2p and libp2p by using the latest Yamux upstream version.
Additionally, it includes various improvements and fixes to boost the stability and performance of the WebSocket stream and the multistream-select protocol.

[#7254]: deprecate AsyncBackingParams

Removes all usage of the static async backing params, replacing them with dynamically computed equivalent values (based on the claim queue and scheduling lookahead).

Adds a new runtime API for querying the scheduling lookahead value. If not present, falls back to 3 (the default value that is backwards compatible with values we have on production networks for allowed_ancestry_len)

Also removes most code that handles async backing not yet being enabled, which includes support for collation protocol version 1 on collators, as it only worked for leaves not supporting async backing (which are none).

[#7338]: [net/libp2p] Use raw `Identify` observed addresses to discover external addresses

Instead of using libp2p-provided external address candidates, susceptible to address translation issues, use litep2p-backend approach based on confirming addresses observed by multiple peers as external.

Fixes #7207.

[#7021]: Improve remote externalities logging

Automatically detect if current env is tty. If not disable the spinner logging.

[#6711]: Expose DHT content providers API from `sc-network`

Expose the Kademlia content providers API for the use by sc-network client code:

Extend the NetworkDHTProvider trait with functions to start/stop providing content and query the DHT for the list of content providers for a given key.
Extend the DhtEvent enum with events reporting the found providers or query failures.
Implement the above for libp2p & litep2p network backends.

[#6215]: Remove `ProspectiveParachainsMode` from backing subsystem

Removes ProspectiveParachainsMode usage from the backing subsystem and assumes
async_backing_params runtime api is always available. Since the runtime api v7 is released on
all networks it should always be true.

[#6455]: Add litep2p network protocol benches

Adds networking protocol benchmarks with litep2p backend

[#7042]: networking::TransactionPool should accept Arc

The sc_network_transactions::config::TransactionPool trait now returns an Arc for transactions.

[#6405]: `fatxpool`: handling limits and priorities improvements

This PR provides a number of improvements and fixes around handling limits and priorities in the fork-aware transaction pool.

[#6284]: backing: improve session buffering for runtime information

This PR implements caching within the backing module for session-stable information,
reducing redundant runtime API calls.

Specifically, it introduces a local cache for the:

validators list;
node features;
executor parameters;
minimum backing votes threshold;
validator-to-group mapping.

Previously, this data was fetched or computed repeatedly each time PerRelayParentState
was built. With this update, the cached information is fetched once and reused throughout
the session.

[#7127]: Forbid v1 descriptors with UMP signals

Adds a check that parachain candidates do not send out UMP signals with v1 descriptors.

[#7014]: Remove `yamux_window_size` from network config

Description

resolve #6468

Checklist

My PR includes a detailed description as outlined in the "Description" and its two subsections above.
My PR follows the labeling requirements of this project (at minimum one label for T required)
- External contributors: ask maintainers to put the right label on your PR.
I have made corresponding changes to the documentation (if applicable)
I have added tests that prove my fix is effective or that my feature works (if applicable)

[#7102]: `fatxpool`: rotator cache size now depends on pool's limits

This PR modifies the hard-coded size of extrinsics cache within PoolRotator to be inline with pool limits. It only applies to fork-aware transaction pool. For the legacy (single-state) transaction pool the logic remains untouched.

[#5842]: Get rid of libp2p dependency in sc-authority-discovery

Removes libp2p types in authority-discovery, and replace them with network backend agnostic types from sc-network-types.
The sc-network interface is therefore updated accordingly.

[#6924]: malus-collator: implement malicious collator submitting same collation to all backing groups

This PR modifies the undying collator to include a malus mode,
enabling it to submit the same collation to all assigned backing groups.

It also includes a test that spawns a network with the malus collator
and verifies that everything functions correctly.

[#6249]: Pure state sync refactoring (part-1)

The pure refactoring of state sync is preparing for #4. This is the first part, focusing on isolating the function process_state_key_values() as the central point for storing received state data in memory. This function will later be adapted to forward the state data directly to the DB layer to resolve the OOM issue and support persistent state sync.

[#7104]: collation-generation: resolve mismatch between descriptor and commitments core index

This PR resolves a bug where collators failed to generate and submit collations,
resulting in the following error:

ERROR tokio-runtime-worker parachain::collation-generation: Failed to construct and
distribute collation: V2 core index check failed: The core index in commitments doesn't
match the one in descriptor.

This issue affects only legacy and test collators that still use the collation function.
It is not a problem for lookahead or slot-based collators.

This fix ensures the descriptor core index contains the value determined by the core
selector UMP signal when the parachain is using RFC103.

[#6983]: cumulus: bump PARENT_SEARCH_DEPTH to allow for 12-core elastic scaling

Bumps the PARENT_SEARCH_DEPTH constant to a larger value (30).
This is a node-side limit that restricts the number of allowed pending availability candidates when choosing the parent parablock during authoring.
This limit is rather redundant, as the parachain runtime already restricts the unincluded segment length to the configured value in the
FixedVelocityConsensusHook.
For 12 cores, a value of 24 should be enough, but bumped it to 30 to have some extra buffer.

[#6897]: Tracing Log for fork-aware transaction pool

Replacement of log crate with tracing crate for better logging.

[#6440]: Remove debug message about pruning active leaves

Removed useless debug message

[#7075]: Snowbridge - Ethereum Electra Upgrade Support

Adds support for the Ethereum Electra hard-fork in the Ethereum light client. Maintains backwards compatibility with the current Deneb hard-fork.
Relayers should update to the latest binary to support sending Electra consensus updates.

[#7073]: Implement NetworkRequest for litep2p

Description

Implements NetworkRequest::request for litep2p that we need for networking benchmarks

Review Notes

Duplicates implementation for NetworkService

polkadot-sdk/substrate/client/network/src/service.rs

Lines 1186 to 1205 in 5bf9dd2

 async fn request( 

 &self, 

 target: sc_network_types::PeerId, 

 protocol: ProtocolName, 

 request: Vec<u8>, 

 fallback_request: Option<(Vec<u8>, ProtocolName)>, 

 connect: IfDisconnected, 

 ) -> Result<(Vec<u8>, ProtocolName), RequestFailure> { 

 let (tx, rx) = oneshot::channel(); 

 self.start_request(target.into(), protocol, request, fallback_request, tx, connect); 

 match rx.await { 

 Ok(v) => v, 

 // The channel can only be closed if the network worker no longer exists. If the 

 // network worker no longer exists, then all connections to `target` are necessarily 

 // closed, and we legitimately report this situation as a "ConnectionClosed". 

 Err(_) => Err(RequestFailure::Network(OutboundFailure::ConnectionClosed)), 

 } 

 }

[#6628]: Remove ReportCollator message

Remove unused message ReportCollator and test related to this message on the collator protocol validator side.

[#7675]: Bandersnatch hot fix version

Essentially, this locks bandersnatch_vrfs to a specific branch of a repository I control.
This is a temporary workaround to avoid issues like
#7653 until
#7669 is ready.

[#7494]: Enhance libp2p logging targets for granular control

This PR modifies the libp2p networking-specific log targets for granular control (e.g., just enabling trace for req-resp).

Previously, all logs were outputted to sub-libp2p target, flooding the log messages on busy validators.

Discovery: sub-libp2p::discovery;
Notification/behaviour: sub-libp2p::notification::behaviour;
Notification/handler: sub-libp2p::notification::handler;
Notification/service: sub-libp2p::notification::service;
Notification/upgrade: sub-libp2p::notification::upgrade;
Request response: sub-libp2p::request-response.

[#7479]: omni-node: add offchain worker

Added support for offchain worker to omni-node-lib for both aura and manual seal nodes.

[#7195]: Unify Import verifier usage across parachain template and omninode

In polkadot-omni-node block import pipeline it uses default aura verifier without checking equivocation,
This Pr replaces the check with full verification with equivocation like in parachain template block import

[#6889]: Remove polkadot-omni-node-lib unused dependency

Removed an unused dependency for polkadot-omni-node-lib.

[#6248]: Upgrade libp2p to 0.54.1

Upgrade libp2p from 0.52.4 to 0.54.1

[#6400]: Remove network starter that is no longer needed

Description

This seems to be an old artifact of the long closed paritytech/substrate#6827 that I noticed when working on related code earlier.

Integration

NetworkStarter was removed, simply remove its usage:

-let (network, system_rpc_tx, tx_handler_controller, start_network, sync_service) =
+let (network, system_rpc_tx, tx_handler_controller, sync_service) =
    build_network(BuildNetworkParams {
...
-start_network.start_network();

Review Notes

Changes are trivial, the only reason for this to not be accepted is if it is desired to not start network automatically for whatever reason, in which case the description of network starter needs to change.

Checklist

My PR includes a detailed description as outlined in the "Description" and its two subsections above.
My PR follows the labeling requirements of this project (at minimum one label for T required)
- External contributors: ask maintainers to put the right label on your PR.

[#6481]: slot-based-collator: Implement dedicated block import

The SlotBasedBlockImport job is to collect the storage proofs of all blocks getting imported. These storage proofs alongside the block are being forwarded to the collation task. Right now they are just being thrown away. More logic will follow later. Basically this will be required to include multiple blocks into one PoV which will then be done by the collation task.

[#6452]: elastic scaling RFC 103 end-to-end tests

Adds end-to-end zombienet-sdk tests for elastic scaling using the RFC103 implementation.
Only notable user-facing change is that the default chain configurations of westend and rococo
now enable by default the CandidateReceiptV2 node feature.

[#7011]: sync: Send already connected peers to new subscribers

Introduce SyncEvent::InitialPeers message sent to new subscribers to allow them correctly tracking sync peers. This resolves a race condition described in #6573 (comment).

Fixes #6573.

[#7504]: Enable approval-voting-parallel by default on polkadot

Enable approval-voting-parallel by default on polkadot

[#6553]: Ensure sync event is processed on unknown peer roles

The GossipEngine::poll_next implementation polls both the notification_service and the sync_event_stream.
This PR ensures both events are processed gracefully.

[#6417]: fix prospective-parachains best backable chain reversion bug

Fixes a bug in the prospective-parachains subsystem that prevented proper best backable chain reorg.

[#6867]: Deprecate ParaBackingState API

Deprecates the para_backing_state API. Introduces and new backing_constraints API that can be used
together with existing candidates_pending_availability to retrieve the same information provided by
para_backing_state.

[#6865]: Rename PanicInfo to PanicHookInfo

Starting with Rust 1.82 PanicInfo is deprecated and will throw warnings when used. The new type is available since Rust 1.81 and should be available on our CI.

[#6963]: grandpa: Ensure `WarpProof` stays in its limits

There was the chance that a WarpProof was bigger than the maximum warp sync proof size. This could have happened when inserting the last justification, which then may pushed the total proof size above the maximum. The solution is simply to ensure that the last justfication also fits into the limits.

Close: #6957

[#6163]: Expose more syncing types to enable custom syncing strategy

Exposes additional syncing types to facilitate the development of a custom syncing strategy.

[#5855]: Remove feature `test-helpers` from sc-service

Removes feature test-helpers from sc-service.

[#6913]: Enable approval-voting-parallel by default on polkadot

Enable approval-voting-parallel by default on polkadot

[#6450]: Add omni-node checks for runtime parachain compatibility

OmniNode parses runtime metadata and checks against the existence of cumulus-pallet-parachain-system
and frame-system, by filtering pallets by names: ParachainSystem and System. It also checks the
frame-system pallet storage Number type, and then uses it to configure AURA if u32 or u64.

[#6534]: Forward logging directives to Polkadot workers

This pull request forward all the logging directives given to the node via RUST_LOG or -l to the workers, instead of only forwarding RUST_LOG.

[#6661]: `txpool api`: `remove_invalid` call improved

Currently the transaction which is reported as invalid by a block builder (or removed_invalid by other components) is silently skipped. This PR improves this behavior. The transaction pool report_invalid function now accepts optional error associated with every reported transaction, and also the optional block hash which both provide hints how reported invalid transaction shall be handled. Depending on error, the transaction pool can decide if transaction shall be removed from the view only or entirely from the pool. Invalid event will be dispatched if required.

[#7488]: Increase litep2p keep-alive to 10 seconds to mirror libp2p

Increase litep2p keep-alive to 10 seconds to mirror libp2p behavior.

[#6521]: Pure state sync refactoring (part-2)

This is the last part of the pure refactoring of state sync, focusing on encapsulating StateSyncMetadata as a separate entity.

[#6528]: TransactionPool API uses async_trait

This PR refactors TransactionPool API to use async_trait, replacing the Pin<Box<...>> pattern. This should improve readability and maintainability.

The change is not altering any functionality.

[#6832]: Remove collation-generation subsystem from validator nodes

Collation-generation is only needed for Collators, and therefore not needed for validators

[#6768]: `basic-authorship`: debug level is now less spammy

The debug level in sc-basic-authorship is now less spammy. Previously it was outputing logs per individual transactions. It made quite hard to follow the logs (and also generates unneeded traffic in grafana).

Now debug level only show some internal details, without spamming output with per-transaction logs. They were moved to trace level.

I also added the EndProposingReason to the summary INFO message. This allows us to know what was the block limit (which is very useful for debugging).

[#5703]: Properly handle block gap created by fast sync

Implements support for handling block gaps generated during fast sync. This includes managing the creation,
updating, and removal of block gaps.
Note that this feature is not fully activated until the body attribute is removed from the LightState
block request in chain sync, which will occur after the issue #5406 is resolved.

[#7505]: `fatxpool`: transaction statuses metrics added

This PR introduces a new mechanism to capture and report Prometheus metrics related to timings of transaction
lifecycle events, which are currently not available. By exposing these timings, we aim to augment transaction-pool
reliability dashboards and extend existing Grafana boards.

A new unknown_from_block_import_txs metric is also introduced. It provides the number of transactions in imported
block which are not known to the node's transaction pool. It allows to monitor alignment of transaction pools
across the nodes in the network.

Changelog for `Runtime Dev`

ℹ️ These changes are relevant to: All of those who rely on the runtime. A parachain team that is using a pallet. A DApp that is using a pallet. These are people who care about the protocol (WASM, not the meta-protocol (client).)

[#6695]: [pallet-revive] bugfix decoding 64bit args in the decoder

The argument index of the next argument is dictated by the size of the current one.

[#7614]: [pallet-revive] tracing improvements

Various pallet-revive improvements

add check for precompiles addresses,
So we can easily identified which one are being called and not supported yet
fixes debug_call for revert call
If a call revert we still want to get the traces for that call, that matches geth behaviors, diff tests will be added to the test suite for this
fixes traces for staticcall
The call type was not always being reported properly.

[#3926]: Introduce pallet-asset-rewards

Introduce pallet-asset-rewards, which allows accounts to be rewarded for freezing fungible
tokens. The motivation for creating this pallet is to allow incentivising LPs.
See the pallet docs for more info about the pallet.

[#7492]: Make `pallet-bridge-rewards` generic over `RewardKind`

The PR enhances the pallet-bridge-rewards by making it generic over the Reward type (previously hardcoded as RewardsAccountParams). This modification allows the pallet to support multiple reward types (e.g., P/K bridge, Snowbridge), increasing its flexibility and applicability across various bridge scenarios.

Other pallets can register rewards using bp_relayers::RewardLedger, which is implemented by the rewards pallet. The runtime can then be configured with different mechanisms for paying/claiming rewards via bp_relayers::PaymentProcedure (e.g., see the pub struct BridgeRewardPayer; implementation for BridgeHubWestend).

[#6673]: chain-spec-guide-runtime: path to wasm blob fixed

In chain-spec-guide-runtime crate's tests, there was assumption that release version of wasm blob exists. This PR uses chain_spec_guide_runtime::runtime::WASM_BINARY_PATH const to use correct path to runtime blob.

[#6059]: [mq pallet] Custom next queue selectors

Changes:

Expose a force_set_head function from the MessageQueue pallet via a new trait: ForceSetHead. This can be used to force the MQ pallet to process this queue next.
The change only exposes an internal function through a trait, no audit is required.

Context

For the Asset Hub Migration (AHM) we need a mechanism to prioritize the inbound upward messages and the inbound downward messages on the AH. To achieve this, a minimal (and no breaking) change is done to the MQ pallet in the form of adding the force_set_head function.

An example use of how to achieve prioritization is then demonstrated in integration_test.rs::AhmPrioritizer. Normally, all queues are scheduled round-robin like this:

| Relay | Para(1) | Para(2) | ... | Relay | ...

The prioritizer listens to changes to its queue and triggers if either:

The queue processed in the last block (to keep the general round-robin scheduling)
The queue did not process since n blocks (to prevent starvation if there are too many other queues)

In either situation, it schedules the queue for a streak of three consecutive blocks, such that it would become:

It basically transforms the round-robin into an elongated round robin. Although different strategies can be injected into the pallet at runtime, this one seems to strike a good balance between general service level and prioritization.

[#7324]: Replace derivative dependency with derive-where

The derivative crate, previously used to derive basic traits for structs with
generics or enums, is no longer actively maintained. It has been replaced with
the derive-where crate, which offers a more straightforward syntax while
providing the same features as derivative.

[#6565]: pallet_revive: Switch to 64bit RISC-V

This PR updates pallet_revive to the newest PolkaVM version and adapts the test fixtures and syscall interface to work under 64bit.

Please note that after this PR no 32bit contracts can be deployed (they will be rejected at deploy time). Pre-deployed 32bit contracts are now considered defunct since we changes how parameters are passed for functions with more than 6 arguments.

Fixtures

The fixtures are now built for the 64bit target. I also removed the temporary directory mechanism that triggered a full rebuild every time. It also makes it easier to find the compiled fixtures since they are now always in target/pallet-revive-fixtures.

Syscall interface

Passing pointer

Registers and pointers are now 64bit wide. This allows us to pass u64 arguments in a single register. Before we needed two registers to pass them. This means that just as before we need one register per pointer we pass. We keep pointers as u32 argument by truncating the register. This is done since the memory space of PolkaVM is 32bit.

Functions with more than 6 arguments

We only have 6 registers to pass arguments. This is why we pass a pointer to a struct when we need more than 6. Before this PR we expected a packed struct and interpreted it as SCALE encoded tuple. However, this was buggy because the MaxEncodedLen returned something that was larger than the packed size of the structure. This wasn't a problem before. But now the memory space changed in a way that things were placed at the edges of the memory space and those extra bytes lead to an out of bound access.

This is why this PR drops SCALE and expects the arguments to be passed as a pointer to a C aligned struct. This avoids unaligned accesses. However, revive needs to adapt its codegen to properly align the structure fields.

TODO

Add multi block migration that wipes all existing contracts as we made breaking changes to the syscall interface

[#6435]: frame-benchmarking: Use correct components for pallet instances

When benchmarking multiple instances of the same pallet, each instance was executed with the components of all instances. While actually each instance should only be executed with the components generated for the particular instance. The problem here was that in the runtime only the pallet-name was used to determine if a certain pallet should be benchmarked. When using instances, the pallet name is the same for both of these instances. The solution is to also take the instance name into account.

The fix requires to change the Benchmark runtime api to also take the instance. The node side is written in a backwards compatible way to also support runtimes which do not yet support the instance parameter.

[#6301]: migrate pallet-nft-fractionalization to benchmarking v2 syntax

Migrates pallet-nft-fractionalization to benchmarking v2 syntax.

Part of:

#6202

[#6509]: Migrate pallet-democracy benchmark to v2

"Part of issue #6202."

[#7048]: [pallet-salary] Migrate to using frame umbrella crate

This PR migrates the pallet-salary to use the FRAME umbrella crate. This is part of the ongoing effort to migrate all pallets to use the FRAME umbrella crate. The effort is tracked here.

[#6890]: Alter semantic meaning of 0 in metering limits of EVM contract calls

A limit of 0, for gas meters and storage meters, no longer has the meaning of unlimited metering.

[#7251]: [pallet-revive] eth-rpc error logging

Log error instead of failing with an error when block processing fails

[#6844]: pallet-revive: disable host functions unused in solidity PolkaVM compiler

Disables host functions in contracts that are not enabled in solidity PolkaVM compiler to reduce surface of possible attack vectors.

[#6367]: Refactor pallet society

Derives MaxEncodedLen implementation for stored types and removes without_storage_info attribute.
Migrates benchmarks from v1 to v2 API.

[#7580]: implement web3_clientVersion

Implements the web3_clientVersion method. This is a common requirement for external Ethereum libraries when querying a client.

Reference issue with more details: paritytech/contract-issues#26.

[#5363]: [pallet-xcm] waive transport fees based on XcmConfig

pallet-xcm::send() no longer implicitly waives transport fees for the local root location,
but instead relies on xcm_executor::Config::FeeManager to determine whether certain locations have free transport.

🚨 Warning: 🚨 If your chain relies on free transport for local root, please make
sure to add Location::here() to the waived-fee locations in your configured xcm_executor::Config::FeeManager.

[#6741]: pallet-revive: Adjust error handling of sub calls

We were trapping the host context in case a sub call was exhausting the storage deposit limit set for this sub call. This prevents the caller from handling this error. In this PR we added a new error code that is returned when either gas or storage deposit limit is exhausted by the sub call.

We also remove the longer used NotCallable error. No longer used because this is no longer an error: It will just be a balance transfer.

We also make set_code_hash infallible to be consistent with other host functions which just trap on any error condition.

[#7319]: [pallet-revive] pack exceeding syscall arguments into registers

This PR changes how we call runtime API methods with more than 6 arguments: They are no longer spilled to the stack but packed into registers instead. Pointers are 32 bit wide so we can pack two of them into a single 64 bit register. Since we mostly pass pointers, this technique effectively increases the number of arguments we can pass using the available registers.

To make this work for instantiate too we now pass the code hash and the call data in the same buffer, akin to how the create family opcodes work in the EVM. The code hash is fixed in size, implying the start of the constructor call data.

[#7176]: [pallet-revive] Bump asset-hub westend spec version

Bump asset-hub westend spec version

[#7763]: [staking] Currency Migration and Overstake fix

Fixes Currency to fungible migration with force-unstake of partial unbonding accounts.
Removes the extrinsic withdraw_overstake which is not useful post fungibe migration of pallet-staking.

[#5723]: Adds `BlockNumberProvider` in multisig, proxy and nft pallets

This PR adds the ability for these pallets to specify their source of the block number.
This is useful when these pallets are migrated from the relay chain to a parachain and
vice versa.

This change is backwards compatible:

If the BlockNumberProvider continues to use the system pallet's block number
When a pallet deployed on the relay chain is moved to a parachain, but still uses the
relay chain's block number

However, we would need migrations if the deployed pallets are upgraded on an existing parachain,
and the BlockNumberProvider uses the relay chain block number.

[#6321]: Utility call fallback

This PR adds the if_else call to pallet-utility
enabling an error fallback when the main call is unsuccessful.

[#7263]: Fix `frame-benchmarking-cli` not buildable without rocksdb

Description

The frame-benchmarking-cli crate has not been buildable without the rocksdb feature since version 1.17.0.

Error:

self.database()?.unwrap_or(Database::RocksDb),
                             ^^^^^^^ variant or associated item not found in `Database`

This issue is also related to the rocksdb feature bleeding (#3793), where the rocksdb feature was always activated even when compiling this crate with --no-default-features.

Fix:

Resolved the error by choosing paritydb as the default database when compiled without the rocksdb feature.
Fixed the issue where the sc-cli crate's rocksdb feature was always active, even compiling frame-benchmarking-cli with --no-default-features.

Review Notes

Fix the crate to be built without rocksdb, not intended to solve #3793.

[#6411]: Support more types in TypeWithDefault

This PR supports more integer types to be used with TypeWithDefault and makes TypeWithDefault<u8/u16/u32, ..>: BaseArithmetic satisfied

[#7507]: Fix compilation warnings

This should fix some compilation warnings discovered under rustc 1.83

[#7690]: Fix DryRunApi client-facing XCM versions

Fixes #7413

This PR updates the DryRunApi. The signature of the dry_run_call is changed, and the XCM version of the return values of dry_run_xcm now follows the version of the input XCM program.

It also fixes xcmp-queue's Router's clear_messages: the channel details first_index and last_index are reset when clearing.

[#7418]: Refactor

This PR contains a small refactor in the logic of #[benchmarks] so if a where clause is included the expanded code set the bound T:Config inside the where clause

[#7928]: Fix pallet-revive-fixtures build.rs

Fix pallet-revive-uapi resolution when building pallet-revive-fixtures
contracts

[#7969]: Fix XCM Barrier Rejection Handling to Return Incomplete with Weight

This PR addresses an issue with the handling of message execution when blocked by the barrier. Instead of returning an Outcome::Error, we modify the behaviour to return Outcome::Incomplete, which includes the weight consumed up to the point of rejection and the error that caused the blockage.

This change ensures more accurate weight tracking during message execution, even when interrupted. It improves resource management and aligns the XCM executor’s behaviour with better error handling practices.

[#6917]: Remove unused dependencies from pallet_revive

Removing apparently unused dependencies from pallet_revive and related crates.

[#7378]: fix pre-dispatch PoV underweight for ParasInherent

This should fix the error log related to PoV pre-dispatch weight being lower than post-dispatch for ParasInherent:

ERROR tokio-runtime-worker runtime::frame-support: Post dispatch weight is greater than pre dispatch weight. Pre dispatch weight may underestimating the actual weight. Greater post dispatch weight components are ignored.
                                        Pre dispatch weight: Weight { ref_time: 47793353978, proof_size: 1019 },
                                        Post dispatch weight: Weight { ref_time: 5030321719, proof_size: 135395 }

[#6604]: dmp: Check that the para exist before delivering a message

Ensure that a para exists before trying to deliver a message to it.
Besides that ensure_successful_delivery function is added to SendXcm. This function
should be used by benchmarks to ensure that the delivery of a Xcm will work in the benchmark.

[#7929]: [pallet-revive] backports 2503

backport revive PRs:

#7928 [pallet-revive] Fix pallet-revive-fixtures build.rs
#7879 [pallet-revive] Support blocktag in eth_getLogs RPC
#7848 [pallet-revive] Add support for eip1898 block notation
#7844 [pallet-revive] fixture as dev dep
#7827 [revive-rpc] allow using legacy data field
#7810 [pallet-revive] precompiles 2->9
bump asset-hub-westend spec version

[#7282]: AHM Multi-block staking election pallet

Multi Block Election Pallet

This PR adds the first iteration of the multi-block staking pallet.

From this point onwards, the staking and its election provider pallets are being customized to work in AssetHub. While usage in solo-chains is still possible, it is not longer the main focus of this pallet. For a safer usage, please fork and user an older version of this pallet.

[#7254]: deprecate AsyncBackingParams

Removes all usage of the static async backing params, replacing them with dynamically computed equivalent values (based on the claim queue and scheduling lookahead).

[#7093]: initial docify readme with some content #6333

Docifying the README.MD under templates/parachain by adding a Docify.
Also Adding the Cargo.toml under the same folder, essentially making it a crate as Docify acts
for Readmes only under the same crate.

[#7662]: pallet_revive: Change address derivation to use hashing

Motivation

Internal auditors recommended to not truncate Polkadot Addresses when deriving Ethereum addresses from it. Reasoning is that they are raw public keys where truncating could lead to collisions when weaknesses in those curves are discovered in the future. Additionally, some pallets generate account addresses in a way where only the suffix we were truncating contains any entropy. The changes in this PR act as a safe guard against those two points.

Changes made

We change the to_address function to first hash the AccountId32 and then use trailing 20 bytes as AccountId20. If the AccountId32 ends with 12x 0xEE we keep our current behaviour of just truncating those trailing bytes.

Security Discussion

This will allow us to still recover the original AccountId20 because those are constructed by just adding those 12 bytes. Please note that generating an ed25519 key pair where the trailing 12 bytes are 0xEE is theoretically possible as 96bits is not a huge search space. However, this cannot be used as an attack vector. It will merely allow this address to interact with pallet_revive without registering as the fallback account is the same as the actual address. The ultimate vanity address. In practice, this is not relevant since the 0xEE addresses are not valid public keys for sr25519 which is used almost everywhere.

tl:dr: We keep truncating in case of an Ethereum address derived account id. This is safe as those are already derived via keccak. In every other case where we have to assume that the account id might be a public key. Therefore we first hash and then take the trailing bytes.

Do we need a Migration for Westend

No. We changed the name of the mapping. This means the runtime will not try to read the old data. Ethereum keys are unaffected by this change. We just advise people to re-register their AccountId32 in case they need to use it as it is a very small circle of users (just 3 addresses registered). This will not cause disturbance on Westend.

[#6820]: Add XCM benchmarks to collectives-westend

Collectives-westend was using FixedWeightBounds, meaning the same weight per instruction. Added proper benchmarks.

[#7230]: revive: Include immutable storage deposit into the contracts `storage_base_deposit`

This PR is centered around a main fix regarding the base deposit and a bunch of drive by or related fixtures that make sense to resolve in one go. It could be broken down more but I am constantly rebasing this PR and would appreciate getting those fixes in as-one.

Record the deposit for immutable data into the `storage_base_deposit`

The storage_base_deposit are all the deposit a contract has to pay for existing. It included the deposit for its own metadata and a deposit proportional (< 1.0x) to the size of its code. However, the immutable code size was not recorded there. This would lead to the situation where on terminate this portion wouldn't be refunded staying locked into the contract. It would also make the calculation of the deposit changes on set_code_hash more complicated when it updates the immutable data (to be done in #6985). Reason is because it didn't know how much was payed before since the storage prices could have changed in the mean time.

In order for this solution to work I needed to delay the deposit calculation for a new contract for after the contract is done executing is constructor as only then we know the immutable data size. Before, we just charged this eagerly in charge_instantiate before we execute the constructor. Now, we merely send the ED as free balance before the constructor in order to create the account. After the constructor is done we calculate the contract base deposit and charge it. This will make set_code_hash much easier to implement.

As a side effect it is now legal to call set_immutable_data multiple times per constructor (even though I see no reason to do so). It simply overrides the immutable data with the new value. The deposit accounting will be done after the constructor returns (as mentioned above) instead of when setting the immutable data.

Don't pre-charge for reading immutable data

I noticed that we were pre-charging weight for the max allowable immutable data when reading those values and then refunding after read. This is not necessary as we know its length without reading the storage as we store it out of band in contract metadata. This makes reading it free. Less pre-charging less problems.

Remove delegate locking

Fixes #7092

This is also in the spirit of making #6985 easier to implement. The locking complicates set_code_hash as we might need to block settings the code hash when locks exist. Check #7092 for further rationale.

Enforce "no terminate in constructor" eagerly

We used to enforce this rule after the contract execution returned. Now we error out early in the host call. This makes it easier to be sure to argue that a contract info still exists (wasn't terminated) when a constructor successfully returns. All around this his just much simpler than dealing this check.

Moved refcount functions to `CodeInfo`

They never really made sense to exist on Stack. But now with the locking gone this makes even less sense. The refcount is stored inside CodeInfo to lets just move them there.

Set `CodeHashLockupDepositPercent` for test runtime

The test runtime was setting CodeHashLockupDepositPercent to zero. This was trivializing many code paths and excluded them from testing. I set it to 30% which is our default value and fixed up all the tests that broke. This should give us confidence that the lockup doeposit collections properly works.

Reworked the `MockExecutable` to have both a `deploy` and a `call` entry point

This type used for testing could only have either entry points but not both. In order to fix the immutable_data_set_overrides I needed to a new function add_both to MockExecutable that allows to have both entry points. Make sure to make use of it in the future :)

[#7482]: [pallet-revive] rpc - gas used fixes

#7463 follow up with RPC fixes

[#7325]: [pallet-revive] eth-rpc minor fixes

Add option to specify database_url from an environment variable
Add a test-deployment.rs rust script that can be used to test deployment and call of a contract before releasing eth-rpc
Make evm_block non fallible so that it can return an Ok response for older blocks when the runtime API is not available
Update subxt version to integrate changes from paritytech/subxt#1904

[#6937]: [pallet-revive] bump polkavm to 0.18

Update to the latest polkavm version, containing a linker fix I need for revive.

[#7652]: [pallet-revive] ecrecover

Add ECrecover 0x1 precompile and remove the unstable equivalent host function.

[#7307]: Bridges small nits/improvements

This PR introduces a new expected_payload_type parameter to the Bridges assert_complete_bridge_types macro.

[#6502]: sp-trie: correctly avoid panicking when decoding bad compact proofs

"Fixed the check introduced in PR #6486. Now sp-trie correctly avoids panicking when decoding bad compact proofs."

[#7167]: [pallet-revive] Add tracing support (2/2)

Add debug endpoint to eth-rpc for capturing a block or a single transaction traces
Use in-memory DB for non-archive node

See:

PR #7166

[#7721]: revive: Rework the instruction benchmark

Fixes #6157

This fixes the last remaining benchmark that was not correct since it was too low level to be written in Rust. Instead, we opted.

This PR changes the benchmark that determines the scaling from ref_time to PolkaVM Gas by benchmarking the absolute worst case of an instruction: One that causes two cache misses by touching two cache lines.

The Contract itself is designed to be as simple as possible. It does random unaligned reads in a loop until the r (repetition) number is reached. The randomness is fully generated by the host and written to the guests memory before the benchmark is run. This allows the benchmark to determine the influence of one loop iteration via linear regression.

[#7170]: Fix reversed error message in DispatchInfo

Fix error message in DispatchInfo where post-dispatch and pre-dispatch weight was reversed.

[#6220]: Fix metrics not shutting down if there are open connections

Fix prometheus metrics not shutting down if there are open connections

[#6954]: [pallet-revive] implement the gas price API

This PR implements the EVM gas price syscall API method. Currently this is a compile time constant in revive, but in the EVM it is an opcode. Thus we should provide an opcode for this in the pallet.

[#7030]: [core-fellowship] Add permissionless import_member

Changes:

Add call import_member to the core-fellowship pallet.
Move common logic between import and import_member into do_import.

This is a minor change as to not impact UI and downstream integration.

`import_member`

Can be used to induct an arbitrary collective member and is callable by any signed origin. Pays no fees upon success.
This is useful in the case that members did not induct themselves and are idling on their rank.

[#6453]: [pallet-revive] breakdown integration tests

Break down the single integration tests into multiple tests, use keccak-256 for tx.hash

[#6715]: Update Nomination Pool Pallet to Support Block Number Provider

This PR makes the block number provider in the Society pallet configurable so that runtimes can choose between using the system block number or an alternative source like the relay chain’s block number.
If you want to keep the existing behavior, simply set the provider to System. For scenarios that require a different notion of block number—such as using a relay chain number you can select another provider,
ensuring flexibility in how the nomination pools pallet references the current block.

[#7430]: [pallet-revive] fix tracing gas used

Charge the nested gas meter for loading the code of the child contract, so that we can properly associate the gas cost to the child call frame.
Move the enter_child_span and exit_child_span around the do_transaction closure to properly capture all failures
Add missing trace capture for call transfer

[#6461]: [pallet-revive] add support for all eth tx types

Add support for 1559, 4844, and 2930 transaction types

[#7656]: Authorize upgrade tests for testnet runtimes + `execute_as_governance` refactor

This PR contains improved test cases that rely on the governance location as preparation for AHM to capture the state as it is.
It introduces execute_as_governance_call, which can be configured with various governance location setups instead of the hard-coded Location::parent().

[#6290]: Migrate pallet-transaction-storage and pallet-indices to benchmark v2

Part of:
#6202

[#7826]: Add asset-hub-next as a trusted teleporter

Asset Hub Next has been deployed on Westend as parachain 1100, but it's not yet a trusted teleporter.
This minimal PR adds it in stable2412 so that it can be deployed right away without waiting for the rest of the release to be finalised and deployed.

[#6986]: [pallet-mixnet] Migrate to using frame umbrella crate

This PR migrates the pallet-mixnet to use the frame umbrella crate. This is part of the ongoing effort to migrate all pallets to use the frame umbrella crate. The effort is tracked here.

[#6419]: Use the custom target riscv32emac-unknown-none-polkavm

Closes: #6335

[#6926]: [pallet-revive] implement the gas limit API

This PR implements the gas limit API, returning the maximum ref_time per block. Solidity contracts only know a single weight dimension and can use this method to get the block ref_time limit.

[#6681]: update scale-info to 2.11.6

Updates scale-info to 2.11.1 from 2.11.5.
Updated version of scale-info annotates generated code with allow(deprecated)

[#5656]: Use Relay Blocknumber in Pallet Broker

Changing sale_start, interlude_length and leading_length in pallet_broker to use relay chain block numbers instead of parachain block numbers.
Relay chain block numbers are almost deterministic and more future proof.

[#7127]: Forbid v1 descriptors with UMP signals

Adds a check that parachain candidates do not send out UMP signals with v1 descriptors.

[#7582]: Implementation of `ah-client` and `rc-client` staking pallets

This PR introduces the initial structure for pallet-ah-client and pallet-rc-client. These
pallets will reside on the relay chain and AssetHub, respectively, and will manage the interaction
between pallet-session on the relay chain and pallet-staking on AssetHub.
Both pallets are experimental and not intended for production use.

[#7760]: Dynamic uncompressed code size limit

Deprecates node constant VALIDATION_CODE_BOMB_LIMIT and introduces
validation_code_bomb_limit runtime API that computes the maximum
uncompressed code size as the maximum code size multiplied by a
compression ratio of 10.

[#7563]: Bump frame-metadata v16 to 19.0.0

Update to latest version of frame-metadata and merkleized-metadata in order to support pallet view function metadata.

[#6796]: pallet-revive: Remove unused dependencies

The dependency on pallet_balances doesn't seem to be necessary. At least everything compiles for me without it. Removed this dependency and a few others that seem to be left overs.

[#7243]: transfer function Preservation is changed to Expendable in fungible and fungibles adapter

The Preservation of transfer method of fungible and fungibles adapters is changed from Preserve to Expendable. So the behavior of the TransferAsset will be consistent with the WithdrawAsset function, as in fungible and fungibles adapter.

[#6338]: Update Referenda to Support Block Number Provider

This PR makes the referenda pallet uses the relay chain as a block provider for a parachain on a regular schedule.
To migrate existing referenda implementations, simply add type BlockNumberProvider = System to have the same behavior as before.

[#6349]: runtimes: presets are provided as config patches

This PR introduces usage of build_struct_json_patch macro in all
runtimes (also guides) within the code base. It also fixes macro to support
field init shorthand, and Struct Update syntax which were missing in original
implementation.

[#7579]: [AHM] Make pallet types public

Preparation for AHM and making stuff public.

[#7379]: Add support for feature pallet_balances/insecure_zero_ed in benchmarks and testing

Currently benchmarks and tests on pallet_balances would fail when the feature insecure_zero_ed is enabled. This PR allows to run such benchmark and tests keeping into account the fact that accounts would not be deleted when their balance goes below a threshold.

[#6836]: [pallet-revive-eth-rpc] persist eth transaction hash

Add an option to persist EVM transaction hash to a SQL db.
This make it possible to run a full archive ETH RPC node (assuming the substrate node is also a full archive node)

Some queries such as eth_getTransactionByHash, eth_getBlockTransactionCountByHash, and other need to work with a transaction hash index, which is not available in Substrate and need to be stored by the eth-rpc proxy.

The refactoring break down the Client into a BlockInfoProvider and ReceiptProvider

BlockInfoProvider does not need any persistence data, as we can fetch all block info from the source substrate chain
ReceiptProvider comes in two flavor,
- An in memory cache implementation - This is the one we had so far.
- A DB implementation - This one persist rows with the block_hash, the transaction_index and the transaction_hash, so that we can later fetch the block and extrinsic for that receipt and reconstruct the ReceiptInfo object.

[#7164]: [pallet-revive] Remove revive events

Remove all pallet::events except for the ContractEmitted event that is emitted by contracts

[#6302]: migrate pallet-nomination-pool-benchmarking to benchmarking syntax v2

migrate pallet-nomination-pool-benchmarking to benchmarking syntax v2

[#2072]: Return iterator in pallet_referenda::TracksInfo::tracks

Change the return type of the trait method pallet_referenda::TracksInfo::tracks to return an
iterator of Cow<'static, Tracks<_, _, _>> instead of a static slice in order to support more
flexible implementations that can define referenda tracks dynamically.

[#7377]: Add missing events to nomination pool extrinsics

Introduces events to extrinsics from pallet_nomination_pools that previously had none:

set_metadata
nominate
chill
set_configs
set_claim_permission

[#7676]: [pallet-revive] tracing should wrap around call stack execution

Fix tracing should wrap around the entire call stack execution

[#7376]: Documentation update for weight.

Document the usage of #[pallet::call(weight = <T as Config>::WeightInfo)] within FRAME macros.
This update enhances the documentation for #[pallet::call] and #[pallet::weight], providing
examples and clarifying weight specifications for dispatchable functions, ensuring consistency
with existing guidelines.

[#7568]: pallet-revive: Fix the contract size related benchmarks

Partly addresses #6157

The benchmarks measuring the impact of contract sizes on calling or instantiating a contract were bogus because they needed to be written in assembly in order to tightly control the basic block size.

This fixes the benchmarks for:

call_with_code_per_byte
upload_code
instantiate_with_code

And adds a new benchmark that accounts for the fact that the interpreter will always compile whole basic blocks:

basic_block_compilation

After this PR only the weight we assign to instructions need to be addressed.

[#6562]: Hide nonce implementation details in metadata

Use custom implementation of TypeInfo for TypeWithDefault to show inner value's type info.
This should bring back nonce to u64 in metadata.

[#7327]: Correctly register the weight n `set_validation_data` in `cumulus-pallet-parachain-system`

The actual weight of the call was register as a refund, but the pre-dispatch weight is 0,
and we can't refund from 0. Now the actual weight is registered manually instead of ignored.

[#7424]: Bounded Slashing: Paginated Offence Processing & Slash Application

This PR refactors the slashing mechanism in pallet-staking to be bounded by introducing paged offence processing and paged slash application.

  ### Key Changes
  - Offences are queued instead of being processed immediately.
  - Slashes are computed in pages, stored as a `StorageDoubleMap` with `(Validator, SlashFraction, PageIndex)` to uniquely identify them.
  - Slashes are applied incrementally across multiple blocks instead of a single unbounded operation.
  - New storage items: `OffenceQueue`, `ProcessingOffence`, `OffenceQueueEras`.
  - Updated API for cancelling and applying slashes.
  - Preliminary benchmarks added; further optimizations planned.

  This enables staking slashing to scale efficiently and removes a major blocker for staking migration to a parachain (AH).

[#7848]: [pallet-revive] Add support for eip1898 block notation

[pallet-revive] Add support for eip1898 block notation
https://eips.ethereum.org/EIPS/eip-1898

[#7206]: Add an extra_constant to pallet-collator-selection

Allows to query collator-selection's pot account via extra constant.

[#6393]: [pallet-revive] adjust fee dry-run calculation

Fix bare_eth_transact so that it estimate more precisely the transaction fee
Add some context to the build.rs to make it easier to troubleshoot errors
Add TransactionBuilder for the RPC tests.
Tweaked some error message, We will need to wait for the next subxt release to properly downcast some errors and
adopt MM error code (https://eips.ethereum.org/EIPS/eip-1474#error-codes)

[#6928]: [Backport] Version bumps and `prdocs` reordering form 2412

This PR includes backport of the regular version bumps and prdocs reordering from the stable2412 branch back ro master

[#7281]: [pallet-revive] fix eth fee estimation

Fix EVM fee cost estimation.
The current estimation was shown in Native and not EVM decimal currency.

[#6267]: Allow configurable number of genesis accounts with specified balances for benchmarking.

This pull request adds an additional field dev_accounts to the GenesisConfig
of the balances pallet, feature gated by runtime-benchmarks.

Bringing about an abitrary number of derived dev accounts when building the genesis
state. Runtime developers should supply a derivation path that includes an index placeholder
(i.e. "//Sender/{}") to generate multiple accounts from the same root in a consistent
manner.

[#6623]: Update Society Pallet to Support Block Number Provider

[#7571]: frame-benchmarking: Improve macro hygiene

Improve macro hygiene of benchmarking macros.

[#7493]: [pallet-revive] fix eth-rpc indexing

Fix a deadlock on the RWLock cache
Remove eth-indexer, we won't need it anymore, the indexing will be started from within eth-rpc directly

[#6184]: Remove pallet::getter from pallet-staking

This PR removes all pallet::getter occurrences from pallet-staking and replaces them with explicit implementations.
It also adds tests to verify that retrieval of affected entities works as expected so via storage::getter.

[#7922]: Use non-native token to benchmark on asset hub

Asset Hub was using the native token for benchmarking xcm instructions. This is not the best since it's cheaper than using something in pallet-assets for example.
Had to remove some restrictive checks from pallet-xcm-benchmarks.

[#7200]: XCM: Deny barrier checks for nested XCMs with specific instructions to be executed on the local chain

This PR improves the validation of nested XCM instructions by introducing a
new barrier, DenyRecursively, which provides more precise control over
instruction denial. Previously, DenyThenTry<Deny, Allow>`` was used, which primarily applied denial rules at the top level. This has now been replaced with DenyThenTry<DenyRecursively, Allow>`, ensuring that both
top-level and nested local instructions are properly checked. This change
enhances the security and predictability of XCM execution by enforcing
consistent denial policies across all levels of message execution. If you
need to deny instructions recursively make sure to change your barrier in
the XCM configuration.

[#7506]: [pallet-revive] Add eth_get_logs

Add support for eth_get_logs rpc method

[#7081]: [pallet-mmr] Migrate to using frame umbrella crate

This PR migrates the pallet-mmr to use the frame umbrella crate. This is part of the ongoing effort to migrate all pallets to use the frame umbrella crate. The effort is tracked here.

[#7198]: [pallet-revive] implement the block author API

This PR implements the block author API method. Runtimes ought to implement it such that it corresponds to the coinbase EVM opcode.

[#6140]: Accurate weight reclaim with frame_system::WeightReclaim and cumulus `StorageWeightReclaim` transaction extensions

Since the introduction of transaction extension, the transaction extension weight is no longer part of base extrinsic weight. As a consequence some weight of transaction extensions are missed when calculating post dispatch weight and reclaiming unused block weight.

For solo chains, in order to reclaim the weight accurately frame_system::WeightReclaim transaction extension must be used at the end of the transaction extension pipeline.

For para chains StorageWeightReclaim in cumulus-primitives-storage-weight-reclaim is deprecated.
A new transaction extension StorageWeightReclaim in cumulus-pallet-weight-reclaim is introduced.
StorageWeightReclaim is meant to be used as a wrapping of the whole transaction extension pipeline, and will take into account all proof size accurately.

The new wrapping transaction extension is used like this:

/// The TransactionExtension to the basic transaction logic.
pub type TxExtension = cumulus_pallet_weight_reclaim::StorageWeightReclaim<
       Runtime,
       (
               frame_system::CheckNonZeroSender<Runtime>,
               frame_system::CheckSpecVersion<Runtime>,
               frame_system::CheckTxVersion<Runtime>,
               frame_system::CheckGenesis<Runtime>,
               frame_system::CheckEra<Runtime>,
               frame_system::CheckNonce<Runtime>,
               pallet_transaction_payment::ChargeTransactionPayment<Runtime>,
               BridgeRejectObsoleteHeadersAndMessages,
               (bridge_to_rococo_config::OnBridgeHubWestendRefundBridgeHubRococoMessages,),
               frame_metadata_hash_extension::CheckMetadataHash<Runtime>,
               frame_system::CheckWeight<Runtime>,
       ),
>;

NOTE: prior to transaction extension, StorageWeightReclaim also missed the some proof size used by other transaction extension prior to itself. This is also fixed by the wrapping StorageWeightReclaim.

[#6896]: pallet-revive: Fix docs.rs

Fixed failing docs.rs build for pallet-revive-uapi by fixing a writing attribute in the manifest (we were using default-target instead of targets)
Removed the macros defining host functions because the cfg attributes introduced in #6866 won't work on them
Added an docs.rs specific attribute so that the unstable-hostfn feature tag will show up on the functions that are guarded behind it.

[#6920]: [pallet-revive] change some getter APIs to return value in register

Call data, return data and code sizes can never exceed u32::MAX; they are also not generic. Hence we know that they are guaranteed to always fit into a 64bit register and revive can just zero extend them into a 256bit integer value. Which is slightly more efficient than passing them on the stack.

[#6608]: [pallet-revive] eth-prc fix geth diff

Add a bunch of differential tests to ensure that responses from eth-rpc matches the one from geth
EVM RPC server will not fail gas_estimation if no gas is specified, I updated pallet-revive to add an extra skip_transfer boolean check to replicate this behavior in our pallet
eth_transact and bare_eth_transact api have been updated to use GenericTransaction directly as this is what is used by eth_estimateGas and eth_call

[#7260]: [eth-indexer] subscribe to finalize blocks instead of best blocks

For eth-indexer, it's probably safer to use subscribe_finalized and index these blocks into the DB rather than subscribe_best

[#6486]: sp-trie: minor fix to avoid panic on badly-constructed proof

"Added a check when decoding encoded proof nodes in sp-trie to avoid panicking when receiving a badly constructed proof, instead erroring out."

[#7729]: [pallet-revive] allow delegate calls to non-contract accounts

This PR changes the behavior of delegate calls when the callee is not a contract account: Instead of returning a CodeNotFound error, this is allowed and the caller observes a successful call with empty output.

The change makes for example the following contract behave the same as on EVM:

contract DelegateCall {
    function delegateToLibrary() external returns (bool) {
        address testAddress = 0x0000000000000000000000000000000000000000;
        (bool success, ) = testAddress.delegatecall(
            abi.encodeWithSignature("test()")
        );
        return success;
    }
}

Closes paritytech/revive#235

[#6503]: xcm: minor fix for compatibility with V4

Following the removal of Rococo, Westend and Wococo from NetworkId, fixed xcm::v5::NetworkId encoding/decoding to be compatible with xcm::v4::NetworkId

[#7451]: omni-node: Adjust manual seal parameters

This PR restores compatibility of older runtimes with the dev mode of omni-node. Before, runtimes built without the changes in #6825 were failing.

[#7407]: Fixes #219

Add a new extrinsic dispatch_as_fallible.

It's almost the same as dispatch_as but check the result of the call.

Closes #219.

And add more unit tests to cover dispatch_as and dispatch_as_fallible.

Polkadot address: 156HGo9setPcU2qhFMVWLkcmtCEGySLwNqa3DaEiYSWtte4Y

[#7590]: [pallet-revive] move exec tests

Moving exec tests into a new file

[#6880]: [pallet-revive] implement the call data copy API

This PR implements the call data copy API by adjusting the input method.

Closes #6770

[#7126]: xcm: Fixes for `UnpaidLocalExporter`

This PR deprecates UnpaidLocalExporter in favor of the new LocalExporter. First, the name is misleading, as it can be used in both paid and unpaid scenarios. Second, it contains a hard-coded channel 0, whereas LocalExporter uses the same algorithm as xcm-exporter.

[#7685]: Introduce filters to restrict accounts from staking

Introduce filters to restrict accounts from staking.
This is useful for restricting certain accounts from staking, for example, accounts staking via pools, and vice
versa.

[#7091]: [pallet-revive] Add new host function `to_account_id`

A new host function to_account_id is added. It allows retrieving the account id for a H160 address.

[#7675]: Bandersnatch hot fix version

Essentially, this locks bandersnatch_vrfs to a specific branch of a repository I control.
This is a temporary workaround to avoid issues like
#7653 until
#7669 is ready.

[#7570]: [pallet-revive] fix subxt version

Cargo.lock change to subxt were rollback
Fixing it and updating it in Cargo.toml so it does not happen again

[#6368]: Migrate inclusion benchmark to v2

Migrate inclusion benchmark to v2.

[#6460]: [pallet-revive] set logs_bloom

Set the logs_bloom in the transaction receipt

[#7069]: Fix defensive! macro to be used in umbrella crates

PR for #7054

Replaced frame_support with $crate from @gui1117 's suggestion to fix the dependency issue

[#6835]: [pallet-revive] implement the call data load API

This PR implements the call data load API akin to how it works on ethereum.

[#6533]: Migrate executor into PolkaVM 0.18.0

Bump polkavm to 0.18.0, and update sc-polkavm-executor to be
compatible with the API changes. In addition, bump also polkavm-derive
and polkavm-linker in order to make sure that the all parts of the
Polkadot SDK use the exact same ABI for .polkavm binaries.

Purely relying on RV32E/RV64E ABI is not possible, as PolkaVM uses a
RISCV-V alike ISA, which is derived from RV32E/RV64E but it is still its
own microarchitecture, i.e. not fully binary compatible.

[#7479]: omni-node: add offchain worker

Added support for offchain worker to omni-node-lib for both aura and manual seal nodes.

[#6459]: Fix version conversion in XcmPaymentApi::query_weight_to_asset_fee.

The query_weight_to_asset_fee function of the XcmPaymentApi was trying
to convert versions in the wrong way.
This resulted in all calls made with lower versions failing.
The version conversion is now done correctly and these same calls will now succeed.

[#6759]: pallet-revive: Statically verify imports on code deployment

Previously, we failed at runtime if an unknown or unstable host function was called. This requires us to keep track of when a host function was added and when a code was deployed. We used the api_version to track at which API version each code was deployed. This made sure that when a new host function was added that old code won't have access to it. This is necessary as otherwise the behavior of a contract that made calls to this previously non existent host function would change from "trap" to "do something".

In this PR we remove the API version. Instead, we statically verify on upload that no non-existent host function is ever used in the code. This will allow us to add new host function later without needing to keep track when they were added.

This simplifies the code and also gives an immediate feedback if unknown host functions are used.

[#6446]: Make pallet-recovery supports `BlockNumberProvider`

pallet-recovery now allows configuring the block provider to be utilized within this pallet. This block is employed for the delay in the recovery process.

A new associated type has been introduced in the Config trait: BlockNumberProvider. This can be assigned to System to maintain the previous behavior, or it can be set to another block number provider, such as RelayChain.

If the block provider is configured with a value different from System, a migration will be necessary for the Recoverable and ActiveRecoveries storage items.

[#6526]: sp-runtime: Be a little bit more functional :D

Some internal refactorings in the Digest code.

[#6792]: Add fallback_max_weight to snowbridge Transact

We removed the require_weight_at_most field and later changed it to fallback_max_weight.
This was to have a fallback when sending a message to v4 chains, which happens in the small time window when chains are upgrading.
We originally put no fallback for a message in snowbridge's inbound queue but we should have one.
This PR adds it.

[#7581]: Move validator disabling logic to pallet-session

This decouples disabling logic from staking, and moves it to session. This ensures validators can be disabled
directly when staking transitions to the system parachain and offences are reported on RC, eliminating
cross-network hops.

[#7203]: pallet_revive: Bump PolkaVM

Update to PolkaVM 0.19. This version renumbers the opcodes in order to be in-line with the grey paper. Hopefully, for the last time. This means that it breaks existing contracts.

[#6311]: Migrate pallet-fast-unstake and pallet-babe benchmark to v2

Migrate pallet-fast-unstake and pallet-babe benchmark to v2

[#6583]: Bump Westend AH

Bump Asset-Hub westend spec version

[#5899]: Remove usage of AccountKeyring

Compared with AccountKeyring, Sr25519Keyring and Ed25519Keyring are more intuitive.
When both Sr25519Keyring and Ed25519Keyring are required, using AccountKeyring bring confusion.
There are two AccountKeyring definitions, it becomes more complex if export two AccountKeyring from frame.

[#6452]: elastic scaling RFC 103 end-to-end tests

[#6981]: [pallet-revive] fix file case

fix #6970

[#7008]: feat(wasm-builder): add support for new `wasm32v1-none` target

Resolves #5777

Previously wasm-builder used hacks such as -Zbuild-std (required rust-src component) and RUSTC_BOOTSTRAP=1 to build WASM runtime without WASM features: sign-ext, multivalue and reference-types, but since Rust 1.84 (will be stable on 9 January, 2025) the situation has improved as there is new wasm32v1-none target that disables all "post-MVP" WASM features except mutable-globals.

Wasm builder requires the following prerequisites for building the WASM binary:

Rust >= 1.68 and Rust < 1.84:
- wasm32-unknown-unknown target
- rust-src component
Rust >= 1.84:
- wasm32v1-none target
- no more -Zbuild-std and RUSTC_BOOTSTRAP=1 hacks and rust-src component requirements!

[#6867]: Deprecate ParaBackingState API

[#7589]: [pallet-revive] rpc add --earliest-receipt-block

Add a cli option to skip searching receipts for blocks older than the specified limit

[#6856]: Enable report_fork_voting()

This PR enables calling report_fork_voting.
In order to do this we needed to also check that the ancestry proof is optimal.

[#6665]: Fix runtime api impl detection by construct runtime

Construct runtime uses autoref-based specialization to fetch the metadata about the implemented runtime apis. This is done to not fail to compile when there are no runtime apis implemented. However, there was an issue with detecting runtime apis when they were implemented in a different file. The problem is solved by moving the trait implemented by impl_runtime_apis! to the metadata ir crate.

Closes: #6659

[#6466]: [pallet-revive] add piggy-bank sol example

This PR update the pallet to use the EVM 18 decimal balance in contracts call and host functions instead of the native balance.

It also updates the js example to add the piggy-bank solidity contract that expose the problem

[#6425]: Introduce `ConstUint` to make dependent types in `DefaultConfig` more adaptable

Introduce ConstUint that is a unified alternative to ConstU8, ConstU16, and
similar types, particularly useful for configuring DefaultConfig in pallets.
It enables configuring the underlying integer for a specific type without the need
to update all dependent types, offering enhanced flexibility in type management.

[#7359]: Improve `set_validation_data` error message.

Adds a more elaborate error message to the error that appears when set_validation_data is missing in a parachain block.

[#5724]: Validator Re-Enabling (master PR)

Implementation of the Stage 3 for the New Disabling Strategy: #4359

This PR changes when an active validator node gets disabled for comitting offences.
When Byzantine Threshold Validators (1/3) are already disabled instead of no longer
disabling the highest offenders will be disabled potentially re-enabling low offenders.

[#6522]: Removes constraint in BlockNumberProvider from treasury

#3970 updated the treasury pallet to support
relay chain block number provider. However, it added a constraint to the BlockNumberProvider
trait to have the same block number type as frame_system:

type BlockNumberProvider: BlockNumberProvider<BlockNumber = BlockNumberFor<Self>>;

This PR removes that constraint and allows the treasury pallet to use any block number type.

[#6621]: Update Conviction Voting Pallet to Support Block Number Provider

This PR makes the block number provider used in the society pallet configurable. Before this PR, society pallet always used the system block number,
with this PR some runtime can opt to use the relay chain block number instead.

[#7109]: Add "run to block" tools

Introduce frame_system::Pallet::run_to_block, frame_system::Pallet::run_to_block_with, and frame_system::RunToBlockHooks to establish a generic run_to_block mechanism for mock tests, minimizing redundant implementations across various pallets.

Closes #299.

[#6908]: [pallet-revive] implement the ref_time_left API

This PR implements the ref_time_left API method. Solidity knows only a single "gas" dimension; Solidity contracts will use this to query the gas left.

[#7318]: revive: Fix compilation of `uapi` crate when `unstable-hostfn` is not set

This regression was introduced with some of the recent PRs. Regression fixed and test added.

[#6310]: Migrate pallet-child-bounties benchmark to v2

Part of:

#6202.

[#7163]: [pallet-revive] Remove debug buffer

Remove the debug_buffer feature

[#6728]: [pallet-revive] eth-rpc add missing tests

Add tests for #6608

fix paritytech/contract-issues#12

[#7194]: [FRAME] `pallet_asset_tx_payment`: replace `AssetId` bound from `Copy` to `Clone`

OnChargeAssetTransaction's associated type AssetId is bounded by Copy which makes it impossible
to use staging_xcm::v4::Location as AssetId. This PR bounds AssetId to Clone instead, which is
more lenient.

[#6450]: Add omni-node checks for runtime parachain compatibility

[#7383]: Bridges small nits/improvements

This PR contains small fixes and backwards compatibility issues identified during work on the larger PR: #6906.

[#7414]: [pallet-revive] do not trap the caller on instantiations with duplicate contracts

This PR changes the behavior of instantiate when the resulting contract address already exists (because the caller tried to instantiate the same contract with the same salt multiple times): Instead of trapping the caller, return an error code.

Solidity allows catching this, which doesn't work if we are trapping the caller. For example, the change makes the following snippet work:

try new Foo{salt: hex"00"}() returns (Foo) {
    // Instantiation was successful (contract address was free and constructor did not revert)
} catch {
    // This branch is expected to be taken if the instantiation failed because of a duplicate salt
}

[#4722]: Implement pallet view functions

Read-only view functions can now be defined on pallets. These functions provide an interface for querying state,
from both outside and inside the runtime. Common queries can be defined on pallets, without users having to
access the storage directly.

[#6866]: Refactor `pallet-revive-uapi` pallet

Puts unstable host functions in uapi under unstable-api feature while moving those functions after stable functions.

[#7784]: [pallet-revive] block.timestamps should return seconds

In solidity block.timestamp should be expressed in seconds
see https://docs.soliditylang.org/en/latest/units-and-global-variables.html#block-and-transaction-properties

[#4529]: Removed `pallet::getter` usage from pallet-grandpa

This PR removed the pallet::getters from pallet-grandpa.
The syntax StorageItem::<T, I>::get() should be used instead

[#7889]: Remove execute_with_origin implementation in the XCM executor

The XCM executor will not support the ExecuteWithOrigin instruction from the start.
It might be added later when more time can be spent on it.

[#6034]: Adds multi-block election types and refactors current single logic to support it

This PR adds election types and structs required to run a multi-block election. In addition,
it modifies EPM, staking pallet and all dependent pallets and logic to use the multi-block types.

[#7043]: Remove usage of `sp-std` from Substrate

Description

This PR removes usage of deprecated sp-std from Substrate. (following PR of #5010)

Integration

This PR doesn't remove re-exported sp_std from any crates yet, so downstream projects using re-exported sp_std will not be affected.

Review Notes

The existing code using sp-std is refactored to use alloc and core directly. The key-value maps are instantiated from an array of tuples directly instead of using sp_std::map! macro.

This PR replaces sp_std::Writer, a helper type for using Vec<u8> with core::fmt::Write trait, with alloc::string::String.

[#7412]: Pallet view functions: improve metadata, API docs and testing

refactor view functions metadata according to #6833 in preparation for V16, and move them to pallet-level metadata
add view_functions_experimental macro to pallet_macros with API docs
improve UI testing for view functions

[#7695]: XCM: Process PayFees only once

The PayFees instruction should only ever be used once. If it's used more than once, it's just a noop.

[#6857]: [pallet-revive] implement the call data size API

This PR adds an API method to query the contract call data input size.

Part of #6770

[#7562]: pallet-revive: Add env var to allow skipping of validation for testing

When trying to reproduce bugs we sometimes need to deploy code that wouldn't pass validation. This PR adds a new environment variable REVIVE_SKIP_VALIDATION that when set will skip all validation except the contract blob size limit.

Please note that this only applies to when the pallet is compiled for std and hence will never be part of on-chain.

[#7879]: [pallet-revive] Support blocktag in eth_getLogs RPC

Support "latest" blocktag in ethGetLogs from_block and to_block parameters

[#7086]: [pallet-revive] Fix `caller_is_root` return value

The return type of the host function caller_is_root was denoted as u32 in pallet_revive_uapi. This PR fixes the return type to bool. As a drive-by, the PR re-exports pallet_revive::exec::Origin to extend what can be tested externally.

[#6111]: [pallet-revive] Update delegate_call to accept address and weight

Enhance the delegate_call function to accept an address target parameter instead of a code_hash.
This allows direct identification of the target contract using the provided address.
Additionally, introduce parameters for specifying a customizable ref_time limit and proof_size limit,
thereby improving flexibility and control during contract interactions.

[#7463]: [pallet-revive] tx fee fixes

Apply some fixes to properly estimate ethereum tx fees:

Set the extension_weight on the dispatch_info to properly calculate the fee with pallet_transaction_payment
Expose the gas_price through Runtime API, just in case we decide to tweak the value in future updates, it should be read from the chain rather than be a shared constant exposed by the crate
add a evm_gas_to_fee utility function to properly convert gas to substrate fee
Fix some minor gas encoding for edge cases

[#6743]: umbrella: Remove `pallet-revive-fixtures`

No need to have them in the umbrella crate also by having them in the umbrella crate they are bleeding into the normal build.

[#5990]: On-demand credits

The PR implements functionality on the relay chain for purchasing on-demand
Coretime using credits. This means on-demand Coretime should no longer be
purchased with the relay chain balance but rather with credits acquired
on the Coretime chain. The extrinsic to use for purchasing Coretime is
place_order_with_credits. It is worth noting that the PR also introduces
a minimum credit purchase requirement to prevent potential attacks.

[#7441]: Update Scheduler to have a configurable block number provider

This PR makes pallet_scheduler configurable by introducing BlockNumberProvider in
pallet_scheduler::Config. Instead of relying solely on
frame_system::Pallet::<T>::block_number(), the scheduler can now use any block number source,
including external providers like the relay chain.

Parachains can continue using frame_system::Pallet::<Runtime> without issue. To retain the
previous behavior, set BlockNumberProvider to frame_system::Pallet::<Runtime>.

[#6964]: [pallet-revive] implement the base fee API

This PR implements the base fee syscall API method. Currently this is implemented as a compile time constant in the revive compiler, returning 0. However, since this is an opocde, if we ever need to implement it for compatibility reasons with EIP-1559, it would break already deployed contracts. Thus we provide a syscall method instead.

[#7177]: Make frame crate not experimental

Frame crate may still be unstable, but it is no longer feature gated by the feature experimental.

[#7046]: adding warning when using default substrateWeight in production

PR for #3581
Added a cfg to show a deprecated warning message when using std

[#7423]: Fix issue with InitiateTransfer and UnpaidExecution

Fix issue where setting the remote_fees field of InitiateTransfer to None could lead to unintended bypassing of fees in certain conditions. UnpaidExecution is now appended after origin alteration. If planning to use UnpaidExecution, you need to set preserve_origin = true.
The AllowExplicitUnpaidExecutionFrom barrier now allows instructions for receiving funds before origin altering instructions before the actual UnpaidExecution. It takes a new generic, Aliasers, needed for executing AliasOrigin to see if the effective origin is allowed to use UnpaidExecution. This should be set to the same value as Aliasers in the XCM configuration.

[#6439]: pallet-membership: Do not verify the `MembershipChanged` in bechmarks

There is no need to verify in the pallet-membership benchmark that the MemembershipChanged implementation works as the pallet thinks it should work. If you for example set it to (), get_prime() will always return None.

TLDR: Remove the checks of MembershipChanged in the benchmarks to support any kind of implementation.

[#6689]: [pallet-revive] Update gas encoding

Update the current approach to attach the ref_time, pov and deposit parameters to an Ethereum transaction.
Previously, these three parameters were passed along with the signed payload, and the fees resulting from gas × gas_price were checked to ensure they matched the actual fees paid by the user for the extrinsic
This approach unfortunately can be attacked. A malicious actor could force such a transaction to fail by injecting low values for some of these extra parameters as they are not part of the signed payload.
The new approach encodes these 3 extra parameters in the lower digits of the transaction gas, using the log2 of the actual values to encode each components on 2 digits

[#7040]: [pallet-node-authorization] Migrate to using frame umbrella crate

This PR migrates the pallet-node-authorization to use the frame umbrella crate. This is part of the ongoing effort to migrate all pallets to use the frame umbrella crate. The effort is tracked here.

[#6544]: Add and test events to conviction voting pallet

Add event for the unlocking of an expired conviction vote's funds, and test recently added
voting events.

[#7844]: [pallet-revive] Update fixture build script

Update the fixture build script so that it can be built from crates.io registry

[#7365]: Use checked math in frame-balances named_reserve

This PR modifies named_reserve() in frame-balances to use checked math instead of defensive saturating math.

The use of saturating math relies on the assumption that the value will always fit in u128::MAX. However, there is nothing preventing the implementing pallet from passing a larger value which overflows. This can happen if the implementing pallet does not validate user input and instead relies on named_reserve() to return an error (this saves an additional read)

This is not a security concern, as the method will subsequently return an error thanks to <Self as ReservableCurrency<_>>::reserve(who, value)?;. However, the defensive_saturating_add will panic in --all-features, creating false positive crashes in fuzzing operations.

[#6624]: Use `cmd_lib` instead of `std::process::Command` when using `#[docify::export]`

Simplified the display of commands and ensured they are tested for chain spec builder's polkadot-sdk reference docs.

[#7810]: [pallet-revive] precompiles 2->9

Add missing pre-compiles 02 -> 09

Changelog for `Node Operator`

ℹ️ These changes are relevant to: Those who don't write any code and only run code.

[#7866]: Make litep2p the default backend in Kusama

This PR makes the litep2p backend the default network backend in Kusama, but also for system chains.
We performed a gradual rollout in Kusama by asking validators to manually switch to litep2p.
The rollout went smoothly, with 250 validators running litep2p without issues. This PR represents the next step in testing the backend at scale.

[#7640]: Bring the latest compatibility fixes via litep2p v0.9.1

[#6923]: omni-node: Tolerate failing metadata check

#6450 introduced metadata checks. Supported are metadata v14 and higher.

However, of course old chain-specs have a genesis code blob that might be on older version. This needs to be tolerated. We should just skip the checks in that case.

Fixes #6921

[#6605]: Notify telemetry only every second about the tx pool status

Before this was done for every imported transaction. When a lot of transactions got imported, the import notification channel was filled. The underlying problem was that the status call is read locking the validated_pool which will be write locked by the internal submitting logic. Thus, the submitting and status reading was interferring which each other.

[#7020]: Remove warning log from frame-omni-bencher CLI

Description

This PR removes the outdated warning message from the frame-omni-bencher CLI that states the tool is "not yet battle tested". Fixes #7019

Integration

No integration steps are required.

Review Notes

The functionality of the tool remains unchanged. Removes the warning message from the CLI output.

[#7494]: Enhance libp2p logging targets for granular control

This PR modifies the libp2p networking-specific log targets for granular control (e.g., just enabling trace for req-resp).

Previously, all logs were outputted to sub-libp2p target, flooding the log messages on busy validators.

Discovery: sub-libp2p::discovery;
Notification/behaviour: sub-libp2p::notification::behaviour;
Notification/handler: sub-libp2p::notification::handler;
Notification/service: sub-libp2p::notification::service;
Notification/upgrade: sub-libp2p::notification::upgrade;
Request response: sub-libp2p::request-response.

[#7353]: Shorter availability data retention period for testnets

Allows specifying a shorter availability data retention period for testnets.

[#7479]: omni-node: add offchain worker

Added support for offchain worker to omni-node-lib for both aura and manual seal nodes.

[#6248]: Upgrade libp2p to 0.54.1

Upgrade libp2p from 0.52.4 to 0.54.1

[#5724]: Validator Re-Enabling (master PR)

Implementation of the Stage 3 for the New Disabling Strategy: #4359

This PR changes when an active validator node gets disabled within parachain consensus (reduced responsibilities and
reduced rewards) for comitting offences. This should not affect active validators on a day-to-day basis and will only
be relevant when the network is under attack or there is a wide spread malfunction causing slashes. In that case
lowest offenders might get eventually re-enabled (back to normal responsibilities and normal rewards).

[#6546]: Increase default trie cache size to 1GiB

The default trie cache size before was set to 64MiB, which is quite low to achieve real speed ups. 1GiB should be a reasonable number as the requirements for validators/collators/full nodes are much higher when it comes to minimum memory requirements. Also the cache will not use 1GiB from the start and fills over time. The setting can be changed by setting --trie-cache-size BYTE_SIZE.The CLI option --state-cache-size is also removed, which was not having any effect anymore.

[#7488]: Increase litep2p keep-alive to 10 seconds to mirror libp2p

Increase litep2p keep-alive to 10 seconds to mirror libp2p behavior.

[#7266]: Add `offchain_localStorageClear` RPC method

Adds RPC method offchain_localStorageClear to clear the offchain local storage.

Changelog for `Runtime User`

ℹ️ These changes are relevant to: Anyone using the runtime. This can be a token holder or a dev writing a front end for a chain.

[#6890]: Alter semantic meaning of 0 in metering limits of EVM contract calls

A limit of 0, for gas meters and storage meters, no longer has the meaning of unlimited metering.

[#6989]: paras-registrar: Improve error reporting

This pr improves the error reporting by paras registrar when an owner wants to access a locked parachain.

Closes: #6745

[#7169]: xcm: fix DenyThenTry when work with multiple Deny tuples

This PR changes the behavior of DenyThenTry to fix #7148
If any of the tuple elements returns Err(()), the execution stops.
Else, Ok(_) is returned if all elements accept the message.

[#7320]: Add view functions to Proxy pallet for runtime-specific type configuration.

Adds two view functions to pallet-proxy:
check_permissions: Checks if a given RuntimeCall is allowed for a specific ProxyType
using the InstanceFilter trait.
is_superset: Checks if one ProxyType is a superset of another ProxyType by comparing
them using the PartialOrd trait.

[#7030]: [core-fellowship] Add permissionless import_member

Changes:

Add call import_member to the core-fellowship pallet.
Move common logic between import and import_member into do_import.

This is a minor change as to not impact UI and downstream integration.

`import_member`

[#4273]: [pallet-broker] add extrinsic to reserve a system core without having to wait two sale boundaries

When calling the reserve extrinsic after sales have started, the assignment will be reserved,
but two sale period boundaries must pass before the core is actually assigned. A new
force_reserve extrinsic is introduced to allow a core to be immediately assigned.

[#6506]: Zero refund check for FungibleAdapter

FungibleAdapter will now check if the refund amount is zero before calling deposit & emitting an event.

Fixes #6469.

[#2072]: Return iterator in pallet_referenda::TracksInfo::tracks

There is a change in pallet-referenda. Now, the tracks are retrieved as a list of Tracks. Also, the names of
the tracks might have some trailing null values (\0). This means display representation of the tracks' names
must be sanitized.

[#7377]: Add missing events to nomination pool extrinsics

Introduces events to extrinsics from pallet_nomination_pools that previously had none:

set_metadata
nominate
chill
set_configs
set_claim_permission

[#6540]: Only allow apply slash to be executed if the slash amount is atleast ED

This change prevents pools::apply_slash from being executed when the pending slash amount of the member is lower
than the ED. With this change, such small slashes will still be applied but only when member funds are withdrawn.

[#6486]: sp-trie: minor fix to avoid panic on badly-constructed proof

"Added a check when decoding encoded proof nodes in sp-trie to avoid panicking when receiving a badly constructed proof, instead erroring out."

[#6503]: xcm: minor fix for compatibility with V4

Following the removal of Rococo, Westend and Wococo from NetworkId, fixed xcm::v5::NetworkId encoding/decoding to be compatible with xcm::v4::NetworkId

[#6856]: Enable report_fork_voting()

This PR enables calling report_fork_voting.
In order to do this we needed to also check that the ancestry proof is optimal.

[#4722]: Implement pallet view functions

Querying the runtime state is now easier with the introduction of pallet view functions. Clients can call commonly
defined view functions rather than accessing the storage directly. These are similar to the Runtime APIs, but
are defined within the runtime itself.

[#5501]: Currency to Fungible migration for pallet-staking

Lazy migration of staking balance from Currency::locks to Fungible::holds. New extrinsic
staking::migrate_currency removes the old lock along with other housekeeping. Additionally, any ledger mutation
creates hold if it does not exist.

The pallet-staking configuration item Currency is updated to use fungible::hold::Mutate type while still
requiring LockableCurrency type to be passed as OldCurrency for migration purposes.

[#6995]: added new proxy ParaRegistration to Westend

This adds a new Proxy type to Westend Runtime called ParaRegistration. This is related to: polkadot-fellows/runtimes#520.

This new proxy allows:

Reserve paraID
Register Parachain
Leverage Utilites pallet
Remove proxy.

[#5990]: On-demand credits

[#7134]: xcm: convert properly assets in xcmpayment apis

Port #6459 changes to relays as well, which were probably forgotten in that PR.
Thanks!

Rust compiler versions

This release was built and tested against the following versions of rustc.
Other versions may work.

Rust Stable: "1.81.0"

Runtimes

The information about the runtimes included in this release can be found below.
The runtimes have been built using srtool v0.18.2 and rustc 1.84.1 (e71f9a9a9 2025-01-27).

Westend

🏋️ Runtime Size:          2.12 MB (2217921 bytes)
🔥 Core Version:          westend-1018002 (parity-westend-0.tx27.au2)
🗜 Compressed:            Yes: 73.47%
🎁 Metadata version:      V14
🗳️ system.setCode hash:   0xbc2999f8ba90baa86a64d83b1d9ea08965ed390eeacc0af6d8419dc71423c266
🗳️ authorizeUpgrade hash: 0x4ac3c769d6b12579840d1186b665bb3e84ad9ffdda81823b02853294baed3a7c
🗳️ Blake2-256 hash:       0xab7b7477971b14e138340ef81b6308a1a455120922e91f04765284ccc8cf855c
📦 IPFS:                  Qmb9jrosaiYNe3d7qLo915fcphFNEWcEFzUqJ6LdLAByQs

Westend Bridgehub

🏋️ Runtime Size:          1.54 MB (1612870 bytes)
🔥 Core Version:          bridge-hub-westend-1018001 (bridge-hub-westend-0.tx6.au1)
🗜 Compressed:            Yes: 72.32%
🎁 Metadata version:      V14
🗳️ system.setCode hash:   0x9052e9ca38ee20e03ff899fdcfb11558c13471cfd19d3b64c8a5c0b62dc3b0e2
🗳️ authorizeUpgrade hash: 0xdd91656acb8ce631d20d95d802567bb0fdfbb7c4f3bc3c11e48d5bdd2c958dac
🗳️ Blake2-256 hash:       0xea34241226368b03a5326dcbad8e7837f98b54f7dd9ce325eeaa294fa0b08031
📦 IPFS:                  QmSszimEJev1BWfc8dtHxW9BPA9kon4yirWfdzDRvGzAAj

Westend Collectives

🏋️ Runtime Size:          1.36 MB (1428351 bytes)
🔥 Core Version:          collectives-westend-1018001 (collectives-westend-0.tx6.au1)
🗜 Compressed:            Yes: 73.95%
🎁 Metadata version:      V14
🗳️ system.setCode hash:   0x0b57aa077c37d3abcaa03dfe6beb86ac282ed90e4d995eb9d7729cb2e546f37f
🗳️ authorizeUpgrade hash: 0xa0677b2b58f8ebe9466969769718161de90aad3a01b09ec7f231da3dab03b8dc
🗳️ Blake2-256 hash:       0xb80c304b5af22a8e391cd4c9e9618a86a23ce19f085c13a6ecf1f815972f2c45
📦 IPFS:                  Qmak8MffqaKJpKrQUGs36gfFx7nusLJ6XsLQ12YGHbsKv6

Westend Coretime

🏋️ Runtime Size:          1.17 MB (1228231 bytes)
🔥 Core Version:          coretime-westend-1018001 (coretime-westend-0.tx2.au1)
🗜 Compressed:            Yes: 73.09%
🎁 Metadata version:      V14
🗳️ system.setCode hash:   0x9024fc5541843d0d4f476efc7f0cad3b25bcdd438b328aba97059d29843db732
🗳️ authorizeUpgrade hash: 0xe78af2dfd82a3797f858704ca1340f9a9fe374bb9bddb981c18d4bb49bc250a4
🗳️ Blake2-256 hash:       0x81740d7ffed237a9d68aa1a2780cb733a6a0205afb4c552d5bb665ff52005f7c
📦 IPFS:                  QmZDpj6RGBLArC1yRBgvoqrGjTnT1oQ2myGuofBgYFHbnv

Westend Glutton

🏋️ Runtime Size:          595.24 kB (609521 bytes)
🔥 Core Version:          glutton-westend-1018001 (glutton-westend-0.tx1.au1)
🗜 Compressed:            Yes: 71.84%
🎁 Metadata version:      V14
🗳️ system.setCode hash:   0xf0d5046c8c070eadc7e43cf68cf812c8296c8044564be568904d005986eb5af2
🗳️ authorizeUpgrade hash: 0xf545c5ee79503ca758dd76aa1bd533a0952ca58f7795fd728bf19e85a47579ba
🗳️ Blake2-256 hash:       0x2077228269c00e69acbf4dd1984d50976f0b7381d0791795987bcd701d57ff8f
📦 IPFS:                  Qmd7TY5dLRiCv2vLZavhYaaqTTXY6xf8iPe44a8ZV4Y8rw

Westend People

🏋️ Runtime Size:          1.17 MB (1228920 bytes)
🔥 Core Version:          people-westend-1018001 (people-westend-0.tx2.au1)
🗜 Compressed:            Yes: 73.01%
🎁 Metadata version:      V14
🗳️ system.setCode hash:   0x15c8b1465a17040a1afa65ccfa47fe149312954553c15766d235d46b90391327
🗳️ authorizeUpgrade hash: 0x22572bbe011dd4ab1b79e991d7ab048944ffc4908fd79ba216e6e592fe97e5e9
🗳️ Blake2-256 hash:       0xd9df666cd7ad9f4170e1d9dde800617ac6348fdfc733daf5609695f55018f249
📦 IPFS:                  QmRTvnCJ4zhhiCayKoJAL9FnGRcUCjyDS5VqqRSF65ZsQQ

Docker images

The docker images for the polkadot node binary and the polkadot-parachain binary can be found at Docker hub (will be available a few minutes after the release has been published):

You may also pull it with:

docker pull parity/polkadot:stable2503-rc2

docker pull parity/polkadot-parachain:stable2503-rc2

	async fn request(
	&self,
	target: sc_network_types::PeerId,
	protocol: ProtocolName,
	request: Vec<u8>,
	fallback_request: Option<(Vec<u8>, ProtocolName)>,
	connect: IfDisconnected,
	) -> Result<(Vec<u8>, ProtocolName), RequestFailure> {
	let (tx, rx) = oneshot::channel();

	self.start_request(target.into(), protocol, request, fallback_request, tx, connect);

	match rx.await {
	Ok(v) => v,
	// The channel can only be closed if the network worker no longer exists. If the
	// network worker no longer exists, then all connections to `target` are necessarily
	// closed, and we legitimately report this situation as a "ConnectionClosed".
	Err(_) => Err(RequestFailure::Network(OutboundFailure::ConnectionClosed)),
	}
	}

paritytech/polkadot-sdk polkadot-stable2503-rc2 Polkadot stable2503-rc2 on GitHub

Changelog

Changelog for Node Dev

[#6561]: slot-based-collator: Move spawning of the futures

[#4880]: Collation fetching fairness in collator protocol

[#6647]: fatxpool: proper handling of priorities when mempool is full

[#7554]: sc-informant: Print full hash when debug logging is enabled

[#6703]: network/libp2p-backend: Suppress warning adding already reserved node as reserved

[#7005]: Log peerset set ID -> protocol name mapping

[#7866]: Make litep2p the default backend in Kusama

[#6636]: Optimize initialization of networking protocol benchmarks

[#6979]: Update prometheus binding failure logging format

[#6262]: Size limits implemented for fork aware transaction pool

[#7368]: Add chain properties to chain-spec-builder

[#7640]: Bring the latest compatibility fixes via litep2p v0.9.1

[#7254]: deprecate AsyncBackingParams

[#7338]: [net/libp2p] Use raw Identify observed addresses to discover external addresses

[#7021]: Improve remote externalities logging

[#6711]: Expose DHT content providers API from sc-network

[#6215]: Remove ProspectiveParachainsMode from backing subsystem

[#6455]: Add litep2p network protocol benches

[#7042]: networking::TransactionPool should accept Arc

[#6405]: fatxpool: handling limits and priorities improvements

[#6284]: backing: improve session buffering for runtime information

[#7127]: Forbid v1 descriptors with UMP signals

[#7014]: Remove yamux_window_size from network config

Description

Checklist

[#7102]: fatxpool: rotator cache size now depends on pool's limits

[#5842]: Get rid of libp2p dependency in sc-authority-discovery

[#6924]: malus-collator: implement malicious collator submitting same collation to all backing groups

[#6249]: Pure state sync refactoring (part-1)

[#7104]: collation-generation: resolve mismatch between descriptor and commitments core index

[#6983]: cumulus: bump PARENT_SEARCH_DEPTH to allow for 12-core elastic scaling

[#6897]: Tracing Log for fork-aware transaction pool

[#6440]: Remove debug message about pruning active leaves

[#7075]: Snowbridge - Ethereum Electra Upgrade Support

[#7073]: Implement NetworkRequest for litep2p

Description

Review Notes

[#6628]: Remove ReportCollator message

[#7675]: Bandersnatch hot fix version

[#7494]: Enhance libp2p logging targets for granular control

[#7479]: omni-node: add offchain worker

[#7195]: Unify Import verifier usage across parachain template and omninode

[#6889]: Remove polkadot-omni-node-lib unused dependency

[#6248]: Upgrade libp2p to 0.54.1

[#6400]: Remove network starter that is no longer needed

Description

Integration

Review Notes

Checklist

[#6481]: slot-based-collator: Implement dedicated block import

[#6452]: elastic scaling RFC 103 end-to-end tests

[#7011]: sync: Send already connected peers to new subscribers

[#7504]: Enable approval-voting-parallel by default on polkadot

[#6553]: Ensure sync event is processed on unknown peer roles

[#6417]: fix prospective-parachains best backable chain reversion bug

[#6867]: Deprecate ParaBackingState API

[#6865]: Rename PanicInfo to PanicHookInfo

[#6963]: grandpa: Ensure WarpProof stays in its limits

[#6163]: Expose more syncing types to enable custom syncing strategy

[#5855]: Remove feature test-helpers from sc-service

[#6913]: Enable approval-voting-parallel by default on polkadot

[#6450]: Add omni-node checks for runtime parachain compatibility

[#6534]: Forward logging directives to Polkadot workers

[#6661]: txpool api: remove_invalid call improved

[#7488]: Increase litep2p keep-alive to 10 seconds to mirror libp2p

[#6521]: Pure state sync refactoring (part-2)

[#6528]: TransactionPool API uses async_trait

[#6832]: Remove collation-generation subsystem from validator nodes

[#6768]: basic-authorship: debug level is now less spammy

[#5703]: Properly handle block gap created by fast sync

[#7505]: fatxpool: transaction statuses metrics added

Changelog for Runtime Dev

[#6695]: [pallet-revive] bugfix decoding 64bit args in the decoder

[#7614]: [pallet-revive] tracing improvements

[#3926]: Introduce pallet-asset-rewards

[#7492]: Make pallet-bridge-rewards generic over RewardKind

[#6673]: chain-spec-guide-runtime: path to wasm blob fixed

paritytech/polkadot-sdk polkadot-stable2503-rc2
Polkadot stable2503-rc2

on GitHub

Changelog for `Node Dev`

[#6647]: `fatxpool`: proper handling of priorities when mempool is full

[#7338]: [net/libp2p] Use raw `Identify` observed addresses to discover external addresses

[#6711]: Expose DHT content providers API from `sc-network`

[#6215]: Remove `ProspectiveParachainsMode` from backing subsystem

[#6405]: `fatxpool`: handling limits and priorities improvements

[#7014]: Remove `yamux_window_size` from network config

[#7102]: `fatxpool`: rotator cache size now depends on pool's limits

[#6963]: grandpa: Ensure `WarpProof` stays in its limits

[#5855]: Remove feature `test-helpers` from sc-service

[#6661]: `txpool api`: `remove_invalid` call improved

[#6768]: `basic-authorship`: debug level is now less spammy

[#7505]: `fatxpool`: transaction statuses metrics added

Changelog for `Runtime Dev`

[#7492]: Make `pallet-bridge-rewards` generic over `RewardKind`

[#5723]: Adds `BlockNumberProvider` in multisig, proxy and nft pallets

[#7263]: Fix `frame-benchmarking-cli` not buildable without rocksdb

[#7230]: revive: Include immutable storage deposit into the contracts `storage_base_deposit`

Record the deposit for immutable data into the `storage_base_deposit`

Moved refcount functions to `CodeInfo`

Set `CodeHashLockupDepositPercent` for test runtime

Reworked the `MockExecutable` to have both a `deploy` and a `call` entry point

`import_member`