ethereum/execution-spec-tests benchmark@v0.0.4

on GitHub

Benchmark Tests

Contains consolidated blockchain tests that can be used for benchmarking, as well as the original blockchain_tests and blockchain_tests_engine, but deprecates state_test format (only for benchmark releases, this format will still be available for normal releases).

This release also deprecates Cancun fork and contains only Prague.

Single consolidated genesis

This release contains a single genesis file for the Engine X format. For every single test contained in this release, only one genesis file has to be loaded and, theoretically, all tests should run with a single client instance without having to restart at any point.

For more information on the new format see Blockchain Engine X Tests in the EEST documentation.

Forks Included

• Prague
• Cancun (Deprecated)

Note: Osaka is still not supported for benchmarks, but will be in a future release.

What's Changed

• fix(github): Remove --solc-version flag from fixture building action by @marioevz in #1937
• fix(docs): fix broken link that leads to auto-issue generation #1930 by @felix314159 in #1938
• (feat)ci: pypy3.11 support by @kclowes in #1854
• fix(plugins): Remove cwd change before running pytest by @marioevz in #1943
• fix(clis/besu): blob count max exception mapper by @Gabriel-Trintinalia in #1944
• feat(tests): remove Blake2b max rounds static test by @chfast in #1941
• feat(tests): add eip7702 test case for clz opcode by @LouisTsai-Csie in #1853
• fix(mypy): adds mypy ignore comments and add ignore rule for imports by @felix314159 in #1951
• chore: bump eels resolver by @spencer-tb in #1950
• fix(plugins): Don't copy test list during collection by @marioevz in #1936
• chore(tests): remove slow/zkevm markers from benchmark tests by @spencer-tb in #1953
• fix(unit_tests): fixed blob unit tests when run with xdist by @felix314159 in #1946
• feat(tests): remove the static performance test calling SHA256 5000x by @chfast in #1949
• fix(static_tests): adjusted gasLimit to only allow gasLimit of up to 30mil by @felix314159 in #1924
• feat(consume): add --extract-to parameter for direct fixture extraction by @richardgreg in #1861
• fix(tests): EIP-7825: Pre-Osaka tests consuming tx gas above cap by @marioevz in #1928
• chore(benchmark): don't fill benchmark tests by default by @spencer-tb in #1920
• chore(clis): update geth exception mapper for TYPE_3_TX_BLOB_COUNT_EXCEEDED by @MariusVanDerWijden in #1958
• chore(docs): add benchmark marker to test case reference by @spencer-tb in #1960
• feat(specs,tests): Update all benchmark tests to use gas_benchmark_value by @marioevz in #1935
• chore(ci): bump to use latest eels forks/osaka by @spencer-tb in #1962
• feat(tests): extra clz param for scenarios test case by @spencer-tb in #1965
• chore(fill): remove fill warnings by @spencer-tb in #1964
• feat(tests): add call memory expansion with no return size test by @JereSalo in #1952
• feat(cli): resolve updated execution_spec exceptions by @souradeep-das in #1784
• tests(benchmark): expand worst case modexp benchmarks by @LouisTsai-Csie in #1780
• chore(tests|forks): add max blobs per tx limit by @spencer-tb in #1884
• fix(fill): allow lowercase fork declaration in fill command by @felix314159 in #1969
• feat(tests): add initcode context extra test for clz opcode by @LouisTsai-Csie in #1831
• chore(ci|cli): improve coverage ci and compare fixtures python script by @spencer-tb in #1970
• chore(clis): add erigon exception mapper for TYPE_3_TX_BLOB_COUNT_EXCEEDED by @yperbasis in #1974
• feat(tests): add call operation test case for CLZ by @LouisTsai-Csie in #1829
• feat(tests): extra stack operation test for CLZ opcode by @LouisTsai-Csie in #1821
• refactor(specs/static_state): Fork string parsing by @marioevz in #1977
• fix(consume): fixes issue where nethtest would not be recognized as valid t8n on certain systems by @felix314159 in #1973
• feat(benchmark): add clz opcode test case by @LouisTsai-Csie in #1845
• fix(tests): Issues with rlp block limit tests and --generate-pre-alloc-groups by @fselmo in #1989
• feat(ci): Build releases in self-hosted-ghr, fetch solidity, lllc from github releases by @marioevz in #1990
• test: add max block size test using access lists by @nerolation in #1932
• feat(checklist): Add Modified Transaction Validity Constraint by @marioevz in #1988
• chore(tests): fix pre alloc grouping remaining fails by @spencer-tb in #1975
• fix(tests/static): Fix all static tests for Osaka fork by @marioevz in #1980
• feat(test): add test for memory expansion on early revert in CALL by @JereSalo in #1967
• chore(tests): remove temp fix for block rlp type 4 by @spencer-tb in #1995
• chore: align gas limit with mainnet by @LouisTsai-Csie in #1994
• chore(benchmark): update gas limit configuration by @LouisTsai-Csie in #1983
• fix(github): Set x-dist to auto for all evm impl by @marioevz in #1996
• chore(tests): update max blob per tx test errors by @spencer-tb in #1997
• chore(ci): preparation for next full release by @spencer-tb in #1954
• feat(fill): Disable --generate-all-formats for static tests by @marioevz in #1998
• feat(tests): add critical BLS tests with invalid point coordinates by @chfast in #2000
• feat(tests): extra BLS tests with invalid point coords by @spencer-tb in #2004
• feat(tests): add case for large exp with zero mod/base length by @spencer-tb in #2005
• chore(benchmark): update max env gas by @spencer-tb in #2001
• fix(tests): Fix ModExpInput.from_bytes by @marioevz in #1999
• test(osaka): add edge case test vectors for EIP-7883 MODEXP gas calculation by @bshastry in #1993
• refactor(fill): remove spaces from test ids/fixture names by @LouisTsai-Csie in #2009
• chore(tests): exception hotfix by @spencer-tb in #2011
• bug(cli): fixed bug where --trace would never be enabled for geth's evm by @felix314159 in #2012
• feat(tests): Test 256verify modular comparison by @kclowes in #2015
• feat(specs,filler,ci): Merge static tests into stable/develop, make --generate-all-formats compatible with static tests by @marioevz in #2006

New Contributors

• @Gabriel-Trintinalia made their first contribution in #1944
• @souradeep-das made their first contribution in #1784
• @nerolation made their first contribution in #1932
• @bshastry made their first contribution in #1993

Full Changelog: https://github.com/ethereum/execution-spec-tests/compare/benchmark@v0.0.3...benchmark@v0.0.4