Skip to content

perf(levm): add sstore bench, allow unoptimized bench contracts and improve bench makefile #3552

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 1 commit into
base: main
Choose a base branch
from

Conversation

edg-l
Copy link
Contributor

@edg-l edg-l commented Jul 8, 2025

Motivation

  • Adds a sstore benchmark, however we need to disable solc optimizations for this contract otherwise it removes most code.
  • Improved the makefile adding a command to samply an individual benchmark

https://share.firefox.dev/44MVD2V

Description

Closes #issue_number

@edg-l edg-l requested a review from a team as a code owner July 8, 2025 11:29
@github-actions github-actions bot added levm Lambda EVM implementation performance labels Jul 8, 2025
Copy link

github-actions bot commented Jul 8, 2025

Lines of code report

Total lines added: 10
Total lines removed: 0
Total lines changed: 10

Detailed view
+------------------------------------------------------------+-------+------+
| File                                                       | Lines | Diff |
+------------------------------------------------------------+-------+------+
| ethrex/crates/vm/levm/bench/revm_comparison/src/compile.rs | 53    | +10  |
+------------------------------------------------------------+-------+------+

Copy link

github-actions bot commented Jul 8, 2025

No significant difference was registered for any benchmark run.

Detailed Results

Benchmark Results: BubbleSort

Command Mean [s] Min [s] Max [s] Relative
main_revm_BubbleSort 3.193 ± 0.038 3.162 3.278 1.00 ± 0.01
main_levm_BubbleSort 4.398 ± 0.017 4.377 4.435 1.38 ± 0.01
pr_revm_BubbleSort 3.184 ± 0.025 3.145 3.238 1.00
pr_levm_BubbleSort 4.402 ± 0.014 4.378 4.421 1.38 ± 0.01

Benchmark Results: ERC20Approval

Command Mean [s] Min [s] Max [s] Relative
main_revm_ERC20Approval 1.027 ± 0.006 1.020 1.043 1.00
main_levm_ERC20Approval 1.526 ± 0.006 1.519 1.536 1.49 ± 0.01
pr_revm_ERC20Approval 1.032 ± 0.017 1.017 1.075 1.01 ± 0.02
pr_levm_ERC20Approval 1.523 ± 0.012 1.510 1.551 1.48 ± 0.01

Benchmark Results: ERC20Mint

Command Mean [ms] Min [ms] Max [ms] Relative
main_revm_ERC20Mint 137.4 ± 0.6 136.6 138.7 1.00 ± 0.01
main_levm_ERC20Mint 258.6 ± 1.6 255.1 260.9 1.88 ± 0.02
pr_revm_ERC20Mint 137.2 ± 1.4 136.1 141.1 1.00
pr_levm_ERC20Mint 259.7 ± 12.3 251.5 283.4 1.89 ± 0.09

Benchmark Results: ERC20Transfer

Command Mean [ms] Min [ms] Max [ms] Relative
main_revm_ERC20Transfer 241.0 ± 3.4 238.2 248.3 1.00 ± 0.02
main_levm_ERC20Transfer 402.0 ± 6.4 396.4 414.1 1.67 ± 0.03
pr_revm_ERC20Transfer 240.1 ± 2.7 238.4 247.5 1.00
pr_levm_ERC20Transfer 397.2 ± 3.3 394.0 406.2 1.65 ± 0.02

Benchmark Results: Factorial

Command Mean [ms] Min [ms] Max [ms] Relative
main_revm_Factorial 232.8 ± 1.5 230.4 235.2 1.00
main_levm_Factorial 441.5 ± 1.4 439.7 443.2 1.90 ± 0.01
pr_revm_Factorial 235.6 ± 3.8 232.9 243.5 1.01 ± 0.02
pr_levm_Factorial 452.7 ± 23.3 443.5 518.9 1.94 ± 0.10

Benchmark Results: FactorialRecursive

Command Mean [s] Min [s] Max [s] Relative
main_revm_FactorialRecursive 1.638 ± 0.022 1.588 1.666 1.02 ± 0.06
main_levm_FactorialRecursive 2.718 ± 0.026 2.681 2.759 1.69 ± 0.10
pr_revm_FactorialRecursive 1.609 ± 0.099 1.339 1.684 1.00
pr_levm_FactorialRecursive 2.649 ± 0.014 2.634 2.681 1.65 ± 0.10

Benchmark Results: Fibonacci

Command Mean [ms] Min [ms] Max [ms] Relative
main_revm_Fibonacci 207.0 ± 1.0 205.4 208.6 1.00
main_levm_Fibonacci 438.4 ± 4.0 435.1 446.9 2.12 ± 0.02
pr_revm_Fibonacci 208.0 ± 4.4 203.0 220.0 1.00 ± 0.02
pr_levm_Fibonacci 475.1 ± 86.6 427.9 700.9 2.30 ± 0.42

Benchmark Results: FibonacciRecursive

Command Mean [ms] Min [ms] Max [ms] Relative
main_revm_FibonacciRecursive 881.6 ± 8.5 869.8 893.6 1.01 ± 0.02
main_levm_FibonacciRecursive 1428.4 ± 4.5 1422.5 1436.0 1.64 ± 0.03
pr_revm_FibonacciRecursive 869.3 ± 15.7 844.6 888.0 1.00
pr_levm_FibonacciRecursive 1396.4 ± 8.0 1384.6 1409.6 1.61 ± 0.03

Benchmark Results: ManyHashes

Command Mean [ms] Min [ms] Max [ms] Relative
main_revm_ManyHashes 8.6 ± 0.1 8.5 8.7 1.00
main_levm_ManyHashes 13.7 ± 0.1 13.5 13.8 1.58 ± 0.02
pr_revm_ManyHashes 8.7 ± 0.1 8.6 8.8 1.00 ± 0.01
pr_levm_ManyHashes 12.9 ± 0.1 12.8 13.1 1.50 ± 0.02

Benchmark Results: MstoreBench

Command Mean [ms] Min [ms] Max [ms] Relative
main_revm_MstoreBench 282.1 ± 31.3 266.3 370.0 1.03 ± 0.12
main_levm_MstoreBench 947.8 ± 5.8 939.5 961.2 3.45 ± 0.10
pr_revm_MstoreBench 274.8 ± 8.0 266.8 290.8 1.00
pr_levm_MstoreBench 936.8 ± 4.9 930.4 947.7 3.41 ± 0.10

Benchmark Results: Push

Command Mean [ms] Min [ms] Max [ms] Relative
main_revm_Push 301.2 ± 5.8 296.7 317.3 1.00
main_levm_Push 1062.1 ± 6.3 1058.0 1079.4 3.53 ± 0.07
pr_revm_Push 301.7 ± 7.3 297.9 321.9 1.00 ± 0.03
pr_levm_Push 1039.6 ± 3.6 1035.9 1046.7 3.45 ± 0.07

Benchmark Results: SstoreBench_no_opt

Command Mean [ms] Min [ms] Max [ms] Relative
main_revm_SstoreBench_no_opt 158.8 ± 4.8 155.7 172.1 1.00 ± 0.04
main_levm_SstoreBench_no_opt 377.5 ± 4.2 371.3 383.7 2.38 ± 0.07
pr_revm_SstoreBench_no_opt 158.5 ± 4.2 156.0 170.3 1.00
pr_levm_SstoreBench_no_opt 375.5 ± 2.8 373.0 382.8 2.37 ± 0.07

@edg-l edg-l changed the title perf(levm): add sstore bench, allow unoptimized bench contracts and improve bench makefile perf(levm): add more benches, allow unoptimized bench contracts and improve bench makefile Jul 8, 2025
@edg-l edg-l changed the title perf(levm): add more benches, allow unoptimized bench contracts and improve bench makefile perf(levm): add sstore bench, allow unoptimized bench contracts and improve bench makefile Jul 8, 2025
@mpaulucci mpaulucci removed this from ethrex_l1 Jul 8, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
levm Lambda EVM implementation performance
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant