Skip to content

feat(test-specs): Test mutators#2860

Open
marioevz wants to merge 7 commits into
ethereum:forks/amsterdamfrom
marioevz:test-mutators
Open

feat(test-specs): Test mutators#2860
marioevz wants to merge 7 commits into
ethereum:forks/amsterdamfrom
marioevz:test-mutators

Conversation

@marioevz
Copy link
Copy Markdown
Member

🗒️ Description

Introduce a spec-test mutator framework that lets fill apply systematic mutations to existing spec tests so we can hunt for edge cases without writing new tests. Mutators are fork-gated: each fork declares which mutators it supports via BaseFork.spec_test_mutators(), and the user opts in per fill invocation.

Usage

fill ... --test-mutators=EIP_7702_ALL_CONTRACTS_AS_DELEGATIONS
fill ... --test-mutators=all

The value selected for a given test is the intersection of what the user requested with what the test's fork supports — so requesting a mutator on a fork that doesn't enable it is a no-op rather than an error, and tests under a fork range get mutated only on the forks where the mutation is meaningful.

First mutator: EIP_7702_ALL_CONTRACTS_AS_DELEGATIONS (Prague+)

Every pre.deploy_contract(...) call is rewritten so that:

  1. The original bytecode is placed in a separate account.
  2. The address returned to the test is a delegated EOA (0xef0100 || <code-address>) pointing at that account.

This stresses the EIP-7702 delegation path against the full body of existing state/blockchain tests. Edge cases surfaced this way are expected to be promoted into standalone tests as a follow-up — the flag is an exploration aid, not part of regular fill runs.

🔗 Related Issues or PRs

N/A.

✅ Checklist

  • All: Ran fast static checks to avoid unnecessary CI fails, see also Code Standards and Enabling Pre-commit Checks:
    just static
  • All: PR title adheres to the repo standard - it will be used as the squash commit message and should start type(scope):.
  • All: Considered updating the online docs in the ./docs/ directory.
  • All: Set appropriate labels for the changes (only maintainers can apply labels).
  • Tests: Ran mkdocs serve locally and verified the auto-generated docs for new tests in the Test Case Reference are correctly formatted.
  • Tests: For PRs implementing a missed test case, update the post-mortem document to add an entry the list.
  • Ported Tests: All converted JSON/YML tests from ethereum/tests or tests/static have been assigned @ported_from marker.

Cute Animal Picture

Put a link to a cute animal picture inside the parenthesis-->

@codecov
Copy link
Copy Markdown

codecov Bot commented May 14, 2026

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 86.94%. Comparing base (b3f5df5) to head (5ff1185).

Additional details and impacted files
@@               Coverage Diff                @@
##           forks/amsterdam    #2860   +/-   ##
================================================
  Coverage            86.94%   86.94%           
================================================
  Files                  586      586           
  Lines                35777    35777           
  Branches              3363     3363           
================================================
  Hits                 31106    31106           
  Misses                4012     4012           
  Partials               659      659           
Flag Coverage Δ
unittests 86.94% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant