feat(ci,tooling): improve static check developer experience by danceratopz · Pull Request #2099 · ethereum/execution-specs

danceratopz · 2026-01-29T09:24:18Z

🗒️ Description

This PR improves static check developer experience and introduces finer-grained tox environments for static checks as were configured in ethereum/execution-spec-tests pre-Weld.

It offers 4 main improvements to dev-ex:

Flexible entry point - User can choose their preferred "check" (aka recipe) runner (esp. helpful for non-Python devs):
```
tox -e <env/check-name>  # or uvx tox -e <env/check-name>
just <check-name>
make <check-name>
```
With this PR, I recommend replacing uvx tox -e static with:
```
just check
# or
make check
```
These run: uvx tox --parallel -e lint,format,typecheck,markdownlint,spec-lint,spellcheck,actionlint,changelog,lockcheck, NOT uvx tox -e check which is not parallelized.
Parallelization - Each one of the commands in the tox static env is now a separate tox env. This means they can be ran in parallel:

i. Faster (~6s → ~3s warm, with mypy cache and .tox/ present).
ii. All checks run; they don't stop upon first fail (particularly annoying in CI).
Tips upon fail - Checks are wrapped in a helper scripts/fast_checks.py that:

i. Provides auto-fix hints on failure (shown on command-line and in CI job summaries, screenshot below).
ii. Uses a soft-fail mode for markdown-cli2 (locally, this was previously in packages/testing/src/execution_testing/cli/tox_helpers.py, now delted).
Fix helper recipe - Adds a "fix" just/make "recipe" for safe ruff (ruff check --fix and ruff format):
```
just fix
make fix
```

This PR also:

The static checks moved out of test.yaml to check.yaml and is configured to ensures no workflows get triggered if check.yaml fails. This involves some unavoidable duplication as each workflow must call check.yaml independently to preserve path filtering. This needs discussion.
Maintains backwards compatibility with tox -e static; which also has check as alias.
Updates documentation for the new commands and build tools.

Fix Hint Github Summary Screenshot

Fix Hint Console Output (also available locally)

From: https://github.com/danceratopz/execution-specs/actions/runs/21474528622

🔗 Related Issues or PRs

Closes #2041

Refactor static into separate tox environments & dedicated fast-checks workflow #2041

✅ Checklist

All: Ran fast tox checks to avoid unnecessary CI fails, see also Code Standards and Enabling Pre-commit Checks:
```
uvx tox -e static
```
All: PR title adheres to the repo standard - it will be used as the squash commit message and should start type(scope):.
All: Considered updating the online docs in the ./docs/ directory.
All: Set appropriate labels for the changes (only maintainers can apply labels).
Tests: Ran mkdocs serve locally and verified the auto-generated docs for new tests in the Test Case Reference are correctly formatted.

Cute Animal Picture

Split the monolithic [testenv:static] into individual check environments for finer-grained control and parallel execution: - spellcheck: codespell. - lint: ruff check. - format: ruff format --check. - typecheck: mypy. - spec-lint: ethereum-spec-lint. - lockcheck: uv lock --check. - actionlint: GitHub Actions workflow linting. Add fast-checks aggregate alias and keep static for backwards compat. Ref: ethereum#2041

Add scripts/fast_checks.py that wraps each static check with actionable fix hints on failure: - Prints clear "Check failed" header with tool name. - Shows how to auto-fix (where applicable). - Shows verify command. - Writes to GITHUB_STEP_SUMMARY in CI for job summaries. Update tox environments to use the wrapper script. Ref: ethereum#2041

Remove tox_helpers.py and its entry points, now replaced by scripts/fast_checks.py: - pyspelling_soft_fail: replaced by spellcheck (via codespell). - markdownlintcli2_soft_fail: replaced by markdownlint check. - validate_changelog: replaced by changelog check. Ref: ethereum#2041

Add .github/workflows/fast-checks.yaml that runs all static checks in parallel via matrix jobs: - Spellcheck, Lint, Format, Typecheck, Spec Lint, Lock Check. - Action Lint, Changelog. - Markdown Lint (via dedicated GitHub Action). - SHA Pinned Actions check. Update workflows to use fast-checks as a prerequisite: - test.yaml: removed monolithic static job. - test-docs.yaml: removed duplicate changelog/markdownlint jobs. - benchmark.yaml: added fast-checks prerequisite. - hive-consume.yaml: added fast-checks prerequisite.

- Add `fast-checks` to tox command table in code_standards.md. - Document fix hints feature with example output. - Update recommended commands to use `fast-checks`. - Add pre-commit hooks table to precommit.md. - Update PR template to use `fast-checks`. - Update EIP author manual to use `fast-checks`. - Add .pre-commit-config.yaml for local development.

- Rename tox env `fast-checks` → `check` for consistency - Rename workflow `fast-checks.yaml` → `check.yaml` - Update all workflow refs (`uses:`, `needs:`) to use `check` - Update GitHub Actions to use `tox -e check` - Add fix hints showing `just fix` / `make fix` for lint/format - Create Justfile with recipes delegating to tox - Keep `static` as backwards-compat alias for `check`

Add Makefile as alternative entry point for running checks. All targets delegate to tox (source of truth). Includes `make help` for discoverability.

- Update all `fast-checks` → `check` references in docs - Update CI workflow link to check.yaml - Show just/make/tox options in code standards table - Create docs/dev/build_tools.md with command reference - Update PR template and EIP authors manual

Run all static checks in parallel via just/make for ~2x speedup (~6s → ~3s) and better error reporting (all checks run even if one fails).

- Enable auto-fix hints to appear in CI job summaries by passing the GITHUB_STEP_SUMMARY env var through to tox environments. - fix(ci): don't double-wrap fix hints in code blocks. The fix_hint field already contains markdown code blocks, so write_github_summary shouldn't wrap it in another code block. - fix(ci): simplify verify hint to match PR template style. Show single line: just check # or: make check, uvx tox -e <check>. - fix(ci): make render workflow depend on check workflow.

This reverts commit b669a88.

danceratopz · 2026-01-29T13:02:18Z

Someone asked an excellent question via DM:

i still don't really get what value tox brings locally

Essentially we could consider removing tox and rely on just or make as a check runner. However, as-is, I would keep tox as it still provides some useful functionality in both the old and new setups.

First some background, as I understand it:

venv creation and management in Python was painful.
one of the main value props of tox is/was helping with venvs.
uv fixes this (coz venvs are free/fast), eroding the tox value prop.

I'd keep tox coz it still has some nice orchestration features:

run envs in parallel (this pr splits static to make this possible) this is now used by just/make (they call tox) to:
- be fast,
- get feedback on all checks in one command.
helps configure testing w/diff Python flavors, e.g., pypy (could be 3.{12,13,14}, but we don't).

Perhaps I'm missing something else?

SamWilsn · 2026-01-29T15:29:33Z

+  sha-pinned-actions:
+    name: SHA Pinned Actions
+    runs-on: ubuntu-latest
+    steps:
+      - uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683
+      - name: Ensure SHA pinned actions
+        uses: zgosalvez/github-actions-ensure-sha-pinned-actions@6124774845927d14c601359ab8138699fa5b70c3


This is available in the repository settings now, unless this serves a different purpose?

Good point.

SamWilsn · 2026-01-29T15:33:13Z

+  check:
+    uses: ./.github/workflows/check.yaml
+


Hm, instead of duplicating check, would on: workflow_run work?

Hm, workflow_run has a few security considerations. Perhaps workflow_call might be safer.

SamWilsn · 2026-01-29T15:54:44Z

We should pick exactly one build system. I understand the desire to make our project as accessible as possible to external developers, but adding a Makefile/Justfile/meson.build/CMakeLists.txt/package.json/pom.xml/ExecutionSpecs.sln/Cargo.toml file just to support muscle memory for another language is silly.

Every build we add is more maintenance for us. If I want to change the build system, I don't want to have to test N different entry points.

Ideally we should have a nice short "Getting Started" section of our readme that includes installation instructions, common commands, and links to further documentation for advanced usage. If a developer can't be bothered to look in our readme for how to run our tooling, I don't really want their contributions.

Good point, got a bit carried away here! 😄

SamWilsn · 2026-01-29T16:05:29Z

I've done a bit more reading into this tool. It's not as bad as I feared. You can't sneak a hook into post-checkout silently; the user needs to rerun pre-commit install.

It still makes me uncomfortable, and I would really rather not promote it, but I'll no longer die on this hill.

Thanks for checking it out, this was unintentionally committed, sorry about that!

SamWilsn · 2026-01-29T16:06:51Z

If we're doing something like this anyway, why not make a build.py in the root directory and just use that instead of just/make/tox?

I think this approach is now overkill.

SamWilsn · 2026-01-29T16:07:54Z

 ## Formatting and Line Length

-The Python code in @ethereum/execution-spec-tests is formatted with `ruff` with a line length of 100 characters.
+The Python code in @ethereum/execution-specs is formatted with `ruff` with a line length of 100 characters.


Suggested change

The Python code in @ethereum/execution-specs is formatted with `ruff` with a line length of 100 characters.

The Python code in @ethereum/execution-specs is formatted with `ruff` with a line length of 79 characters.

SamWilsn

Very old review I forgot to submit ^^;;

SamWilsn · 2026-01-29T16:15:35Z

+  check:
+    uses: ./.github/workflows/check.yaml
+


Hm, workflow_run has a few security considerations. Perhaps workflow_call might be safer.

felix314159 · 2026-03-02T13:42:57Z

Will this PR get updated to remove tox? The PR was opened 2026-01-29, which is the day where we decided in the team meeting to get rid of tox (see gemini summary of 2026-01-29, although gemini phrased it differently), but it still includes tox. Issue #2041 is a bit older so that probably can be closed/reworked when tox is removed

danceratopz · 2026-03-03T07:38:59Z

Will this PR get updated to remove tox? The PR was opened 2026-01-29, which is the day where we decided in the team meeting to get rid of tox (see gemini summary of 2026-01-29, although gemini phrased it differently), but it still includes tox. Issue #2041 is a bit older so that probably can be closed/reworked when tox is removed

Yes, I'd like to update this PR to remove tox.

In the same testing call, this was deemed low priority and added undue stress on the team, so it was put on hold. I can update if we'd like to prioritize it. Feel free to bring it up in the testing meeting.

We did agree to remove tox, but we didn't agree what to replace it with. Currently, my plan would be to update this PR to use just, https://just.systems/man/en/, in order to see how feasible a bare uv + just setup is. just has the simplicity of make but with cleaner syntax. Other build systems, e.g., nox, waf seem overkill for our purposes. You or anyone else should feel free to chime in if you/they have an opinion here!

felix314159 · 2026-03-03T10:19:18Z

I personally would prefer uv + make. I looked at some specific code examples of make vs just and the difference sometimes is just one line of code. No strong feelings though. make has the advantage of having 50 years of training data, so that could make things easier for 'us' even when it at first might seem otherwise

danceratopz · 2026-03-25T10:08:40Z

Thanks a lot @SamWilsn for your review and @felix314159 for your comments!

I personally would prefer uv + make. I looked at some specific code examples of make vs just and the difference sometimes is just one line of code. No strong feelings though. make has the advantage of having 50 years of training data, so that could make things easier for 'us' even when it at first might seem otherwise

Appreciate the feedback, I personally feel that just's UX improvements make it worth it (simple argument pass-through, tasks split by group in help output, no .PHONY, no tabs) and we don't need the additional complexity of a build system.

I definitely got a bit carried away in this PR. And made sweeping changes across the build system and CI. I made a more focussed proposal (just a suggestion!) for replacing tox with just in:

feat(ci,tooling): replace tox with just #2555

danceratopz added 9 commits January 29, 2026 00:45

feat(tooling): add Makefile

e3fdd4d

Add Makefile as alternative entry point for running checks. All targets delegate to tox (source of truth). Includes `make help` for discoverability.

feat(tooling): run checks in parallel for faster feedback

6cc0385

Run all static checks in parallel via just/make for ~2x speedup (~6s → ~3s) and better error reporting (all checks run even if one fails).

danceratopz added C-feat Category: an improvement or new feature A-tooling Area: Improvements or changes to auxiliary tooling such as uv, ruff, mypy, ... A-ci Area: Continuous Integration labels Jan 29, 2026

danceratopz requested review from SamWilsn, felix314159, fselmo and spencer-tb January 29, 2026 09:24

oops broke ci

b669a88

danceratopz force-pushed the split-static branch 2 times, most recently from c7ebf91 to c02c126 Compare January 29, 2026 10:25

danceratopz added 2 commits January 29, 2026 11:28

Revert "oops broke ci"

c309c56

This reverts commit b669a88.

danceratopz force-pushed the split-static branch from c02c126 to c309c56 Compare January 29, 2026 10:28

SamWilsn reviewed Jan 29, 2026

View reviewed changes

marioevz assigned felix314159 Feb 18, 2026

SamWilsn reviewed Feb 23, 2026

View reviewed changes

felix314159 mentioned this pull request Mar 2, 2026

Refactor static into separate tox environments & dedicated fast-checks workflow #2041

Open

danceratopz closed this Mar 25, 2026

	The Python code in @ethereum/execution-specs is formatted with `ruff` with a line length of 100 characters.
	The Python code in @ethereum/execution-specs is formatted with `ruff` with a line length of 79 characters.

Conversation

danceratopz commented Jan 29, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

🗒️ Description

Fix Hint Github Summary Screenshot

Fix Hint Console Output (also available locally)

🔗 Related Issues or PRs

✅ Checklist

Cute Animal Picture

Uh oh!

danceratopz commented Jan 29, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

SamWilsn left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

felix314159 commented Mar 2, 2026

Uh oh!

danceratopz commented Mar 3, 2026

Uh oh!

felix314159 commented Mar 3, 2026

Uh oh!

danceratopz commented Mar 25, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

danceratopz commented Jan 29, 2026 •

edited

Loading

danceratopz commented Jan 29, 2026 •

edited

Loading