Add comprehensive execution tests

[JasonTheAdams]

Summary

• Expand the wp-core-v1 execution suite to 150 deterministic tests across core runtime, content/media/platform, Gutenberg, and AI/platform APIs.
• Remove unused AI judge/quality scaffolding and make expected_behavior reviewer-facing documentation.
• Add REST/output verifier support, dataset validation, and WordPress 7.0 wp-env configuration.

Verification

• .venv/bin/python -m pytest python
• .venv/bin/python -m ruff check python
• .venv/bin/python -m compileall -q python/wp_bench
• .venv/bin/wp-bench dry-run --config wp-bench.yaml --test-type execution
• .venv/bin/python datasets/export_dataset.py
• git diff --check
• python3 /private/tmp/verify_wpbench_execution_refs.py --mode reference
• python3 /private/tmp/verify_wpbench_execution_refs.py --mode wrong

[github-actions]

The following accounts have interacted with this PR and/or linked issues. I will continue to update these lists as activity occurs. You can also manually ask me to refresh this list by adding the props-bot label.

If you're merging code through a pull request on GitHub, copy and paste the following into the bottom of the merge commit message.

Co-authored-by: JasonTheAdams <jason_the_adams@git.wordpress.org>
Co-authored-by: lezama <migueluy@git.wordpress.org>

To understand the WordPress project's expectations around crediting contributors, please review the Contributor Attribution page in the Core Handbook.

[lezama]

nice, this is a great expansion — the suite feels solid and the judge/quality cleanup is nice and tidy 👍

couple of things i'd tweak before merging, the rest can iterate:

• correctness now averages static + runtime 50/50, and an empty static check scores 1.0 — so a runtime-only test gets a free 0.5. all 150 tests have both today so it's fine in practice, but i'd guard it (only fold static in when there actually are static checks, and probably weight runtime a bit higher).
• assert_rest_response passes anything when the assertion has no expected_status/expected_data/body_* (falls through to passed = true). i'd default status to 200 or treat an empty assertion as a fail.
• heads up: dockerfile is still on WP 6.9 while .wp-env.json moved to 7.0 — the docker grader would run the wrong core against the 7.0 tests.

smaller stuff, totally fine to defer: loose == in expected_data ({"id":42} also accepts "42"), the match block is over-indented, and output_not_contains isn't used by any test yet.

would also love a CI smoke that runs the reference solutions through wp bench verify at some point — 147/150 are custom assertions so it'd catch a lot. not blocking 🙂

great work overall!

[JasonTheAdams]

Thanks, @lezama! Something I'm considering is whether the test "difficulty" property is meaningful. It got added quite a bit ago, and somehow made sense at the time. But looking at it now and seeing how the difficulty was selected for various tests. It feels too subjective.

Do you see a reason to keep it? Otherwise, I'm inclined to remove it.

[lezama]

yeah i'd drop it. poked at the data and a few things stood out:

• it's not actually used anywhere — we parse, store, validate and export it, but nothing reads it for scoring, filtering, or in the report. pure dead metadata right now.
• in the execution suite it barely varies anyway (123 intermediate / 27 hard, no easy tier), so even as a label it's not saying much.
• and the taxonomies don't even match across suites (execution is intermediate/hard, knowledge is basic/intermediate/hard) — kind of proves your point about it being subjective.

so unless we want difficulty-sliced reporting soon, i'd remove it and re-add later with a real rubric if we ever need per-difficulty breakdowns. happy to take the removal on if it helps 🙂