Skip to content

[antithesis] Enable reuse of banff e2e test for antithesis testing #3554

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 14 commits into from
Jul 21, 2025

Conversation

marun
Copy link
Contributor

@marun marun commented Nov 19, 2024

Why this should be merged

Having to duplicate existing test coverage to benefit from execution with antithesis is not ideal. Better to be able to trivially refactor compatible e2e coverage for reuse. This PR refactors the banff e2e test for reuse with antithesis to demonstrate that this is possible.

How this works

  • adds new APITestFunction type that a compatible e2e test can implement to allow for execution with both ginkgo and antithesis
  • adds ExecuteAPITest helper to simplify execution of an APITestFunction with ginkgo
  • Refactors the banff e2e test into an APITestFunction executed by ExecuteAPITest
  • Refactors the avalanchego antithesis test setup to support execution of the banff test
    • Moves test dispatch into a new function so that panics can be recovered via a deferred function
      • This supports use of require by e2e tests without risking an unhandled panic exiting the test process

How this was tested

CI

Need to be documented in RELEASES.md?

N/A

@marun marun marked this pull request as draft November 19, 2024 15:24
@marun marun changed the title bE2e reuse [antithesis] Enable reuse of banff e2e test for antithesis testing Nov 19, 2024
@marun marun added the testing This primarily focuses on testing label Nov 19, 2024
@marun marun self-assigned this Nov 19, 2024
@marun marun marked this pull request as ready for review November 19, 2024 18:14
@marun marun marked this pull request as draft November 20, 2024 21:59
@marun
Copy link
Contributor Author

marun commented Nov 20, 2024

Switched to draft in light of wanting to rebase on #3557

@marun marun changed the base branch from master to testing-switch-to-zap November 21, 2024 13:52
@marun marun force-pushed the testing-switch-to-zap branch from cf8a4d4 to 292016a Compare November 21, 2024 19:27
Base automatically changed from testing-switch-to-zap to master November 21, 2024 22:39
@marun marun marked this pull request as ready for review November 22, 2024 16:12
@marun marun marked this pull request as draft December 5, 2024 13:44
@marun
Copy link
Contributor Author

marun commented Dec 5, 2024

Moving to draft and adding a bunch of TODOs prompted by a manually-triggered antithesis run. The error handling of the e2e test is not sufficiently robust, so there are a lot of unhelpful errors that need to be addressed before this PR will be mergeable.

@marun marun force-pushed the e2e-reuse branch 3 times, most recently from 6e0b41c to 981663d Compare December 14, 2024 20:33
Copy link

This PR has become stale because it has been open for 30 days with no activity. Adding the lifecycle/frozen label will cause this PR to ignore lifecycle events.

Copy link

github-actions bot commented Mar 9, 2025

This PR has become stale because it has been open for 30 days with no activity. Adding the lifecycle/frozen label will cause this PR to ignore lifecycle events.

@maru-ava
Copy link
Contributor

Approving my own PR is a treat (because my personal account proposed it), but I'll hold off on merging until someone else approves.

@maru-ava maru-ava removed the request for review from Elvis339 July 17, 2025 04:52
// execTestWithRecovery ensures assertion-related panics encountered during test execution are recovered
// and that deferred cleanups are always executed before returning.
func execTestWithRecovery(ctx context.Context, log logging.Logger, test Test, wallet *Wallet) {
tc := tests.NewTestContext(log)
Copy link
Contributor

@maru-ava maru-ava Jul 18, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Creating a new TestContext here ensures that recovery executes only the cleanups registered to this context.

@@ -85,10 +85,7 @@ func (l LoadGenerator) Run(
default:
}

ctx, cancel := context.WithTimeout(ctx, testTimeout)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Moved to execTestWithRecovery to ensure that cancelation is localized to test execution rather than only being performed on goroutine exit.

Copy link
Contributor

@RodrigoVillar RodrigoVillar left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nit: since the test will now gracefully recover in the case of a context cancellation, we should modify the following:

childCtx := ctx
if loadTimeout != 0 {
ctx, cancel := context.WithTimeout(ctx, loadTimeout)
childCtx = ctx
defer cancel()
}

to just:

if loadTimeout != 0 {
    childCtx, cancel := context.WithTimeout(ctx, loadTimeout)
    ctx = childCtx
    defer cancel()
}

This way, Run() only deals with just one context which is easier to read + having two separate contexts isn't necessary as a result of this PR.

@maru-ava
Copy link
Contributor

This way, Run() only deals with just one context which is easier to read + having two separate contexts isn't necessary as a result of this PR.

Done

@maru-ava
Copy link
Contributor

I'd like to rename SimpleTestContext to RecoverableTestContext, but would prefer to do that in a follow-on PR.

Copy link
Contributor

@StephenButtolph StephenButtolph left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A couple minor comments / questions. Feel free to merge after resolving.

@StephenButtolph StephenButtolph added this pull request to the merge queue Jul 21, 2025
Merged via the queue into master with commit c82d683 Jul 21, 2025
29 checks passed
@StephenButtolph StephenButtolph deleted the e2e-reuse branch July 21, 2025 20:46
@github-project-automation github-project-automation bot moved this from In Progress 🏗️ to Done 🎉 in avalanchego Jul 21, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
testing This primarily focuses on testing tooling Build, test and development tooling
Projects
Status: Done 🎉
Development

Successfully merging this pull request may close these issues.

4 participants