Skip to content

Consolidate ask_user E2E snapshots into single canonical folder#1311

Merged
stephentoub merged 1 commit into
mainfrom
stephentoub/fix-issue-314
May 16, 2026
Merged

Consolidate ask_user E2E snapshots into single canonical folder#1311
stephentoub merged 1 commit into
mainfrom
stephentoub/fix-issue-314

Conversation

@stephentoub
Copy link
Copy Markdown
Collaborator

Fixes #314

test/snapshots/ had three sibling folders for the same scenario (ask-user/, ask_user/, askuser/), which caused confusion and snapshot drift across SDKs. This PR consolidates them.

What's actually used

All five SDK E2E harnesses already resolve their snapshot path to ask_user/:

SDK How the path is derived
Node.js sdkTestContext.ts strips .test.ts / .e2e and replaces - with _, so ask_user.e2e.test.ts -> ask_user
Python conftest.py strips test_ and _e2e, so test_ask_user_e2e.py -> ask_user
Go testharness/context.go strips _test.go / _e2e, so ask_user_e2e_test.go -> ask_user
.NET AskUserE2ETests.cs passes "ask_user" explicitly
Rust tests/e2e/ask_user.rs passes "ask_user" explicitly

No harness has any fallback / alternative-spelling logic, and the replaying CAPI proxy uses the exact path the harness gives it. So ask-user/ and askuser/ were dead snapshots that only existed because earlier captures were regenerated under different folder names.

The change

Deletions only, no source/test/harness code changes:

  • Removed test/snapshots/ask-user/ (3 should_*.yaml files)
  • Removed test/snapshots/askuser/ (3 should_*.yaml files)
  • Removed 3 stale files inside test/snapshots/ask_user/ left over from before PR Fail CI if snapshots aren't present #304 normalized test names to use the should_ prefix:
    • handle_freeform_user_input_response.yaml
    • invoke_user_input_handler_when_model_uses_ask_user_tool.yaml
    • receive_choices_in_user_input_request.yaml

Kept the 3 canonical ask_user/should_*.yaml files that every SDK's tests actually replay against.

Validation

All ask_user E2E suites pass in snapshot-replay mode (GITHUB_ACTIONS=true) against the consolidated folder: Node.js 3/3, Python 3/3, Go 3/3, .NET 3/3, Rust 3/3.

Scope

This is scoped to ask_user per #314. The remaining mcp_and_agents / mcp-and-agents sibling pair is tracked separately as the related issue #315 and is intentionally left alone here.

Three sibling folders existed for the same scenario in test/snapshots/:

- ask_user/ (canonical, targeted by all five SDK harnesses)

- ask-user/ (unused legacy duplicate)

- askuser/ (unused legacy duplicate)

All five SDK E2E harnesses (Node, Python, Go, .NET, Rust) resolve their

snapshot path to ask_user/. Removed the two duplicate folders and three

stale files inside ask_user/ that were left over from before the test

names were normalized to use a should_ prefix (PR #304). Only the three

canonical should_*.yaml files remain.

Fixes #314

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@stephentoub stephentoub requested a review from a team as a code owner May 16, 2026 18:35
Copilot AI review requested due to automatic review settings May 16, 2026 18:35
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Consolidates duplicate ask_user E2E snapshot folders into the single canonical test/snapshots/ask_user/ directory that all five SDK harnesses actually resolve to, eliminating dead snapshots and snapshot drift.

Changes:

  • Deletes the unused test/snapshots/ask-user/ folder (3 should_*.yaml files).
  • Deletes the unused test/snapshots/askuser/ folder (3 should_*.yaml files).
  • Removes 3 stale pre-should_-prefix files from test/snapshots/ask_user/, keeping only the canonical should_*.yaml snapshots.
Show a summary per file
File Description
test/snapshots/ask-user/should_receive_choices_in_user_input_request.yaml Removes unused duplicate snapshot.
test/snapshots/ask-user/should_invoke_user_input_handler_when_model_uses_ask_user_tool.yaml Removes unused duplicate snapshot.
test/snapshots/ask-user/should_handle_freeform_user_input_response.yaml Removes unused duplicate snapshot.
test/snapshots/askuser/should_receive_choices_in_user_input_request.yaml Removes unused duplicate snapshot.
test/snapshots/askuser/should_invoke_user_input_handler_when_model_uses_ask_user_tool.yaml Removes unused duplicate snapshot.
test/snapshots/askuser/should_handle_freeform_user_input_response.yaml Removes unused duplicate snapshot.
test/snapshots/ask_user/receive_choices_in_user_input_request.yaml Removes stale pre-should_-prefix snapshot.
test/snapshots/ask_user/invoke_user_input_handler_when_model_uses_ask_user_tool.yaml Removes stale pre-should_-prefix snapshot.
test/snapshots/ask_user/handle_freeform_user_input_response.yaml Removes stale pre-should_-prefix snapshot.

Copilot's findings

  • Files reviewed: 9/9 changed files
  • Comments generated: 0

@stephentoub stephentoub merged commit bc0f1c4 into main May 16, 2026
33 checks passed
@stephentoub stephentoub deleted the stephentoub/fix-issue-314 branch May 16, 2026 18:47
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Test Snapshots for Ask User has three folders

3 participants