feat!: Merge 1.0-dev branch to main - #182
Closed
jmesnil wants to merge 199 commits into
Closed
Conversation
* Rename REST to HTTP+JSON to be consistent with the spec. * a2a.json is derived from a2a.proto (not pulled from GitHub) * maybe generate junitreport.xml and use existing tool for HTML reports There are other things we discussed: * [ ] having the tck query the agent to find the exposed capabilities, create a test plan to run (so that any capabilities requirements would be tested) and then run the tests * [ ] having the grpc stubs be versioned * [ ] keep our own mapping of the spec requirements (to avoid having to section numbers diverge) * [ ] having the ability to periodically pull the latest a2a.proto and compare what's new/updated/removed with the current requirements tested by the tck Signed-off-by: Jeff Mesnil <jmesnil@ibm.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
Signed-off-by: Jeff Mesnil <jmesnil@ibm.com>
Signed-off-by: Jeff Mesnil <jmesnil@ibm.com>
Signed-off-by: Jeff Mesnil <jmesnil@ibm.com>
Signed-off-by: Jeff Mesnil <jmesnil@ibm.com>
Signed-off-by: Jeff Mesnil <jmesnil@ibm.com>
Add RequirementSpec base class with RequirementLevel enum, OperationType enum, TransportBinding dataclass, and RequirementSpec dataclass for defining testable A2A protocol requirements per PRD Section 5.1.1. Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Generate Python proto stubs from a2a.proto using buf. Added buf.gen.yaml config, generated a2a_pb2.py and a2a_pb2_grpc.py stubs, and created __init__.py that adds the generated dir to sys.path for proper imports. Also added googleapis-common-protos dependency and Makefile with proto target. Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Signed-off-by: Jeff Mesnil <jmesnil@ibm.com>
Signed-off-by: Jeff Mesnil <jmesnil@ibm.com>
Signed-off-by: Jeff Mesnil <jmesnil@ibm.com>
Signed-off-by: Jeff Mesnil <jmesnil@ibm.com>
Create requirement registry skeleton with ALL_REQUIREMENTS list and helper functions for filtering by level, section, operation, and tag. Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Signed-off-by: Jeff Mesnil <jmesnil@ibm.com>
Signed-off-by: Jeff Mesnil <jmesnil@ibm.com>
Add JSON Schema validator that validates responses against a2a.json schema definitions. Supports custom $ref resolution, collects all errors with JSON paths, and uses Draft202012Validator. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Add proto schema validator that validates protobuf messages against message descriptors. Supports type checking, required field validation using google.api.field_behavior annotation, and recursive nested message validation. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Add error validator that validates JSON-RPC error responses against the error code mappings defined in A2A spec Section 9. Includes all A2A-specific and standard JSON-RPC error codes. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Add HTTP+JSON error validator with RFC 7807 Problem Details support. Rename validators/rest to validators/http_json for consistency with the A2A specification terminology. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Mark TASK-2 and TASK-2.5 as done with all acceptance criteria verified. All 105 validator unit tests pass across all four validators. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Add JsonRpcClient (JSON-RPC 2.0 over HTTP with SSE streaming) - Update BaseTransportClient with full proto parameter signatures - Update GrpcClient to match new base class signatures - Fix all ruff lint issues in tck/ and tests/ directories - Update .ruff.toml with project-appropriate ignores and exclusions Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Add HTTP+JSON (REST) transport client per A2A spec Section 11 with RESTful URL patterns, RFC 9457 Problem Details error handling, and SSE streaming support. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Add transport manager for orchestrating client creation, selection, and lifecycle across gRPC, JSON-RPC, and HTTP+JSON transports. Archive completed tasks 3.1-3.4. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Add a Python code emitter that generates a runnable SUT project from Gherkin scenarios using the a2a-python SDK, mirroring the existing a2a-java pipeline. New files: - codegen/python_emitter.py — maps Actions to TaskUpdater API calls - codegen/a2a-python/*.j2 — Jinja2 templates (sut_agent.py, pyproject.toml) - tests/unit/codegen/test_python_emitter.py — 6 unit tests - sut/a2a-python/ — generated SUT project (16 scenarios) Modified: - codegen/generator.py — register a2a-python emitter - Makefile — add codegen-a2a-python-sut target - AGENTS.md — document new emitter and commands - .gitignore — scope a2a-python ignore to root only TCK result: 76.5% compatibility (remaining failures are SDK-level). --------- Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
…2aproject#153) + fix: use DataPart.fromJson() instead of constructor in a2a-java SUT --------- Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
The gRPC SubscribeToTask hang was a TCK bug (missing timeout on streaming RPCs fixed in a2aproject#153 ), not a SUT bug. Remove the --deselect workaround from the a2a-java-sut skill and update the run-tck skill to reflect the built-in 30s timeout. Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
The gRPC SubscribeToTask hang was a TCK bug (missing timeout on streaming RPCs that was fixed in a2aproject#153 ). Remove the --deselect workaround from the a2a-java-sut skill and update the run-tck skill to reflect the built-in 30s timeout. Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
…2aproject#156) MUST and SHOULD test runners were missing the capability-based skip logic that MAY tests already had, causing false failures for requirements like CARD-EXT-001/002 when the SUT doesn't declare the capability. Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
Runs ruff check and pytest unit tests on pushes and PRs to main and 1.0-dev branches. Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
Runs ruff check and pytest unit tests on pushes and PRs to main and 1.0-dev branches. Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
…tor (a2aproject#159) The `_is_repeated_field` helper assumed `is_repeated` was a callable method, but on newer UPB-based protobuf descriptors it is a property. The fallback to `.label` also failed because UPB descriptors don't expose that attribute, causing GRPC-SVC-001 to fail with an AttributeError. Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
Signed-off-by: Jeff Mesnil <jmesnil@ibm.com>
Signed-off-by: Jeff Mesnil <jmesnil@ibm.com>
Signed-off-by: Jeff Mesnil <jmesnil@ibm.com>
…t contamination (a2aproject#164) When parametrized requirement tests run across multiple transports, they share the same messageId and taskId values. If the SUT persists state (e.g. creates a task from a client-provided taskId), later transports hit different code paths and produce misleading error messages. Append the transport name to messageId and taskId in sample_input so each transport gets its own isolated state on the SUT. Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
CORE-SEND-002 requires a completed task to exist before it can test sending a message to a terminal task. The parametrized runner in test_requirements.py dispatches a single operation and cannot create prerequisite tasks. The dedicated test in test_task_lifecycle.py handles the setup correctly. Tag the requirement as multi-operation so _parametrize_requirements skips it. Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
…ients (a2aproject#167) - grpc: rewrite _streaming_call to peek at the first event so that UNIMPLEMENTED / connection errors surface as success=False instead of being silently deferred; fix retry bug where rpc was not re-fetched from the new stub after reconnect - jsonrpc: check Content-Type before entering SSE mode so that plain JSON-RPC error responses are parsed and reflected in success/error_code - tests: replace raw response inspection with response.success / response.error_code; drop unused contextlib import Signed-off-by: Emmanuel Hugonnet <ehugonne@redhat.com>
Add a new code generation target (a2a-jakarta) that produces a WildFly WAR application from Gherkin scenarios, alongside the existing Quarkus (a2a-java) and Python SUTs. The Jakarta emitter reuses action-to-Java logic from java_emitter.py and adds WildFly-specific templates (Galleon provisioning, Maven profiles for jsonrpc/grpc/rest transports, JAX-RS Application class, and beans.xml). Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> --------- Signed-off-by: Jeff Mesnil <jmesnil@ibm.com> Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com> Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
…project#172) * add TASK-11.7 for push notification conformance tests * Implements TASK-11.7: the TCK now spins up a lightweight HTTP server during tests to receive push notification webhooks, making all three PUSH-DELIVER-* requirements automatable. * enable push notification capability in a2a-java SUT Changes: - Add tck/webhook/server.py with WebhookReceiver (captures requests for assertion in tests) - Add delivery tests for PUSH-DELIVER-001 (auth headers), PUSH-DELIVER-002 (at-least-once delivery), PUSH-DELIVER-003 (StreamResponse payload format) - Add --webhook-host CLI option for remote SUT setups - Remove NOT_AUTOMATABLE tag from PUSH-DELIVER-* requirements - Add unit tests for WebhookReceiver - Add has_push_notifications flag to Java emitter context - Update agent card template with .pushNotifications(true) - Regenerate a2a-java SUT --------- Signed-off-by: Jeff Mesnil <jmesnil@ibm.com> Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com> Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
Define schema ref constants in tck.validators so all validators share a common vocabulary. ProtoSchemaValidator now accepts string refs (e.g. "Stream Response") and resolves them internally, matching the interface JSONSchemaValidator and JsonRpcResponseValidator already use. Streaming tests validate every event against the schema via validators[transport].validate(event, STREAM_RESPONSE) with no transport-specific branching. Also unifies the two ordering check functions into one, using a single state-order map keyed by json_value. --------- Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
Full TCK run against Java SUT identified 27 requirements with no test coverage. Added tasks for auth (TASK-27), binding equivalence (TASK-28), agent card signing (TASK-29), versioning (TASK-30), and misc gaps (TASK-31). --------- Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
…#176) Tests that bypass the transport client and make direct httpx calls were missing the A2A-Version header, which could cause failures on SUTs that enforce version checking. Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
…ct#175) Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
Also, don't propagate errors when draining SSE events during cleanup
…ject#179) The A2A spec permits extension fields on agent cards, but the JSON schema's additionalProperties:false was rejecting them. Add an allow_additional parameter to JSONSchemaValidator.validate() that strips that constraint, so the TCK validates structure and types without rejecting unknown fields. Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
![gemini-code-assist[bot]](https://avatars.githubusercontent.com/in/956858?s=60&v=4){kind=small}
Contributor
There was a problem hiding this comment.
Code Review
This pull request establishes the A2A Protocol Technology Compatibility Kit (TCK) v1.0, providing a comprehensive framework for validating agent compliance across gRPC, JSON-RPC, and HTTP+JSON transports. The changes introduce a Python-based code generator for various SUT targets, a suite of specialized agent skills, and extensive documentation including a PRD and contribution guides. Feedback identifies critical merge conflict markers in the .gitignore file that must be resolved, along with a typo in AGENTS.md and an inconsistent function reference in the PRD documentation.
Signed-off-by: Jeff Mesnil <jmesnil@ibm.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
Contributor
Author
|
I used a merge commit to merge this PR in |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
4 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.