Add conformance test suites from September 2025 GraphQL specification - #910
Open
hugo-vrijswijk wants to merge 1 commit into
Open
Add conformance test suites from September 2025 GraphQL specification#910hugo-vrijswijk wants to merge 1 commit into
hugo-vrijswijk wants to merge 1 commit into
Conversation
Closes #26 Introduces a `ConformanceSuite` and a number of test suites based on examples from the September 2025 GraphQL specification. Roughly, each suite corresponds to a section of the specification, and each test case corresponds to an example or counter-example in the specification text. The suites cover every example, including the examples which currently fail. Each failing case carries a `.fail` marker. The markers record the current conformance status as a baseline. Out of 221 cases, 31 currently fail. Issue #908 will close 3 of them. Some cases fail because of small bugs. Others fail because a feature is missing, such as allowing `description` on the root schema. Others fail because of missing minor validations. The suites use these test helpers: - `parses` to check that a document parses - `validSchema`/`invalidSchema` to check the validation of a schema - `validQuery`/`invalidQuery` to check the validation of a query - `yields` to run a query against a mapping and compare the response - `yieldsFieldOrder` to run a query against a mapping and compare the order of the response fields
hugo-vrijswijk
force-pushed
the
conformance-suite-2
branch
from
August 24, 2026 15:06
1e9565c to
3f30715
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #26
Introduces a
ConformanceSuiteand a number of test suites based on examples from the September 2025 GraphQL specification. Roughly, each suite corresponds to a section of the specification, and each test case corresponds to an example or counter-example in the specification text.The suites cover every example, including the examples which currently fail. Each failing case carries a
.failmarker. The markers record the current conformance status as a baseline. Out of 221 cases, 31 currently fail. Issue #908 will close 3 of them. Some cases fail because of small bugs. Others fail because a feature is missing, such as allowingdescriptionon the root schema. Others fail because of missing minor validations.The suites use these test helpers:
parsesto check that a document parsesvalidSchema/invalidSchemato check the validation of a schemavalidQuery/invalidQueryto check the validation of a queryyieldsto run a query against a mapping and compare the responseyieldsFieldOrderto run a query against a mapping and compare the order of the response fieldsExtracting the conformance suite from the spec was done by AI, but all code has been read, reviewed and understood by me.