Correction, 14 Aug 2026: the paragraph beginning "One of the four is this class" misdescribes the delivered artefact. In the delivered contract both enumerations are of six and agree with each other; the enumeration of two existed earlier in the run and should not have been presented as the shipped contract. Errata: https://doi.org/10.5281/zenodo.21926137 — see my comment below for what this does and does not cost the request. The original text is left intact.
Disclosure, per the Spec Kit AI contributions policy: this issue was drafted with AI assistance. Extent: I directed the investigation and chose the framing; an AI assistant read the source material and wrote the draft; I reviewed every claim against the sources cited and I am responsible for the content. Replies from me are my own.
Problem statement
When a spec declares a closed set of values — statuses, categories, admissible kinds — and more than one part of the spec refers to that set, the set gets restated in each place. There is usually nowhere to declare it once. So it multiplies, and any error in it multiplies with it.
This comes from a measured case rather than a hunch.
I ran an experiment where a 40-module system was generated entirely from a contract — no module written or edited by hand, 44 model calls — and verified against reference trajectories published beforehand in an independent formalism.
Four defects survived complete automated verification. None was in the 391 generated statements. All four were in the 30 hand-written contract signatures, and all four were the same error: mis-transcribing a definition that was available in the project. The model wrote correctly what the contract declared, including what the contract declared wrongly.
One of the four is this class. Three parts of the contract declared which invariant classes were admissible. A correction reached one of the three; the other two kept the original wording. The delivered system carried two incompatible vocabularies for the same closed set — one enumerating two values, the others six — and all three enumerations were wrong, in three different ways.
Two verifications could have caught it and neither did. Structural verification compares the contract against the code: it checks which module decides what, not which vocabulary each signature declares in prose. The trajectory oracle validates the running dynamics against published reference traces: it never reached it, because the reference instances do not exercise that part of the formalism. Trace validation validates what the trace traverses.
The contradiction was real, present in the contract, and inert over the validation corpus. It would have surfaced the first time someone ran an instance outside the two validated ones.
The gap, stated precisely: the verification covered the structure of the artifacts and the dynamics of execution. The vocabulary declared in prose inside the specification was covered by neither. I had the analogous check for numeric constants — a threshold declared twice with two values — and it caught those. The same defect over a set of names went straight through.
Proposed solution
In /speckit.analyze, which already does cross-artifact consistency and coverage, add a check for closed vocabularies: flag when the same named set of values is enumerated in more than one artifact or section and the enumerations are not identical, and report the locations and the difference between them.
It needs no domain understanding — only to notice that a named set is enumerated in more than one place and that the enumerations differ. And it is invisible to every other stage: it is not a coverage gap, since every requirement is covered, and not a structural inconsistency between spec, plan and tasks, since the artifacts agree about what exists. It is two statements of the same closed set that disagree about its members, sitting inert until an execution path finally traverses it.
A warning rather than a gate seems right. Legitimate divergence exists — one section sometimes intends a subset of another — and the point is that today the divergence is invisible either way.
A second signal in the same family, if it helps: the same multiplication happened to a procedure rather than a vocabulary. Six signatures repeated the identical construction recipe for one object, because the contract language had no place to declare how a structure is built. Fields had somewhere to go; the construction procedure did not, so it migrated into the signatures and multiplied. Wherever a spec language lacks a home for something, that thing ends up copied, and the copies drift.
Reference, if the case is of interest: When the Defect Is in the Specification: Trajectory-Level Validation of Generated Code, https://doi.org/10.5281/zenodo.21879238 — §8.3 is this defect with its three transcriptions, §10.1 states the check as pending work. The report declares its own limits plainly: one formalism, one model, one operator, and the author of the oracle is the operator of the experiment.
Correction, 14 Aug 2026: the paragraph beginning "One of the four is this class" misdescribes the delivered artefact. In the delivered contract both enumerations are of six and agree with each other; the enumeration of two existed earlier in the run and should not have been presented as the shipped contract. Errata: https://doi.org/10.5281/zenodo.21926137 — see my comment below for what this does and does not cost the request. The original text is left intact.
Disclosure, per the Spec Kit AI contributions policy: this issue was drafted with AI assistance. Extent: I directed the investigation and chose the framing; an AI assistant read the source material and wrote the draft; I reviewed every claim against the sources cited and I am responsible for the content. Replies from me are my own.
Problem statement
When a spec declares a closed set of values — statuses, categories, admissible kinds — and more than one part of the spec refers to that set, the set gets restated in each place. There is usually nowhere to declare it once. So it multiplies, and any error in it multiplies with it.
This comes from a measured case rather than a hunch.
I ran an experiment where a 40-module system was generated entirely from a contract — no module written or edited by hand, 44 model calls — and verified against reference trajectories published beforehand in an independent formalism.
Four defects survived complete automated verification. None was in the 391 generated statements. All four were in the 30 hand-written contract signatures, and all four were the same error: mis-transcribing a definition that was available in the project. The model wrote correctly what the contract declared, including what the contract declared wrongly.
One of the four is this class. Three parts of the contract declared which invariant classes were admissible. A correction reached one of the three; the other two kept the original wording. The delivered system carried two incompatible vocabularies for the same closed set — one enumerating two values, the others six — and all three enumerations were wrong, in three different ways.
Two verifications could have caught it and neither did. Structural verification compares the contract against the code: it checks which module decides what, not which vocabulary each signature declares in prose. The trajectory oracle validates the running dynamics against published reference traces: it never reached it, because the reference instances do not exercise that part of the formalism. Trace validation validates what the trace traverses.
The contradiction was real, present in the contract, and inert over the validation corpus. It would have surfaced the first time someone ran an instance outside the two validated ones.
The gap, stated precisely: the verification covered the structure of the artifacts and the dynamics of execution. The vocabulary declared in prose inside the specification was covered by neither. I had the analogous check for numeric constants — a threshold declared twice with two values — and it caught those. The same defect over a set of names went straight through.
Proposed solution
In
/speckit.analyze, which already does cross-artifact consistency and coverage, add a check for closed vocabularies: flag when the same named set of values is enumerated in more than one artifact or section and the enumerations are not identical, and report the locations and the difference between them.It needs no domain understanding — only to notice that a named set is enumerated in more than one place and that the enumerations differ. And it is invisible to every other stage: it is not a coverage gap, since every requirement is covered, and not a structural inconsistency between spec, plan and tasks, since the artifacts agree about what exists. It is two statements of the same closed set that disagree about its members, sitting inert until an execution path finally traverses it.
A warning rather than a gate seems right. Legitimate divergence exists — one section sometimes intends a subset of another — and the point is that today the divergence is invisible either way.
A second signal in the same family, if it helps: the same multiplication happened to a procedure rather than a vocabulary. Six signatures repeated the identical construction recipe for one object, because the contract language had no place to declare how a structure is built. Fields had somewhere to go; the construction procedure did not, so it migrated into the signatures and multiplied. Wherever a spec language lacks a home for something, that thing ends up copied, and the copies drift.
Reference, if the case is of interest: When the Defect Is in the Specification: Trajectory-Level Validation of Generated Code, https://doi.org/10.5281/zenodo.21879238 — §8.3 is this defect with its three transcriptions, §10.1 states the check as pending work. The report declares its own limits plainly: one formalism, one model, one operator, and the author of the oracle is the operator of the experiment.