Skip to content

[WIP] Verify deterministic issue authoring across models #1805

Description

@marcelo-cjl

Summary

This temporary tracking issue verifies that different AI models consuming the shared skills produce consistent, repository-compatible GitHub issue drafts and publication results.

Current behavior

The first trial produced a Chinese free-form issue body and then closed the issue when its plan was incomplete, so language, structure, and lifecycle handling were not consistent.

Expected behavior

Every model should render the same English structure, keep incomplete work open with a leading WIP marker, and publish or update only the canonical controller-generated draft.

Environment

  • Repository: zilliztech/knowhere
  • Authoring workflow: provider-neutral shared skills

Reproduction

  1. Ask a model consuming the shared skills to draft a Knowhere issue from the same structured facts.
  2. Compare the generated title, language, section order, and lifecycle decision across models.
  3. Publish or update through the authoring controller and compare the live issue with the draft.

Evidence

  • The initial issue body used Chinese headings and described command execution as reproduction steps.
  • The issue was closed as not planned even though incomplete work should remain open as WIP.

Impact

Different models can otherwise publish materially different or misleading issues from the same request, reducing the reliability of the shared skills workflow.

Affected areas

  • Cross-model skill consumption
  • GitHub issue drafting and lifecycle handling
  • Remote publication read-back

Acceptance criteria

  • Public issue content is written in English.
  • Incomplete content has WIP as the first title marker and remains open.
  • The same structured facts render the same title and section order across models.
  • Remote publication or update is read back and matches the canonical draft.

Root cause

Authoring constraints were previously advisory prose rather than one executable renderer with explicit ready and WIP states.

Proposed direction

Route drafting, publication, update, and read-back through the deterministic issue authoring controller while models provide only structured facts.

Limitations

This issue tracks the authoring workflow and does not report a Knowhere product defect.

Related links

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

No labels
No labels

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions