Skip to content

chore: standardize mise quality task contract - #716

Open
binaryaaron wants to merge 2 commits into
mainfrom
binaryaaron/mise-quality-contract
Open

chore: standardize mise quality task contract#716
binaryaaron wants to merge 2 commits into
mainfrom
binaryaaron/mise-quality-contract

Conversation

@binaryaaron

@binaryaaron binaryaaron commented Aug 12, 2026

Copy link
Copy Markdown
Collaborator

Summary

  • Replace legacy quality tasks with read-only check:* leaves, aggregate check, and mutating lock:update.
  • Move CI, pre-commit, contributor docs, and bundled skills to the canonical task contract, and remove the matching Make aliases.
  • Add regression coverage for task descriptions, dependencies, retired names and alias, lock commands, hook inputs, and local-gate composition.

Dependency

Test plan

  • mise tasks validate
  • mise run check ::: test (1,843 passed)
  • uv-lock hook against pyproject.toml and cuda_deps.toml
  • stale-lock failure probe and identical tracked state before and after mise run check

Summary by CodeRabbit

  • Chores

    • Modernized quality-check workflows with dedicated formatting, linting, type, license-header, lockfile, and task-validation checks.
    • Added a unified local check-and-test gate for pre-PR verification.
    • Added a command for updating dependency lockfiles.
    • Removed deprecated validation and Make targets.
  • Documentation

    • Updated contributor, CI, workflow, and development guidance to use the current commands and quality-check process.
  • Tests

    • Added coverage validating quality-task definitions, dependencies, hooks, and local check-and-test composition.

Signed-off-by: Aaron Gonzales <aagonzales@nvidia.com>
@binaryaaron
binaryaaron requested review from a team as code owners August 12, 2026 20:19
@github-actions github-actions Bot added area:dev-ex Affects build or dev experience area:ci area:tests area:docs labels Aug 12, 2026
@coderabbitai

coderabbitai Bot commented Aug 12, 2026

Copy link
Copy Markdown
Contributor

Review Change Stack

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: ac2323d5-bb6f-4aa5-ae5e-dda13da6fb97

📥 Commits

Reviewing files that changed from the base of the PR and between 6f60ce8 and a02f36a.

📒 Files selected for processing (1)
  • Makefile
💤 Files with no reviewable changes (1)
  • Makefile
📜 Recent review details
⏰ Context from checks skipped due to timeout. (1)
  • GitHub Check: Greptile Review

Walkthrough

The PR replaces legacy Mise quality tasks with dedicated read-only checks, adds a lock update task, updates CI and pre-commit integrations, removes deprecated Make targets, and revises contributor guidance and task validation tests.

Changes

Mise quality task migration

Layer / File(s) Summary
Quality task definitions and validation
.mise/tasks/quality.toml, tests/tools/test_mise_tasks.py
Quality checks are split into dedicated formatting, lint, type, lock, license-header, and task checks. lock:update regenerates uv.lock. Tests validate task definitions and check/test composition.
CI and hook integration
.github/workflows/ci-checks.yml, .github/workflows/README.md, .github/workflows/dependabot-sync-lock.yml, .pre-commit-config.yaml
CI and pre-commit configuration use the dedicated check tasks and validate both pyproject.toml and cuda_deps.toml for lock changes.
Legacy command targets
Makefile
Usage text references check, and deprecated format-check, lock-check, and typecheck targets are removed.
Contributor and agent guidance
.agents/skills/*, .claude/commands/lint.md, .github/PULL_REQUEST_TEMPLATE.md, AGENTS.md, CONTRIBUTING.md, README.md, tools/codestyle/*
Repository guidance and command references use the new task names and the mise run check ::: test gate.

Estimated code review effort: 3 (Moderate) | ~20 minutes

Mergeability Score: ⚪ Minimal · up to a02f3

This PR standardizes development and CI quality tasks while updating related documentation and hooks; no actionable merge-blocking risk remains after normal checks and review.

Suggested labels: chore

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 0.00% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (4 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title clearly summarizes the main change: standardizing the mise quality task contract across tasks, CI, documentation, and tests.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
✨ Finishing Touches 💡 1
🛠️ Fix failing CI checks 💡
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch binaryaaron/mise-quality-contract

Comment @coderabbitai help to get the list of available commands.

@greptile-apps

greptile-apps Bot commented Aug 12, 2026

Copy link
Copy Markdown
Contributor

Greptile Summary

The PR standardizes repository quality automation around read-only check:* tasks, a composite check task, and a mutating lock:update task.

  • Migrates CI, pre-commit hooks, contributor documentation, and agent skills to the new task names.
  • Removes retired Mise task names and corresponding Make aliases.
  • Adds regression coverage for task registration, dependencies, lock commands, hook inputs, and local-gate composition.

Confidence Score: 5/5

The PR appears safe to merge.

No blocking failure remains.

Important Files Changed

Filename Overview
.mise/tasks/quality.toml Defines the canonical read-only quality-task graph and separates lock validation from lock mutation.
.github/workflows/ci-checks.yml Migrates CI quality steps to the corresponding atomic check:* tasks.
.pre-commit-config.yaml Routes lock validation through check:lock for both dependency source files.
tests/tools/test_mise_tasks.py Adds regression tests enforcing the new task contract, retired names, lock behavior, and local-gate composition.
Makefile Removes forwarding aliases for the retired quality-task names.

Flowchart

%%{init: {'theme': 'neutral'}}%%
flowchart LR
  Gate["mise run check ::: test"] --> Check["check"]
  Gate --> Test["test"]
  Check --> Format["check:format"]
  Check --> Lint["check:lint"]
  Check --> Type["check:type"]
  Check --> Lock["check:lock"]
  Check --> License["check:license:headers"]
  Check --> Tasks["check:tasks"]
  Update["lock:update"] --> UVLock["Regenerate uv.lock"]
Loading

Reviews (2): Last reviewed commit: "chore: remove retired make quality alias..." | Re-trigger Greptile

@coderabbitai coderabbitai Bot added the chore Maintenance not tied to a user-visible change label Aug 12, 2026

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1


ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: 384a4f8c-d9cb-4d3e-8e92-4980b478c435

📥 Commits

Reviewing files that changed from the base of the PR and between edaa85f and 6f60ce8.

📒 Files selected for processing (16)
  • .agents/skills/github-cli/SKILL.md
  • .agents/skills/uv-build/SKILL.md
  • .claude/commands/lint.md
  • .github/PULL_REQUEST_TEMPLATE.md
  • .github/workflows/README.md
  • .github/workflows/ci-checks.yml
  • .github/workflows/dependabot-sync-lock.yml
  • .mise/tasks/quality.toml
  • .pre-commit-config.yaml
  • AGENTS.md
  • CONTRIBUTING.md
  • Makefile
  • README.md
  • tests/tools/test_mise_tasks.py
  • tools/codestyle/format.sh
  • tools/codestyle/ruff_check.sh
📜 Review details
⏰ Context from checks skipped due to timeout. (5)
  • GitHub Check: Unit Tests (3.13)
  • GitHub Check: Unit Tests (3.12)
  • GitHub Check: Smoke Tests
  • GitHub Check: End-user Wheel Install
  • GitHub Check: Greptile Review
🧰 Additional context used
📓 Path-based instructions (19)
**/*.sh

📄 CodeRabbit inference engine (STYLE_GUIDE.md)

Use #!/usr/bin/env bash, set -euo pipefail unless incompatible, quote variables, use ${VAR:-default}, use snake_case functions with _ for private helpers, and lint with ShellCheck.

Shell scripting: never use ~ inside double-quoted strings -- it does not expand. Use $HOME or an absolute path instead.

Files:

  • tools/codestyle/ruff_check.sh
  • tools/codestyle/format.sh

⚙️ CodeRabbit configuration file

Review shell scripts for #!/usr/bin/env bash, set -euo pipefail where appropriate, quoting, repo root detection, and shellcheck compliance.

Files:

  • tools/codestyle/ruff_check.sh
  • tools/codestyle/format.sh
**/*.{py,sh,yaml,yml,md}

📄 CodeRabbit inference engine (STYLE_GUIDE.md)

Include the required SPDX copyright and Apache-2.0 license headers, using comment syntax appropriate to the file format.

**/*.{py,sh,yaml,yml,md}: All source files (.py, .sh, .yaml, .yml, .md) require SPDX copyright headers.
Before contributing, run mise run format, review its changes, then run mise run check ::: test.

Files:

  • tools/codestyle/ruff_check.sh
  • README.md
  • tools/codestyle/format.sh
  • AGENTS.md
  • tests/tools/test_mise_tasks.py
  • CONTRIBUTING.md
**/*

📄 CodeRabbit inference engine (STYLE_GUIDE.md)

End files with a newline, contain no trailing whitespace, use one space between sentences, and keep code, comments, and docstrings within 120 characters.

Files:

  • tools/codestyle/ruff_check.sh
  • README.md
  • tools/codestyle/format.sh
  • AGENTS.md
  • Makefile
  • tests/tools/test_mise_tasks.py
  • CONTRIBUTING.md

⚙️ CodeRabbit configuration file

**/*: Review as a senior maintainer for NeMo Safe Synthesizer. Prioritize issues that can change behavior, break user workflows, weaken privacy guarantees, hide failures, make tests unreliable, or create maintenance risk. Avoid generic style commentary unless it points to a concrete project convention that automated tools will not catch.
Comment only when the finding is actionable and tied to changed code. For each finding, state the impact, the condition that triggers it, and the smallest practical fix. Prefer one precise comment over broad advice. Do not ask for refactors outside the PR scope unless the changed code creates the problem.
Review type guidance: - Potential issue: use for correctness bugs, data loss, privacy leaks,
security risks, broken public APIs, invalid config behavior, missing
validation, hidden failures, nondeterministic tests, or CI breakage.

  • Refactor suggestion: use for local maintainability problems introduced
    by the diff when they have clear future cost, such as duplicated setup,
    unclear boundaries, over-mocking, avoidable complexity, or opaque test
    helpers.
  • Nitpick: avoid in chill mode. Do not emit formatting, import-order,
    wording, or style-only comments unless automated tools cannot catch the
    issue and it affects maintainability.

Severity guidance: - Critical: security/privacy leaks, data loss, training/test/holdout
contamination, or broken release/package/core pipeline execution.

  • Major: incorrect generation/training/evaluation behavior, broken
    CLI/SDK public API, invalid config defaults or validators, or GPU/vLLM
    cleanup and process-isolation bugs likely to fail CI or production
    runs.
  • Minor: localized bugs, missing focused tests for changed behavior, or
    bad test patterns that weaken regression coverage.
  • Trivial: small cleanup with no behavior impact. Usually suppress in
    chill mode.
  • Info: context only. Avoid unless it helps reviewers understand risk.
    Safe-Synthesizer-specific review focus: - Data ...

Files:

  • tools/codestyle/ruff_check.sh
  • README.md
  • tools/codestyle/format.sh
  • AGENTS.md
  • Makefile
  • tests/tools/test_mise_tasks.py
  • CONTRIBUTING.md
tools/**

⚙️ CodeRabbit configuration file

Review tools as developer and CI infrastructure. Check that scripts use uv or Makefile wrappers instead of ad hoc python/pip commands, preserve read-only behavior for check targets, fail with clear messages, avoid hidden network or filesystem side effects, and stay consistent with STYLE_GUIDE.md and CONTRIBUTING.md. Tooling may use print() when it is a standalone script or intentional CLI output.

Files:

  • tools/codestyle/ruff_check.sh
  • tools/codestyle/format.sh
tools/codestyle/**

⚙️ CodeRabbit configuration file

Treat codestyle wrappers as CI-critical. Check consistency with Makefile targets, ruff.toml, ty configuration, copyright handling, staged-file behavior, read-only check modes, and whether fixes mutate only expected files.

Files:

  • tools/codestyle/ruff_check.sh
  • tools/codestyle/format.sh
.github/**

⚙️ CodeRabbit configuration file

Review GitHub configuration for branch protection expectations, CODEOWNERS alignment, least privilege permissions, pinned actions where practical, and consistency with CONTRIBUTING.md.

Files:

  • .github/workflows/dependabot-sync-lock.yml
  • .github/workflows/README.md
  • .github/PULL_REQUEST_TEMPLATE.md
  • .github/workflows/ci-checks.yml
**/*.{md,markdown,py}

📄 CodeRabbit inference engine (.cursor/rules/agent-markdown-style.mdc)

**/*.{md,markdown,py}: Avoid decorative bold (**text**) in list items, body text, and docstrings; use structural cues (headers, list markers, colons, backticks) for emphasis instead
Use backticks for code identifiers, paths, and CLI commands in markdown and docstrings

Files:

  • README.md
  • AGENTS.md
  • tests/tools/test_mise_tasks.py
  • CONTRIBUTING.md
**/*.{md,markdown}

📄 CodeRabbit inference engine (.cursor/rules/agent-markdown-style.mdc)

**/*.{md,markdown}: Bold is acceptable only in markdown tables where it's the conventional way to mark header-like cells in the body
Use ## headers to segment markdown sections instead of bold text
Use -- (em-dash) instead of - (hyphen) for asides in markdown

Files:

  • README.md
  • AGENTS.md
  • CONTRIBUTING.md
**/*.md

📄 CodeRabbit inference engine (STYLE_GUIDE.md)

Do not use decorative bold text in body content; use single backticks for inline code and -- for asides. In Python docstrings, use double backticks and MkDocs autorefs rather than Sphinx roles.

Files:

  • README.md
  • AGENTS.md
  • CONTRIBUTING.md
README.md

⚙️ CodeRabbit configuration file

Treat README.md as the project overview. Check that setup, usage, and links stay consistent with CONTRIBUTING.md, Makefile, and docs/.

Files:

  • README.md
.agents/skills/**

📄 CodeRabbit inference engine (.cursor/rules/repo-navigation.mdc)

Store skills in canonical location .agents/skills/ with each skill containing a SKILL.md file and optional references/

Files:

  • .agents/skills/github-cli/SKILL.md
  • .agents/skills/uv-build/SKILL.md
.claude/commands/**

📄 CodeRabbit inference engine (.cursor/rules/claude-commands.mdc)

Pre-defined command files live in .claude/commands/. When a user asks about a development task, read the corresponding file for instructions. Available commands include: unit-test (run unit tests), test-slow (all tests including slow), test-ci-container (CI tests in a container), gpu-test (GPU integration and e2e tests), format (format code with ruff + copyright), lint (lint and typecheck), bootstrap (bootstrap dev environment), build-wheel (build Python wheel), build-docs (build documentation site), start-docs-server (local docs dev server).

Files:

  • .claude/commands/lint.md
AGENTS.md

📄 CodeRabbit inference engine (.cursor/rules/repo-navigation.mdc)

Maintain agent guide with module map and conventions in AGENTS.md

Files:

  • AGENTS.md
.pre-commit-config.yaml

📄 CodeRabbit inference engine (.cursor/rules/repo-navigation.mdc)

Configure pre-commit hooks (ruff, ty, copyright, uv-lock) in .pre-commit-config.yaml

Files:

  • .pre-commit-config.yaml

⚙️ CodeRabbit configuration file

Treat pre-commit configuration as a developer and CI consistency layer. Check hook stages, file filters, local hook commands, DCO checks, and consistency with Makefile validation targets.

Files:

  • .pre-commit-config.yaml
Makefile

⚙️ CodeRabbit configuration file

Treat Makefile targets as the validation source of truth. Check that new targets are documented with target-name: ## Description, declared .PHONY where appropriate, and use uv/mise conventions.

Files:

  • Makefile
tests/**

📄 CodeRabbit inference engine (.cursor/rules/repo-navigation.mdc)

tests/**: Mirror src/ directory structure in tests/ directory for test organization
Auto-mark tests by directory: tests/e2e/e2e, tests/smoke/smoke, otherwise default to unit

Files:

  • tests/tools/test_mise_tasks.py
**/*.py

📄 CodeRabbit inference engine (STYLE_GUIDE.md)

**/*.py: Use American English spelling in Python code, comments, and documentation; use from __future__ import annotations in every module.
Use BaseSettings for environment/CLI settings; prefer AliasChoices for fields accepting both Python and environment-variable names.
Pydantic model fields must include Field(description=...); prefer assignment-style Field() and use Annotated only for additional metadata or constraints.
Use @dataclass(frozen=True) for immutable value objects and validators, and field(default_factory=...) for mutable defaults; never use mutable default values directly.
Use StrEnum for string-valued configuration or serialization enums and plain Enum for internal-only constants.
Obtain loggers with observability.get_logger(__name__); do not call logging.getLogger() or structlog.get_logger() directly.
Do not use print() for operational library output; use the approved logger, click.echo() for CLI output, or sys.stdout.write() for raw tool output.
Use logger extra={} for metrics, counts, durations, and other data intended for machine querying; use f-strings for human-readable context.
Raise known errors through the Safe Synthesizer custom hierarchy, using dual inheritance where callers should also catch a built-in exception.
Use X | Y, built-in generic types, Self for fluent returns, collection ABCs for arguments, Protocol for structural boundaries, and avoid Any.
Prefer match/case for dispatch on types or tagged values, and use comprehensions when clearer without multiple for clauses.
Keep functions flat: more than two indentation levels beyond def indicates the function should be refactored using guard clauses, helpers, generators, or named predicates.
Error messages must accurately describe the condition and make interpolated values identifiable, typically with !r; chain replacement exceptions with raise ... from e.
Use PascalCase for classes, snake_case for functions and variables, UPPER_SNAK...

Files:

  • tests/tools/test_mise_tasks.py
tests/**/*.py

📄 CodeRabbit inference engine (STYLE_GUIDE.md)

tests/**/*.py: Use absolute imports in tests.
Name test files test_*.py, classes Test*, and functions test_<module>_<expected_behavior>; prefix fixtures with fixture_ and give each fixture a one-line purpose docstring.
Use function-scoped fixtures by default, bare assert, pytest.raises(match=...), pytest.approx(), and pytest.mark.parametrize for input combinations.
Use tmp_path for file operations, mock only external boundaries, avoid shared mutable state and order dependencies, and mark CUDA-dependent tests appropriately.

tests/**/*.py: Every test must have exactly one category marker: unit, smoke, or e2e; use slow and requires_gpu as modifiers where applicable.
Use pytest.importorskip for optional dependencies that require specific extras, such as sentence_transformers and vllm.
Use the shared fixture_mock_processor or fixture_mock_processor_without_valid_records fixtures for processor mocks, with ParsedResponse fields valid_records, invalid_records, errors, and prompt_number.
Use load_test_dataset(filename) and load_test_dataframe(filename) for loading shared test datasets.
Keep tokenizers function-scoped unless there is a specific reason to change scope; fixture_session_cache_dir is session-scoped.
Tests should mirror the source structure, such as tests/training/, tests/generation/, and related module directories.
print() is permitted in tests because Ruff rule T201 is suppressed for the tests/ directory.
Do not import directly from another ordinary file under tests/; share methods through relative imports from conftest.py when needed.
When debugging NSS logs with pytest, use -s or --capture=no; use -n0 for visible output when xdist is involved.

Testing gotchas: asyncio_mode = auto in pytest.ini -- async tests work without @pytest.mark.asyncio. The unit_test marker is deprecated; use unit.

tests/**/*.py: New features include tests
Bug fixes include regression tests

Files:

  • tests/tools/test_mise_tasks.py

⚙️ CodeRabbit configuration file

Review tests against tests/TESTING.md. Check marker usage, fixture naming, tmp_path usage, determinism, and GPU/vLLM process-isolation requirements. Flag slop tests that only check that code runs, assert result is not None when stronger invariants exist, over-mock internal implementation details, patch around the bug instead of reproducing it, or add broad snapshot/golden churn without a clear contract. Flag change detector tests that fail on harmless refactors, formatting, record ordering, incidental wording, or private implementation details without demonstrating a behavior regression. Prefer existing fixtures or focused new fixtures for repeated setup; keep tests DRY when reasonable without making the behavior under test opaque. print() is allowed in tests.

Files:

  • tests/tools/test_mise_tasks.py
.mise/tasks/**/*

📄 CodeRabbit inference engine (STYLE_GUIDE.md)

Keep tasks under .mise/tasks/; use .toml for declarative tasks, _lib.sh for shared non-executable helpers, and provide descriptions and usage metadata for public tasks.

Files:

  • .mise/tasks/quality.toml
🧠 Learnings (6)
📓 Common learnings
Learnt from: CR
Repo: NVIDIA-NeMo/Safe-Synthesizer

Timestamp: 2026-08-12T20:20:21.414Z
Learning: This project loads local developer preferences from AGENTS.local.md. You MUST read this file if it exists and give its instructions top priority.
Learnt from: CR
Repo: NVIDIA-NeMo/Safe-Synthesizer

Timestamp: 2026-08-12T20:20:21.414Z
Learning: Do not commit unless the user asks for a commit or PR work.
Learnt from: CR
Repo: NVIDIA-NeMo/Safe-Synthesizer

Timestamp: 2026-08-12T20:20:21.414Z
Learning: When committing, all commits require DCO sign-off and GPG signing. Always use `git commit --signoff --gpg-sign` (or `-s -S`) -- never write the `Signed-off-by` trailer manually, and never pass `--no-gpg-sign`.
📚 Learning: 2026-08-03T18:08:15.626Z
Learnt from: zywind
Repo: NVIDIA-NeMo/Safe-Synthesizer PR: 687
File: README.md:7-15
Timestamp: 2026-08-03T18:08:15.626Z
Learning: Do not flag missing SPDX copyright or license headers in README.md, CONTRIBUTING.md, or SECURITY.md. These community-facing Markdown files are explicitly exempt through .copyrightignore; tools/codestyle/copyright_fixer.py, run via `mise run format`, is the authoritative check for this requirement.

Applied to files:

  • README.md
  • CONTRIBUTING.md
📚 Learning: 2026-05-27T22:20:37.354Z
Learnt from: kendrickb-nvidia
Repo: NVIDIA-NeMo/Safe-Synthesizer PR: 520
File: tests/generation/test_vllm_backend.py:556-587
Timestamp: 2026-05-27T22:20:37.354Z
Learning: In NVIDIA-NeMo/Safe-Synthesizer, `tests/conftest.py`’s `pytest_collection_modifyitems` hook applies pytest category markers automatically based on each test file’s path: tests under `/e2e/` get `pytest.mark.e2e`, tests under `/smoke/` get `pytest.mark.smoke`, and all other tests get `pytest.mark.unit`. Therefore, when reviewing pytest tests outside `tests/e2e/` and `tests/smoke/`, do not flag missing explicit `pytest.mark.unit` decorators on test classes/functions as an issue (the hook will add them during collection). If a new test directory/category is introduced, ensure the hook is updated so it’s categorized correctly.

Applied to files:

  • tests/tools/test_mise_tasks.py
📚 Learning: 2026-07-27T22:07:22.590Z
Learnt from: binaryaaron
Repo: NVIDIA-NeMo/Safe-Synthesizer PR: 673
File: tests/pii_replacer/test_edit.py:327-327
Timestamp: 2026-07-27T22:07:22.590Z
Learning: When tests read structured logging context from Python `logging.LogRecord` instances, don’t access `record.ctx` directly (it isn’t declared on `LogRecord` and will break static typing). Instead, use `getattr(record, "ctx", default)` (or an appropriate fallback) to safely handle cases where `ctx` may or may not be attached. This applies even if Ruff rule `B009` isn’t enabled in the repo.

Applied to files:

  • tests/tools/test_mise_tasks.py
📚 Learning: 2026-07-29T17:12:32.642Z
Learnt from: zywind
Repo: NVIDIA-NeMo/Safe-Synthesizer PR: 654
File: tests/config/test_parameters.py:116-118
Timestamp: 2026-07-29T17:12:32.642Z
Learning: In Pydantic validation tests (e.g., models configured with `from_attributes`), when asserting failures from `model_validate(...)`, assert the structured error details (such as `ValidationError.errors()[0]["type"]`, e.g. `"model_attributes_type"`) rather than relying on the human-readable error message text. This keeps tests stable even if wording changes, while still verifying the correct validation rule is triggered.

Applied to files:

  • tests/tools/test_mise_tasks.py
📚 Learning: 2026-08-05T19:07:15.856Z
Learnt from: binaryaaron
Repo: NVIDIA-NeMo/Safe-Synthesizer PR: 679
File: tests/conftest.py:89-96
Timestamp: 2026-08-05T19:07:15.856Z
Learning: In the Safe Synthesizer test suite, treat CPU and CUDA installation profiles as the supported pytest collection profiles because they install PyTorch. A bare installation without PyTorch is an incomplete, unsupported profile. GPU-marked test modules may import PyTorch before tests/conftest.py::pytest_collection_modifyitems executes, so do not require that collection hook to prevent such imports.

Applied to files:

  • tests/tools/test_mise_tasks.py
🪛 ast-grep (0.45.1)
tests/tools/test_mise_tasks.py

[error] 29-35: Command coming from incoming request
Context: subprocess.run(
["mise", *args],
cwd=repo_root,
check=False,
capture_output=True,
text=True,
)
Note: [CWE-78] Improper Neutralization of Special Elements used in an OS Command ('OS Command Injection').

(subprocess-from-request)

🪛 Ruff (0.16.1)
tests/tools/test_mise_tasks.py

[error] 30-30: subprocess call: check for execution of untrusted input

(S603)


[error] 31-31: Starting a process with a partial executable path

(S607)

🪛 SkillSpector (2.5.1)
.agents/skills/github-cli/SKILL.md

[warning] 215: [EA2] Autonomous Decision Making: Skill enables autonomous high-impact decisions without human-in-the-loop verification. Critical operations (destructive commands, financial transactions, data deletion) should require explicit user confirmation.

Remediation: Add human-in-the-loop confirmation for destructive, irreversible, or high-impact operations. Never auto-execute commands that modify files, send data, or alter system state.

(Excessive Agency (EA2))

🔇 Additional comments (18)
Makefile (3)

23-25: LGTM!


78-79: LGTM!


101-101: LGTM!

.mise/tasks/quality.toml (1)

12-17: LGTM!

Also applies to: 19-25, 27-40, 42-56

tests/tools/test_mise_tasks.py (1)

6-42: LGTM!

Also applies to: 139-182

.github/workflows/ci-checks.yml (1)

91-101: LGTM!

Also applies to: 126-126

.github/workflows/README.md (1)

121-123: LGTM!

Also applies to: 136-137

.github/workflows/dependabot-sync-lock.yml (1)

17-17: LGTM!

.pre-commit-config.yaml (1)

63-66: LGTM!

.agents/skills/github-cli/SKILL.md (1)

134-139: LGTM!

.agents/skills/uv-build/SKILL.md (1)

76-81: LGTM!

Also applies to: 143-143

.claude/commands/lint.md (1)

9-9: LGTM!

.github/PULL_REQUEST_TEMPLATE.md (1)

16-17: LGTM!

AGENTS.md (1)

32-32: LGTM!

CONTRIBUTING.md (1)

424-424: LGTM!

Also applies to: 561-575, 591-595, 681-681

README.md (1)

68-69: LGTM!

tools/codestyle/format.sh (1)

43-43: LGTM!

tools/codestyle/ruff_check.sh (1)

10-10: LGTM!

Comment thread AGENTS.md
Use `uv` for everything -- never `pip` or raw `python`. Python 3.11–3.14 with modern syntax (`X | Y`, `list[str]`, `Self`).

Common commands: `mise run test` (unit tests), `mise run format` (auto-fix formatting + lint + copyright), `mise run check` (read-only local quality checks), `mise run validate` (pre-PR quality, lock, and CI unit checks), `mise run typecheck` (ty only). Always use mise tasks or the wrapper scripts in `tools/` instead of running `ruff` or `ty` directly. Use `uv run` for Python execution. When in doubt, inspect `mise tasks` and `pytest --markers`.
Common commands: `mise run test` (unit tests), `mise run format` (auto-fix formatting + lint + copyright), `mise run check` (all read-only local quality checks), `mise run check ::: test` (the local pre-PR gate), and `mise run check:type` (ty only). Always use mise tasks or the wrapper scripts in `tools/` instead of running `ruff` or `ty` directly. Use `uv run` for Python execution. When in doubt, inspect `mise tasks` and `pytest --markers`.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🎯 Functional Correctness | 🟡 Minor | ⚡ Quick win

Run marker inspection through uv.

AGENTS.md tells agents to use uv run for Python execution but still documents pytest --markers directly. This can use an unrelated environment or fail when pytest is not on PATH. Replace it with uv run --frozen pytest --markers.

As per coding guidelines, use the repository's uv workflow for tool execution. As per path instructions, targeted pytest commands use uv run --frozen pytest ....

Sources: Coding guidelines, Path instructions

@codecov

codecov Bot commented Aug 12, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

@binaryaaron
binaryaaron marked this pull request as draft August 12, 2026 20:33
Signed-off-by: Aaron Gonzales <aagonzales@nvidia.com>
@binaryaaron
binaryaaron marked this pull request as ready for review August 12, 2026 22:50

@zywind zywind left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There's just a merge conflict. But other than that, it's ready to go.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area:ci area:dev-ex Affects build or dev experience area:docs area:tests chore Maintenance not tied to a user-visible change

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants