fix(analyzer): retain fenced SKILL.md findings - #276
Conversation
rng1995
left a comment
There was a problem hiding this comment.
[Automated SkillSpector Review]
Approved. Canonical SKILL.md fenced instructions no longer create a runner-level detection blind spot, while ordinary Markdown and noncanonical suffix names retain the existing example filtering. Runner and CLI regressions cover the boundary without changing report schemas.
|
@rodboev - Please address review comments and resolve merge conflicts. |
Signed-off-by: Rod Boev <rod.boev@gmail.com>
f1010a7 to
b95352d
Compare
|
Thanks, I've rebased onto current
|
rng1995
left a comment
There was a problem hiding this comment.
Re-review approved at the current head. The previously reviewed fenced-SKILL.md behavior remains acceptable and there are no unresolved review threads. Merge remains blocked: the merge-updated head has nine unit-test regressions in CLI JSON/exit-code behavior plus an unsigned merge commit; the owner must repair CI before merge.
|
@rodboev Re-review is approved, but please fix the current CI regressions before merge. The merge-updated head fails 9 unit tests: |
Signed-off-by: Rod Boev <rod.boev@gmail.com>
Signed-off-by: Rod Boev <rod.boev@gmail.com>
…ample-filtering Signed-off-by: Rod Boev <rod.boev@gmail.com>
Signed-off-by: Rod Boev <rod.boev@gmail.com>
…ample-filtering Signed-off-by: Rod Boev <rod.boev@gmail.com>
9cf9891 to
636bb8e
Compare
|
Thanks for the CI details. I repaired the unsigned merge commit and narrowed the I also added a regression beside the existing fenced |
…ample-filtering Signed-off-by: Rod Boev <rod.boev@gmail.com>
fb6f3da to
18f1836
Compare
rng1995
left a comment
There was a problem hiding this comment.
Re-review approved at the final automatic base-update head. The previously reviewed fenced-SKILL.md behavior remains acceptable and there are no unresolved review threads. Merge remains gated on the active full CI run.
|
@rodboev - Fix the CI issues |
…ample-filtering Signed-off-by: Rod Boev <rod.boev@gmail.com>
f12b67d to
941854d
Compare
|
I rewrote the latest merge-only refreshes on the current head so every commit in the PR range carries a |
Summary
The static runner now retains findings located inside fenced commands in canonical
SKILL.md. Inline and prose examples inSKILL.mdremain subject to the generic code-example filter, as do examples in ordinary Markdown and non-canonical*skill.mdfiles.Root cause
_scan_path()treatedSKILL.mdas non-executable Markdown, then dropped findings whose context contained triple backticks. The TM1 analyzer emitted the finding, but the shared per-path runner removed it before reporting. The first exception was path-only, so inline examples could bypass the same filter after the rework.Changes
_is_skill_md(path)helper at the three legacySKILL.mddocumentation-filter sites, and retained_is_canonical_skill_md(path)for the exact-basename runner boundary shared byrun_static_patterns()andrun_static_patterns_with_ledger()._is_fenced_code_block()and made the canonical exception depend on the finding's actual source line being inside a triple-backtick block.SKILL.md, inline and post-fenceSKILL.mdexamples, fencedguide.md, non-canonical*skill.mdpaths, and the loose-versus-canonical path-helper boundary.tmp_path / "SKILL.md"and asserts TM1 plus nonzero before and after filtering counts.Scope
This implements only the fenced
SKILL.mdexample-filtering slice of issue #268. It doesn't changemeta_analyzer.py, scoring bands, report schema, analyzer registration, provider code, or other issue slices. The retained finding remains in the current LOW/SAFE aggregate band; this change surfaces and counts it without retuning scoring. Analyzer-local fenced-example policies remain unchanged.Verification
pytest tests/nodes/analyzers/test_static_runner_filtering.py tests/nodes/analyzers/test_static_patterns_anti_refusal.py tests/nodes/analyzers/test_static_false_positive_controls.py tests/unit/test_cli.py -q-> 190 passed, 4 xfaileduv run ruff check src/ tests/-> All checks passeduv run ruff format --check src/ tests/-> 168 files already formattedgit diff --checkSigned-off-by:trailerBase reproduction:
issues: [],findings_before_filtering: 0, andfindings_after_filtering: 0.Head reproduction: the CLI regression asserts TM1 in
issuesand both filtering counts greater than zero.Upstream
Refs #268