- Translate instruction files into ESLint configs, detect drift, and extract rules.
-
-
-
-
-
-
-
-
-
+
-## Why
+# RuleProbe
-AI coding agents read your `CLAUDE.md`, `AGENTS.md`, or `.cursorrules`, but nothing checks whether your ESLint config actually enforces the same rules. Drift accumulates: the instruction file says "use camelCase" but ESLint never checks it, or ESLint enforces rules the instruction file never mentioned.
+When your `CLAUDE.md` says "use camelCase" but ESLint doesn't enforce it, drift has already happened. RuleProbe reads an instruction file and translates it to an ESLint config. It detects what each enforces but the other misses, or converts ESLint rules back to instruction prose.
-RuleProbe closes that gap. It reads your instruction file, extracts machine-verifiable rules, and bridges them to your tooling in three ways:
+[](https://www.npmjs.com/package/ruleprobe)
+[](https://github.com/moonrunnerkc/ruleprobe/actions/workflows/self-check.yml)
+[](https://github.com/moonrunnerkc/ruleprobe/blob/main/LICENSE)
+[](https://www.typescriptlang.org)
+[](https://nodejs.org)
+[](https://github.com/moonrunnerkc/ruleprobe/stargazers)
-**Translate.** Generate an ESLint config from your instruction file. `ruleprobe lint-config CLAUDE.md` produces a flat or legacy config you can drop into your project immediately.
+## Installation
-**Detect drift.** Compare your instruction file against an existing ESLint config. `ruleprobe drift CLAUDE.md .eslintrc.json` reports rules present in one but missing from the other, severity mismatches, and config argument differences.
-
-**Extract.** Pull a rules section out of an ESLint config for pasting into an instruction file. `ruleprobe extract .eslintrc.json` converts ESLint rules back into human-readable instruction prose.
-
-## Quick Start
+Requires Node.js 18 or later.
```bash
npm install -g ruleprobe
```
-Or run it directly:
+Or try it without installing:
```bash
npx ruleprobe --help
```
-**Translate an instruction file to ESLint config:**
+Confirm it's working:
+
+```bash
+ruleprobe --version
+# 4.5.0
+```
+
+## Usage
+
+For maintainers who use AI coding agents and ESLint. Works with any instruction file format your agents read.
+
+**Translate an instruction file to an ESLint config:**
```bash
ruleprobe lint-config CLAUDE.md
ruleprobe lint-config AGENTS.md --format legacy --output .eslintrc.json
```
-**Detect drift between instructions and ESLint:**
+**Detect drift between an instruction file and an existing ESLint config:**
```bash
ruleprobe drift CLAUDE.md .eslintrc.json
ruleprobe drift CLAUDE.md .eslintrc.json --format markdown
```
-**Extract rules from an ESLint config:**
+**Convert ESLint rules back to instruction prose:**
```bash
ruleprobe extract .eslintrc.json
ruleprobe extract .eslintrc.json --output rules-section.md
```
-**Parse an instruction file** to see what rules RuleProbe can extract:
+**See what rules RuleProbe can parse from an instruction file:**
```bash
-ruleprobe parse CLAUDE.md
-ruleprobe parse AGENTS.md --show-unparseable
+ruleprobe parse CLAUDE.md --show-unparseable
```
-**Verify agent output** against extracted rules (legacy mode). Valid formats: `text`, `json`, `markdown`, `rdjson`, `summary`, `detailed`, `ci`.
+**Check code against extracted rules (legacy verify mode):**
```bash
-ruleprobe verify CLAUDE.md ./agent-output --format text
ruleprobe verify AGENTS.md ./src --changed-since origin/main
```
-## What It Does
+**Discover and cross-reference all instruction files in a project:**
-**Translate.** Reads 7 instruction file formats (CLAUDE.md, AGENTS.md, .cursorrules, copilot-instructions.md, GEMINI.md, .windsurfrules, .rules) and emits an ESLint config. Flat config is the default; pass `--format legacy` for `.eslintrc.json` output. Each extractable rule maps to an ESLint rule with appropriate severity and options. Rules that have no ESLint equivalent appear as comments in the output so you know what wasn't covered.
-
-**Detect drift.** Compares parsed rules against an existing ESLint config. Reports rules in the instruction file but missing from ESLint (you're not enforcing what you said), rules in ESLint but not in the instructions (you're enforcing what you never stated), severity mismatches, and argument differences. Use `--format markdown` for PR-ready output.
-
-**Extract.** Parses an ESLint config and generates a markdown rules section you can paste into an instruction file. Stylistic rules (semicolons, quotes) are reported but skipped from the output by default since they don't carry meaningful instruction content.
-
-**Parse.** Extracts machine-verifiable rules from instruction files with qualifiers (`always`, `prefer`, `when-possible`, `avoid-unless`, `try-to`, `never`) and section context. Subjective lines like "write clean code" are reported as unparseable.
-
-**Verify.** Runs extracted rules against a directory of code. Deterministic by default, optional semantic analysis available. The original mode, still supported but no longer the primary focus.
+```bash
+ruleprobe analyze ./my-project --format json
+```
-**Analyze.** Discovers all instruction files in a project, parses each, and cross-references them for conflicts and redundancies. Pass `--semantic` for structural pattern analysis.
+Seven instruction file formats are supported: `CLAUDE.md`, `AGENTS.md`, `.cursorrules`, `copilot-instructions.md`, `GEMINI.md`, `.windsurfrules`, `.rules`. Full flag reference: [docs/cli-reference.md](docs/cli-reference.md)
## Configuration
-RuleProbe auto-discovers a config file in the working directory (or any parent). You can also pass `--config ` explicitly. Supported file names, in priority order:
-
-- `ruleprobe.config.ts`
-- `ruleprobe.config.js`
-- `ruleprobe.config.json`
-- `.ruleproberc.json`
-
-A config file lets you add custom rules, override extracted rules, or exclude rules entirely:
+RuleProbe auto-discovers a config file in the working directory or any parent. Pass `--config ` to override. Supported names, in priority order: `ruleprobe.config.ts`, `ruleprobe.config.js`, `ruleprobe.config.json`, `.ruleproberc.json`.
```typescript
// ruleprobe.config.ts
import { defineConfig } from 'ruleprobe';
export default defineConfig({
- // Add rules that the parser can't extract from your instruction file
+ // Rules the parser can't extract from your instruction file
rules: [
{
id: 'custom-no-lodash',
@@ -113,7 +99,7 @@ export default defineConfig({
},
],
- // Change severity or expected values on extracted rules
+ // Change severity or thresholds on extracted rules
overrides: [
{ ruleId: 'naming-camelcase', severity: 'warning' },
{ ruleId: 'structure-max-file-length', expected: '500' },
@@ -124,43 +110,14 @@ export default defineConfig({
});
```
-`defineConfig()` is a no-op passthrough that provides type checking in TypeScript configs. JSON configs work without it.
-
-Custom rules use the same verifier types (`ast`, `regex`, `filesystem`) and pattern types as extracted rules.
-
-## CLI Reference
-
-Six commands: `parse`, `verify`, `analyze`, `lint-config`, `drift`, `extract`.
-
-```bash
-ruleprobe parse CLAUDE.md --show-unparseable
-ruleprobe verify AGENTS.md ./src --format detailed --threshold 0.9
-ruleprobe lint-config CLAUDE.md --format flat --output eslint.config.js
-ruleprobe drift CLAUDE.md .eslintrc.json --format markdown
-ruleprobe extract .eslintrc.json --output rules-section.md
-ruleprobe analyze ./my-project --format json
-```
-
-Full command reference with all options: [docs/cli-reference.md](docs/cli-reference.md)
-
-### Incremental verification with `--changed-since`
-
-`ruleprobe verify` accepts `--changed-since ` to limit per-file checks to files changed between the given ref and `HEAD`. Useful on PRs where you only want to enforce rules on the new diff and not surface pre-existing violations.
-
-```bash
-ruleprobe verify AGENTS.md ./src --changed-since origin/main
-ruleprobe verify AGENTS.md ./src --changed-since $(git merge-base HEAD origin/main)
-```
-
-The flag runs `git diff --name-only --diff-filter=ACMR ...HEAD` (added, copied, modified, renamed; deleted files are excluded). Project-level rules (`changelog-exists`, `strict-mode`, etc.) still run regardless of the diff. Cross-file rules like `test-files-exist` see the full file list so they can find a test file even when the test itself was not changed.
-
-If git is not on `PATH` or the ref is invalid, `verify` exits with code 2 and a descriptive message.
+`defineConfig()` is a no-op passthrough that provides TypeScript type checking.
## GitHub Action
-Drop this into `.github/workflows/ruleprobe.yml`:
+Add drift detection to every pull request (PR):
```yaml
+# .github/workflows/ruleprobe.yml
name: RuleProbe Drift
on: [pull_request]
jobs:
@@ -178,57 +135,45 @@ jobs:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
```
-No API keys needed, deterministic results, runs in seconds. The action only runs drift detection when instruction files or ESLint config files change in the PR. `@v4` tracks the latest v4.x release; pin to a specific tag (e.g., `@v4.5.0`) for reproducible builds.
+No API keys needed. The action runs only when instruction files or ESLint configs change in the PR. Pin to `@v4.5.0` for reproducible builds.
Full action options
-```yaml
-- uses: moonrunnerkc/ruleprobe@v4
- with:
- mode: drift
- instruction-file: CLAUDE.md
- eslint-file: .eslintrc.json
- regenerate-on-drift: "false"
- comment-on-pr: "true"
- fail-on-drift: "false"
-```
-
| Input | Default | Description |
|-------|---------|-------------|
-| `mode` | `drift` | Run mode: `drift` (default) or `verify` (legacy) |
-| `instruction-file` | (required) | Path to instruction file |
-| `eslint-file` | (auto-detected) | Path to ESLint config file |
-| `regenerate-on-drift` | `false` | Open a follow-up PR with regenerated config when drift is detected |
+| `mode` | `drift` | `drift` (default) or `verify` (legacy) |
+| `instruction-file` | required | Path to instruction file |
+| `eslint-file` | auto-detected | Path to ESLint config |
+| `regenerate-on-drift` | `false` | Open a follow-up PR with the regenerated config |
| `comment-on-pr` | `true` | Post drift results as a PR comment |
| `fail-on-drift` | `false` | Fail the action if drift is detected |
-| `changed-since` | (unset) | Verify mode only. Git ref to diff against; only files changed in `...HEAD` are checked per-file |
+| `changed-since` | unset | Verify mode only: git ref to diff against |
Drift mode outputs: `drift-count`, `has-drift`.
-### Verify mode with incremental diff
-
-To run `verify` on the PR diff only — surfacing new violations without flagging pre-existing ones — set `mode: verify` and pass the PR base SHA via `changed-since`:
+## How It Works
-```yaml
-- uses: actions/checkout@v4
- with:
- fetch-depth: 0 # full history so the base SHA is reachable
-- uses: moonrunnerkc/ruleprobe@v4
- with:
- mode: verify
- instruction-file: AGENTS.md
- output-dir: src
- changed-since: ${{ github.event.pull_request.base.sha }}
+```
+Instruction File --> Parser --> RuleSet --> Mapper --> ESLint Config
+Instruction File --> Parser --> RuleSet --.
+ESLint Config --> Parser --> Parsed --+--> Drift Detector --> Drift Report
+ESLint Config --> Extractor --> Markdown Rules Section
+Agent Output --> Verifier (verify mode, legacy)
```
-`fetch-depth: 0` is required because `actions/checkout` defaults to a shallow clone and the base SHA may not be present in a shallow history.
+| Engine | What it checks |
+|--------|----------------|
+| AST (Abstract Syntax Tree) via ts-morph | TypeScript and JavaScript structure, naming, imports, type safety |
+| Tree-sitter | Python and Go: function naming, function length |
+| Regex | Line-level patterns across any text file |
+| Filesystem | File existence, naming conventions, directory structure |
-## Programmatic API
+34 ESLint-mappable matchers across 7 categories (`naming`, `forbidden-pattern`, `structure`, `import-pattern`, `error-handling`, `type-safety`, `code-style`). Rules with no ESLint equivalent appear as comments in generated configs. Full matcher table: [docs/matchers.md](docs/matchers.md)
-Core pipeline functions are exported for programmatic use:
+## Programmatic API
```typescript
import { parseInstructionFile, verifyOutput, generateReport, formatReport } from 'ruleprobe';
@@ -246,69 +191,14 @@ console.log(formatReport(report, 'summary'));
Full API reference: [docs/api-reference.md](docs/api-reference.md)
-## How It Works
-
-```
-Instruction File --> Rule Parser --> RuleSet --+
- +--> Verifier --> Adherence Report
- Agent Output ---------+
-
-Instruction File --> Rule Parser --> RuleSet --> Mapper --> ESLint Config
-
-Instruction File --> Rule Parser --> RuleSet --+
-ESLint Config --> Parser --> Parsed --+--> Drift Report
-
-ESLint Config --> Extractor --> Markdown Rules Section
-```
-
-The parser reads your instruction file and identifies lines that map to deterministic checks. Each rule gets a category, a verifier type, a pattern, and a qualifier (how strictly the instruction is worded). Three verifier engines handle different rule types:
-
-| Engine | What it checks |
-|--------|---------------|
-| AST (ts-morph) | Code structure, naming, type safety, imports for TypeScript/JavaScript |
-| Filesystem | File existence, naming conventions, directory structure |
-| Regex | Content patterns, forbidden strings |
-
-The mapper translates extractable rules into ESLint rule entries. The drift detector compares parsed rules against an existing ESLint config. The extractor reverses the mapping: ESLint rules become human-readable instruction prose.
-
-## Supported Rule Types
-
-34 mappable matchers across 7 categories produce ESLint config entries:
-
-| Category | Count | Verifier | Examples |
-|----------|------:|----------|----------|
-| naming | 5 | AST, Filesystem | camelCase variables, PascalCase types, kebab-case files, UPPER_CASE constants |
-| forbidden-pattern | 5 | AST, Regex | no `any`, no `console.log`, no `var`, max line length, no `TODO` comments |
-| structure | 4 | AST, Filesystem | named exports, JSDoc required, max file length, no unused exports |
-| import-pattern | 4 | AST | no path aliases, no deep relative imports, no namespace imports, no wildcard exports |
-| error-handling | 2 | AST | no empty catch, throw Error only |
-| type-safety | 5 | AST, Regex | no enums, no type assertions, no non-null assertions, no implicit any, no ts directives |
-| code-style | 9 | AST, Regex | prefer const, no else after return, no nested ternary, no magic numbers, semicolons, quotes, max function length, max params, no TODO comments |
-
-Rules that can't map to ESLint (test file requirements, project config, git conventions) are reported as unmappable so you can enforce them through other tooling.
-
-Full matcher details: [docs/matchers.md](docs/matchers.md)
-
## Security
-RuleProbe never executes scanned code, never makes network calls (unless you opt in with `--llm-extract`, `--rubric-decompose`, or `--semantic`), and never modifies files in the scanned directory. User-supplied paths are resolved and bounded to the working directory; symlinks outside the project are skipped unless you pass `--allow-symlinks`. All dependencies are pinned to exact versions.
-
-When `--semantic` is enabled, all analysis runs locally. The only network calls are to the Anthropic API for LLM judgment when vector similarity is ambiguous (using your own `ANTHROPIC_API_KEY`). Only numeric AST vectors, opaque sub-tree hashes, boolean flags, and rule text are sent to the LLM. No source code, variable names, comments, import paths, or file paths leave the machine. See [SECURITY.md](SECURITY.md) for the full model.
+RuleProbe never executes scanned code, makes no network calls by default, and never writes to the scanned directory. The optional `--llm-extract`, `--rubric-decompose`, and `--semantic` flags call external APIs using your own keys. User-supplied paths are resolved and restricted to the working directory; symlinks outside it are skipped unless you pass `--allow-symlinks`. All runtime dependencies are pinned to exact versions; see [SECURITY.md](SECURITY.md) for the full model.
## Limitations
-- **Not all rules map to ESLint.** Test file requirements, project config conventions, git workflow rules, and preference pairs don't have ESLint equivalents. These are reported as unmappable so you can enforce them through other tooling.
-- **Monorepo support is limited.** Drift detection and lint-config scan from the repository root and use the first ESLint config found. Monorepos with per-package instruction files or configs need to specify paths explicitly.
-
-## What's New in v4.5.0
-
-v4.5.0 pivots RuleProbe from "verify adherence" to "translate instruction files into ESLint configs." The core value proposition is now: translate, detect drift, extract.
-
-Key changes:
-- **New commands**: `lint-config` (translate instructions → ESLint config), `drift` (compare instructions vs an existing ESLint config), `extract` (ESLint config → instruction prose).
-- **Removed**: `compare`, `tasks`, `task`, `run`, the runner module exports (`buildAgentConfig`, `invokeAgent`, `watchForCompletion`, `countCodeFiles`).
-- **Deprecated**: `verify` still works, but the primary workflow is now lint-config, drift, and extract.
-- **Matcher audit**: 34 ESLint-mappable matchers remain across 7 categories (`naming`, `forbidden-pattern`, `structure`, `import-pattern`, `error-handling`, `type-safety`, `code-style`). 67 unmappable matchers and the `test-requirement`, `dependency`, `preference`, `file-structure`, `tooling`, `testing`, and `workflow` categories were removed.
+- Not all rules map to ESLint. Test file requirements, git conventions, and preference pairs are reported as unmappable so you can enforce them through other tooling.
+- Monorepo drift detection scans from the repo root and uses the first ESLint config found. Specify paths explicitly for per-package instruction files or configs.
## Contributing
@@ -322,4 +212,4 @@ Issues and pull requests welcome at [github.com/moonrunnerkc/ruleprobe](https://
## License
-[MIT](LICENSE)
\ No newline at end of file
+[MIT](LICENSE)
diff --git a/SECURITY.md b/SECURITY.md
index dacfbdd..3ff90fb 100644
--- a/SECURITY.md
+++ b/SECURITY.md
@@ -5,7 +5,7 @@
RuleProbe reads files and produces reports. That is the entire operational scope.
- **No code execution.** ts-morph parses TypeScript into ASTs for structural analysis. It never runs the TypeScript compiler's emit pipeline and never executes scanned code.
-- **No network calls by default.** RuleProbe has zero runtime network dependencies. It does not phone home, fetch updates, or transmit any data. Network calls happen only when you explicitly opt in with `--llm-extract`, `--rubric-decompose`, `--semantic`, or `ruleprobe run`.
+- **No network calls by default.** RuleProbe has zero runtime network dependencies. It does not phone home, fetch updates, or transmit any data. Network calls happen only when you explicitly opt in with `--llm-extract`, `--rubric-decompose`, or `--semantic`.
- **No file modification.** RuleProbe never writes to the scanned directory. Output goes to stdout or to a user-specified `--output` path, nowhere else.
- **No auth, no database, no state.** Each invocation is stateless. Nothing is persisted between runs.
@@ -23,6 +23,24 @@ What is NEVER sent:
- Source code, variable names, function names, string literals
- Comments, import paths, module names, file paths, scope names
+## LLM Call Budgets
+
+External LLM calls happen only on the opt-in `--llm-extract`,
+`--rubric-decompose`, and `--semantic` paths, and each has an upper
+bound on calls per invocation:
+
+- `--llm-extract`: at most one OpenAI call per invocation. Unparseable
+ lines are sent as a single batch (default 50 lines). Transient 429
+ and 503 responses are retried up to 3 times with exponential
+ backoff; non-transient errors fail immediately.
+- `--rubric-decompose`: at most one OpenAI call per invocation, batch
+ of 20 unparseable lines.
+- `--semantic`: capped by `--max-llm-calls` (default 20). The semantic
+ engine logs and stops escalation when the budget is hit.
+
+If you supply your own API key, these budgets define the maximum cost
+ceiling for a single ruleprobe invocation against your account.
+
## Path Traversal Protection
User-supplied paths (instruction files and output directories) are resolved and bounded to the current working directory before any filesystem operation.
@@ -47,7 +65,7 @@ npm run audit
### Current audit status
-As of v0.1.0, `npm audit` reports 5 moderate advisories in `esbuild`, a transitive dev dependency of vitest. These affect the vitest development server only and have no impact on RuleProbe's runtime behavior. esbuild is not bundled in the published package.
+As of v4.5.0, `npm audit` reports 7 moderate advisories in the `vitest` dev-dependency chain (`vite`, `esbuild`, `postcss`, `brace-expansion`). All four are dev-tooling only. None are reachable at runtime and none are included in the published package; the `files` field in package.json restricts the npm artifact to `dist/`, `action.yml`, and metadata.
## Reporting Security Issues
diff --git a/action.yml b/action.yml
index ca37001..51d0f36 100644
--- a/action.yml
+++ b/action.yml
@@ -94,7 +94,7 @@ runs:
using: "composite"
steps:
- name: Set up Node.js
- uses: actions/setup-node@v4
+ uses: actions/setup-node@49933ea5288caeca8642d1e84afbd3f7d6820020 # v4.4.0
with:
node-version: "22"
diff --git a/assets/cover.svg b/assets/cover.svg
new file mode 100644
index 0000000..6020922
--- /dev/null
+++ b/assets/cover.svg
@@ -0,0 +1,81 @@
+
diff --git a/docs/api-reference.md b/docs/api-reference.md
index 579af85..2914dcc 100644
--- a/docs/api-reference.md
+++ b/docs/api-reference.md
@@ -45,17 +45,6 @@ RuleProbe exports core pipeline functions, project analysis, configuration, and
Semantic analysis runs entirely in-process. Raw source code never leaves the machine; only numeric AST vectors, opaque hashes, boolean flags, and rule text are sent to the Anthropic API for LLM-assisted judgments.
-## Agent Invocation
-
-| Function | Purpose |
-|----------|---------|
-| `buildAgentConfig(options)` | Build an agent invocation configuration |
-| `invokeAgent(config)` | Invoke an AI agent via SDK |
-| `isAgentSdkAvailable()` | Check if the Claude Agent SDK is installed |
-| `hasAgentOutput(dir)` | Check if a directory contains agent output |
-| `watchForCompletion(options)` | Watch a directory for agent output |
-| `countCodeFiles(dir)` | Count code files in a directory |
-
---
## Usage Examples
@@ -129,21 +118,7 @@ export default defineConfig({
### Semantic analysis (v4.0.0+)
-```typescript
-import { analyzeProject } from 'ruleprobe';
-import { analyzeProjectSemantic, integrateSemanticResults } from 'ruleprobe/semantic';
-import { resolveSemanticConfig } from 'ruleprobe/semantic/config';
-
-const analysis = analyzeProject('./my-project');
-const config = resolveSemanticConfig('./my-project', { anthropicKey: process.env.ANTHROPIC_API_KEY });
-
-if (config) {
- const rules = analysis.files.flatMap(f => f.ruleSet.rules);
- const semanticResult = await analyzeProjectSemantic('./my-project', config, rules);
- const enhanced = integrateSemanticResults(analysis, semanticResult);
- console.log(`Semantic verdicts: ${semanticResult.report.verdicts.length}`);
-}
-```
+Semantic analysis runs via the `--semantic` flag on the `analyze` command. The underlying `analyzeProjectSemantic`, `integrateSemanticResults`, and `resolveSemanticConfig` functions live in `src/semantic/` but are not part of the published package's main export. Call semantic analysis through the CLI rather than the programmatic API.
---
@@ -195,14 +170,16 @@ interface ProjectAnalysis {
}
type RuleCategory =
- | 'naming' | 'forbidden-pattern' | 'structure' | 'test-requirement'
- | 'import-pattern' | 'error-handling' | 'type-safety' | 'code-style'
- | 'dependency' | 'preference' | 'file-structure' | 'tooling'
- | 'testing' | 'workflow' | 'agent-behavior';
-
-type VerifierType =
- | 'ast' | 'regex' | 'filesystem' | 'treesitter'
- | 'preference' | 'tooling' | 'config-file' | 'git-history';
+ | 'naming'
+ | 'forbidden-pattern'
+ | 'structure'
+ | 'import-pattern'
+ | 'error-handling'
+ | 'type-safety'
+ | 'code-style'
+ | 'agent-behavior';
+
+type VerifierType = 'ast' | 'regex' | 'filesystem' | 'treesitter';
type QualifierType =
| 'always' | 'prefer' | 'when-possible'
diff --git a/src/commands/lint-config.ts b/src/commands/lint-config.ts
index 3b1b840..605e10a 100644
--- a/src/commands/lint-config.ts
+++ b/src/commands/lint-config.ts
@@ -8,7 +8,7 @@
import { parseInstructionFile } from '../parsers/index.js';
import { mapRuleSetToEslintConfig } from '../mapper/index.js';
-import { emitEslintConfig } from '../emitter/eslint.js';
+import { emitEslintConfig, formatUnmappableSummary } from '../emitter/eslint.js';
import type { EslintFormat } from '../mapper/types.js';
import { resolveSafePath } from '../utils/safe-path.js';
import { writeFileSync } from 'node:fs';
@@ -54,4 +54,14 @@ export async function handleLintConfig(
} else {
process.stdout.write(output + '\n');
}
+
+ // Legacy JSON output cannot carry unmappable rules inline (no JSON
+ // comment syntax). Surface them on stderr so the user still sees
+ // which instructions had no ESLint equivalent.
+ if (format === 'legacy') {
+ const summary = formatUnmappableSummary(eslintConfig);
+ if (summary !== '') {
+ process.stderr.write(summary + '\n');
+ }
+ }
}
\ No newline at end of file
diff --git a/src/emitter/eslint.ts b/src/emitter/eslint.ts
index 503b4ea..875b264 100644
--- a/src/emitter/eslint.ts
+++ b/src/emitter/eslint.ts
@@ -4,12 +4,14 @@
* Serializes an EslintConfig to a runnable ESLint configuration string.
* Supports flat config (default) and legacy .eslintrc format.
*
- * Flat config emits valid ES module JavaScript that can be imported by ESLint.
- * Legacy config emits valid JSON suitable for .eslintrc.json files.
+ * Flat config emits valid ES module JavaScript that can be imported by
+ * ESLint. Unmappable rules are appended as JS comments, which remain
+ * valid JavaScript.
*
- * Unmappable rules are emitted as commented sections with the original
- * instruction text and a one-line reason explaining why no ESLint rule
- * can enforce them.
+ * Legacy config emits strictly valid JSON suitable for .eslintrc.json
+ * files. Unmappable rules are not included in the JSON output because
+ * JSON has no comment syntax; callers should surface them through a
+ * sidecar channel via formatUnmappableSummary().
*/
import type { EslintConfig, EslintFormat, EslintRuleEntry } from '../mapper/types.js';
@@ -159,14 +161,10 @@ function emitLegacyConfig(config: EslintConfig): string {
configObj['plugins'] = config.plugins;
}
- // Extends
- if (config.plugins.length > 0) {
- const extendsList = ['eslint:recommended'];
- for (const plugin of config.plugins) {
- extendsList.push(`plugin:${plugin}/recommended`);
- }
- configObj['extends'] = extendsList;
- }
+ // No automatic `extends` block. The flat config emitter does not add
+ // one and emitting eslint:recommended plus plugin recommended sets
+ // here would enable hundreds of unrelated rules. Users who want
+ // recommended sets can extend them in their own config.
// Rules as an object with severity + options
const rulesObj: Record = {};
@@ -179,27 +177,36 @@ function emitLegacyConfig(config: EslintConfig): string {
}
configObj['rules'] = rulesObj;
- // Serialize to JSON with indentation
- const jsonStr = JSON.stringify(configObj, null, 2);
+ // Legacy output is strictly JSON. Unmappable rules cannot be encoded
+ // inside the JSON without breaking JSON.parse, so they are omitted
+ // here. Callers that want to surface them should use
+ // formatUnmappableSummary() and write to stderr or a sidecar path.
+ return JSON.stringify(configObj, null, 2);
+}
- // Append unmappable rules as a comment block after the JSON
+/**
+ * Build a human-readable summary of unmappable rules.
+ *
+ * Returns an empty string when there are no unmappable rules. Otherwise
+ * returns a multi-line block listing each rule's id, reason, and the
+ * original instruction text. Safe to print to stderr or write alongside
+ * the legacy JSON output as a sidecar file.
+ *
+ * @param config - The mapped ESLint config
+ * @returns A summary block or empty string when nothing is unmappable
+ */
+export function formatUnmappableSummary(config: EslintConfig): string {
if (config.unmappable.length === 0) {
- return jsonStr;
+ return '';
}
-
- const commentLines: string[] = [
- '',
- '// Unmappable rules (not part of the JSON config):',
- '// The following rules have no direct ESLint equivalent.',
- '',
+ const lines: string[] = [
+ 'Unmappable rules (not enforceable by ESLint):',
];
for (const rule of config.unmappable) {
- commentLines.push(`// [${rule.sourceRuleId}] ${rule.reason}`);
- commentLines.push(`// Original: ${rule.sourceText}`);
- commentLines.push('');
+ lines.push(` [${rule.sourceRuleId}] ${rule.reason}`);
+ lines.push(` Original: ${rule.sourceText}`);
}
-
- return jsonStr + commentLines.join('\n');
+ return lines.join('\n');
}
/**
diff --git a/src/llm/known-patterns.ts b/src/llm/known-patterns.ts
new file mode 100644
index 0000000..c6b713e
--- /dev/null
+++ b/src/llm/known-patterns.ts
@@ -0,0 +1,60 @@
+/**
+ * Single source of truth for the pattern types LLM prompts may reference.
+ *
+ * Both the LLM extraction pipeline and the rubric decomposition
+ * pipeline send this list to the model so it knows which checks the
+ * project can actually verify. Previously each pipeline kept its own
+ * private list, and the two drifted; the rubric list carried type-aware
+ * and tree-sitter entries that the extraction list did not. Keep this
+ * file in sync with the verifier switch statements in
+ * src/verifier/ast-verifier.ts, src/verifier/regex-verifier.ts, and
+ * src/verifier/file-verifier.ts.
+ */
+
+/** Pattern types verified by the AST verifier (ts-morph). */
+const AST_PATTERN_TYPES = [
+ 'camelCase', 'PascalCase', 'no-any', 'no-console-log', 'named-exports',
+ 'jsdoc-public', 'no-path-aliases', 'no-deep-relative-imports',
+ 'no-empty-catch', 'no-enum', 'no-type-assertions', 'no-non-null-assertions',
+ 'throw-error-only', 'no-console-extended', 'no-nested-ternary',
+ 'no-magic-numbers', 'no-else-after-return', 'max-function-length',
+ 'max-params', 'no-namespace-imports', 'no-barrel-files', 'no-settimeout-in-tests',
+ 'no-var', 'prefer-const', 'no-wildcard-exports',
+] as const;
+
+/** Pattern types verified by the regex verifier. */
+const REGEX_PATTERN_TYPES = [
+ 'line-length', 'no-ts-directives', 'no-test-only', 'no-test-skip',
+ 'quote-style', 'banned-import', 'no-todo-comments', 'consistent-semicolons',
+] as const;
+
+/** Pattern types verified by the filesystem verifier. */
+const FILESYSTEM_PATTERN_TYPES = [
+ 'kebab-case', 'test-files-exist', 'max-file-length', 'test-file-naming',
+ 'strict-mode', 'file-exists', 'formatter-config', 'pinned-dependencies',
+] as const;
+
+/** Pattern types verified by the type-aware checks. */
+const TYPE_AWARE_PATTERN_TYPES = [
+ 'no-implicit-any', 'no-unused-exports', 'no-unresolved-imports',
+] as const;
+
+/** Pattern types verified by the tree-sitter verifier. */
+const TREE_SITTER_PATTERN_TYPES = [
+ 'python-snake-case', 'python-class-naming', 'go-naming', 'function-length',
+] as const;
+
+/**
+ * The full set of known pattern types passed to LLM prompts.
+ *
+ * Spans AST, regex, filesystem, type-aware, and tree-sitter checks.
+ * Used by both the extraction and rubric decomposition pipelines so
+ * the model is told the same surface in either path.
+ */
+export const KNOWN_PATTERN_TYPES: string[] = [
+ ...AST_PATTERN_TYPES,
+ ...REGEX_PATTERN_TYPES,
+ ...FILESYSTEM_PATTERN_TYPES,
+ ...TYPE_AWARE_PATTERN_TYPES,
+ ...TREE_SITTER_PATTERN_TYPES,
+];
diff --git a/src/llm/openai-provider.ts b/src/llm/openai-provider.ts
index 197402f..3cf5ecf 100644
--- a/src/llm/openai-provider.ts
+++ b/src/llm/openai-provider.ts
@@ -19,6 +19,47 @@ export interface OpenAiProviderConfig {
baseUrl?: string;
/** Request timeout in milliseconds. Defaults to 30000. */
timeoutMs?: number;
+ /**
+ * Total attempts on transient errors (429 and 503). Defaults to 3.
+ * Set to 1 to disable retries. The first attempt counts.
+ */
+ maxAttempts?: number;
+ /**
+ * Initial backoff in milliseconds between retries. Doubled each
+ * subsequent attempt. Honored only when the response has no
+ * Retry-After header. Defaults to 1000.
+ */
+ retryBaseDelayMs?: number;
+ /**
+ * Sleep implementation, overridable for tests. Defaults to
+ * setTimeout-based wait.
+ */
+ sleep?: (ms: number) => Promise;
+ /**
+ * Fetch implementation, overridable for tests. Defaults to global fetch.
+ */
+ fetchImpl?: typeof fetch;
+}
+
+/** Parse a Retry-After header into milliseconds. Returns null if absent or invalid. */
+function parseRetryAfter(header: string | null): number | null {
+ if (header === null || header === '') {
+ return null;
+ }
+ const seconds = Number(header);
+ if (Number.isFinite(seconds) && seconds >= 0) {
+ return Math.round(seconds * 1000);
+ }
+ const date = Date.parse(header);
+ if (!Number.isNaN(date)) {
+ return Math.max(0, date - Date.now());
+ }
+ return null;
+}
+
+/** Default sleep implementation using setTimeout. */
+function defaultSleep(ms: number): Promise {
+ return new Promise((resolve) => setTimeout(resolve, ms));
}
/**
@@ -42,6 +83,10 @@ export function createOpenAiProvider(config: OpenAiProviderConfig = {}): LlmProv
const model = config.model ?? 'gpt-4o-mini';
const baseUrl = (config.baseUrl ?? 'https://api.openai.com/v1').replace(/\/$/, '');
const timeoutMs = config.timeoutMs ?? 30000;
+ const maxAttempts = Math.max(1, config.maxAttempts ?? 3);
+ const retryBaseDelayMs = config.retryBaseDelayMs ?? 1000;
+ const sleep = config.sleep ?? defaultSleep;
+ const fetchFn = config.fetchImpl ?? fetch;
return {
name: `openai/${model}`,
@@ -55,33 +100,54 @@ export function createOpenAiProvider(config: OpenAiProviderConfig = {}): LlmProv
}
const prompt = buildExtractionPrompt(lines, knownPatternTypes);
+ const body = JSON.stringify({
+ model,
+ messages: [
+ { role: 'system', content: prompt.system },
+ { role: 'user', content: prompt.user },
+ ],
+ temperature: 0,
+ response_format: { type: 'json_object' },
+ });
- const controller = new AbortController();
- const timer = setTimeout(() => controller.abort(), timeoutMs);
-
- try {
- const response = await fetch(`${baseUrl}/chat/completions`, {
- method: 'POST',
- headers: {
- 'Content-Type': 'application/json',
- 'Authorization': `Bearer ${apiKey}`,
- },
- body: JSON.stringify({
- model,
- messages: [
- { role: 'system', content: prompt.system },
- { role: 'user', content: prompt.user },
- ],
- temperature: 0,
- response_format: { type: 'json_object' },
- }),
- signal: controller.signal,
- });
+ for (let attempt = 1; attempt <= maxAttempts; attempt++) {
+ const controller = new AbortController();
+ const timer = setTimeout(() => controller.abort(), timeoutMs);
+
+ let response: Response;
+ try {
+ response = await fetchFn(`${baseUrl}/chat/completions`, {
+ method: 'POST',
+ headers: {
+ 'Content-Type': 'application/json',
+ 'Authorization': `Bearer ${apiKey}`,
+ },
+ body,
+ signal: controller.signal,
+ });
+ } finally {
+ clearTimeout(timer);
+ }
+
+ // Retry only on 429 (rate limit) and 503 (service unavailable).
+ // Other non-ok statuses surface as errors immediately because
+ // they indicate a config or request problem retrying cannot
+ // fix.
+ const retryable = response.status === 429 || response.status === 503;
+ if (!response.ok && retryable && attempt < maxAttempts) {
+ const retryAfter = parseRetryAfter(response.headers.get('Retry-After'));
+ const backoff = retryAfter ?? retryBaseDelayMs * Math.pow(2, attempt - 1);
+ process.stderr.write(
+ `OpenAI API returned ${response.status}; retrying in ${backoff}ms (attempt ${attempt}/${maxAttempts})\n`,
+ );
+ await sleep(backoff);
+ continue;
+ }
if (!response.ok) {
- const body = await response.text();
+ const errBody = await response.text();
throw new Error(
- `OpenAI API error ${response.status}: ${body.slice(0, 200)}`,
+ `OpenAI API error ${response.status}: ${errBody.slice(0, 200)}`,
);
}
@@ -95,9 +161,10 @@ export function createOpenAiProvider(config: OpenAiProviderConfig = {}): LlmProv
}
return parseExtractionResponse(content, lines, knownPatternTypes);
- } finally {
- clearTimeout(timer);
}
+
+ // Loop exits via return or throw; unreachable in practice.
+ throw new Error('OpenAI API retries exhausted without a final response');
},
};
}
diff --git a/src/llm/pipeline.ts b/src/llm/pipeline.ts
index 8e30af4..5150ad4 100644
--- a/src/llm/pipeline.ts
+++ b/src/llm/pipeline.ts
@@ -8,31 +8,7 @@
import type { Rule, RuleSet } from '../types.js';
import type { LlmExtractOptions, LlmRuleCandidate } from './types.js';
-
-/**
- * All known pattern types across all verifiers.
- *
- * This list is used in LLM prompts so the model knows which
- * checks are available. Keep in sync with the verifier switch
- * statements in ast-verifier.ts, regex-verifier.ts, and
- * file-verifier.ts.
- */
-const KNOWN_PATTERN_TYPES: string[] = [
- // AST checks
- 'camelCase', 'PascalCase', 'no-any', 'no-console-log', 'named-exports',
- 'jsdoc-public', 'no-path-aliases', 'no-deep-relative-imports',
- 'no-empty-catch', 'no-enum', 'no-type-assertions', 'no-non-null-assertions',
- 'throw-error-only', 'no-console-extended', 'no-nested-ternary',
- 'no-magic-numbers', 'no-else-after-return', 'max-function-length',
- 'max-params', 'no-namespace-imports', 'no-barrel-files', 'no-settimeout-in-tests',
- 'no-var', 'prefer-const', 'no-wildcard-exports',
- // Regex checks
- 'line-length', 'no-ts-directives', 'no-test-only', 'no-test-skip',
- 'quote-style', 'banned-import', 'no-todo-comments', 'consistent-semicolons',
- // Filesystem checks
- 'kebab-case', 'test-files-exist', 'max-file-length', 'test-file-naming',
- 'strict-mode', 'file-exists', 'formatter-config', 'pinned-dependencies',
-];
+import { KNOWN_PATTERN_TYPES } from './known-patterns.js';
/**
* Run LLM extraction on a RuleSet's unparseable lines.
diff --git a/src/llm/rubric-pipeline.ts b/src/llm/rubric-pipeline.ts
index ee03931..dec93b8 100644
--- a/src/llm/rubric-pipeline.ts
+++ b/src/llm/rubric-pipeline.ts
@@ -9,31 +9,7 @@
import type { Rule, RuleSet } from '../types.js';
import type { RubricDecomposer, DecomposedRubric, RubricCheck } from './rubric-types.js';
-
-/**
- * All known pattern types that rubric checks can reference.
- * Kept in sync with the verifier switch statements.
- */
-const KNOWN_PATTERN_TYPES: string[] = [
- // AST checks
- 'camelCase', 'PascalCase', 'no-any', 'no-console-log', 'named-exports',
- 'jsdoc-public', 'no-path-aliases', 'no-deep-relative-imports',
- 'no-empty-catch', 'no-enum', 'no-type-assertions', 'no-non-null-assertions',
- 'throw-error-only', 'no-console-extended', 'no-nested-ternary',
- 'no-magic-numbers', 'no-else-after-return', 'max-function-length',
- 'max-params', 'no-namespace-imports', 'no-barrel-files', 'no-settimeout-in-tests',
- 'no-var', 'prefer-const', 'no-wildcard-exports',
- // Regex checks
- 'line-length', 'no-ts-directives', 'no-test-only', 'no-test-skip',
- 'quote-style', 'banned-import', 'no-todo-comments', 'consistent-semicolons',
- // Filesystem checks
- 'kebab-case', 'test-files-exist', 'max-file-length', 'test-file-naming',
- 'strict-mode', 'file-exists', 'formatter-config', 'pinned-dependencies',
- // Type-aware checks
- 'no-implicit-any', 'no-unused-exports', 'no-unresolved-imports',
- // Tree-sitter checks
- 'python-snake-case', 'python-class-naming', 'go-naming', 'function-length',
-];
+import { KNOWN_PATTERN_TYPES } from './known-patterns.js';
/** Options for rubric decomposition. */
export interface RubricDecomposeOptions {
diff --git a/src/mappings/index.ts b/src/mappings/index.ts
index dea49f7..c535279 100644
--- a/src/mappings/index.ts
+++ b/src/mappings/index.ts
@@ -75,11 +75,11 @@ export const MAPPINGS: MappingEntry[] = [
{ patternType: 'throw-error-only', eslintRuleName: 'no-throw-literal', defaultSeverity: 'error', description: 'Only Error objects may be thrown' },
// type safety
- { patternType: 'no-enum', eslintRuleName: '@typescript-eslint/no-enum', plugin: '@typescript-eslint', defaultSeverity: 'warn', description: 'Enums must not be used; prefer union types' },
+ { patternType: 'no-enum', eslintRuleName: 'no-restricted-syntax', defaultSeverity: 'warn', description: 'Enums must not be used; prefer union types' },
{ patternType: 'no-type-assertions', eslintRuleName: '@typescript-eslint/consistent-type-assertions', plugin: '@typescript-eslint', defaultSeverity: 'warn', description: 'Type assertions (as casts) must not be used' },
{ patternType: 'no-non-null-assertions', eslintRuleName: '@typescript-eslint/no-non-null-assertion', plugin: '@typescript-eslint', defaultSeverity: 'warn', description: 'Non-null assertions (!) must not be used' },
{ patternType: 'no-implicit-any', eslintRuleName: '@typescript-eslint/no-implicit-any', plugin: '@typescript-eslint', defaultSeverity: 'warn', description: 'No implicit any types' },
- { patternType: 'no-unused-exports', eslintRuleName: 'no-unused-vars', plugin: '@typescript-eslint', defaultSeverity: 'warn', description: 'Exported declarations must be imported by other files' },
+ { patternType: 'no-unused-exports', eslintRuleName: 'import/no-unused-modules', plugin: 'import', defaultSeverity: 'warn', description: 'Exported declarations must be imported by other files' },
{ patternType: 'no-ts-directives', eslintRuleName: '@typescript-eslint/ban-ts-comment', plugin: '@typescript-eslint', defaultSeverity: 'error', description: 'TypeScript suppression directives must not be used' },
// function limits
diff --git a/src/types-core.ts b/src/types-core.ts
new file mode 100644
index 0000000..ded859b
--- /dev/null
+++ b/src/types-core.ts
@@ -0,0 +1,152 @@
+/**
+ * Core type primitives for RuleProbe.
+ *
+ * These types describe instruction files, the rules extracted from
+ * them, and the matchers that drive extraction. Result and report
+ * types live in types-results.ts so that adding a new result field
+ * does not push the file past the 300-line self-check limit.
+ *
+ * Re-exported by types.ts; prefer importing from '../types.js' in
+ * most call sites so callers see a single surface.
+ */
+
+/** Categories of machine-verifiable rules extracted from instruction files. */
+export type RuleCategory =
+ | 'naming'
+ | 'forbidden-pattern'
+ | 'structure'
+ | 'import-pattern'
+ | 'error-handling'
+ | 'type-safety'
+ | 'code-style'
+ | 'agent-behavior';
+
+/** Which verification engine handles a given rule. */
+export type VerifierType = 'ast' | 'regex' | 'filesystem' | 'treesitter';
+
+/**
+ * Qualifier describing the strength of an instruction.
+ *
+ * Detected via deterministic keyword/phrase matching on the rule text
+ * during extraction. Rules with no qualifier keyword default to 'always'.
+ */
+export type QualifierType =
+ | 'always'
+ | 'prefer'
+ | 'when-possible'
+ | 'avoid-unless'
+ | 'try-to'
+ | 'never';
+
+/** Instruction file format detected from the file path. */
+export type InstructionFileType =
+ | 'claude.md'
+ | 'agents.md'
+ | 'cursorrules'
+ | 'copilot-instructions'
+ | 'gemini.md'
+ | 'windsurfrules'
+ | 'rules'
+ | 'generic-markdown'
+ | 'unknown';
+
+/** Describes the specific check a verifier runs for a rule. */
+export interface VerificationPattern {
+ /** The kind of check, e.g. "camelCase", "no-any", "file-exists". */
+ type: string;
+ /** What to check, e.g. "variables", "*.ts", "src/". */
+ target: string;
+ /** The expected value, pattern, or boolean condition. */
+ expected: string | boolean;
+ /** Whether the check applies per-file or across the whole project. */
+ scope: 'file' | 'project';
+}
+
+/** A single machine-verifiable rule extracted from an instruction file. */
+export interface Rule {
+ /** Unique identifier, e.g. "naming-camelcase-variables". */
+ id: string;
+ /** Which category this rule belongs to. */
+ category: RuleCategory;
+ /** The raw text from the instruction file that produced this rule. */
+ source: string;
+ /** Human-readable summary of what the rule checks. */
+ description: string;
+ /** Whether a violation is an error or a warning. */
+ severity: 'error' | 'warning';
+ /** Which verification engine handles this rule. */
+ verifier: VerifierType;
+ /** The specific check to run. */
+ pattern: VerificationPattern;
+ /** Confidence level of the extraction (high = exact keyword match). */
+ confidence?: 'high' | 'medium' | 'low';
+ /** How this rule was extracted. */
+ extractionMethod?: 'static' | 'llm' | 'rubric' | 'rubric-deterministic' | 'custom';
+ /** Weight within a rubric (0-1). Only set for rubric-decomposed rules. */
+ rubricWeight?: number;
+ /** The markdown section header this rule was found under. */
+ section?: string;
+ /** Qualifier strength detected from the instruction text. */
+ qualifier?: QualifierType;
+}
+
+/** A complete set of rules extracted from a single instruction file. */
+export interface RuleSet {
+ /** Path to the instruction file that was parsed. */
+ sourceFile: string;
+ /** Detected file format. */
+ sourceType: InstructionFileType;
+ /** All machine-verifiable rules that were extracted. */
+ rules: Rule[];
+ /** Lines from the instruction file that could not be converted to rules. */
+ unparseable: string[];
+}
+
+/** A parsed section from a markdown instruction file. */
+export interface MarkdownSection {
+ /** The header text (without leading # characters). */
+ header: string;
+ /** Header depth (1 for #, 2 for ##, etc). */
+ depth: number;
+ /** The body content under this header, as raw text. */
+ body: string;
+ /** Lines within the body, trimmed and filtered for empties. */
+ lines: string[];
+}
+
+/**
+ * A matcher definition that maps natural language patterns in instruction
+ * files to structured, machine-verifiable rules.
+ */
+export interface RuleMatcher {
+ /** Unique identifier prefix for rules produced by this matcher. */
+ id: string;
+ /** Regex patterns that match instruction lines this rule covers. */
+ patterns: RegExp[];
+ /** The rule category. */
+ category: RuleCategory;
+ /** Which verifier handles this rule. */
+ verifier: VerifierType;
+ /** Human-readable description of what this rule checks. */
+ description: string;
+ /** Default severity. */
+ severity: 'error' | 'warning';
+ /** Confidence level for rules produced by this matcher. */
+ confidence?: 'high' | 'medium' | 'low';
+ /** Build the verification pattern from the matched line. */
+ buildPattern: (line: string, match: RegExpMatchArray) => VerificationPattern;
+}
+
+/**
+ * Recognized instruction file names.
+ * Used by project-level discovery to find all instruction files in a repo.
+ */
+export const INSTRUCTION_FILE_NAMES = [
+ 'CLAUDE.md',
+ 'AGENTS.md',
+ '.cursorrules',
+ '.github/copilot-instructions.md',
+ 'GEMINI.md',
+ '.windsurfrules',
+ '.rules',
+] as const;
diff --git a/src/types-results.ts b/src/types-results.ts
new file mode 100644
index 0000000..0f4d0ca
--- /dev/null
+++ b/src/types-results.ts
@@ -0,0 +1,161 @@
+/**
+ * Result, report, and project-analysis types for RuleProbe.
+ *
+ * Built on the primitives in types-core.ts. Split from types.ts to
+ * keep both files under the 300-line self-check limit. Re-exported
+ * by types.ts; prefer importing from '../types.js' in most call
+ * sites so callers see a single surface.
+ */
+
+import type {
+ Rule,
+ RuleCategory,
+ RuleSet,
+ InstructionFileType,
+} from './types-core.js';
+
+/** Valid output format for adherence reports. */
+export type ReportFormat = 'text' | 'json' | 'markdown' | 'rdjson' | 'summary' | 'detailed' | 'ci';
+
+/** A standardized coding task designed to exercise rule categories. */
+export interface TaskTemplate {
+ /** Unique identifier, e.g. "rest-endpoint". */
+ id: string;
+ /** Human-readable name, e.g. "REST API Endpoint". */
+ name: string;
+ /** The full prompt given to the coding agent. */
+ prompt: string;
+ /** Files the agent output should contain. */
+ expectedFiles: string[];
+ /** Which rule categories this task exercises. */
+ exercises: RuleCategory[];
+}
+
+/** Metadata about a single agent run. */
+export interface AgentRun {
+ /** Agent identifier, e.g. "claude-code", "copilot", "cursor". */
+ agent: string;
+ /** Model version, e.g. "opus-4.6". */
+ model: string;
+ /** Which task template was given to the agent. */
+ taskTemplateId: string;
+ /** Path to the directory containing agent output files. */
+ outputDir: string;
+ /** ISO 8601 timestamp of when the run started. */
+ timestamp: string;
+ /** How long the agent took, or null if not measured. */
+ durationSeconds: number | null;
+}
+
+/** A piece of evidence supporting a rule result (pass or fail). */
+export interface Evidence {
+ /** The file where the check was performed. */
+ file: string;
+ /** Line number of the finding, or null for file-level checks. */
+ line: number | null;
+ /** What was actually found in the code. */
+ found: string;
+ /** What the rule required. */
+ expected: string;
+ /** Surrounding code for readability. */
+ context: string;
+}
+
+/** The result of checking a single rule against agent output. */
+export interface RuleResult {
+ /** The rule that was checked. */
+ rule: Rule;
+ /** Whether the agent output conformed to this rule. */
+ passed: boolean;
+ /** Compliance ratio from 0 to 1. Binary checks return 0 or 1. Pattern checks return the ratio. */
+ compliance: number;
+ /** Evidence of what was checked and found. */
+ evidence: Evidence[];
+ /** Whether this rule was skipped because it has no concrete implementation. */
+ skipped?: boolean;
+}
+
+/** Per-category breakdown of pass/total counts. */
+export interface CategoryScore {
+ passed: number;
+ total: number;
+}
+
+/** Summary statistics for an adherence report. */
+export interface ReportSummary {
+ /** Total number of rules checked. */
+ totalRules: number;
+ /** Number of rules that passed. */
+ passed: number;
+ /** Number of rules that failed. */
+ failed: number;
+ /** Number of rules skipped (present in ruleset but excluded from verification, e.g. by severity filter). */
+ skipped: number;
+ /** Number of warnings (failed rules with severity "warning"). */
+ warnings: number;
+ /** Adherence score as a percentage (passed / totalRules * 100). */
+ adherenceScore: number;
+ /** Pass/total breakdown by rule category. */
+ byCategory: Record;
+}
+
+/** A complete adherence report for a single agent run. */
+export interface AdherenceReport {
+ /** Metadata about the agent run. */
+ run: AgentRun;
+ /** The rules that were checked. */
+ ruleset: RuleSet;
+ /** Individual results for each rule. */
+ results: RuleResult[];
+ /** Aggregate summary. */
+ summary: ReportSummary;
+}
+
+/** A conflict between rules in different instruction files. */
+export interface CrossFileConflict {
+ /** Topic or pattern category the conflict relates to. */
+ topic: string;
+ /** Rules from different files that contradict each other. */
+ rules: Array<{ file: string; rule: Rule }>;
+ /** Description of the conflict. */
+ description: string;
+}
+
+/** A redundancy: the same instruction appearing in multiple files. */
+export interface CrossFileRedundancy {
+ /** Normalized text of the redundant instruction. */
+ normalizedText: string;
+ /** Occurrences across files. */
+ occurrences: Array<{ file: string; originalText: string }>;
+}
+
+/** Per-file analysis result within a project. */
+export interface FileAnalysis {
+ /** Path to the instruction file. */
+ filePath: string;
+ /** Detected file format. */
+ fileType: InstructionFileType;
+ /** Rules extracted from this file. */
+ ruleSet: RuleSet;
+ /** Verification results (populated after verification). */
+ results: RuleResult[];
+}
+
+/** Complete project-level analysis across all instruction files. */
+export interface ProjectAnalysis {
+ /** Root directory of the project. */
+ projectDir: string;
+ /** Per-file analysis results. */
+ files: FileAnalysis[];
+ /** Cross-file conflicts (same topic, different instructions). */
+ conflicts: CrossFileConflict[];
+ /** Cross-file redundancies (same instruction, different wording). */
+ redundancies: CrossFileRedundancy[];
+ /** Map of rule categories to which files contain rules in that category. */
+ coverageMap: Record;
+ /** Aggregate summary across all files. */
+ summary: ReportSummary;
+}
+
+/** Default compliance threshold for determining pass/fail from compliance ratios. */
+export const DEFAULT_COMPLIANCE_THRESHOLD = 0.8;
diff --git a/src/types.ts b/src/types.ts
index 6d708f1..50d3a85 100644
--- a/src/types.ts
+++ b/src/types.ts
@@ -1,293 +1,39 @@
/**
- * All shared types for RuleProbe.
+ * Public type surface for RuleProbe.
*
- * These types define the data structures flowing through the core pipeline:
- * instruction file parsing, rule extraction, verification, and reporting.
+ * Re-exports the core type primitives and the result/report types
+ * from their respective modules. Existing imports `from '../types.js'`
+ * keep working; new code may import from types-core.ts or
+ * types-results.ts directly when only one half is needed.
*/
-/** Valid output format for adherence reports. */
-export type ReportFormat = 'text' | 'json' | 'markdown' | 'rdjson' | 'summary' | 'detailed' | 'ci';
-
-/** Categories of machine-verifiable rules extracted from instruction files. */
-export type RuleCategory =
- | 'naming'
- | 'forbidden-pattern'
- | 'structure'
- | 'import-pattern'
- | 'error-handling'
- | 'type-safety'
- | 'code-style'
- | 'agent-behavior';
-
-/** Which verification engine handles a given rule. */
-export type VerifierType = 'ast' | 'regex' | 'filesystem' | 'treesitter';
-
-/**
- * Qualifier describing the strength of an instruction.
- *
- * Detected via deterministic keyword/phrase matching on the rule text
- * during extraction. Rules with no qualifier keyword default to 'always'.
- */
-export type QualifierType =
- | 'always'
- | 'prefer'
- | 'when-possible'
- | 'avoid-unless'
- | 'try-to'
- | 'never';
-
-/** Instruction file format detected from the file path. */
-export type InstructionFileType =
- | 'claude.md'
- | 'agents.md'
- | 'cursorrules'
- | 'copilot-instructions'
- | 'gemini.md'
- | 'windsurfrules'
- | 'rules'
- | 'generic-markdown'
- | 'unknown';
-
-/** Describes the specific check a verifier runs for a rule. */
-export interface VerificationPattern {
- /** The kind of check, e.g. "camelCase", "no-any", "file-exists". */
- type: string;
- /** What to check, e.g. "variables", "*.ts", "src/". */
- target: string;
- /** The expected value, pattern, or boolean condition. */
- expected: string | boolean;
- /** Whether the check applies per-file or across the whole project. */
- scope: 'file' | 'project';
-}
-
-/** A single machine-verifiable rule extracted from an instruction file. */
-export interface Rule {
- /** Unique identifier, e.g. "naming-camelcase-variables". */
- id: string;
- /** Which category this rule belongs to. */
- category: RuleCategory;
- /** The raw text from the instruction file that produced this rule. */
- source: string;
- /** Human-readable summary of what the rule checks. */
- description: string;
- /** Whether a violation is an error or a warning. */
- severity: 'error' | 'warning';
- /** Which verification engine handles this rule. */
- verifier: VerifierType;
- /** The specific check to run. */
- pattern: VerificationPattern;
- /** Confidence level of the extraction (high = exact keyword match). */
- confidence?: 'high' | 'medium' | 'low';
- /** How this rule was extracted. */
- extractionMethod?: 'static' | 'llm' | 'rubric' | 'rubric-deterministic' | 'custom';
- /** Weight within a rubric (0-1). Only set for rubric-decomposed rules. */
- rubricWeight?: number;
- /** The markdown section header this rule was found under. */
- section?: string;
- /** Qualifier strength detected from the instruction text. */
- qualifier?: QualifierType;
-}
-
-/** A complete set of rules extracted from a single instruction file. */
-export interface RuleSet {
- /** Path to the instruction file that was parsed. */
- sourceFile: string;
- /** Detected file format. */
- sourceType: InstructionFileType;
- /** All machine-verifiable rules that were extracted. */
- rules: Rule[];
- /** Lines from the instruction file that could not be converted to rules. */
- unparseable: string[];
-}
-
-/** A standardized coding task designed to exercise rule categories. */
-export interface TaskTemplate {
- /** Unique identifier, e.g. "rest-endpoint". */
- id: string;
- /** Human-readable name, e.g. "REST API Endpoint". */
- name: string;
- /** The full prompt given to the coding agent. */
- prompt: string;
- /** Files the agent output should contain. */
- expectedFiles: string[];
- /** Which rule categories this task exercises. */
- exercises: RuleCategory[];
-}
-
-/** Metadata about a single agent run. */
-export interface AgentRun {
- /** Agent identifier, e.g. "claude-code", "copilot", "cursor". */
- agent: string;
- /** Model version, e.g. "opus-4.6". */
- model: string;
- /** Which task template was given to the agent. */
- taskTemplateId: string;
- /** Path to the directory containing agent output files. */
- outputDir: string;
- /** ISO 8601 timestamp of when the run started. */
- timestamp: string;
- /** How long the agent took, or null if not measured. */
- durationSeconds: number | null;
-}
-
-/** A piece of evidence supporting a rule result (pass or fail). */
-export interface Evidence {
- /** The file where the check was performed. */
- file: string;
- /** Line number of the finding, or null for file-level checks. */
- line: number | null;
- /** What was actually found in the code. */
- found: string;
- /** What the rule required. */
- expected: string;
- /** Surrounding code for readability. */
- context: string;
-}
-
-/** The result of checking a single rule against agent output. */
-export interface RuleResult {
- /** The rule that was checked. */
- rule: Rule;
- /** Whether the agent output conformed to this rule. */
- passed: boolean;
- /** Compliance ratio from 0 to 1. Binary checks return 0 or 1. Pattern checks return the ratio. */
- compliance: number;
- /** Evidence of what was checked and found. */
- evidence: Evidence[];
- /** Whether this rule was skipped because it has no concrete implementation. */
- skipped?: boolean;
-}
-
-/** Per-category breakdown of pass/total counts. */
-export interface CategoryScore {
- passed: number;
- total: number;
-}
-
-/** Summary statistics for an adherence report. */
-export interface ReportSummary {
- /** Total number of rules checked. */
- totalRules: number;
- /** Number of rules that passed. */
- passed: number;
- /** Number of rules that failed. */
- failed: number;
- /** Number of rules skipped (present in ruleset but excluded from verification, e.g. by severity filter). */
- skipped: number;
- /** Number of warnings (failed rules with severity "warning"). */
- warnings: number;
- /** Adherence score as a percentage (passed / totalRules * 100). */
- adherenceScore: number;
- /** Pass/total breakdown by rule category. */
- byCategory: Record;
-}
-
-/** A complete adherence report for a single agent run. */
-export interface AdherenceReport {
- /** Metadata about the agent run. */
- run: AgentRun;
- /** The rules that were checked. */
- ruleset: RuleSet;
- /** Individual results for each rule. */
- results: RuleResult[];
- /** Aggregate summary. */
- summary: ReportSummary;
-}
-
-/** A parsed section from a markdown instruction file. */
-export interface MarkdownSection {
- /** The header text (without leading # characters). */
- header: string;
- /** Header depth (1 for #, 2 for ##, etc). */
- depth: number;
- /** The body content under this header, as raw text. */
- body: string;
- /** Lines within the body, trimmed and filtered for empties. */
- lines: string[];
-}
-
-/**
- * A matcher definition that maps natural language patterns in instruction
- * files to structured, machine-verifiable rules.
- */
-export interface RuleMatcher {
- /** Unique identifier prefix for rules produced by this matcher. */
- id: string;
- /** Regex patterns that match instruction lines this rule covers. */
- patterns: RegExp[];
- /** The rule category. */
- category: RuleCategory;
- /** Which verifier handles this rule. */
- verifier: VerifierType;
- /** Human-readable description of what this rule checks. */
- description: string;
- /** Default severity. */
- severity: 'error' | 'warning';
- /** Confidence level for rules produced by this matcher. */
- confidence?: 'high' | 'medium' | 'low';
- /** Build the verification pattern from the matched line. */
- buildPattern: (line: string, match: RegExpMatchArray) => VerificationPattern;
-}
-
-/**
- * Recognized instruction file names.
- * Used by project-level discovery to find all instruction files in a repo.
- */
-export const INSTRUCTION_FILE_NAMES = [
- 'CLAUDE.md',
- 'AGENTS.md',
- '.cursorrules',
- '.github/copilot-instructions.md',
- 'GEMINI.md',
- '.windsurfrules',
- '.rules',
-] as const;
-
-/** A conflict between rules in different instruction files. */
-export interface CrossFileConflict {
- /** Topic or pattern category the conflict relates to. */
- topic: string;
- /** Rules from different files that contradict each other. */
- rules: Array<{ file: string; rule: Rule }>;
- /** Description of the conflict. */
- description: string;
-}
-
-/** A redundancy: the same instruction appearing in multiple files. */
-export interface CrossFileRedundancy {
- /** Normalized text of the redundant instruction. */
- normalizedText: string;
- /** Occurrences across files. */
- occurrences: Array<{ file: string; originalText: string }>;
-}
-
-/** Per-file analysis result within a project. */
-export interface FileAnalysis {
- /** Path to the instruction file. */
- filePath: string;
- /** Detected file format. */
- fileType: InstructionFileType;
- /** Rules extracted from this file. */
- ruleSet: RuleSet;
- /** Verification results (populated after verification). */
- results: RuleResult[];
-}
-
-/** Complete project-level analysis across all instruction files. */
-export interface ProjectAnalysis {
- /** Root directory of the project. */
- projectDir: string;
- /** Per-file analysis results. */
- files: FileAnalysis[];
- /** Cross-file conflicts (same topic, different instructions). */
- conflicts: CrossFileConflict[];
- /** Cross-file redundancies (same instruction, different wording). */
- redundancies: CrossFileRedundancy[];
- /** Map of rule categories to which files contain rules in that category. */
- coverageMap: Record;
- /** Aggregate summary across all files. */
- summary: ReportSummary;
-}
-
-/** Default compliance threshold for determining pass/fail from compliance ratios. */
-export const DEFAULT_COMPLIANCE_THRESHOLD = 0.8;
+export type {
+ RuleCategory,
+ VerifierType,
+ QualifierType,
+ InstructionFileType,
+ VerificationPattern,
+ Rule,
+ RuleSet,
+ MarkdownSection,
+ RuleMatcher,
+} from './types-core.js';
+
+export { INSTRUCTION_FILE_NAMES } from './types-core.js';
+
+export type {
+ ReportFormat,
+ TaskTemplate,
+ AgentRun,
+ Evidence,
+ RuleResult,
+ CategoryScore,
+ ReportSummary,
+ AdherenceReport,
+ CrossFileConflict,
+ CrossFileRedundancy,
+ FileAnalysis,
+ ProjectAnalysis,
+} from './types-results.js';
+
+export { DEFAULT_COMPLIANCE_THRESHOLD } from './types-results.js';
diff --git a/tests/emitter/eslint.test.ts b/tests/emitter/eslint.test.ts
index 449eb14..e9b4ec0 100644
--- a/tests/emitter/eslint.test.ts
+++ b/tests/emitter/eslint.test.ts
@@ -8,7 +8,7 @@
*/
import { describe, it, expect } from 'vitest';
-import { emitEslintConfig } from '../../src/emitter/eslint.js';
+import { emitEslintConfig, formatUnmappableSummary } from '../../src/emitter/eslint.js';
import type { EslintConfig } from '../../src/mapper/types.js';
function makeConfig(overrides: Partial = {}): EslintConfig {
@@ -210,10 +210,9 @@ describe('emitEslintConfig', () => {
const parsed = JSON.parse(output);
expect(parsed.rules['@typescript-eslint/no-explicit-any']).toBe('error');
expect(parsed.plugins).toEqual(['@typescript-eslint']);
- expect(parsed.extends).toEqual([
- 'eslint:recommended',
- 'plugin:@typescript-eslint/recommended',
- ]);
+ // Legacy emitter must not auto-enable plugin recommended sets:
+ // it would enable hundreds of rules the flat emitter does not.
+ expect(parsed.extends).toBeUndefined();
});
it('emits rules with options in legacy format', () => {
@@ -269,7 +268,7 @@ describe('emitEslintConfig', () => {
expect(parsed.rules['prefer-const']).toEqual(['warn', { destructuring: 'all' }]);
});
- it('emits unmappable rules in a comment block after the JSON', () => {
+ it('keeps legacy output as pure JSON even when unmappable rules exist', () => {
const config = makeConfig({
unmappable: [
{
@@ -281,16 +280,32 @@ describe('emitEslintConfig', () => {
});
const output = emitEslintConfig(config, 'legacy');
- // Find the JSON portion (ends before the unmappable comment block, or at end)
- const commentMarker = '// Unmappable rules';
- const commentStart = output.indexOf(commentMarker);
- const jsonPart = commentStart >= 0 ? output.substring(0, commentStart).trimEnd() : output;
- const parsed = JSON.parse(jsonPart);
+ // The entire output must parse as JSON. No trailing comment block,
+ // no JS-style // markers anywhere.
+ const parsed = JSON.parse(output);
expect(parsed).toHaveProperty('rules');
+ expect(output).not.toContain('//');
+ });
+
+ it('exposes unmappable rules through formatUnmappableSummary', () => {
+ const config = makeConfig({
+ unmappable: [
+ {
+ sourceRuleId: 'strict-mode-1',
+ sourceText: 'Use TypeScript strict mode',
+ reason: 'TypeScript strict mode is a tsconfig setting',
+ },
+ ],
+ });
+ const summary = formatUnmappableSummary(config);
+ expect(summary).toContain('strict-mode-1');
+ expect(summary).toContain('TypeScript strict mode is a tsconfig setting');
+ expect(summary).toContain('Use TypeScript strict mode');
+ });
- // The unmappable section must appear after the JSON
- expect(output).toContain('// [strict-mode-1]');
- expect(output).toContain('TypeScript strict mode is a tsconfig setting');
+ it('returns an empty summary when no rules are unmappable', () => {
+ const config = makeConfig();
+ expect(formatUnmappableSummary(config)).toBe('');
});
});
});
\ No newline at end of file
diff --git a/tests/llm/openai-provider.test.ts b/tests/llm/openai-provider.test.ts
new file mode 100644
index 0000000..d8fb337
--- /dev/null
+++ b/tests/llm/openai-provider.test.ts
@@ -0,0 +1,145 @@
+/**
+ * Tests for the OpenAI provider's retry behavior on transient errors.
+ *
+ * These tests inject a fake fetch implementation so we can drive the
+ * retry path deterministically without touching the network.
+ */
+
+import { describe, it, expect } from 'vitest';
+import { createOpenAiProvider } from '../../src/llm/openai-provider.js';
+
+const PROMPT_PATTERNS = ['camelCase', 'no-any'];
+
+/** Build a Response-like object with the given status, body, and headers. */
+function makeResponse(
+ status: number,
+ body: unknown,
+ headers: Record = {},
+): Response {
+ return new Response(typeof body === 'string' ? body : JSON.stringify(body), {
+ status,
+ headers,
+ });
+}
+
+/** Body that parses through the extractor without errors. */
+const SUCCESS_BODY = {
+ choices: [
+ {
+ message: {
+ content: JSON.stringify({ rules: [], unparseable_lines: [] }),
+ },
+ },
+ ],
+};
+
+describe('createOpenAiProvider retry behavior', () => {
+ it('retries on 429 and succeeds on a follow-up 200', async () => {
+ const calls: number[] = [];
+ const fetchImpl = async (): Promise => {
+ calls.push(Date.now());
+ if (calls.length === 1) {
+ return makeResponse(429, '', { 'Retry-After': '0' });
+ }
+ return makeResponse(200, SUCCESS_BODY);
+ };
+
+ const provider = createOpenAiProvider({
+ apiKey: 'test-key',
+ maxAttempts: 3,
+ sleep: async () => undefined,
+ fetchImpl,
+ });
+
+ await provider.extractRules(['some unparseable line'], PROMPT_PATTERNS);
+ expect(calls.length).toBe(2);
+ });
+
+ it('retries on 503 and succeeds on a follow-up 200', async () => {
+ const responses: Response[] = [
+ makeResponse(503, ''),
+ makeResponse(200, SUCCESS_BODY),
+ ];
+ let i = 0;
+ const fetchImpl = async (): Promise => {
+ const next = responses[i] ?? makeResponse(500, 'unexpected');
+ i += 1;
+ return next;
+ };
+
+ const provider = createOpenAiProvider({
+ apiKey: 'test-key',
+ maxAttempts: 3,
+ sleep: async () => undefined,
+ fetchImpl,
+ });
+
+ await provider.extractRules(['line'], PROMPT_PATTERNS);
+ expect(i).toBe(2);
+ });
+
+ it('does not retry on 400 (non-transient error)', async () => {
+ let calls = 0;
+ const fetchImpl = async (): Promise => {
+ calls += 1;
+ return makeResponse(400, 'bad request');
+ };
+
+ const provider = createOpenAiProvider({
+ apiKey: 'test-key',
+ maxAttempts: 3,
+ sleep: async () => undefined,
+ fetchImpl,
+ });
+
+ await expect(provider.extractRules(['line'], PROMPT_PATTERNS)).rejects.toThrow(
+ /OpenAI API error 400/,
+ );
+ expect(calls).toBe(1);
+ });
+
+ it('gives up after maxAttempts when all attempts return 429', async () => {
+ let calls = 0;
+ const fetchImpl = async (): Promise => {
+ calls += 1;
+ return makeResponse(429, '', { 'Retry-After': '0' });
+ };
+
+ const provider = createOpenAiProvider({
+ apiKey: 'test-key',
+ maxAttempts: 2,
+ sleep: async () => undefined,
+ fetchImpl,
+ });
+
+ await expect(provider.extractRules(['line'], PROMPT_PATTERNS)).rejects.toThrow(
+ /OpenAI API error 429/,
+ );
+ expect(calls).toBe(2);
+ });
+
+ it('honors the Retry-After header value when present', async () => {
+ const sleeps: number[] = [];
+ const responses: Response[] = [
+ makeResponse(429, '', { 'Retry-After': '2' }),
+ makeResponse(200, SUCCESS_BODY),
+ ];
+ let i = 0;
+ const fetchImpl = async (): Promise => {
+ const next = responses[i] ?? makeResponse(500, 'unexpected');
+ i += 1;
+ return next;
+ };
+
+ const provider = createOpenAiProvider({
+ apiKey: 'test-key',
+ maxAttempts: 3,
+ retryBaseDelayMs: 9999,
+ sleep: async (ms) => { sleeps.push(ms); },
+ fetchImpl,
+ });
+
+ await provider.extractRules(['line'], PROMPT_PATTERNS);
+ expect(sleeps).toEqual([2000]);
+ });
+});
- 
- 
- 
- 
- 
- 
-## Why
+# RuleProbe
-AI coding agents read your `CLAUDE.md`, `AGENTS.md`, or `.cursorrules`, but nothing checks whether your ESLint config actually enforces the same rules. Drift accumulates: the instruction file says "use camelCase" but ESLint never checks it, or ESLint enforces rules the instruction file never mentioned.
+When your `CLAUDE.md` says "use camelCase" but ESLint doesn't enforce it, drift has already happened. RuleProbe reads an instruction file and translates it to an ESLint config. It detects what each enforces but the other misses, or converts ESLint rules back to instruction prose.
-RuleProbe closes that gap. It reads your instruction file, extracts machine-verifiable rules, and bridges them to your tooling in three ways:
+[](https://www.npmjs.com/package/ruleprobe)
+[](https://github.com/moonrunnerkc/ruleprobe/actions/workflows/self-check.yml)
+[](https://github.com/moonrunnerkc/ruleprobe/blob/main/LICENSE)
+[](https://www.typescriptlang.org)
+[](https://nodejs.org)
+[](https://github.com/moonrunnerkc/ruleprobe/stargazers)
-**Translate.** Generate an ESLint config from your instruction file. `ruleprobe lint-config CLAUDE.md` produces a flat or legacy config you can drop into your project immediately.
+## Installation
-**Detect drift.** Compare your instruction file against an existing ESLint config. `ruleprobe drift CLAUDE.md .eslintrc.json` reports rules present in one but missing from the other, severity mismatches, and config argument differences.
-
-**Extract.** Pull a rules section out of an ESLint config for pasting into an instruction file. `ruleprobe extract .eslintrc.json` converts ESLint rules back into human-readable instruction prose.
-
-## Quick Start
+Requires Node.js 18 or later.
```bash
npm install -g ruleprobe
```
-Or run it directly:
+Or try it without installing:
```bash
npx ruleprobe --help
```
-**Translate an instruction file to ESLint config:**
+Confirm it's working:
+
+```bash
+ruleprobe --version
+# 4.5.0
+```
+
+## Usage
+
+For maintainers who use AI coding agents and ESLint. Works with any instruction file format your agents read.
+
+**Translate an instruction file to an ESLint config:**
```bash
ruleprobe lint-config CLAUDE.md
ruleprobe lint-config AGENTS.md --format legacy --output .eslintrc.json
```
-**Detect drift between instructions and ESLint:**
+**Detect drift between an instruction file and an existing ESLint config:**
```bash
ruleprobe drift CLAUDE.md .eslintrc.json
ruleprobe drift CLAUDE.md .eslintrc.json --format markdown
```
-**Extract rules from an ESLint config:**
+**Convert ESLint rules back to instruction prose:**
```bash
ruleprobe extract .eslintrc.json
ruleprobe extract .eslintrc.json --output rules-section.md
```
-**Parse an instruction file** to see what rules RuleProbe can extract:
+**See what rules RuleProbe can parse from an instruction file:**
```bash
-ruleprobe parse CLAUDE.md
-ruleprobe parse AGENTS.md --show-unparseable
+ruleprobe parse CLAUDE.md --show-unparseable
```
-**Verify agent output** against extracted rules (legacy mode). Valid formats: `text`, `json`, `markdown`, `rdjson`, `summary`, `detailed`, `ci`.
+**Check code against extracted rules (legacy verify mode):**
```bash
-ruleprobe verify CLAUDE.md ./agent-output --format text
ruleprobe verify AGENTS.md ./src --changed-since origin/main
```
-## What It Does
+**Discover and cross-reference all instruction files in a project:**
-**Translate.** Reads 7 instruction file formats (CLAUDE.md, AGENTS.md, .cursorrules, copilot-instructions.md, GEMINI.md, .windsurfrules, .rules) and emits an ESLint config. Flat config is the default; pass `--format legacy` for `.eslintrc.json` output. Each extractable rule maps to an ESLint rule with appropriate severity and options. Rules that have no ESLint equivalent appear as comments in the output so you know what wasn't covered.
-
-**Detect drift.** Compares parsed rules against an existing ESLint config. Reports rules in the instruction file but missing from ESLint (you're not enforcing what you said), rules in ESLint but not in the instructions (you're enforcing what you never stated), severity mismatches, and argument differences. Use `--format markdown` for PR-ready output.
-
-**Extract.** Parses an ESLint config and generates a markdown rules section you can paste into an instruction file. Stylistic rules (semicolons, quotes) are reported but skipped from the output by default since they don't carry meaningful instruction content.
-
-**Parse.** Extracts machine-verifiable rules from instruction files with qualifiers (`always`, `prefer`, `when-possible`, `avoid-unless`, `try-to`, `never`) and section context. Subjective lines like "write clean code" are reported as unparseable.
-
-**Verify.** Runs extracted rules against a directory of code. Deterministic by default, optional semantic analysis available. The original mode, still supported but no longer the primary focus.
+```bash
+ruleprobe analyze ./my-project --format json
+```
-**Analyze.** Discovers all instruction files in a project, parses each, and cross-references them for conflicts and redundancies. Pass `--semantic` for structural pattern analysis.
+Seven instruction file formats are supported: `CLAUDE.md`, `AGENTS.md`, `.cursorrules`, `copilot-instructions.md`, `GEMINI.md`, `.windsurfrules`, `.rules`. Full flag reference: [docs/cli-reference.md](docs/cli-reference.md)
## Configuration
-RuleProbe auto-discovers a config file in the working directory (or any parent). You can also pass `--config