Skip to content

solanabr/Auditor

Repository files navigation

AUDITOR — Open-Source AI Security Audit Skill

Production-grade security audit for any codebase, powered by AI agents. 18 checklists · 1,182 verification items · 100 known attack vectors · Benchmarked against CertiK, SOC 2, OWASP Top 10:2025 Verify COSTS.md as a referecence - Running this audit can burn a lot of credits.


What Is This?

AUDITOR is a skill file (a structured prompt + checklists) that turns any LLM agent (Copilot, Cursor, Windsurf, Claude Code, Codex, etc.) into a professional-grade security auditor. It reads your code file by file, checks 1,182 items across 18 security domains, tests against 100 real-world attack vectors, and produces a structured report with severity scores.

It is not a SaaS product. It's a folder of markdown files you clone into your repo or give to an AI agent.

Adaptation required before use. This skill was originally developed for a specific Solana/Anchor DeFi project and then generalized. Before running it on your project you must update discovery/file-map.md with your actual folder structure, file names, and state variable names. The questionnaire in QUESTIONS.md is a blank template — fill it out for your project before invoking the auditor. Everything else (checklists, known vectors, output rules) is fully portable as-is.


Supported Languages & Frameworks

Language Checklists Items
Rust (Solana/Anchor) 01-07 417
TypeScript / Node.js 08-09 160
React / Next.js 08, 10 122
Python 14 82
Go / Java / Ruby / PHP 15 88
Always applied (any repo) 11-13, 16-18 418
Total 18 1,182

Quick Start

Option 1: Clone into your repo (recommended for private repos)

# From your project root
git clone https://github.com/YOUR_ORG/AUDITOR.git .github/skills/AUDITOR

# Or copy the folder manually
cp -r /path/to/AUDITOR .github/skills/AUDITOR

Then in your AI agent (Copilot, Cursor, etc.):

Audit the entire repository using the AUDITOR skill with FULL scope

Option 2: Point to the repo (recommended for open source)

If your code is in a public repo, you can clone AUDITOR separately and point the agent at your code:

# Clone AUDITOR
git clone https://github.com/YOUR_ORG/AUDITOR.git

# Open your target project in your IDE
# Copy AUDITOR into it, or configure as a skill

Option 3: Feed files to the API directly

If building a service, send the AUDITOR files as system context and the target repo files as user content to any LLM API.


Before Running an Audit

Mandatory Intake Rule (Do Not Skip)

Before any audit output is produced, the agent MUST recursively read all markdown files under AUDITOR/, including:

  • root docs,
  • all checklists/*.md,
  • all discovery/*.md,
  • all templates/*.md,
  • known-vectors/INDEX.md and every file in known-vectors/001..100.

If any file is not loaded, the audit is invalid and must be marked incomplete.

  1. Fill out the questionnaire: Copy QUESTIONS.md and answer all questions. This tells the auditor what checklists to apply, what severity calibration to use, and what compliance frameworks matter.

  2. Review estimated costs: Check COSTS.md for token/dollar estimates based on your repo size and chosen model.

  3. Choose your scope:

Scope What It Covers Estimated Time (50K lines)
FULL Everything — all 18 checklists + 100 vectors 60-90 min
PROGRAM Smart contract only (checklists 01-07) 20-35 min
BACKEND Backend API (checklists 08-09) 15-25 min
FRONTEND Frontend (checklists 08, 10) 15-25 min
DEVOPS Infra + supply chain (checklists 11-13) 10-15 min
QUICK Known vectors only (grep-based scan) 5-10 min

Folder Structure

AUDITOR/
├── README.md                ← YOU ARE HERE
├── SKILL.md                 ← Orchestrator — the AI agent reads this first
├── OUTPUT-RULES.md          ← Mandatory output format, severity scale
├── FULL-AUDIT.md            ← Step-by-step execution plan for complete audits
├── QUESTIONS.md             ← Pre-audit questionnaire (fill before running)
├── COSTS.md                 ← Estimated costs by model and repo size
├── TOP-100-HACKS.md         ← Compatibility pointer (canonical source is known-vectors/)
│
├── known-vectors/           ← Individual attack vector files (for contributors)
│   ├── INDEX.md             ← One-line index of all vectors
│   ├── 001-private-key-leak.md
│   ├── 002-flash-loan-price-manipulation.md
│   ├── ...
│   └── 100-insufficient-backup-disaster-recovery.md
│
├── checklists/              ← 18 micro-checklists (the core verification items)
│   ├── 01-program-account-validation.md    (57 items)
│   ├── 02-program-access-control.md        (50 items)
│   ├── 03-program-arithmetic-safety.md     (61 items)
│   ├── 04-program-cpi-pda.md              (63 items)
│   ├── 05-program-state-machine.md         (56 items)
│   ├── 06-program-economic-logic.md        (62 items)
│   ├── 07-program-opsec-governance.md      (75 items)
│   ├── 08-typescript-safety.md             (60 items)
│   ├── 09-backend-security.md             (100 items)
│   ├── 10-frontend-security.md             (76 items)
│   ├── 11-supply-chain.md                  (43 items)
│   ├── 12-secrets-opsec.md                 (52 items)
│   ├── 13-deployment-infrastructure.md     (77 items)
│   ├── 14-python-safety.md                 (82 items)
│   ├── 15-general-language-safety.md       (88 items)
│   ├── 16-formal-verification-testing.md   (58 items)
│   ├── 17-logging-monitoring-incident-response.md (62 items)
│   └── 18-privacy-compliance-change-management.md (60 items)
│
├── discovery/               ← File patterns and search commands
│   ├── file-map.md          ← Maps checklists → target file patterns
│   └── grep-commands.md     ← All grep/terminal commands by category
│
└── templates/               ← Output templates
    ├── report-template.md   ← Full audit report structure (9 sections)
    └── instruction-worksheet.md  ← Per-instruction deep-review form

Output Format

The audit produces a structured markdown report with:

  1. Executive Summary — risk score (1-10), deploy/no-deploy verdict, severity distribution
  2. Instruction Matrix — every smart contract instruction mapped
  3. State Model — account structs, PDA seeds, relationships
  4. Per-Item Verdicts — all 1,182 checklist items with [PASS], [FAIL-N], [PARTIAL], or [N/A]
  5. Known Vectors Results — each of the 100 attack vectors with explicit verdict and evidence
  6. Findings — deduplicated, severity-sorted
  7. Attack Scenarios — narrative exploitable paths
  8. Aggregate Score — PASS/PARTIAL/FAIL percentages
  9. Recommendations — prioritized fix list

See OUTPUT-RULES.md for the complete specification.


How to Invoke

VS Code / GitHub Copilot

@workspace Audit the entire repository using the AUDITOR skill

Cursor / Windsurf

Read .github/skills/AUDITOR/SKILL.md then audit this repository following the FULL-AUDIT.md execution plan

Claude Code (CLI)

Read the AUDITOR skill files in .github/skills/AUDITOR/ and perform a full security audit of this repository

API (programmatic)

# Send SKILL.md + OUTPUT-RULES.md + FULL-AUDIT.md as system prompt
# Send target files as user messages (chunked)
# Collect structured output

Contributing

Adding a New Attack Vector

  1. Create known-vectors/NNN-short-name.md with the next available number
  2. Use this template:
---
id: NNN
title: "Your Attack Name"
severity: 7
category: crypto|backend|frontend|devops
---

### NNN — Your Attack Name
**Severity: 7** | **Real: Example Incident ($X, Year)**

Brief description of the attack.

#### Verification Procedure

**Step 1: Description**
\```
grep command or check to perform
\```
- ✅ PASS: What passing looks like
- ❌ FAIL: What failing looks like

**Step 2: ...**
(continue with 3-12 steps)

**Overall verdict:**
- ✅: Full mitigation criteria
- ⚠️: Partial mitigation criteria
- ❌: Vulnerability criteria
  1. Add the entry to known-vectors/INDEX.md
  2. Submit a PR

Adding Checklist Items

  1. Open the relevant checklist in checklists/
  2. Add new items following the existing format (ID prefix + sequential number)
  3. Update the item count in SKILL.md

Updating Cost Estimates

If model pricing changes, update COSTS.md with new rates.


Benchmarking

This auditor was designed by comparing methodology from:

Standard Domain What We Took
CertiK Crypto audit On-chain vulnerability taxonomy, formal verification approach
EY / SOC 2 Enterprise IT audit Control objectives, evidence requirements
OWASP Top 10:2025 Web security Attack categories, severity calibration
NIST SP 800-53 Government security Control framework structure
COBIT 2019 IT governance Process maturity model
GDPR / MiCA / DORA EU regulation Compliance checklist items

License

MIT — use it, fork it, improve it, sell services built on it. Attribution appreciated but not required.


FAQ

Q: Does this replace a professional audit? A: It covers more items than most paid audits (1,182 plus 100 known-vector checks vs typical 50-200), but an AI auditor cannot do everything a human can (social engineering assessment, business logic review requiring domain expertise, legal compliance opinions). Use this as a first pass, then hire humans for what it flags.

Q: Which AI model should I use? A: See COSTS.md. For maximum depth, use Opus 4 or o3. For best value, use Sonnet 4 or GPT-4.1. For CI/CD integration (fast, cheap), use Haiku or o4-mini.

Q: How long does an audit take? A: Depends on repo size and model. A 50K-line repo takes 60-90 minutes with Opus 4. See COSTS.md for full estimates.

Q: Can I use this in CI/CD? A: Yes — run the QUICK scope (known vectors grep scan) on every PR, and FULL scope on release branches.

Q: Can I use this for non-Solana projects? A: Yes — checklists 08-18 are language-agnostic or cover Python/Go/Java/Ruby/PHP. Checklists 01-07 are Solana-specific.

About

Open-Source AI Security Audit Skill

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

 
 
 

Contributors