Skip to content

fix(fetchers): enforce docs site outbound policy#100

Merged
chaliy merged 1 commit intomainfrom
fix/issue-94-docs-site-ssrf-policy
Apr 15, 2026
Merged

fix(fetchers): enforce docs site outbound policy#100
chaliy merged 1 commit intomainfrom
fix/issue-94-docs-site-ssrf-policy

Conversation

@chaliy
Copy link
Copy Markdown
Contributor

@chaliy chaliy commented Apr 15, 2026

What

Enforce FetchKit's outbound policy for DocsSiteFetcher, covering direct llms.txt requests, llms probe requests, and docs-page fetches.

Closes #94.

Why

DocsSiteFetcher used its own redirect-following client, which bypassed DNS pinning, private-IP blocking, and manual redirect validation. That let docs and llms.txt requests ignore the default SSRF posture.

How

  • route direct llms.txt fetches through the shared hardened request helper
  • route llms-full.txt / llms.txt probes and fallback docs-page fetches through the same transport path
  • add SSRF regressions for direct loopback llms.txt fetches and localhost -> 127.0.0.1 redirect enforcement

Risk

  • Low
  • docs-site fetches now use the hardened redirect flow; the main regression risk is around llms probing and redirect handling on legitimate docs sites

Checklist

  • Unit tests are passed
  • Smoke tests are passed
  • Documentation is updated
  • Specs are up to date and not in conflict

@chaliy chaliy merged commit eefb3b5 into main Apr 15, 2026
11 checks passed
@chaliy chaliy deleted the fix/issue-94-docs-site-ssrf-policy branch April 15, 2026 13:11
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

docs_site fetcher bypasses SSRF protections on docs and llms.txt URLs

1 participant