Skip to content

Improve RepoStats plugin resilience: Avoid GitHub API rate-limit failures in CI and support GITHUB_TOKEN gracefully #797

@valadas

Description

@valadas

Problem

The current RepoStatsBuildStep DocFX plugin (see logs: https://github.com/DNNCommunity/DNNDocs/actions/runs/25406094109/job/74517256950) fails CI builds due to GitHub API rate limiting, especially when run with the default GITHUB_TOKEN or as a GitHub App installation token.

  • The plugin attempts to gather all contributors/commits via repository statistics endpoints on every CI run.
  • This is excessive, triggers rate limits (see recent failures: API rate limit exceeded for installation...), and currently only skips if a specific env var (SKIP_CONTRIBUTORS) is set.
  • The ThrottleIfNeeded/backoff logic is insufficient: it retries early, but does not sleep until reset when actually exhausted.
  • In CI, using a Personal Access Token works, but we should not require increased credentials just for doc site builds if we can avoid it.

Ideal Solution

  • Use the automatically-provided GITHUB_TOKEN for all plugin GitHub API calls in CI if possible.
  • When rate limited with the GITHUB_TOKEN:
    • If practical, cache the relevant data between runs, or upload it periodically via a scheduled/statistics job instead of inline with every doc build.
  • Log explicit rate limit error summaries in the CI output for later investigation.
  • (Optionally) If running with a high-permission PAT, permit full stats; otherwise, default to lightweight behavior in CI.
  • Improve throttle and error handling: when fully rate limited, detect and sleep-until-reset or just skip, rather than retry immediately or hard fail.

Additional Suggestions

  • Consider a mode that fetches & stores stats once daily (cron), then reuses that cache during most builds.
  • Consider API endpoint selection: prefer endpoints that are more generous (lower cost) if only summary data is needed.
  • Review plugin placement: only gather stats for the main repo (avoid duplicate calls in forks, PRs, etc).

References


Acceptance criteria:

  • Plugin no longer causes CI failures due to GitHub API rate-limits when running in typical PR or branch builds with GITHUB_TOKEN.
  • Rate limit exceeded situations are handled with a warning and fallback, not a job failure.
  • It is clear in logs WHEN and WHY the fallback has occurred.
  • Minimal/no PAT requirement for typical CI runs; full stats only require PAT.

Metadata

Metadata

Labels

No labels
No labels

Type

No type
No fields configured for issues without a type.

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions