Skip to content
Open
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
190 changes: 117 additions & 73 deletions AGENTS.md
Original file line number Diff line number Diff line change
@@ -1,85 +1,129 @@
# FlashDreams Agent Guide

FlashDreams is a GPU-heavy inference and serving library for autoregressive video and world models. Default to inspection, docs, config checks, and CPU tests unless the user explicitly asks to run generation or GPU workflows.

Start here, then use the narrower docs for the task in front of you:

- `skills/` contains repo-authored Agent Skills. Use `skills/README.md` for opt-in setup and skill authoring rules.
- `README.md` covers user-facing setup, requirements, supported models, and first-run commands.
- `CONTRIBUTING.md` covers PR process, DCO sign-off, coding conventions, test markers, and dependency rules.
- `docs/README.md` covers Sphinx docs build and hosting. `tests/README.md` covers local, Docker, and CI-oriented test entry points.

## Repo Map

- `flashdreams/flashdreams/core/`: reusable numerical primitives, checkpoint loading, distributed helpers, attention, and I/O. Keep it model-agnostic.
- `flashdreams/flashdreams/infra/`: framework contracts and orchestration for configs, pipelines, encoders, decoders, diffusion models, schedulers, runners, profiling, and CUDA graph wrapping.
- `flashdreams/flashdreams/recipes/`: built-in reusable recipe code such as WAN, Cosmos, TAEHV, and template wiring.
- `flashdreams/flashdreams/configs/`, `plugins/`, and `scripts/`: runner registry, plugin discovery, and CLI entry points.
- `integrations/<name>/`: workspace-member model/plugin packages with their own configs, runners, tests, README files, and `pyproject.toml` entry points.
- `docs/source/`: Sphinx sources for quickstart, models, developer guides, API, and community docs.
- `tests/`: root test helpers plus package/integration tests. Ignore `.claude/worktrees/` when scanning the source tree; those are nested worktree artifacts, not the repo's current source.

## Skill Map

- `skills/flashdreams-integrations`: read before changing recipe/integration architecture, runner registration, pipeline wiring, or model boundaries.
- `skills/integrate-a-model`: read before porting a new external model or reproducing an integration workflow.
- `skills/profile-model-performance`: read before starting performance work on an existing model integration, demo, runner, or serving path; use it to map execution, add stage timings, and identify decode/model/cache/transfer/presentation bottlenecks.
- `skills/apply-inference-optimizations`: read before porting runtime speedups such as bounded K/V caches, overlap, compile, CUDA graphs, decoder layout changes, or presentation queue tuning into an integration.
- `skills/validate-performance-quality`: read before adding benchmark sweeps, quality comparisons, profiler probes, performance summaries, or docs for a performance change.
- `skills/flashdreams-postprocessing`: read before adding or modifying video post-processors, postprocess presets, `VideoPostprocessStream`, buffering/layout behavior, or runner postprocess wiring.
- `skills/python-docstring-style`: read before adding or polishing Python docstrings, field docstrings, module comments, or SPDX headers.
- `skills/maintaining-oss-state`: read before dependency, license, NOTICE, REUSE, or OSS-release collateral changes.
- When adding a new `skills/<skill-name>/SKILL.md`, update this section so agents can discover when to use it.

## Common Commands

```bash
uv sync --extra dev --extra runners
uv sync --package <integration> --extra dev
uv run flashdreams-run --help
uv run flashdreams-run <runner-name> --help
uv run flashdreams-run --no-instantiate <runner-name>
uv run --group lint pre-commit run -a
uv run pytest -m ci_cpu
uv run pytest -m "not manual"
./tests/run_tests_local.sh [target]
./tests/run_tests_docker.sh [target]
uv run --group docs sphinx-build -b html docs/source docs/_build/html
uv run --group docs sphinx-autobuild -E docs/source docs/_build/html --port 8000
# FlashDreams — AI Agent Context

## Purpose

FlashDreams is a GPU-heavy inference and serving library for autoregressive
video and world models. Researchers will modify the code for new models,
hardware, and experiments, so optimize for code that is easy to understand and
change rather than code that anticipates every possible use.

These rules apply to all newly written or modified code, even when nearby code
uses older patterns.

## Coding Principles

- **Readability over flexibility.** Add an abstraction only when it reduces net
complexity. Prefer direct code over frameworks, registries, wrappers, and
extension points created for hypothetical reuse.
- **Solve the present problem.** Implement the smallest complete behavior the
request needs. Do not bundle adjacent features, compatibility layers, or
speculative generality.
- **One way to do it.** Each concept should have one representation and one
execution path. Remove superseded paths instead of keeping aliases, fallback
branches, or parallel APIs.
- **Locality of behavior.** Keep related state and logic together. Every
indirection must improve readability or provide genuine reuse across several
call sites.
- **Configuration has one owner.** Do not add a CLI flag, environment variable,
constant default, and config field for the same setting. Extend the existing
authoritative configuration path.
- **Fail fast.** Reject malformed or unsupported input at the boundary. Do not
silently substitute defaults, swallow exceptions, or continue through a
partially initialized state.
- **Preserve dependency direction.** Generic runtime code must not know about a
model integration. Model-specific behavior belongs with that integration.
- **Comments describe current code.** Never leave migration history, “temporary”
notes without an issue, or commentary about what code used to do.
- **Google Python Style Guide** is the general baseline.

## Simplicity Rules

### Minimize abstraction layers

Add a class or wrapper only when it removes meaningful duplication across at
least three call sites or represents a real domain object.

```python
# Discouraged: one-use delegation.
class SessionStarter:
def start(self, runtime, inputs):
return runtime.start_session(inputs)

# Preferred.
session = runtime.start_session(inputs)
```

Use `--no-instantiate` before GPU work to inspect the resolved runner config without constructing models or loading checkpoints.
Use dataclasses for domain objects, protocol payloads, persisted artifacts, and
configuration schemas. Do not create dataclasses that merely bundle arguments
for one local call.

## No-GPU Workflow
### Avoid defensive attribute probing

- Inspect available runners with `uv run flashdreams-run --help`, then inspect a specific runner with `uv run flashdreams-run --no-instantiate <runner-name>`.
- Prefer CPU checks first: config imports, checkpoint key-remap shape/bijection tests on CPU or meta tensors, docs builds, `pytest -m ci_cpu`, and static assertions about runner names and pipeline wiring.
- Avoid `ci_gpu`, generation, `torchrun`, Docker GPU tests, large Hugging Face downloads, rollout parity, CUDA graph, WebRTC runtime, and quality-regression tests on CPU-only hosts unless the user requests them or the test explicitly skips cleanly.
Do not use `getattr(...)` chains to guess which interface an object implements.
Define the expected interface and call it directly. Let missing attributes fail
at the actual programming error.

## Testing Guidance
### Avoid unnecessary constants

Every pytest test must carry exactly one of `ci_cpu`, `ci_gpu`, or `manual`; `CONTRIBUTING.md` has the exact rules. Use module-level `pytestmark = pytest.mark.ci_cpu` for pure Python/metadata tests. Keep GPU, `libGL`/`cv2`, large-checkpoint, credential, and download-heavy checks out of `ci_cpu`.
Inline file-local literals by default. Introduce a named constant when multiple
components must agree on the value and drift would cause a subtle failure, such
as a wire-protocol key or environment variable name.

## Boundaries
### No dead-code compatibility

Keep dependency direction strict: `core` -> `infra` -> recipes/integrations. `core` and `infra` must not import from `integrations/`; expose a generic config slot or override hook instead of adding model-specific branches. Built-in reusable model pieces belong in `flashdreams/flashdreams/recipes/`; standalone plugin packages belong in `integrations/<name>/`.
FlashDreams is evolving quickly. Unless the user explicitly requires a staged
migration, delete old constructors, parameters, aliases, adapters, and branches
when replacing them. A second path is a maintenance cost, not free safety.

## Known Pitfalls
### Explicit signatures and useful types

- FlashDreams targets large NVIDIA GPUs; many real rollouts need 80 GB VRAM and CUDA-capable dependencies.
- Full `uv sync`/`uv run` can build CUDA packages such as Transformer Engine. For narrow work, use `uv sync --package <integration> --extra dev` when possible.
- `flashdreams-run --no-instantiate` resolves config only; it does not prove weights, CUDA execution, or parity.
- First GPU runs include compile/autotune warmup, so benchmark harnesses should discard warmup chunks.
- Docs CI mocks heavy GPU packages; local docs work should use the commands in `docs/README.md`.
- Plain `pytest` includes manual tests. Use `pytest -m ci_cpu` or `pytest -m "not manual"` for normal local validation.
- New or moved skills must pass the Agent Skills frontmatter checks in `tests/test_agent_skills.py`.
Prefer explicit keyword parameters over argument bags. Add local annotations
when a boundary returns an untyped value and naming the concrete type improves
understanding. Do not add annotations that force casts or repeat obvious types.

## Troubleshooting Links
## Error Handling

- Setup and requirements: `README.md`, `docs/source/quickstart/installation.rst`
- CLI details: `docs/source/api/cli.rst`
- Integration/plugin layout: `docs/source/api/integrations.rst`
- New integrations: `docs/source/developer_guides/new_integration.rst`
- Docs and CPU autodoc: `docs/README.md`
- Tests and quality regressions: `tests/README.md`
- Security reports: `SECURITY.md`
- Use normal exceptions and preserve their traceback.
- Validate external data, user input, checkpoint metadata, and transport
messages at their boundary.
- Do not catch an exception unless the code can recover, add actionable context,
or complete required cleanup.
- Never use a broad fallback to hide a missing dependency, unsupported backend,
or failed optimization.
- “Auto” behavior must report what it selected and why; it must not silently
change model semantics or output quality.

## Documentation and Comments

- Every public method needs a clear docstring describing behavior, arguments,
return value, and important failure modes.
- Private methods need concise docstrings when their purpose is not obvious.
- Prefer plain language over architecture jargon.
- Explain invariants and non-obvious constraints, not line-by-line mechanics.

## Tests Prove Behavior

Write a small number of high-signal tests for:

- user-visible behavior and regressions;
- numerical or state-machine correctness;
- validation at external boundaries;
- parity between two paths that claim identical semantics;
- failure and cleanup behavior that local reasoning cannot prove.

Do not write tests that:

- restate constants, config defaults, package metadata, or function signatures;
- verify that a mock was called without asserting the resulting behavior;
- snapshot implementation details instead of a public outcome;
- duplicate checks already guaranteed by static typing;
- require a GPU when a CPU/meta-tensor test proves the same contract.

Every pytest test must carry exactly one marker: `ci_cpu`, `ci_gpu`, or
`manual`. Use module-level `pytestmark = pytest.mark.ci_cpu` for CPU-safe test
modules. Plain `pytest` includes manual tests; normal validation should use a
marker expression.

## Source of Truth

- `SECURITY.md`: vulnerability reporting.
Loading