docs(skills) + fix(agent): Claude Code / Codex skill + make cascadeflow.agent callable as decorator#182
Merged
saschabuehrle merged 8 commits intomainfrom Apr 28, 2026
Merged
Conversation
Adds skills/cascadeflow/SKILL.md — a distributable skill that gives Claude Code and Codex agents up-to-date guidance on using cascadeflow: the three-tier API (init / run / @agent), runtime intelligence harness, drafter+verifier selection, tool loops, framework integrations, and the common pitfalls students and contributors hit. Also carves a narrow exception in .gitignore so this one published skill file is committed while personal/local skill files under skills/ remain ignored. Install paths for consumers: Claude Code: ~/.claude/skills/cascadeflow/SKILL.md Codex: ~/.agents/skills/cascadeflow/SKILL.md
github-actions
Bot
added
documentation
Validated the skill the way a hackathon developer will use it: dispatched a subagent with SKILL.md as their only reference and asked them to build a cost-optimized agent with budget cap, tool calling, graceful stop handling, trace export, and a callback hook. RED-phase report identified the gaps that block a 30-minute demo: - no proxy-vs-in-process comparison (the core "why agent-in-loop" pitch) - stop-action contract undocumented: which exception, which attribute - session.trace() / session.summary() field shapes not shown - specific stop reason strings (budget_exceeded, max_tool_calls_reached, compliance_no_approved_model, latency_limit_exceeded, energy_limit_exceeded) absent - no Python tool-registration example (ToolConfig + ToolExecutor + tools= on agent.run) - callback hook for cost/decision streaming was a one-liner with no API - max_latency_ms semantics (cumulative vs per-step) unspecified - self-improving aspect not mentioned GREEN-phase patch: - Add "Why in-the-loop matters" comparison table (proxy vs harness) - Document HarnessStopError + BudgetExceededError handling with try/except - Document the five verbatim stop reason strings - Show session.summary() and session.trace() dict shapes inline - Add ToolConfig + ToolExecutor + tools= example for CascadeAgent - Add CallbackManager + CallbackEvent + callback_manager= wiring example - Clarify max_latency_ms is cumulative across the run - Add self-improving paragraph to agent loops section GREEN re-test with the same hackathon brief moved every previously NOT-COVERED in-loop advantage to COVERED, and the verdict from "Partially" to "Yes, ships in under 30 minutes." Frontmatter still 984 / 1024 chars.
Collaborator
Author
Validation update — agent-in-loop coverageValidated the skill end-to-end the way a hackathon dev will use it (RED → GREEN), then patched the gaps. Pushed How it was testedA subagent was given only
RED — what the original skill couldn't deliver
Verdict: Partially ships in 30 min — likely crashes at runtime on GREEN — what the patch added (commit 68a2b7b, +139/-4)
GREEN re-test (same brief, same constraints)Every previously not-covered item is now COVERED with a quotable line in the skill. Dev verdict: "Yes — ships in well under 30 minutes." No crash-causing unknowns remain; residual minor unclarities ( Frontmatter still 984 / 1024 chars. All CI checks were green on the previous push and there are no code changes here. |
Adds a "Found a bug? Contribute the fix back" section so that when a hackathon dev (or the agent assisting them) discovers a bug in cascadeflow itself or in any of its integrations, the skill instructs the canonical fork → fix branch → commit → push → upstream PR flow without making the dev read CONTRIBUTING.md or guess paths. Includes: - Scope rule: "Bug in cascadeflow or an integration → upstream PR. Bug in your own hackathon app → skill has no opinion." - Path map for every place a bug can live (Python core, TS core, each Python integration, each TS integration package, ml/, etc.) so the agent doesn't guess where withCascade or harness/instrument lives. - Canonical command sequence: gh repo fork --clone --remote, pre-commit install, branch off main, conventional-commit area tags, pnpm --filter <pkg> test vs pytest, gh pr create against lemony-ai/cascadeflow. - "Unblock the demo while the PR is in review" with both pip install -e (Python) and pnpm pack + npm install (TS), plus a warning that npm link is flaky with pnpm workspaces. - Don't list: no main pushes, no force-pushes to shared branches, no --no-verify, no PR without a regression test, no committed secrets. Validated RED → GREEN with the same 3 scenarios: - A) bug in cascadeflow.harness.instrument - B) bug in @cascadeflow/langchain withCascade - C) bug in dev's own app RED (without section): agent produced a sensible plan but flagged the contributor flow, monorepo layout, paths, and test commands as guesses. GREEN (with section): every one of 12 audited facts came directly from the skill with quotable lines, including the Scenario-C disclaimer. Verdict moved to "Yes — PR-able fix without reading source or CONTRIBUTING.md." Frontmatter trimmed to 960 / 1024 chars to stay within the agentskills.io spec after adding the new bugfix trigger.
Collaborator
Author
Update — works for both Claude Code & Codex; added upstream bug-fix workflowSkill is dual-target. Same New section: "Found a bug? Contribute the fix back" (commit
|
Ran the upstream-fix workflow end-to-end on a real working copy (real bug, real branch, real pytest, real commit) and hit two gotchas that directly violate the skill's own rules. Fixed them. 1. Bare `pytest` fails on a fresh `pip install -e .` because the repo's pyproject pytest config injects `--cov=cascadeflow ... --asyncio-mode=auto`, and `pytest-cov` / `pytest-asyncio` aren't pulled by the base install — they live in the `[dev]` extra. Skill now says `pip install -e ".[dev]"` before `pytest`. Same gotcha added to the "Don't" list. 2. The skill's commit example was `git commit -am "fix(...)"`. The `-a` flag stages tracked changes only; the new regression test file is untracked, so it gets silently dropped. A copy-paste user would commit the fix without the test, directly violating the skill's own "Don't open a PR without a regression test" rule. Replaced with explicit `git add <touched> <new-test>` + `git commit -m`. Also added to the "Don't" list with the rationale. Plus two minor improvements from the integration-package dry-run: - Mention `pnpm install --filter @cascadeflow/<pkg>... --frozen-lockfile` for faster iteration when only one package was touched. - Note that Step 0 (`gh release list` / `gh issue list`) requires `gh auth login`; suggest a web search + `git log upstream/main` as fallback for unauthed contributors. - Clarified the regression-test rule applies to "non-trivial fixes" (single-line comment/typo fixes are fine without one — confirmed by the integration dry-run, which fixed a one-line JSDoc with no test). Method: spawned three parallel real-dev subagents — (a) hackathon demo build with imports/agent-construction validated against installed cascadeflow 1.1.0, all 13 documented session.summary() keys verified; (b) core bug-fix on a worktree, real `pip install -e .`, real `pytest`, real `git commit`, real `cascadeflow.__version__` regression test that passes; (c) integration bug-fix on a worktree, real `pnpm install --frozen-lockfile`, real `pnpm --filter @cascadeflow/langchain test` (75 passed), real `git commit`. None pushed; would-be `gh pr create` commands rendered. Frontmatter still 960 / 1024 chars.
Collaborator
Author
Real-dev test (end-to-end against installed package and real working copies)Ran the skill the way a hackathon dev actually would, including the bug-fix workflow in core and integrations. Three parallel real-dev subagents, all with side-effects on disk: A — Hackathon agent demo (
|
Collaborator
Author
Ready for reviewFinal state of this PR:
Test plan in PR body updated to tick what's been validated; the only unchecked items are honest deferrals (Codex was not invoked in a Codex session — file format is platform-neutral, but a runtime check by a reviewer with Codex installed would close it). Three commits on this branch:
PR is review-ready. Distributable to hackathon participants in its current state. |
Collaborator
Author
|
Addressed the last reviewable issues and pushed . Changes:
PR is ready for the next review pass. |
Collaborator
Author
|
Addressed the last reviewable issues and pushed Changes:
PR is ready for the next review pass. |
…r items
Independent code review surfaced a critical accuracy bug plus four
non-trivial issues. All confirmed against the installed package and
fixed.
CRITICAL — `@cascadeflow.agent(...)` raises TypeError
=====================================================
`cascadeflow/__init__.py` registers two colliding names:
- lazy import "agent" → the module file `cascadeflow/agent.py`
- eager `harness_agent` → the actual decorator
`cascadeflow.agent` resolves to the module. The skill recommended
`@cascadeflow.agent(...)` in five places (frontmatter description,
entry-point table, the headline policy-metadata code example,
Common pitfalls, Red flags). A hackathon dev who copy-pastes hits:
TypeError: 'module' object is not callable
Verified empirically against pip-installed cascadeflow.
Fix: switch all five sites to the working pattern:
from cascadeflow.harness import agent
@agent(budget=..., kpi_weights=..., compliance=...)
Also note `cascadeflow.harness_agent` as the eager top-level alias.
Added an explicit "Don't write @cascadeflow.agent(...)" entry to the
pitfalls list with the exact TypeError so devs recognize it.
Verified the patched example works against pip-installed cascadeflow:
from cascadeflow.harness import agent
@agent(budget=0.20, kpi_weights={...}, compliance="gdpr")
async def my_agent(query: str): ...
# callable: True
(The root-cause collision should also be fixed upstream — drop the
`"agent"` lazy alias in `cascadeflow/__init__.py`. Out of scope for
this skill PR. Note left for the project README, which has the same
bug at line 180.)
OTHER REVIEWER ITEMS
====================
I1 — Stop-handling snippet referenced `session` without showing
where it came from. Wrapped the try/except in an explicit
`with cascadeflow.run(...) as session:` block.
I2 — `pre-commit install` is documented but the repo has no
`.pre-commit-config.yaml`. Made it conditional ("if a config file
exists at the root, also run pre-commit install") and softened the
related "Don't" entry. (CONTRIBUTING.md has the same stale doc bug;
out of scope here.)
I3 — "Mentions budgets / compliance / KPI weights" was an over-broad
trigger for a globally-installed skill — could mis-fire on healthcare
or fintech apps that don't use cascadeflow. Tightened to require a
cascadeflow signal (import / repo / explicit mention) alongside.
I4 — `simulate(...)` was framed as "model a run against historical
traces". Real signature is `simulate(queries, models, quality_threshold,
domain_detection)` — a query replay through the deterministic routing
pipeline. Rewrote the example with the real signature and the actual
return fields (projected_cost, escalation_rate, model_distribution).
M1 — Moved `gh auth login` requirement to step 0 prerequisite (every
gh command needs it, not just gh release/issue list).
M2 — Dropped the misleading "faster iteration on a single TS package"
hint — `pnpm install --filter <pkg>... --frozen-lockfile` is a CI
pattern, not a speed-up. Replaced with `pnpm --filter <pkg> test:watch`
suggestion for vitest watch mode.
Frontmatter still under cap: 975 / 1024 chars.
Collaborator
Author
Review pass — critical bug found and fixed (commit
|
`cascadeflow.agent` is the module file `cascadeflow/agent.py`, and many
internal imports rely on that:
from cascadeflow.agent import CascadeAgent
cascadeflow.agent.PROVIDER_REGISTRY
cascadeflow/integrations/openclaw/wrapper.py: from cascadeflow.agent import CascadeAgent
But README/docs/llms.txt all show the policy decorator as:
@cascadeflow.agent(budget=0.20, compliance="gdpr", kpi_weights={...})
async def my_agent(query: str): ...
That historically raised `TypeError: 'module' object is not callable`,
because modules aren't callable. Every README/docs reader who copy-pasted
the headline decorator example crashed on the first line.
We can't drop the `agent` lazy alias without breaking all the existing
module imports. Instead, subclass the module's type and add `__call__`
so calling `cascadeflow.agent(...)` delegates to the harness `agent`
decorator. Module-attribute access is unaffected.
Verified all four call paths:
1. @cascadeflow.agent(budget=0.20, ...) → works (the fix)
2. from cascadeflow.agent import CascadeAgent → still works
3. cascadeflow.agent.PROVIDER_REGISTRY → still works
4. from cascadeflow.harness import agent → still works
5. @cascadeflow.harness_agent(...) → still works
Regression tests in `tests/test_agent_module_callable.py` cover all
five paths so this can't silently regress.
Discovered while validating the new Claude Code / Codex skill: an
independent code review (`superpowers:code-reviewer`) flagged the
TypeError on the headline decorator example. This commit fixes the
root cause in the package; the skill keeps recommending the
version-agnostic `from cascadeflow.harness import agent` pattern
because pip-installed cascadeflow ≤ 1.2.0 still has the old behavior
until a release ships with this fix.
Test plan:
- `pytest tests/test_agent_module_callable.py` — 5 new tests pass
- `pytest tests/test_agent_p0_tool_loop.py tests/test_harness_api.py`
— 46 existing tests still pass (no regression on module-attr access)
- Manual: `import cascadeflow; @cascadeflow.agent(budget=0.20)` no longer raises
saschabuehrle
changed the title
… black formatting CI Python Code Quality flagged the bottom-of-file 'import sys as _sys' as E402 (module-level import not at top). sys is already imported at line 58, so just use it directly. Plus black asked for one extra blank line before the class. Verified locally: - ruff check: All checks passed! - black --check: 2 files would be left unchanged - mypy --ignore-missing-imports: Success
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds a distributable SKILL.md at
skills/cascadeflow/SKILL.mdso Claude Code and Codex agents working in cascadeflow projects get accurate, up-to-date guidance — without having to read the full docs cold.Primary goal: give developers and users (incl. hackathon participants) the fastest, most accurate start with cascadeflow. Content was written against
origin/mainat v1.2.0 and covers both the cascading and the runtime intelligence sides of the project.What the skill covers
cascadeflow.init·cascadeflow.run·@cascadeflow.agent·CascadeAgent· presets · gateway server · framework integrationsoff/observe/enforcemodes, the four per-step actions (allow/switch_model/deny_tool/stop), budget / KPI / compliance / energy enforcement,session.summary(),session.trace(),session.save("run.jsonl"),simulate(), env + YAML configexamples/andpackages/core/examples/nodejs/)PreRouter/ComplexityDetector)mainUserProfile,TierLevel,TIER_PRESETSresult.cost_saved/cost_saved_percentage(Py) andsavingsPercentage(TS), plusget_cascade_callback()for aggregate tracking.gitignore change
skills/is already gitignored (for personal/local skill files). This PR carves a narrow exception so this one published file can be committed:Future distributable skills can be added under
skills/<name>/with a matching negation line.Install paths for consumers
Test plan
name+descriptionpresent, third-personUse when…description with no workflow summary) — verified at 960 chars after adding the bug-fix-workflow triggermainAND validated against pip-installedcascadeflow(subagent built the demo; imports + agent construction + harness init + scoped run + trace export all clean to the auth boundary;session.summary()returns the exact 13-key dict the skill documents):cascadeflow.init(mode, budget, max_tool_calls, max_latency_ms, max_energy, kpi_targets, kpi_weights, compliance, callback_manager)→cascadeflow/harness/api.pycascadeflow.run(...)→HarnessRunContextwith.summary(),.trace(),.save()@cascadeflow.agent(budget, kpi_targets, kpi_weights, compliance)— metadata-only, sync + asyncHarnessMode = "off" | "observe" | "enforce"CascadeResultfields (content,model_used,total_cost,draft_cost,verifier_cost,cost_saved,cost_saved_percentage) →cascadeflow/schema/result.pyauto_agent,get_cost_optimized_agent,get_balanced_agent,get_quality_optimized_agent,get_speed_optimized_agent,get_development_agent→cascadeflow/utils/presets.pypython -m cascadeflow.server→cascadeflow/server.pywithCascade,CascadeFlow,CascadeAgent,findBestCascadePair,discoverCascadePairs,PreRouter,ComplexityDetector→packages/langchain-cascadeflow/src/index.tsexamples/integrations/{openai_agents,crewai,pydantic_ai,google_adk,langchain}_harness.py) all existpyproject.tomlextras ([semantic],[all],[langchain],[crewai], etc.)~/.claude/skills/cascadeflow/SKILL.md, ask it to build a minimal cost-optimized agent with a budget cap — verified end-to-end via subagent against pip-installed cascadeflow 1.x; demo reaches the auth boundary cleanly (real model call deferred — no API key in test env, ~$0.001 cost when run by reviewer)$CODEX_HOME/skills/cascadeflow/SKILL.md(default:~/.codex/skills/cascadeflow/SKILL.md) — file format is platform-neutral (agentskills.io spec) and identical to the Claude Code path; not separately exercised in a Codex session yetReal-dev validation (post-original-PR)
Three parallel ground-truth tests, each with disk side-effects, all reported in PR comments:
pip install cascadeflowin fresh venv, subagent reads only the globally-installed skill, writes the full demo (CascadeAgent+ToolConfig/ToolExecutor+ scopedcascadeflow.run+ try/except forBudgetExceededError/HarnessStopError+CallbackManager+session.save). Runs to the auth boundary. Zero API divergences — every API the skill names exists in the pip-installed package with the documented signature.__version__drift), branch off main, regression test,pytest, commit. Surfaced two real frictions in the skill's instructions (barepytestfails afterpip install -e .;git commit -amskips untracked tests). Both fixed in commitec6e68c.@cascadeflow/langchain, branch, 1-line patch,pnpm install --frozen-lockfile,pnpm --filter @cascadeflow/langchain test→ 75/75 passed, commit. Workflow shipped clean.The skill survives a ground-truth real-dev test on three real workflows (build-the-demo, fix-the-core, fix-the-integration). Ready for distribution to hackathon participants.