diff --git a/docs/diagram1_three_contexts.md b/docs/diagram1_three_contexts.md new file mode 100644 index 0000000..5955c55 --- /dev/null +++ b/docs/diagram1_three_contexts.md @@ -0,0 +1,51 @@ +# diagram1_three_contexts + +```mermaid +flowchart TB + classDef llm fill:#e3f2fd,stroke:#1565c0,stroke-width:2px,color:#000 + classDef human fill:#fff3e0,stroke:#e65100,color:#000 + classDef tool fill:#f1f8e9,stroke:#33691e,color:#000 + classDef delta fill:#fce4ec,stroke:#ad1457,stroke-dasharray:4 3,color:#000 + + Thesis["Today: LLMs inform humans   ━━ 6-month step ━━▶   Tomorrow: LLMs act within workflows"]:::delta + + subgraph P1["① Real-time bot — Mattermost"] + direction TB + U1["ePIC users"]:::human + B1["AI bot
Haiku · cross-session memory
context harness"]:::llm + O1["Q&A, diagnostics, on-the-fly analysis"] + U1 --> B1 --> O1 --> U1 + end + + subgraph P2["② Research orchestrator — corun-ai"] + direction TB + U2["Expert evaluators
(production, user learning)"]:::human + S2["Scheduler
model × sysprompt × MCP set
config compare & annotate"] + B2["Long-latency worker
Opus / Sonnet / Gemini / Gemma
minutes–tens of minutes"]:::llm + O2["Deep research entry
(e.g. Perlmutter performance)"] + U2 --> S2 --> B2 --> O2 --> U2 + end + + subgraph P3["③ Active workflow orchestrator — swf-testbed"] + direction TB + U3["Testbed users"]:::human + B3["LLM orchestrator
launch · run · monitor
assess · summarize"]:::llm + W3["Hybrid workflow
LLM steps ⇄ deterministic agents
DAQ sim → PanDA workers"] + O3["Completed run + summary"] + U3 --> B3 --> W3 --> B3 + W3 --> O3 --> U3 + end + + subgraph MCP["Shared MCP tool ecosystem"] + direction LR + IH["In-house
AskPanDA · PanDA Monitor · Streaming Workflow"]:::tool + AD["Adopted
Rucio · XRootD · uproot · LXR · GitHub · Zenodo"]:::tool + end + + Thesis -.-> P1 + Thesis -.-> P2 + Thesis -.-> P3 + P1 --> MCP + P2 --> MCP + P3 --> MCP +``` diff --git a/docs/diagram1_three_contexts.mmd b/docs/diagram1_three_contexts.mmd new file mode 100644 index 0000000..d648a60 --- /dev/null +++ b/docs/diagram1_three_contexts.mmd @@ -0,0 +1,47 @@ +flowchart TB + classDef llm fill:#e3f2fd,stroke:#1565c0,stroke-width:2px,color:#000 + classDef human fill:#fff3e0,stroke:#e65100,color:#000 + classDef tool fill:#f1f8e9,stroke:#33691e,color:#000 + classDef delta fill:#fce4ec,stroke:#ad1457,stroke-dasharray:4 3,color:#000 + + Thesis["Today: LLMs inform humans   ━━ 6-month step ━━▶   Tomorrow: LLMs act within workflows"]:::delta + + subgraph P1["① Real-time bot — Mattermost"] + direction TB + U1["ePIC users"]:::human + B1["AI bot
Haiku · cross-session memory
context harness"]:::llm + O1["Q&A, diagnostics, on-the-fly analysis"] + U1 --> B1 --> O1 --> U1 + end + + subgraph P2["② Research orchestrator — corun-ai"] + direction TB + U2["Expert evaluators
(production, user learning)"]:::human + S2["Scheduler
model × sysprompt × MCP set
config compare & annotate"] + B2["Long-latency worker
Opus / Sonnet / Gemini / Gemma
minutes–tens of minutes"]:::llm + O2["Deep research entry
(e.g. Perlmutter performance)"] + U2 --> S2 --> B2 --> O2 --> U2 + end + + subgraph P3["③ Active workflow orchestrator — swf-testbed"] + direction TB + U3["Testbed users"]:::human + B3["LLM orchestrator
launch · run · monitor
assess · summarize"]:::llm + W3["Hybrid workflow
LLM steps ⇄ deterministic agents
DAQ sim → PanDA workers"] + O3["Completed run + summary"] + U3 --> B3 --> W3 --> B3 + W3 --> O3 --> U3 + end + + subgraph MCP["Shared MCP tool ecosystem"] + direction LR + IH["In-house
AskPanDA · PanDA Monitor · Streaming Workflow"]:::tool + AD["Adopted
Rucio · XRootD · uproot · LXR · GitHub · Zenodo"]:::tool + end + + Thesis -.-> P1 + Thesis -.-> P2 + Thesis -.-> P3 + P1 --> MCP + P2 --> MCP + P3 --> MCP diff --git a/docs/diagram1_three_contexts.svg b/docs/diagram1_three_contexts.svg new file mode 100644 index 0000000..e97bff6 --- /dev/null +++ b/docs/diagram1_three_contexts.svg @@ -0,0 +1,135 @@ + + + + + + + + + + + + + + + + + + + + + + Today: LLMs inform humans + ━━ 6-month step ━━▶ + Tomorrow: LLMs act within workflows + + three systems in operation today, ordered left → right by increasing LLM autonomy + + + + ① Real-time bot — Mattermost + human-facing · quick turnaround + + + ePIC users + + + + AI bot (Haiku) + cross-session memory + context harness for + O(100) MCP tools + deployed to ePIC community + + + + Q&A · diagnostics · analysis + on-the-fly insights to users + + + + + draws on MCP ↓ + + + + + ② Research orchestrator — corun-ai + long-latency · configurable · R&D testbed + + + Expert evaluators + production · user learning + + + + Scheduler + model × sysprompt × MCP set + compare & annotate configs + + + + Long-latency worker + Opus · Sonnet · Gemini · Gemma + minutes – tens of minutes per study + + + + Deep research entry + e.g. Perlmutter production performance + + + + draws on MCP ↓ + + + + + ③ Active workflow orchestrator + swf-testbed · LLM-driven + + + Testbed users + + + + LLM orchestrator + launch · run · monitor + assess · summarize + + + + Hybrid workflow + LLM steps ⇄ deterministic agents + DAQ sim → PanDA workers + ePIC streaming workflow testbed + + + + Completed run + summary + narrative report · anomaly notes + + + + draws on MCP ↓ + + + + + Shared MCP tool ecosystem + all three systems draw from the same instrumentation layer + + + In-house + AskPanDA · PanDA Monitor · Streaming Workflow + purpose-built against our production WFMS + (PanDA DB, monitor, testbed control) + + + Adopted + Rucio · XRootD · uproot · LXR · GitHub · Zenodo + data catalogs · remote I/O · code XREF · repos + (data products, code base, documentation) + + legend: blue = LLM · orange = human · purple = hybrid (LLM + deterministic) · green = MCP tool surface + diff --git a/docs/diagram2_mcp_ecosystem.md b/docs/diagram2_mcp_ecosystem.md new file mode 100644 index 0000000..b4c854c --- /dev/null +++ b/docs/diagram2_mcp_ecosystem.md @@ -0,0 +1,46 @@ +# diagram2_mcp_ecosystem + +MCP ecosystem: one LLM reaches into the experiment's operational stack +through a two-tier tool set. Answers the reviewer who thinks "everyone +has MCP tools now" by showing depth into production systems. + +```mermaid +flowchart TB + classDef llm fill:#e3f2fd,stroke:#1565c0,stroke-width:2px,color:#000 + classDef ih fill:#f1f8e9,stroke:#33691e,color:#000 + classDef ad fill:#fff8e1,stroke:#f57f17,color:#000 + classDef sys fill:#fafafa,stroke:#888,color:#444 + + LLM["LLM
Opus · Sonnet · Haiku · Gemini · Gemma
sysprompt · effort level · context harness"]:::llm + + subgraph IH["In-house — purpose-built on our production WFMS"] + direction LR + T1["AskPanDA
job diagnostics"]:::ih + T2["PanDA Monitor MCP
operational state"]:::ih + T3["Streaming Workflow MCP
active testbed control"]:::ih + end + + subgraph AD["3rd-party MCP — 6+ community/standard tools"] + direction LR + T4["Rucio MCP"]:::ad + T5["XRootD MCP"]:::ad + T6["uproot MCP"]:::ad + T7["LXR XREF MCP"]:::ad + T8["GitHub MCP"]:::ad + T9["Zenodo MCP"]:::ad + end + + subgraph SYS["Reaches into"] + direction LR + S1["PanDA DB
monitor · testbed"]:::sys + S2["Rucio
data catalogs"]:::sys + S3["XRootD
remote I/O"]:::sys + S4["ePIC codebase
(55+ repos)"]:::sys + S5["Zenodo
official repo"]:::sys + end + + LLM --> IH + LLM --> AD + IH --> SYS + AD --> SYS +``` diff --git a/docs/diagram3_hybrid_workflow.md b/docs/diagram3_hybrid_workflow.md new file mode 100644 index 0000000..652c1b4 --- /dev/null +++ b/docs/diagram3_hybrid_workflow.md @@ -0,0 +1,45 @@ +# diagram3_hybrid_workflow + +```mermaid +flowchart TB + classDef llm fill:#e3f2fd,stroke:#1565c0,stroke-width:2px,color:#000 + classDef det fill:#eeeeee,stroke:#555,color:#000 + classDef hitl fill:#fff3e0,stroke:#e65100,stroke-dasharray:5 3,color:#000 + classDef mcp fill:#f1f8e9,stroke:#33691e,font-size:11px,color:#000 + + U["User prompt
'run a fast-processing test
and summarize results'"]:::hitl + + L1["① Prepare
select config, check prior runs"]:::llm + L1t["swf_list_workflow_executions
pcs_list_tags · swf_get_system_state"]:::mcp + + L2["② Start testbed"]:::llm + L2t["swf_start_user_testbed"]:::mcp + + L3["③ Start workflow"]:::llm + L3t["swf_start_workflow
(stf_count, config, …)"]:::mcp + + D1["④ DAQ simulator
emits STF files"]:::det + D2["⑤ Data agent
STF registration"]:::det + D3["⑥ FastMon agent
samples Time Frames"]:::det + D4["⑦ Fast processing agent
TF slices → PanDA"]:::det + D5["⑧ PanDA workers
EICrecon reconstruction"]:::det + + L4["⑨ Monitor in-flight
errors, throughput, stragglers"]:::llm + L4t["swf_list_logs(level='ERROR')
swf_list_workflow_executions
panda_get_activity"]:::mcp + + G1{"human-in-loop
gate
(scope of 6-mo work)"}:::hitl + + L5["⑩ Assess & summarize
narrative run report,
anomaly notes,
comparison to prior runs"]:::llm + L5t["swf_get_workflow_execution
panda_study_job · lxr_ident"]:::mcp + + O["Run entry + summary
annotated, searchable"]:::hitl + + U --> L1 --> L2 --> L3 --> D1 --> D2 --> D3 --> D4 --> D5 --> L4 + L4 --> G1 --> L5 --> O + + L1 -.- L1t + L2 -.- L2t + L3 -.- L3t + L4 -.- L4t + L5 -.- L5t +``` diff --git a/docs/diagram3_hybrid_workflow.mmd b/docs/diagram3_hybrid_workflow.mmd new file mode 100644 index 0000000..cbfd708 --- /dev/null +++ b/docs/diagram3_hybrid_workflow.mmd @@ -0,0 +1,41 @@ +flowchart TB + classDef llm fill:#e3f2fd,stroke:#1565c0,stroke-width:2px,color:#000 + classDef det fill:#eeeeee,stroke:#555,color:#000 + classDef hitl fill:#fff3e0,stroke:#e65100,stroke-dasharray:5 3,color:#000 + classDef mcp fill:#f1f8e9,stroke:#33691e,font-size:11px,color:#000 + + U["User prompt
'run a fast-processing test
and summarize results'"]:::hitl + + L1["① Prepare
select config, check prior runs"]:::llm + L1t["swf_list_workflow_executions
pcs_list_tags · swf_get_system_state"]:::mcp + + L2["② Start testbed"]:::llm + L2t["swf_start_user_testbed"]:::mcp + + L3["③ Start workflow"]:::llm + L3t["swf_start_workflow
(stf_count, config, …)"]:::mcp + + D1["④ DAQ simulator
emits STF files"]:::det + D2["⑤ Data agent
STF registration"]:::det + D3["⑥ FastMon agent
samples Time Frames"]:::det + D4["⑦ Fast processing agent
TF slices → PanDA"]:::det + D5["⑧ PanDA workers
EICrecon reconstruction"]:::det + + L4["⑨ Monitor in-flight
errors, throughput, stragglers"]:::llm + L4t["swf_list_logs(level='ERROR')
swf_list_workflow_executions
panda_get_activity"]:::mcp + + G1{"human-in-loop
gate
(scope of 6-mo work)"}:::hitl + + L5["⑩ Assess & summarize
narrative run report,
anomaly notes,
comparison to prior runs"]:::llm + L5t["swf_get_workflow_execution
panda_study_job · lxr_ident"]:::mcp + + O["Run entry + summary
annotated, searchable"]:::hitl + + U --> L1 --> L2 --> L3 --> D1 --> D2 --> D3 --> D4 --> D5 --> L4 + L4 --> G1 --> L5 --> O + + L1 -.- L1t + L2 -.- L2t + L3 -.- L3t + L4 -.- L4t + L5 -.- L5t diff --git a/docs/diagram4_delta_before_after.md b/docs/diagram4_delta_before_after.md new file mode 100644 index 0000000..d83e4d7 --- /dev/null +++ b/docs/diagram4_delta_before_after.md @@ -0,0 +1,38 @@ +# diagram4_delta_before_after + +The 6-month delta visualized. Same boxes, one arrow moves, one audit +loop added. Makes the project feel like a bounded increment on an +operational system, not a research leap. + +```mermaid +flowchart LR + classDef llm fill:#e3f2fd,stroke:#1565c0,stroke-width:2px,color:#000 + classDef human fill:#fff3e0,stroke:#e65100,stroke-width:2px,color:#000 + classDef wfms fill:#f3e5f5,stroke:#6a1b9a,stroke-width:2px,color:#000 + classDef audit fill:#fffde7,stroke:#f9a825,stroke-dasharray:4 3,color:#000 + classDef delta fill:#fce4ec,stroke:#ad1457,color:#000 + + subgraph TODAY["Today — LLM informs, human decides"] + direction TB + T_LLM["LLM"]:::llm + T_MCP["MCP tools
reads · analyzes"]:::llm + T_HUM["Human decides"]:::human + T_WFMS["WFMS acts"]:::wfms + T_LLM --> T_MCP --> T_HUM --> T_WFMS + end + + subgraph NEXT["Proposed (6 months) — LLM decides on pre-defined classes"] + direction TB + N_LLM["LLM"]:::llm + N_MCP["MCP tools
reads · analyzes · decides"]:::llm + N_WFMS["WFMS acts"]:::wfms + N_AUD["HITL audit trail
async human review"]:::audit + N_LLM --> N_MCP --> N_WFMS + N_WFMS -.-> N_AUD + N_AUD -.-> N_LLM + end + + DELTA["Delta:
• 'human decides' → 'LLM decides'
• HITL audit loop added
• Scope gated by decision-class allowlist"]:::delta + + TODAY -.-> DELTA -.-> NEXT +``` diff --git a/docs/diagram5_panda_provenance.md b/docs/diagram5_panda_provenance.md new file mode 100644 index 0000000..8cebe39 --- /dev/null +++ b/docs/diagram5_panda_provenance.md @@ -0,0 +1,40 @@ +# diagram5_panda_provenance + +Scale-provenance stack. The "why 6 months is plausible" anchor: we're +layering on a production WFMS with a decade of operational history, +not starting from zero. + +```mermaid +flowchart BT + classDef app fill:#e3f2fd,stroke:#1565c0,stroke-width:2px,color:#000 + classDef ai fill:#f1f8e9,stroke:#33691e,stroke-width:2px,color:#000 + classDef panda fill:#fff3e0,stroke:#e65100,stroke-width:2px,color:#000 + + subgraph L1["Foundation — PanDA production WFMS (operational since 2005)"] + direction LR + P1["ATLAS @ LHC
O(million) jobs/day
200+ institutions"]:::panda + P2["PanDA monitor
deep drill-down
refined 15+ years"]:::panda + P3["ePIC production
(monthly campaigns, OSG, HPC)"]:::panda + P4["ePIC streaming
workflow testbed
(this team, 2025+)"]:::panda + end + + subgraph L2["AI instrumentation today — this team (2024–)"] + direction LR + I1["AskPanDA MCP"]:::ai + I2["PanDA Monitor MCP"]:::ai + I3["VectorDB RAG"]:::ai + I4["Streaming Workflow MCP"]:::ai + I5["3rd-party MCP (6+)"]:::ai + end + + subgraph L3["New application layer — LLM-driven orchestration (proposed, 6 months)"] + direction LR + A1["LLM workflow
orchestrator"]:::app + A2["Hybrid workflows
LLM + deterministic"]:::app + A3["Harnessed autonomous
LLM action"]:::app + A4["LLM research assistant
evolution of Mattermost
bot + codoc-ai"]:::app + end + + L1 --> L2 + L2 --> L3 +``` diff --git a/docs/diagram5_panda_provenance.svg b/docs/diagram5_panda_provenance.svg new file mode 100644 index 0000000..f0fe182 --- /dev/null +++ b/docs/diagram5_panda_provenance.svg @@ -0,0 +1,141 @@ + + + + + + + + + + + + + + PanDA scale provenance + + the 6-month claim rests on a decade-old production foundation — not greenfield work + + + + + + + ③ New application layer + LLM-driven orchestration — proposed, 6 months + + + + LLM workflow + orchestrator + launch · run · monitor + assess · summarize + + + Hybrid workflows + LLM steps ⇄ + deterministic agents + extending present testbed + + + Harnessed autonomous + LLM action + bounded, auditable, + reversible decisions + + + LLM research + assistant + evolution of Mattermost + bot + codoc-ai + + + + + + built on + built on + + + + + + ② AI instrumentation today + this team · 2024– + + + + AskPanDA MCP + job diagnostics + + + PanDA Monitor MCP + operational state + + + VectorDB RAG + embedded documentation + + + Streaming Workflow MCP + active testbed control + + + 3rd-party MCP (6+) + Rucio · XRootD · LXR · … + + + + + + instruments + instruments + + + + + + ① Foundation — PanDA production WFMS + operational since 2005 + + + + ATLAS @ LHC + O(million) jobs/day + 200+ institutions + production scale + provenance + + + PanDA monitor + deep drill-down + refined 15+ years + operational + diagnostics base + + + ePIC production + monthly campaigns + OSG · HPC + live production + workload + + + ePIC streaming + workflow testbed + (this team, 2025+) + hybrid workflow + platform + + + + Foundation + (decades) → + Instrumentation + (this team, recent) → + New application + (6-month delta) + + + each layer is a bounded, incremental extension of the one below it + + diff --git a/docs/diagram5_panda_provenance_epic.svg b/docs/diagram5_panda_provenance_epic.svg new file mode 100644 index 0000000..b34c50f --- /dev/null +++ b/docs/diagram5_panda_provenance_epic.svg @@ -0,0 +1,139 @@ + + + + + + + + + + + + + + ePIC AI infrastructure stack — current status + + + + + + ③ AI services + + + + Mattermost bot + ePIC community Q&A · prod, testbed ops + cross-session memory · lie detector + tiered MCP exposure + MCPs (all): + AskPanDA · PanDA Monitor · PCS + Streaming Workflow · VectorDB RAG + LXR · GitHub · Rucio · XRootD + uproot · Zenodo + + + + Production ops analysis + deep-dive PCS / task / job / site + diagnostics + + https://epic-devcloud.org/prod/alarms/ + + MCPs: + AskPanDA · PanDA Monitor + Streaming Workflow · Rucio + + + + Software analysis + deep-dive documentation study + + https://epic-devcloud.org/doc + + MCPs: + LXR · GitHub · VectorDB RAG + + + + PR review + New: review of PRs + grounded in LXR codebase knowledge + + https://epic-devcloud.org/doc/prs/ + + MCPs: + LXR · GitHub + + + + + + built on + built on + + + + + + ② AI instrumentation — MCP services + AskPanDA (job diagnostics) · PanDA Monitor (operational state) · PCS (physics configuration) · Streaming Workflow (testbed control) · VectorDB RAG (embedded docs) + LXR (code cross-reference) · GitHub (repo inquiries) · Rucio (data management) · XRootD (remote I/O) · uproot (data reading) · Zenodo (documentation) + + + + + + instruments + instruments + + + + + + ① ePIC Services + + + + PanDA monitor + deep drill-down + task · job · site + errors and summaries + + + + PCS + Physics Configuration + Service + from physics need to + production task + + + + Automated production + OSG · HPC + under development + + + + ePIC streaming + workflow testbed + information and controls + + + + Rucio + data management and + cataloging services + at JLab and BNL + + + + LXR + software cross-reference + and browse system + + eic-code-browser.sdcc.bnl.gov + + 62 repositories indexed + + diff --git a/docs/diagram6_corunai_loop.md b/docs/diagram6_corunai_loop.md new file mode 100644 index 0000000..93a0158 --- /dev/null +++ b/docs/diagram6_corunai_loop.md @@ -0,0 +1,28 @@ +# diagram6_corunai_loop + +corun-ai research loop — shows corun-ai is not a chatbot but an +orchestrated research system with config-compare as a first-class +feature. + +```mermaid +sequenceDiagram + autonumber + participant U as Expert evaluator + participant S as Scheduler + participant W as Worker LLM + participant M as MCP tools + participant E as Research entry + + U->>S: submit research prompt
+ config (model · sysprompt · MCP set) + S->>W: spawn worker with config + loop deep analysis — minutes to tens of minutes + W->>M: tool call (PanDA / LXR / Rucio / ...) + M-->>W: results + W->>W: reason · refine · iterate + end + W-->>S: completed analysis + S->>E: write research entry + E-->>U: notify + surface result + U->>E: annotate · thread comments + Note over U,E: config variants compared
side-by-side in threads —
an R&D testbed, not a product +``` diff --git a/docs/diagram6_corunai_loop.svg b/docs/diagram6_corunai_loop.svg new file mode 100644 index 0000000..0238765 --- /dev/null +++ b/docs/diagram6_corunai_loop.svg @@ -0,0 +1,127 @@ + + + + + + + + + + + + + + + + + corun-ai research loop + an orchestrated research system with config-compare threading + + + + Expert + evaluator + + + Scheduler + + + Worker LLM + frontier · open source + + + MCP tools + + + Research + entry + + + + + + + + + + + 1 + submit research prompt + + config (model · sysprompt · MCP set) + + + + + 2 + spawn worker with config + + + + + + loop + deep analysis — minutes to tens of minutes + + + + 3 + tool call + PanDA / LXR / Rucio / … + + + + + 4 + results + + + + + 5 + reason · refine · iterate + + + + + 6 + completed analysis + + + + + 7 + write research entry + + + + + 8 + notify + surface result + + + + + 9 + annotate · thread comments + + + + + + config variants (model · sysprompt · MCP set) compared side-by-side in annotation threads — + + + an R&D testbed measuring LLM configuration performance while addressing + + + real, complex production and software documentation questions + + + + + solid arrow = call  ·  dashed arrow = return  ·  boxed section = loop + + + per-study latency spans minutes to tens of minutes; long-latency by design + + diff --git a/docs/diagram7_corunai_codocai.md b/docs/diagram7_corunai_codocai.md new file mode 100644 index 0000000..45fee7e --- /dev/null +++ b/docs/diagram7_corunai_codocai.md @@ -0,0 +1,50 @@ +# diagram7_corunai_codocai + +corun-ai + codoc-ai: the research orchestrator and its immediate +application, spanning frontier, commercial, and open-source models. The +open-source tier requires local hardware, which is why the team has +implemented a remote-inference bridge — open-source models run on the +user's desktop (Mac Studio / ollama) and plug into corun-ai as a +first-class dispatch target. + +```mermaid +flowchart TB + classDef user fill:#fff3e0,stroke:#e65100,color:#000 + classDef codoc fill:#e3f2fd,stroke:#1565c0,stroke-width:2px,color:#000 + classDef orch fill:#e8eaf6,stroke:#3949ab,stroke-width:2px,color:#000 + classDef frontier fill:#fff8e1,stroke:#f57f17,color:#000 + classDef oss fill:#f1f8e9,stroke:#33691e,stroke-width:2px,color:#000 + classDef remote fill:#fce4ec,stroke:#ad1457,stroke-width:2px,stroke-dasharray:6 3,color:#000 + classDef tool fill:#f1f8e9,stroke:#33691e,color:#000 + classDef out fill:#ffffff,stroke:#555,color:#000 + + U["Expert evaluator
documentation questions · production examination"]:::user + + CODOC["codoc-ai — immediate application
documentation drafting · production analysis"]:::codoc + + CORUN["corun-ai scheduler
config = model × sysprompt × MCP set
configure · dispatch · compare"]:::orch + + subgraph MODELS["Model ensemble — three providers, common scheduler"] + direction LR + M1["Anthropic
Claude Opus · Sonnet · Haiku
Anthropic API · claude -p CLI"]:::frontier + M2["Google
Gemini 2.5 Pro · Flash
Google API"]:::frontier + M3["Open source
Gemma 4 via ollama
needs local GPU hardware"]:::oss + end + + REMOTE["Remote inference worker
open-source models hosted on
user desktop (Mac Studio · ollama)
bridged to corun-ai as a scheduler target
enables commercial vs. open-source comparison"]:::remote + + MCP["MCP tool ecosystem
in-house: AskPanDA · PanDA Monitor · Streaming Workflow · VectorDB RAG
3rd-party (6+): Rucio · XRootD · LXR · GitHub · Zenodo · …"]:::tool + + OUT["codoc-ai outputs
documentation drafts · production-examination reports ·
cross-model comparison threads (user annotations)"]:::out + + U --> CODOC + CODOC --> CORUN + CORUN --> M1 + CORUN --> M2 + CORUN --> M3 + M3 -. hosted by .-> REMOTE + M1 --> MCP + M2 --> MCP + M3 --> MCP + MCP --> OUT +``` diff --git a/docs/diagram7_corunai_codocai.svg b/docs/diagram7_corunai_codocai.svg new file mode 100644 index 0000000..0e59c27 --- /dev/null +++ b/docs/diagram7_corunai_codocai.svg @@ -0,0 +1,112 @@ + + + + + + + + + + + + + + + + + + + + corun-ai + codoc-ai — research orchestrator and its immediate application + + three model providers under one scheduler; open-source bridged in via remote inference on user hardware + + + + + Expert evaluator + documentation questions · production examination + + + + + codoc-ai — immediate application + documentation drafting + production-examination reports + + + + + corun-ai scheduler + config = model × sysprompt × MCP set + configure · dispatch · compare across configurations + + + + + + + + model ensemble — three providers, common scheduler + + + + Anthropic + Claude Opus · Sonnet · Haiku + Anthropic API + claude -p CLI (subscription) + + + + Google + Gemini 2.5 Pro · Flash + Google API + + + + Open source + Gemma 4 via ollama + needs local GPU hardware + (cannot run on vendor APIs) + + + + Remote inference + worker + Mac Studio · ollama + open-source models + bridged to corun-ai as + first-class dispatch target + + + + hosted by + + + + + + + + + MCP tool ecosystem — shared across all providers + + + In-house + AskPanDA · PanDA Monitor · Streaming Workflow + VectorDB RAG for embedded documentation + + + 3rd-party MCP (6+) + Rucio · XRootD · uproot · LXR · GitHub · Zenodo + data catalogs · remote I/O · code XREF · repos + + + + + + + codoc-ai outputs + documentation drafts · production-examination reports + cross-model comparison threads (user annotations) + diff --git a/docs/proposal_diagrams.md b/docs/proposal_diagrams.md new file mode 100644 index 0000000..cbcfa83 --- /dev/null +++ b/docs/proposal_diagrams.md @@ -0,0 +1,286 @@ +# AI-enabled WFMS proposal — draft diagrams + +Mermaid prototypes to support the "Why us?" section. Finished versions will +graduate to hand-authored SVG in `swf-testbed/docs/images/` style. + +Open this file's preview (`Ctrl+Shift+V`) to render. + +--- + +## Diagram 1 — Three Contexts + +The thesis picture: three LLM-integrated systems running today, ordered +left-to-right by increasing LLM autonomy. Shared MCP ecosystem feeds all +three. Top banner carries the 6-month claim. + +```mermaid +flowchart TB + classDef llm fill:#e3f2fd,stroke:#1565c0,stroke-width:2px,color:#000 + classDef human fill:#fff3e0,stroke:#e65100,color:#000 + classDef tool fill:#f1f8e9,stroke:#33691e,color:#000 + classDef delta fill:#fce4ec,stroke:#ad1457,stroke-dasharray:4 3,color:#000 + + Thesis["Today: LLMs inform humans   ━━ 6-month step ━━▶   Tomorrow: LLMs act within workflows"]:::delta + + subgraph P1["① Real-time bot — Mattermost"] + direction TB + U1["ePIC users"]:::human + B1["AI bot
Haiku · cross-session memory
context harness"]:::llm + O1["Q&A, diagnostics, on-the-fly analysis"] + U1 --> B1 --> O1 --> U1 + end + + subgraph P2["② Research orchestrator — corun-ai"] + direction TB + U2["Expert evaluators
(production, user learning)"]:::human + S2["Scheduler
model × sysprompt × MCP set
config compare & annotate"] + B2["Long-latency worker
Opus / Sonnet / Gemini / Gemma
minutes–tens of minutes"]:::llm + O2["Deep research entry
(e.g. Perlmutter performance)"] + U2 --> S2 --> B2 --> O2 --> U2 + end + + subgraph P3["③ Active workflow orchestrator — swf-testbed"] + direction TB + U3["Testbed users"]:::human + B3["LLM orchestrator
launch · run · monitor
assess · summarize"]:::llm + W3["Hybrid workflow
LLM steps ⇄ deterministic agents
DAQ sim → PanDA workers"] + O3["Completed run + summary"] + U3 --> B3 --> W3 --> B3 + W3 --> O3 --> U3 + end + + subgraph MCP["Shared MCP tool ecosystem"] + direction LR + IH["In-house
AskPanDA · PanDA Monitor · Streaming Workflow"]:::tool + AD["Adopted
Rucio · XRootD · uproot · LXR · GitHub · Zenodo"]:::tool + end + + Thesis -.-> P1 + Thesis -.-> P2 + Thesis -.-> P3 + P1 --> MCP + P2 --> MCP + P3 --> MCP +``` + +Legend: blue = LLM, orange = human, green = MCP/tool surface, pink-dashed = thesis / 6-month delta. + +--- + +## Diagram 3 — Hybrid Workflow Anatomy + +One real swf-testbed streaming run as a pipeline of alternating LLM and +deterministic steps, with the MCP tools each LLM step actually calls. +Human-in-loop gate between ⑨ and ⑩ is where the 6-month scope lands. + +```mermaid +flowchart TB + classDef llm fill:#e3f2fd,stroke:#1565c0,stroke-width:2px,color:#000 + classDef det fill:#eeeeee,stroke:#555,color:#000 + classDef hitl fill:#fff3e0,stroke:#e65100,stroke-dasharray:5 3,color:#000 + classDef mcp fill:#f1f8e9,stroke:#33691e,font-size:11px,color:#000 + + U["User prompt
'run a fast-processing test
and summarize results'"]:::hitl + + L1["① Prepare
select config, check prior runs"]:::llm + L1t["swf_list_workflow_executions
pcs_list_tags · swf_get_system_state"]:::mcp + + L2["② Start testbed"]:::llm + L2t["swf_start_user_testbed"]:::mcp + + L3["③ Start workflow"]:::llm + L3t["swf_start_workflow
(stf_count, config, …)"]:::mcp + + D1["④ DAQ simulator
emits STF files"]:::det + D2["⑤ Data agent
STF registration"]:::det + D3["⑥ FastMon agent
samples Time Frames"]:::det + D4["⑦ Fast processing agent
TF slices → PanDA"]:::det + D5["⑧ PanDA workers
EICrecon reconstruction"]:::det + + L4["⑨ Monitor in-flight
errors, throughput, stragglers"]:::llm + L4t["swf_list_logs(level='ERROR')
swf_list_workflow_executions
panda_get_activity"]:::mcp + + G1{"human-in-loop
gate
(scope of 6-mo work)"}:::hitl + + L5["⑩ Assess & summarize
narrative run report,
anomaly notes,
comparison to prior runs"]:::llm + L5t["swf_get_workflow_execution
panda_study_job · lxr_ident"]:::mcp + + O["Run entry + summary
annotated, searchable"]:::hitl + + U --> L1 --> L2 --> L3 --> D1 --> D2 --> D3 --> D4 --> D5 --> L4 + L4 --> G1 --> L5 --> O + + L1 -.- L1t + L2 -.- L2t + L3 -.- L3t + L4 -.- L4t + L5 -.- L5t +``` + +Legend: blue = LLM step, grey = deterministic agent, dashed orange = human-in-loop / user edge, green captions = MCP tool calls. + +--- + +## Diagram 2 — MCP Tool Ecosystem + +One LLM reaches into the experiment's operational stack through a two-tier +tool set. Counters "everyone has MCP now" by showing depth into production +systems. + +```mermaid +flowchart TB + classDef llm fill:#e3f2fd,stroke:#1565c0,stroke-width:2px,color:#000 + classDef ih fill:#f1f8e9,stroke:#33691e,color:#000 + classDef ad fill:#fff8e1,stroke:#f57f17,color:#000 + classDef sys fill:#fafafa,stroke:#888,color:#444 + + LLM["LLM
Opus · Sonnet · Haiku · Gemini · Gemma
sysprompt · effort level · context harness"]:::llm + + subgraph IH["In-house — purpose-built on our production WFMS"] + direction LR + T1["AskPanDA
job diagnostics"]:::ih + T2["PanDA Monitor MCP
operational state"]:::ih + T3["Streaming Workflow MCP
active testbed control"]:::ih + end + + subgraph AD["3rd-party MCP — 6+ community/standard tools"] + direction LR + T4["Rucio MCP"]:::ad + T5["XRootD MCP"]:::ad + T6["uproot MCP"]:::ad + T7["LXR XREF MCP"]:::ad + T8["GitHub MCP"]:::ad + T9["Zenodo MCP"]:::ad + end + + subgraph SYS["Reaches into"] + direction LR + S1["PanDA DB
monitor · testbed"]:::sys + S2["Rucio
data catalogs"]:::sys + S3["XRootD
remote I/O"]:::sys + S4["ePIC codebase
(55+ repos)"]:::sys + S5["Zenodo
official repo"]:::sys + end + + LLM --> IH + LLM --> AD + IH --> SYS + AD --> SYS +``` + +--- + +## Diagram 4 — 6-month Delta (before / after) + +Same boxes, one arrow moves, one audit loop added. Makes the project +feel like a bounded increment on an operational system, not a research +leap. + +```mermaid +flowchart LR + classDef llm fill:#e3f2fd,stroke:#1565c0,stroke-width:2px,color:#000 + classDef human fill:#fff3e0,stroke:#e65100,stroke-width:2px,color:#000 + classDef wfms fill:#f3e5f5,stroke:#6a1b9a,stroke-width:2px,color:#000 + classDef audit fill:#fffde7,stroke:#f9a825,stroke-dasharray:4 3,color:#000 + classDef delta fill:#fce4ec,stroke:#ad1457,color:#000 + + subgraph TODAY["Today — LLM informs, human decides"] + direction TB + T_LLM["LLM"]:::llm + T_MCP["MCP tools
reads · analyzes"]:::llm + T_HUM["Human decides"]:::human + T_WFMS["WFMS acts"]:::wfms + T_LLM --> T_MCP --> T_HUM --> T_WFMS + end + + subgraph NEXT["Proposed (6 months) — LLM decides on pre-defined classes"] + direction TB + N_LLM["LLM"]:::llm + N_MCP["MCP tools
reads · analyzes · decides"]:::llm + N_WFMS["WFMS acts"]:::wfms + N_AUD["HITL audit trail
async human review"]:::audit + N_LLM --> N_MCP --> N_WFMS + N_WFMS -.-> N_AUD + N_AUD -.-> N_LLM + end + + DELTA["Delta:
• 'human decides' → 'LLM decides'
• HITL audit loop added
• Scope gated by decision-class allowlist"]:::delta + + TODAY -.-> DELTA -.-> NEXT +``` + +--- + +## Diagram 5 — PanDA Scale Provenance + +The "why 6 months is plausible" anchor: we're layering on a production +WFMS with a decade of operational history, not starting from zero. + +```mermaid +flowchart BT + classDef app fill:#e3f2fd,stroke:#1565c0,stroke-width:2px,color:#000 + classDef ai fill:#f1f8e9,stroke:#33691e,stroke-width:2px,color:#000 + classDef panda fill:#fff3e0,stroke:#e65100,stroke-width:2px,color:#000 + + subgraph L1["Foundation — PanDA production WFMS (operational since 2005)"] + direction LR + P1["ATLAS @ LHC
O(million) jobs/day
200+ institutions"]:::panda + P2["PanDA monitor
deep drill-down
refined 15+ years"]:::panda + P3["ePIC production
(monthly campaigns, OSG, HPC)"]:::panda + P4["ePIC streaming
workflow testbed
(this team, 2025+)"]:::panda + end + + subgraph L2["AI instrumentation today — this team (2024–)"] + direction LR + I1["AskPanDA MCP"]:::ai + I2["PanDA Monitor MCP"]:::ai + I3["VectorDB RAG"]:::ai + I4["Streaming Workflow MCP"]:::ai + I5["3rd-party MCP (6+)"]:::ai + end + + subgraph L3["New application layer — LLM-driven orchestration (proposed, 6 months)"] + direction LR + A1["LLM workflow
orchestrator"]:::app + A2["Hybrid workflows
LLM + deterministic"]:::app + A3["Harnessed autonomous
LLM action"]:::app + A4["LLM research assistant
evolution of Mattermost
bot + codoc-ai"]:::app + end + + L1 --> L2 + L2 --> L3 +``` + +--- + +## Diagram 6 — corun-ai Research Loop + +Shows corun-ai as an orchestrated research system, not a chatbot. +Config-compare in annotation threads is the R&D-testbed feature. + +```mermaid +sequenceDiagram + autonumber + participant U as Expert evaluator + participant S as Scheduler + participant W as Worker LLM + participant M as MCP tools + participant E as Research entry + + U->>S: submit research prompt
+ config (model · sysprompt · MCP set) + S->>W: spawn worker with config + loop deep analysis — minutes to tens of minutes + W->>M: tool call (PanDA / LXR / Rucio / ...) + M-->>W: results + W->>W: reason · refine · iterate + end + W-->>S: completed analysis + S->>E: write research entry + E-->>U: notify + surface result + U->>E: annotate · thread comments + Note over U,E: config variants compared
side-by-side in threads —
an R&D testbed, not a product +``` + +--- + +Fill in concrete numbers (PanDA jobs/day, testbed run count, corun-ai prompt count, etc.) before these go into proposal figures.