Add t2v API and integrations and benchmark support - #493
Conversation
|
Auto-sync is disabled for draft pull requests in this repository. Workflows must be run manually. Contributors can view more details about this message here. |
4a058ac to
7c4b0d3
Compare
d8e49b8 to
fc7be28
Compare
77934ab to
84a0d5d
Compare
The first model behind the v2 protocol and the first user of the batch runner and Mp4OutputSink. Output is MP4 only, so there is no input side yet. The check it runs is shared, in flashdreams.testing_v2.t2v_conformance, since every t2v model wants the same coverage. CPU tests use a stand-in model; the real one carries ci_gpu and skips unless T2V_SELF_FORCING_REAL_MODEL_RUN is set, because pytest-manual-marker xfails manual tests at setup and they never run.
The v2 sink writes the measurements a step reports, and a benchmark is only one thing that reads them, so MetricsOutputSink says what it is where BenchmarkStatsOutputSink said who asked for it. The v1 sink keeps its name, and the artifact type both write is unchanged, so a report can still hold runs of either API.
The base branch removed run_batch, so the two places here that drove it now drive run_session: the command line against an Mp4ClientWindow, and the shared t2v check against a window of its own that wraps its frame inspector. A benchmark still wants the clip and the measurements from one run, and a window owns what it writes, so Mp4ClientWindow takes the stats path and adds a MetricsOutputSink beside the MP4 writer. That is the last caller CompositeOutputSink had, so it goes.
84a0d5d to
0ac8471
Compare
| @@ -0,0 +1,414 @@ | |||
| { | |||
There was a problem hiding this comment.
This file is just the benchmark config stuff for running the new integrations.
Greptile SummaryThis PR adds the v2 text-to-video application protocol, model integrations, MP4/metrics output, CLI discovery, and benchmark scenarios. It also extends lifecycle ownership so windows are closed when application initialization or session creation fails.
Confidence Score: 4/5The PR is not yet safe to merge because a timed-out WebRTC construction can leave an unreachable server thread and listening socket alive. WebRTCServer starts its thread before waiting for readiness, but a timeout raises through mode.create without stopping that thread or returning an object the CLI can close. Files Needing Attention: flashdreams/flashdreams/runtime_v2/cli.py and flashdreams/flashdreams/runtime_v2/serving/webrtc_server.py Important Files Changed
Sequence DiagramsequenceDiagram
participant User
participant CLI as flashdreams-run-v2
participant Mode as ClientWindowMode
participant Server as WebRTCServer
participant Runner as ApplicationRunner
participant App as T2VApplication
participant Session as T2VSession
User->>CLI: select application and mode
CLI->>Mode: create(parsed)
Mode->>Server: construct and start thread
alt startup succeeds
Server-->>Mode: live window
Mode-->>CLI: window
CLI->>Runner: run(session_desc, args)
Runner->>App: init and create_session
Runner->>Session: run_session
Runner-->>CLI: cleanup complete
else startup times out
Server--xMode: TimeoutError
Note over Server: Thread may continue without a cleanup owner
end
Reviews (2): Last reviewed commit: "Simplified readmes" | Re-trigger Greptile |
The first model behind the v2 protocol and the first user of the batch runner and Mp4OutputSink. Output is MP4 only, so there is no input side yet. The check it runs is shared, in flashdreams.testing_v2.t2v_conformance, since every t2v model wants the same coverage. CPU tests use a stand-in model; the real one carries ci_gpu and skips unless T2V_SELF_FORCING_REAL_MODEL_RUN is set, because pytest-manual-marker fails manual tests at setup and they never run.