feat: research grant video pipeline — timeline-based 15-30s announcement#1
feat: research grant video pipeline — timeline-based 15-30s announcement#1devin-ai-integration[bot] wants to merge 9 commits into
Conversation
… additional archival b-roll - build_ito.py: v2 narrative assembler with ordered clip placement, crossfade transitions, section-specific cut lengths, and tighter voiceover sync - assemble_final.py: v2 final mix with sidechain ducking fallback, refined music bed (v2), proper fade envelopes, and faststart for streaming - edl.json: updated pool with 24 selects including archival NYSE footage, Ken Burns portrait clips, synthetic b-roll (sacred geo, stochastic paths, basket convergence, market web, ticker overlay, fragment venues), and endcard - gen_music_v2.py: 6-layer cinematic ambient bed (sub-bass pad, harmonic pad, high shimmer, rhythmic pulse, pink noise texture, tension rise/resolve) mixed with loudnorm normalization - gen_synthetic_broll.py: generates all synthetic b-roll clips via pure FFmpeg (sacred geometry, basket convergence, market web, ticker overlay, stochastic paths, fragment venues, endcard, title cards, names montage, archival trims) Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
🤖 Devin AI EngineerI'll be helping with this pull request! Here's what you should know: ✅ I will automatically:
Note: I can only respond to comments from users who have write access to this repository. ⚙️ Control Options:
|
![greptile-apps[bot]](https://avatars.githubusercontent.com/in/867647?s=60&v=4){kind=small}
![devin-ai-integration[bot]](https://avatars.githubusercontent.com/in/811515?s=60&v=4){kind=small}
| f"[1:a]volume=1.0,apad=whole_dur={vid_dur}[vo];" | ||
| f"[2:a]volume=-18dB," | ||
| f"afade=t=in:ss=0:d=4," | ||
| f"afade=t=out:st={fade_out_start:.1f}:d=8," | ||
| f"apad=whole_dur={vid_dur}[mu_raw];" | ||
| f"[mu_raw][vo]sidechaincompress=threshold=0.02:ratio=4:attack=50:release=300:level_sc=1[mu];" | ||
| f"[vo][mu]amix=inputs=2:duration=first:dropout_transition=2:weights=1 0.6[a]" |
There was a problem hiding this comment.
🔴 FFmpeg filter pad [vo] consumed twice causes sidechain ducking command to always fail
The [vo] output pad from line 46 is used as an input to sidechaincompress on line 51, which consumes it. Then line 52 tries to use [vo] again as input to amix. In FFmpeg's filtergraph, each output pad can only be linked to one downstream input pad — reusing [vo] will cause FFmpeg to error (e.g. "pad already linked"), so r.returncode != 0 is always true and the code always falls through to the simple-mix fallback at assemble_final.py:66-87. The headline v2 feature (sidechain-style ducking) silently never works.
Fix: use asplit to duplicate the voiceover stream
The voiceover stream needs to be split so one copy feeds the sidechain key and another feeds the final mix:
[1:a]volume=1.0,apad=whole_dur=...,asplit=2[vo][vo_sc];
[mu_raw][vo_sc]sidechaincompress=...[mu];
[vo][mu]amix=...[a]
| f"[1:a]volume=1.0,apad=whole_dur={vid_dur}[vo];" | |
| f"[2:a]volume=-18dB," | |
| f"afade=t=in:ss=0:d=4," | |
| f"afade=t=out:st={fade_out_start:.1f}:d=8," | |
| f"apad=whole_dur={vid_dur}[mu_raw];" | |
| f"[mu_raw][vo]sidechaincompress=threshold=0.02:ratio=4:attack=50:release=300:level_sc=1[mu];" | |
| f"[vo][mu]amix=inputs=2:duration=first:dropout_transition=2:weights=1 0.6[a]" | |
| f"[1:a]volume=1.0,apad=whole_dur={vid_dur},asplit=2[vo][vo_sc];" | |
| f"[2:a]volume=-18dB," | |
| f"afade=t=in:ss=0:d=4," | |
| f"afade=t=out:st={fade_out_start:.1f}:d=8," | |
| f"apad=whole_dur={vid_dur}[mu_raw];" | |
| f"[mu_raw][vo_sc]sidechaincompress=threshold=0.02:ratio=4:attack=50:release=300:level_sc=1[mu];" | |
| f"[vo][mu]amix=inputs=2:duration=first:dropout_transition=2:weights=1 0.6[a]" |
Was this helpful? React with 👍 or 👎 to provide feedback.
There was a problem hiding this comment.
Fixed in aedab28 — added asplit to split [vo] into [vo] + [vo_sc] before feeding sidechaincompress and amix separately. Sidechain ducking now activates successfully (confirmed: "sidechain ducking OK" in output instead of fallback message).
| # Skip motion for pre-rendered synthetic clips (titles, endcard, etc.) | ||
| if s.get("grade"): | ||
| fc = (f"[0:v]{rot}scale={MW}:{MH}:force_original_aspect_ratio=increase," | ||
| f"crop={MW}:{MH},setsar=1,setpts={ptsf:.4f}*PTS,fps=30," | ||
| f"{motion_expr(idx, T, zoom)},scale={CW}:{CH},setsar=1,format=gbrp," | ||
| f"{grade_chain(s.get('grade', []))},{TAG}{fade_filters}[v]") | ||
| else: | ||
| fc = (f"[0:v]{rot}scale={CW}:{CH}:force_original_aspect_ratio=increase," | ||
| f"crop={CW}:{CH},setsar=1,setpts={ptsf:.4f}*PTS,fps=30," | ||
| f"format=gbrp,{TAG}{fade_filters}[v]") |
There was a problem hiding this comment.
🚩 Grade-based branching for motion vs. simplified pipeline is inverted from what the comment suggests
The comment at build_ito.py:130 says "Skip motion for pre-rendered synthetic clips (titles, endcard, etc.)" but the condition if s.get("grade"): on line 131 routes clips with grade tags into the full motion pipeline, and clips without grade (empty list []) into the simplified path. After cross-referencing edl.json, this is actually correct: synthetic title/endcard clips have "grade": [] (falsy → simplified path), while clips needing motion+grading have non-empty grade lists like ["crush_black"]. The comment is misleading but the logic is correct. However, some synthetic clips like m_web (market_web.mp4, generated by gen_synthetic_broll.py) have "grade": ["cool"] and will get the full motion pipeline including upscale to 2208x1242 and Ken Burns crop, which may produce unexpected results on content already rendered at 1920x1080.
Was this helpful? React with 👍 or 👎 to provide feedback.
There was a problem hiding this comment.
Fixed in aedab28 — replaced fade=t=in/out + concat with proper xfade=transition=fade filter that blends overlapping clips. The xfade_concat() function builds a chain of xfade filters across all 22 segments with 0.5s crossfade duration. Also fixed the misleading comment — now reads "Clips with grade tags get full motion pipeline; clips with empty grade get simplified scale-only path."
| "problem": ["p_fragment", "broll_problem_traders"], | ||
| "solution": ["m_web", "broll_solution_floor", "broll_solution_trading"], | ||
| "product": ["m_ticker", "broll_product_exchange"], |
There was a problem hiding this comment.
🚩 Several edl.json pool clips referenced in SECTION_ORDER lack generation code
SECTION_ORDER references clip IDs broll_problem_traders, broll_solution_floor, broll_solution_trading, and broll_product_exchange (build_ito.py:45-47), and these are present in edl.json with src paths under assets/gen/. However, gen_synthetic_broll.py has no functions to generate these four files — gen_stock_exchange_trim() only produces broll_vidsplay_stock_exchange, broll_archive_vista_nyse_floor, and broll_archive_vista_nyse_open. If these clips aren't generated by another tool or manually placed, resolve_src will return a non-existent path, render() will print "MISSING" and return (False, ''), and build_ito.py will break out of the section entirely (line 203-204), skipping remaining clips for that section.
Was this helpful? React with 👍 or 👎 to provide feedback.
There was a problem hiding this comment.
Fixed in aedab28 — added the 4 missing clips (problem_traders, solution_floor, solution_trading, product_exchange) to gen_stock_exchange_trim() in gen_synthetic_broll.py. They were generated separately during the session but the generation code wasn't included in the committed script. Now all referenced broll clips are generated by the same function that trims the archival footage.
…chain, missing broll gen - build_ito.py: replace fade-to-black with real xfade filter for smooth crossfade transitions between clips (was producing black frames at cuts) - build_ito.py: fix misleading comment about grade-based branching - assemble_final.py: use asplit to split [vo] stream before feeding both sidechaincompress and amix (was always falling back to simple mix) - assemble_final.py: check ffprobe returncode and warn on fallback - gen_synthetic_broll.py: add missing broll generation (problem_traders, solution_floor, solution_trading, product_exchange) to gen_stock_exchange_trim - gen_synthetic_broll.py: clean up temp part files in gen_names_montage Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
New pipeline for 26s brand film v3: - gen_manim_math.py: Ito's lemma, stochastic paths, SDE formula animations - gen_v3_clips.py: 23 clip types (typography cards, art Ken Burns, portraits, platform UI, data buffer, gold particles, chromatic aberration) - build_v3.py: xfade assembly + sidechain-ducked audio mix Brand spec sourced from ito-cloud-runtime: - Libre Baskerville serif headlines, IBM Plex Mono labels - Colors: #0A0E17 bg, #8D7A50 gold, #E8E4DE white - Japanese art: Hokusai Red Fuji, gold-leaf screens, winter Fuji, peony garden 24 shots across 6 narrative beats: open(art+identity) → math+data → hedge → trade → construct → close(endcard) Output: out/ito_brand_film_v3.mp4 (26.3s, 1920x1080, 30fps) Regenerate: python3 gen_v3_clips.py && python3 gen_manim_math.py && python3 build_v3.py Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
devin-ai-integration
Bot
changed the title
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
devin-ai-integration
Bot
changed the title
Adds a new sourcing pack from the May 26 Intently direction: - articles/ news and context links - clips/ 16 temp/reference MP4 segments with URL + log sidecars - source-pages/ style/source reference pages - Updated REFERENCES.md and references/README.md Rights note: reference/temp-edit assets only. Final usage must be cleared. Generated with [Devin](https://devin.ai) Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
| GOLD = (141, 122, 80) # #8D7A50 | ||
| DARK_GOLD = (100, 86, 56) | ||
| FAINT = (50, 48, 44) # very dim text | ||
| INK = (17, 24, 39) # #111827 |
There was a problem hiding this comment.
REPO is fixed to /home/ubuntu/repos/ito-video, so the script immediately fails on any machine that isn't the original dev box — every file path derived from it (font files, clip output dirs, art assets) resolves to a non-existent path, and font loading falls back to the PIL default while CLIPS_DIR.mkdir() succeeds under the wrong tree. Use Path(__file__).resolve().parent (the same pattern used in build_v3.py and gen_v3_clips.py) instead.
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
|
Too many files changed for review. ( |
…ildup Changes: - Increased all subtitle/secondary text sizes (18→24, 14→20, etc.) - Added sourced reference clip overlays on text cards (Itô lemma, TMX, Jensen, Pentagon, Simons, Itô paths) - New 006b_inspirational_montage clip: fast 10-cut montage cycling through sourced clips with gold tint - Assembly arc restructured: quiet open → problem → offerings → math beauty → MONTAGE ENERGY → authority → resolve - Reference assembly: 27.7s (10 clips, up from 9) - Main body text: 78pt (was 72), hero stat: 130pt (was 120), audience: 96pt (was 80), offerings: 42pt (was 36) Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
devin-ai-integration
Bot
changed the title
- Removed non-thematic clips from montage (Hegseth, Pentagon, Jensen, Satya, Trump, SOTU) — now only markets/math: Simons, Itô, TMX - Title card restructured: Itô cropped logo → 'RESEARCH GRANT' → tagline - Endcard: larger Itô wordmark, 'MARKETS' below, bigger tagline (28pt), URL, all visually centered - URL card: APPLY label 20→26pt, URL 34→40pt - Offerings overlay: Jensen → TMX ETF (thematic) - Audience overlay: Pentagon → Itô paths (thematic) - Added cropped Itô logo asset Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
1 participant
Summary
Full pipeline for the Itô Research Grant 15-30s announcement video, staged as individual timeline clips for CapCut post-editing.
v2 (latest): Incorporated user feedback — bigger secondary text, sourced reference clip overlays, fast-cut montage, buildup arc.
Key changes in
gen_research_grant.py:Reference clip overlays —
extract_ref_frame()pulls stills from the institutional-refresh clips (Jensen, TMX, Simons, Itô lemma/paths, Pentagon) andoverlay_ref_clip()blends them as subtle desaturated backdrops on text cards at 6-10% opacity with gold tint.New
gen_006b_inspirational_montage()— fast 10-cut montage cycling through all sourced reference clips (~0.3s each), desaturated with gold tint and cross-dissolves. Creates an energy buildup between the quiet math section and the "23 endpoints" authority beat.Text size increases — all secondary/subtitle text bumped for readability:
Assembly arc (27.7s, 10 clips):
14 total clips generated for timeline flexibility, with
edl.jsonmanifest, trimmed music stem, and editing guide.Link to Devin session: https://app.devin.ai/sessions/cf5f76993c6749dca92686c13da692b0
Requested by: @affaan-m