Skip to content

fix(608): anchor pop-on->roll-up transition start time to first character, not CR - #2290

Open
x15sr71 wants to merge 2 commits into
CCExtractor:masterfrom
x15sr71:fix/608-rollup-transition-start-time
Open

fix(608): anchor pop-on->roll-up transition start time to first character, not CR#2290
x15sr71 wants to merge 2 commits into
CCExtractor:masterfrom
x15sr71:fix/608-rollup-transition-start-time

Conversation

@x15sr71

@x15sr71 x15sr71 commented Jul 2, 2026

Copy link
Copy Markdown
Contributor

In raising this pull request, I confirm the following (please check boxes):

Reason for this PR:

  • This PR adds new functionality.
  • This PR fixes a bug that I have personally experienced or that a real user has reported and for which a sample exists.
  • This PR is porting code from C to Rust.

Sanity check:

  • I have read and understood the contributors guide.
  • I have checked that another pull request for this purpose does not exist.
  • If the PR adds new functionality, I've added it to the changelog. If it's just a bug fix, I have NOT added it to the changelog.
  • I am NOT adding new C code unless it's to fix an existing, reproducible bug.

Repro instructions:

This PR fixes Regression Test 84 which is currently failing on both linux and windows. Reference Test Run - Windows, Linux.

Problem
Regression test 84 (sample 79, f23a544b..., --out=srt --latin1) fails exact SHA-256 comparison against reference e5c9d9ec... Confirmed via CI history (sample-platform DB) failing continuously since test_id 6698 (Dec 2025, shortly after 54df50f merged), including at the v0.96.6 release build (test 8446/8447).

8 caption entries in an 11,000-line file show incorrect start times when a 608 stream transitions from pop-on to roll-up mode mid-broadcast:

Entry Reference (correct) Current output
20 00:01:06,499 00:01:09,102
2164 00:19:03,299 00:19:05,768
(6 more, same pattern)

End times match exactly in all 8 cases; only start times diverge, by 2-4.5s.

Root cause
The bug is in COM_CARRIAGERETURN's handling of the rollup_from_popon transition-timing mechanism, added by PR #1808.

54df50f ("preserve CR time during pop-on to roll-up transition") added:

if (context->rollup_from_popon && !changes)
    context->ts_start_of_current_line = get_fts(...);

During a transition, CRs with changes==0 fire repeatedly while lines accumulate; this reassigns on every one, not just the first — the eventual changes==1 CR inherits the last CR's time instead of the transition's actual start (per --608 trace on sample 79).

The correct anchor is the fts of the first character written after the transition, via write_char()'s guard: if (ts_start_of_current_line == -1) ts_start_of_current_line = get_fts(...). This guard is not new either — present, unmodified, at least since this repo's first commit; its actual origin predates that and isn't determinable from git history.

Fix
Two changes, both required:

In the COM_CARRIAGERETURN handler: for the rollup_from_popon && !changes case, only set ts_start_of_current_line to the CR time when output_format == CCX_OF_TRANSCRIPT. For SRT, leave it untouched so write_char()'s existing == -1 guard anchors to the first character typed — which is what the SRT reference corpus (RT84) anchors to. Transcript mode needs CR time (RT29/RT30); SRT needs first-char time (RT84). A single unconditional change breaks one or the other.

In write_cc_buffer_as_transcript2: change write_cc_line_as_transcript2 from void to int (returns 1 if content was written, 0 if not) and use wrote_something |= in the caller. PR #2105 (03ad9e8e) set wrote_something = 1 unconditionally even when nothing was written for a blank cursor row, causing a bare newline to be emitted for every blank row. The --null-terminated WebSocket feature introduced by #2105 is unaffected.

Verification

  • sha256sum of full ccextractor output on sample 79 matches reference e5c9d9ec... exactly. Diff before fix: 8 lines. After: 0, full file.
  • RT29 / RT30 (--out=ttxt --latin1): transcript timing restored. 0-line diff vs reference.
  • RT41 (01509...ts, --out=ttxt --latin1): two spurious blank lines removed. Output is byte-identical to accepted variant ref 9f139b8c....
  • RT11 (--out=srt): first-line timing error from current master (00:00:06,173 vs reference 00:00:01,868) is fixed. Two residual sub-frame deltas (~68ms, ~34ms) are pre-existing and unrelated to this PR.
  • Regression test 98 (725a49f871.mpg) and 156 (c83f765c.ts), 54df50f's own motivating samples: neither currently hash-matches its stored reference, and this predates this PR (confirmed via CI history, both platforms). This PR does not regress either: test 98's affected line is restored to exact match; test 156's largest timing error improves from 1301ms to 67ms. Remaining deltas in both (an XDS-path timing offset, a # vs music-note character substitution from 300f8ca, and an unexplained residual ~67ms/2-field offset) are unrelated to this fix and out of scope for this PR.
  • Not a fix for the WTV 751ms global-offset limitation noted in fix(timing): correct caption start/end times to match video frame PTS fix(timing): correct caption start/end times to match video frame PTS #1808 — that's a uniform epoch mismatch across a whole file; this PR's target file hash-matches its reference in full, ruling out a uniform offset here. Different mechanism, likely addressed separately by 300f8ca.

@x15sr71
x15sr71 force-pushed the fix/608-rollup-transition-start-time branch from 072f7df to 3442bcf Compare August 9, 2026 19:39
…tion

In the pop-on -> roll-up transition with changes==0 (roll-up window not yet
full), only set ts_start_of_current_line to the CR time when in transcript
mode. SRT leaves it for write_char()'s existing == -1 guard to set on the
first character typed, which is what the SRT reference corpus anchors to.

Fixes a 133 ms transcript timing regression on RT29/RT30 introduced by the
earlier condition flip, while preserving the RT84 fix.
…ipt2 writes nothing

03ad9e8 refactored write_cc_buffer_as_transcript2 to write encoded_end_frame
at the end of each caption block. It set wrote_something = 1 unconditionally
after calling write_cc_line_as_transcript2, even when that function returned
without writing anything (blank cursor row, length = 0). Result: a bare newline
(encoded_end_frame) was emitted for every blank cursor row, producing spurious
blank lines in transcript output.

Fix: change write_cc_line_as_transcript2 from void to int (returns 1 if it
wrote, 0 if not) and use wrote_something |= to only set the flag when content
was actually emitted.

Fixes RT41 blank line regression introduced by PR CCExtractor#2105.
@x15sr71
x15sr71 force-pushed the fix/608-rollup-transition-start-time branch from 3b1ee1e to 1782e17 Compare August 9, 2026 19:44
@ccextractor-bot

Copy link
Copy Markdown
Collaborator
CCExtractor CI platform finished running the test files on linux. Below is a summary of the test results, when compared to test for commit e7ad5c7...:
Report Name Tests Passed
Broken 9/13
CEA-708 2/14
DVB 1/7
DVD 3/3
DVR-MS 1/2
General 24/27
Hardsubx 1/1
Hauppage 2/3
MP4 3/3
NoCC 10/10
Options 29/86
Teletext 20/21
WTV 11/13
XDS 30/34

Your PR breaks these cases:

NOTE: The following tests have been failing on the master branch as well as the PR:

Congratulations: Merging this PR would fix the following tests:

  • ccextractor --autoprogram --out=srt --latin1 1d9731bd80..., Last passed: Never
  • ccextractor --autoprogram --out=ttxt --latin1 --ucla --xds 8e8229b88b..., Last passed: Never
  • ccextractor --out=srt --latin1 06b3a9237d..., Last passed: Never
  • ccextractor --out=srt --latin1 b46e9e8e3f..., Last passed: Never
  • ccextractor --datapid 5603 --autoprogram --out=srt --latin1 --teletext 85c7fc1ad7..., Last passed: Never
  • ccextractor --wtvconvertfix --autoprogram --out=srt --latin1 acf871cbfd..., Last passed: Never
  • ccextractor --autoprogram --out=ttxt --latin1 99e5eaafdc..., Last passed: Never
  • ccextractor --autoprogram --out=ttxt --latin1 --ucla 70000200c0..., Last passed: Never
  • ccextractor --autoprogram --out=ttxt --latin1 01509e4d27..., Last passed: Never
  • ccextractor --autoprogram --out=ttxt --latin1 --ucla ab9cf8cfad..., Last passed: Never
  • ccextractor --autoprogram --out=ttxt --latin1 --ucla --output-field 2 5d3a29f9f8..., Last passed: Never
  • ccextractor --autoprogram --out=ttxt --latin1 --ucla --output-field 2 c41f73056a..., Last passed: Never
  • ccextractor --autoprogram --out=srt --latin1 --sentencecap c032183ef0..., Last passed: Never
  • ccextractor --dru c83f765c66..., Last passed: Never
  • ccextractor --startat 4 --endat 7 c83f765c66..., Last passed: Never
  • ccextractor --codec dvbsub --out=spupng 85271be4d2..., Last passed: Never
  • ccextractor --startcreditstext "CCextractor Start crdit Testing" c4dd893cb9..., Last passed: Never
  • ccextractor --startcreditsnotbefore 1 --startcreditstext "CCextractor Start crdit Testing" c4dd893cb9..., Last passed: Never
  • ccextractor --startcreditsnotafter 2 --startcreditstext "CCextractor Start crdit Testing" c4dd893cb9..., Last passed: Never
  • ccextractor --startcreditsforatleast 1 --startcreditstext "CCextractor Start crdit Testing" c4dd893cb9..., Last passed: Never
  • ccextractor --startcreditsforatmost 2 --startcreditstext "CCextractor Start crdit Testing" c4dd893cb9..., Last passed: Never
  • ccextractor --autoprogram --out=ttxt --latin1 efbe129086..., Last passed: Never
  • ccextractor --out=srt --latin1 f23a544ba8..., Last passed: Never
  • ccextractor --out=srt --latin1 0e5e6b26be..., Last passed: Never
  • ccextractor --autoprogram --out=ttxt --xds --latin1 --ucla e274a73653..., Last passed: Never
  • ccextractor --autoprogram --out=ttxt --latin1 --xds --ucla c813e713a0..., Last passed: Never
  • ccextractor --autoprogram --out=ttxt --latin1 --ucla --xds 27fab4dbb6..., Last passed: Never
  • ccextractor --autoprogram --out=ttxt --latin1 --ucla --xds bbd5bb52fc..., Last passed: Never
  • ccextractor --autoprogram --out=ttxt --latin1 --ucla --xds b992e0cccb..., Last passed: Never
  • ccextractor --autoprogram --out=ttxt --latin1 --ucla --xds d0291cdcf6..., Last passed: Never
  • ccextractor --autoprogram --out=ttxt --latin1 --ucla --xds c8dc039a88..., Last passed: Never
  • ccextractor --autoprogram --out=ttxt --latin1 --ucla --xds 53339f3455..., Last passed: Never
  • ccextractor --autoprogram --out=srt --latin1 --ucla 7d3f25c32c..., Last passed: Never
  • ccextractor --autoprogram --out=ttxt --latin1 --ucla --xds 88cd42b89a..., Last passed: Never
  • ccextractor --autoprogram --out=ttxt --latin1 --ucla --xds 7f41299cc7..., Last passed: Never

It seems that not all tests were passed completely. This is an indication that the output of some files is not as expected (but might be according to you).

Check the result page for more info.

@ccextractor-bot

Copy link
Copy Markdown
Collaborator
CCExtractor CI platform finished running the test files on windows. Below is a summary of the test results, when compared to test for commit e7ad5c7...:
Report Name Tests Passed
Broken 9/13
CEA-708 2/14
DVB 2/7
DVD 3/3
DVR-MS 2/2
General 24/27
Hardsubx 1/1
Hauppage 2/3
MP4 3/3
NoCC 10/10
Options 28/86
Teletext 20/21
WTV 10/13
XDS 33/34

Your PR breaks these cases:

NOTE: The following tests have been failing on the master branch as well as the PR:

Congratulations: Merging this PR would fix the following tests:

  • ccextractor --autoprogram --out=ttxt --latin1 1020459a86..., Last passed: Never
  • ccextractor --autoprogram --out=ttxt --latin1 132d7df7e9..., Last passed: Never
  • ccextractor --autoprogram --out=ttxt --latin1 99e5eaafdc..., Last passed: Never
  • ccextractor --autoprogram --out=ttxt --latin1 01509e4d27..., Last passed: Never
  • ccextractor --dru c83f765c66..., Last passed: Never
  • ccextractor --startat 4 --endat 7 c83f765c66..., Last passed: Never
  • ccextractor --startcreditstext "CCextractor Start crdit Testing" c4dd893cb9..., Last passed: Never
  • ccextractor --startcreditsnotbefore 1 --startcreditstext "CCextractor Start crdit Testing" c4dd893cb9..., Last passed: Never
  • ccextractor --startcreditsnotafter 2 --startcreditstext "CCextractor Start crdit Testing" c4dd893cb9..., Last passed: Never
  • ccextractor --startcreditsforatleast 1 --startcreditstext "CCextractor Start crdit Testing" c4dd893cb9..., Last passed: Never
  • ccextractor --startcreditsforatmost 2 --startcreditstext "CCextractor Start crdit Testing" c4dd893cb9..., Last passed: Never
  • ccextractor --autoprogram --out=srt --latin1 --ucla d037c7509e..., Last passed: Never
  • ccextractor --autoprogram --out=srt --latin1 --ucla 7d3f25c32c..., Last passed: Never
  • ccextractor --autoprogram --out=ttxt --latin1 --ucla --xds 7f41299cc7..., Last passed: Never

It seems that not all tests were passed completely. This is an indication that the output of some files is not as expected (but might be according to you).

Check the result page for more info.

@cfsmp3
cfsmp3 self-requested a review August 10, 2026 14:19

@cfsmp3 cfsmp3 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

One thing that needs fixing

On sample 7236304cfc, SRT output regresses:

master: 1 00:00:05,922 --> 00:00:07,089
2 00:00:07,091 --> 00:00:08,273

this PR: 1 00:00:00,000 --> 00:00:07,089
2 00:00:07,273 --> 00:00:08,273

Two problems. The first cue anchors to 00:00:00,000, so it claims to be on screen from the very start of the file. And the second cue's start moves 182 ms later, which is the opposite
direction from everything else in the corpus and opens a gap where master had a 2 ms join.

This is the only sample of the 25 where either happens, so it looks like an edge case in how ts_start_of_current_line is anchored when the transition happens near the start of the stream
— plausibly it is still 0/unset rather than a real first-character time, and the > 0 guard elsewhere in the file doesn't cover this path.

Please note the regression suite cannot catch this: 7236304cfc is only exercised as --autoprogram --out=ttxt --latin1 (rt5), which passes precisely because transcript output is unchanged
now. So a green run here would not mean it is fixed.

Two smaller points

Please split the transcript fix out. Making write_cc_line_as_transcript2() return whether it wrote, and folding that into wrote_something, is a correct fix and has nothing to do with
pop-on→roll-up timing. On its own it is small, low-risk and I would merge it right away. Bundled here, it is stuck behind the timing discussion.

Please justify the output_format gate in the description. After this change the 608 decoder produces different start times depending on which encoder is selected, so --out=srt and
--out=ttxt disagree on when the same caption begins. That may well be right, given transcript times individual lines rather than buffers, but it is surprising enough that it should be
written down — otherwise it gets reported as a bug later, and whoever looks at it will not know it was deliberate.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants