8331458: Replace qtdemux with MP4 demux based on Media Foundation

[sashamatveev]

• Replaced qtdemux with a Media Foundation-based MP4 demuxer on Windows.
• The Media Foundation-based MP4 demuxer supports pull mode only due to the design of the Media Foundation APIs. For fragmented MP4 (fMP4) segments, each segment is demuxed independently because of API limitations. After processing each segment, the demuxer internally creates a new Source Reader.
• HLSConnectionHolder: Headers are now sent with every segment because the MP4 demuxer reloads the Source Reader for each segment. Headers are intentionally not cached to avoid unnecessary connections. Additionally, HLS streams with separate audio renditions now seek to the last segment when seeking to the end of the stream, matching the behavior of standard HLS streams.
• JavaSource: Removed the unused update flag and moved caps setting into a dedicated switch case.
• Added trace support for Media Foundation-based plugins (both the demuxer and decoder). The tracing infrastructure is still a work in progress and will be expanded as needed for debugging. It is disabled by default and can only be enabled at compile time.
• H.264 video from MP4 and fMP4 (HLS) is now decoded using Media Foundation instead of DirectShow. H.264 video in TS-based HLS streams continues to use DirectShow. This change avoids several DirectShow issues encountered with the new MP4 demuxer and reduces implementation complexity.
• Updated MFWrapper to properly support H.264 decoding. Unlike H.265, H.264 requires using the display aperture for the display resolution, does not signal frame availability, requires handling format changes, and must fully drain output before accepting new input samples.
• Modified the cache implementation to support pull mode for the HLS progress buffer.
• Added pull mode support to the HLS progress buffer.
• Changed progressbuffer to use a static source pad instead of a dynamic one, simplifying pipeline construction. The need for a dynamic source pad is unclear.
• Performed minor cleanup by removing dead and obsolete code.
• The progress buffer is now forced when JavaSource does not support pull mode (for example, with jar and jrt protocols). This provides a simple workaround until native pull mode support is implemented for these protocols. In particular, jar does not provide true random access at the Java layer anyway.

---

• I confirm that I make this contribution in accordance with the OpenJDK Interim AI Policy.

---

Progress

• Change must not contain extraneous whitespace
• Commit message must refer to an issue
• Change must be properly reviewed (2 reviews required, with at least 1 Reviewer, 1 Author)

Issues

• JDK-8331458: Replace qtdemux with MP4 demux based on Media Foundation (Enhancement - P3)
• JDK-8305842: Video sometimes does not start when reinitializing in Windows 11 (Bug - P3)

Reviewing

Using git

Checkout this PR locally:
$ git fetch https://git.openjdk.org/jfx.git pull/2206/head:pull/2206
$ git checkout pull/2206

Update a local copy of the PR:
$ git checkout pull/2206
$ git pull https://git.openjdk.org/jfx.git pull/2206/head

Using Skara CLI tools

Checkout this PR locally:
$ git pr checkout 2206

View PR using the GUI difftool:
$ git pr show -t 2206

Using diff file

Download this PR as a diff file:
https://git.openjdk.org/jfx/pull/2206.diff

Using Webrev

Link to Webrev Comment

[bridgekeeper]

👋 Welcome back almatvee! A progress list of the required criteria for merging this PR into master will be added to the body of your pull request. There are additional pull request commands available for use with this pull request.

[openjdk]

❗ This change is not yet ready to be integrated.
See the Progress checklist in the description for automated requirements.

[openjdk]

The total number of required reviews for this PR has been set to 2 based on the presence of this label: rfr. This can be overridden with the /reviewers command.

[mlbridge]

Webrevs

• 06: Full - Incremental (354991a0)
• 05: Full - Incremental (3610b66c)
• 04: Full - Incremental (d363edbc)
• 03: Full (874938f5)
• 02: Full - Incremental (132d5157)
• 01: Full - Incremental (aa3a7a77)
• 00: Full (57b35bef)

[sashamatveev]

/issue add JDK-8305842

[openjdk]

@sashamatveev
Adding additional issue to issue list: 8305842: Video sometimes does not start when reinitializing in Windows 11.

[sashamatveev]

8305842: Replace qtdemux with MP4 demux based on Media Foundation [v4]

• Fixed mimetype issue on Linux. qtdemux only accepts video/quicktime, so we need to map video/mp4 and video/x-m4v to video/quicktime.

[openjdk]

@sashamatveev this pull request can not be integrated into master due to one or more merge conflicts. To resolve these merge conflicts and update this pull request you can run the following commands in the local repository for your personal fork:

git checkout JDK-8331458-2
git fetch https://git.openjdk.org/jfx.git master
git merge FETCH_HEAD
# resolve conflicts and follow the instructions given by git merge
git commit -m "Merge master"
git push

[sashamatveev]

8305842: Replace qtdemux with MP4 demux based on Media Foundation [v5]

• Changed on how HLS select media file index when variant playlist is switched. It was based on file name, but it is not reliable, since file names are not guarantee to match. Instead index is adjusted based on sequence number which should not drift more then +/- 1 when switch is done. Sequence number is like index of first segment file in live streams.
• Fixed issue when variant playlist was not detected as live even it was live. Playlist with variant playlists was detected correctly as live.
• Fixed reload time. Thread sleep is expecting milliseconds and we provided (seconds / 2) as milliseconds without conversion. As are result we where requesting playlist way too often.
• Redone mechanism on how we waiting for live segments to be added. Old approach had a bug with semaphore and if we added two segments in one run it will cause code to assume that we have segments when we did not. As a result EOS was signaled for live streams. This issue does not reproduce often, but might happen.
• Old playlist data will be cleared when we switching bitrate, otherwise we will attempt to play old segments which are gone long time.
• Fixed removing old segments not to remove init header for fMP4.
• Added additional tracing to mfdemux and mfwrapper.
• mfdemux will not push caps event if caps did not change. When event is pushed decoder will flush and cause video glitch.
• mfdemux will adjust segment event to first PTS. Needed for live streams, since their PTS does not start with 0.
• Increase queue buffers for HLS Live on Windows with mfdemux since we reload source reader for each segment to eliminate stream break ups.

[sashamatveev]

8305842: Replace qtdemux with MP4 demux based on Media Foundation [v6]

• Fixed merge issues.

[sashamatveev]

8305842: Replace qtdemux with MP4 demux based on Media Foundation [v7]

• Undo unrelated HLS fixes and moved them under 8389211: Fix HLS Live issues found during testing of JDK-8331458 and various live streams #2227

[sashamatveev]

8305842: Replace qtdemux with MP4 demux based on Media Foundation [v8]

• mfdemux will convert BYTE segments to TIME to handle HLS Live change introduced in 8389211: Fix HLS Live issues found during testing of JDK-8331458 and various live streams #2227. With 8389211: Fix HLS Live issues found during testing of JDK-8331458 and various live streams #2227 hlsprogressbuffer will send BYTE segment, so qtdemux or any another demux will handle PTS at demux level.