[Android] RTMP HEVC video stream black (audio working)

[studio4evr]

Unity version

2022.3.62f2

Unity editor platform

Windows

AVPro Video edition

Trial

AVPro Video version

3.3.3

Device hardware

Meta Quest 3S

Which Android OS version are you using?

13

Unity Graphics API

OpenGLES 3

Video API

media3/ExoPlayer

Texture format

BGRA

Audio output

System Direct

Any other Media Player component configuration required to reproduce the issue.

No response

Which output component(s) are you using?

Apply to Mesh

Any other component configuration required to reproduce the issue.

No response

The issue

I'm streaming 180 VR video from a SLAM camera over RTMP. The stream works with windows, but not in android. I'm using the 360 VR demo scene with an open XR XR rig. I'm getting black video and an audio stream on the build in the headset. I've copied the logs from the meta developer logging app under the search Unity and AVPro.

Media information

Video is HEVX main L6.0, yuv40p, 6144x3072, A aac lc, 48khz stereo

Logcat output

1:54.072
Unity
[XR] [481668483568] [16:21:54.072][Info ]
16:21:54.072
Unity
16:21:54.072
Unity
==== End Unity OpenXR Diagnostic Report ====
16:21:54.072
Unity
16:21:54.072
Unity
16:21:58.393
Unity
playerID: 0000B762
16:21:58.393
Unity
RenderHeads.Media.AVProVideo.PlatformMediaPlayer:CreateCommandBuffers()
16:21:58.393
Unity
RenderHeads.Media.AVProVideo.PlatformMediaPlayer:.ctor(PlatformOptions)
16:21:58.393
Unity
RenderHeads.Media.AVProVideo.MediaPlayer:CreateMediaPlayer(OptionsAndroid)
16:21:58.393
Unity
RenderHeads.Media.AVProVideo.MediaPlayer:CreateMediaPlayer()
16:21:58.393
Unity
RenderHeads.Media.AVProVideo.MediaPlayer:Initialise()
16:21:58.393
Unity
RenderHeads.Media.AVProVideo.MediaPlayer:Start()
16:21:58.393
Unity
16:21:58.393
Unity
CreateCommandBuffers - flags: 00000000
16:21:58.393
Unity
RenderHeads.Media.AVProVideo.PlatformMediaPlayer:CreateCommandBuffers()
16:21:58.393
Unity
RenderHeads.Media.AVProVideo.PlatformMediaPlayer:.ctor(PlatformOptions)
16:21:58.393
Unity
RenderHeads.Media.AVProVideo.MediaPlayer:CreateMediaPlayer(OptionsAndroid)
16:21:58.393
Unity
RenderHeads.Media.AVProVideo.MediaPlayer:CreateMediaPlayer()
16:21:58.393
Unity
RenderHeads.Media.AVProVideo.MediaPlayer:Initialise()
16:21:58.393
Unity
RenderHeads.Media.AVProVideo.MediaPlayer:Start()
16:21:58.393
Unity
16:21:58.393
Unity
flags: 0000B762
16:21:58.393
Unity
RenderHeads.Media.AVProVideo.PlatformMediaPlayer:CreateCommandBuffers()
16:21:58.393
Unity
RenderHeads.Media.AVProVideo.PlatformMediaPlayer:.ctor(PlatformOptions)
16:21:58.393
Unity
RenderHeads.Media.AVProVideo.MediaPlayer:CreateMediaPlayer(OptionsAndroid)
16:21:58.393
Unity
RenderHeads.Media.AVProVideo.MediaPlayer:CreateMediaPlayer()
16:21:58.393
Unity
RenderHeads.Media.AVProVideo.MediaPlayer:Initialise()
16:21:58.393
Unity
RenderHeads.Media.AVProVideo.MediaPlayer:Start()
16:21:58.393
Unity
16:21:58.395
Unity
[AVProVideo] Initialising AVPro Video v3.3.3 (native plugin v3.3.3-trial) on Adreno (TM) 740/Vulkan 1.1.0 [512.819.2] (MT True) on Android
16:21:58.395
Unity
RenderHeads.Media.AVProVideo.MediaPlayer:Initialise()
16:21:58.395
Unity
RenderHeads.Media.AVProVideo.MediaPlayer:Start()
16:21:58.395
Unity
16:21:58.396
Unity
[AVProVideo] Opening rtmp://192.168.0.254/live/stream (offset 0) with API ExoPlayer
16:21:58.396
Unity
RenderHeads.Media.AVProVideo.MediaPlayer:InternalOpenMedia()
16:21:58.396
Unity
RenderHeads.Media.AVProVideo.MediaPlayer:Start()
16:21:58.396
Unity
16:21:58.417
Unity
Ignoring depth surface store action as it is memoryless
16:21:58.417
Unity
UnityEngine.Rendering.Universal.UniversalRenderPipeline:RenderSingleCamera(ScriptableRenderContext, CameraData&)
16:21:58.417
Unity
UnityEngine.Rendering.Universal.UniversalRenderPipeline:RenderCameraStack(ScriptableRenderContext, Camera)
16:21:58.417
Unity
UnityEngine.Rendering.Universal.UniversalRenderPipeline:Render(ScriptableRenderContext, List`1)
16:21:58.417
Unity
UnityEngine.Rendering.RenderPipelineManager:DoRenderLoop_Internal(RenderPipelineAsset, IntPtr, Object)
16:21:58.417
Unity
16:21:58.417
Unity
[ line 631998104]
16:21:58.417
Unity

AVPro logs:

16:21:58.403
AVProVideo
💬️ Player_ExoPlayer.CreateExoPlayer - CREATED new ExoPlayer object
16:21:58.405
AVProVideo
💬️ Player_ExoPlayer.BuildMediaSource - Video Loading: RTSP/RTMP
16:21:59.698
AVProVideo
AVPPlayerAudioCaptureBuffer: capacity = 960000
16:21:59.700
AVProVideo
AVPPlayerAudioCaptureBuffer: 1
16:22:08.353
AVProVideo
💬️ Player_ExoPlayer.DoReadyStateSetup - DoReadyStateSetup : Audio : m_bVideo_AcceptCommands.set( true )
16:22:22.909
AVProVideo
💬️ Player_ExoPlayer.DoReadyStateSetup - DoReadyStateSetup : Audio : m_bVideo_AcceptCommands.set( true )
16:22:45.875
AVProVideo
💬️ Player_ExoPlayer.DoReadyStateSetup - DoReadyStateSetup : Audio : m_bVideo_AcceptCommands.set( true )
16:23:09.966
AVProVideo
💬️ Player_ExoPlayer.DoReadyStateSetup - DoReadyStateSetup : Audio : m_bVideo_AcceptCommands.set( true )
16:23:35.271
AVProVideo
💬️ Player_ExoPlayer.DoReadyStateSetup - DoReadyStateSetup : Audio : m_bVideo_AcceptCommands.set( true )

[Chris-RH]

A full, unfiltered logcat would be helpful.

I would suggest testing an H264 stream, to see if its a codec issue.

[studio4evr]

tested a H264 stream, and that was working.

[studio4evr]

I will send a unfiltered logcat later today!

[Chris-RH]

How does the H265 stream run in editor on your desktop?

[studio4evr]

How does the H265 stream run in editor on your desktop?

Works fine!

[Chris-RH]

OK, so it does seem to be the Quest 3 having issues decoding the H265. I would recommend sticking with H264 in that case. We rely on the hardware for decoding, so its unlikely that we can do anything else in this instance.

[studio4evr]

It's a 8k 180 VR stream, so I'm trying to keep it h265 - would the OES texture format make any difference? that's the only setting i can't test as it is deactivated in the trial version.

[Chris-RH]

OES is unlikely to make any difference.
Have you tried a different streaming format?
Have you tried reducing resolution?