Weird jankiness with certain TS live streams over UDP

[signumnova]

Relatively new to exoplayer3 but coming up the curve. Currently using 1.5.1.

I have a test player app that plays a live stream delivered via udp (unicast, but have successfully tested multicast delivery as well).

The stream is a MPEG-TS stream, H264 encoding.

The target Android device is a QBIC BXP320 which has a ARM Dual-Core A72 + Quad-core A53, 64 bit 1.8GHz
Embedded Platform is Android 10. This thing is capable of 4Kp30 H264 playback, but my test stream is only 1080i50

Most of the time, the first time I run the player and start the stream playing it plays beautifully. No jank.

If I then call player.stop(), player.release and then recreate it, I start getting janky playback. Regular pauses in the renderer and then a bit of p-frame type "catchup" (not quite the same as green decode errors).

The analytics stats says there are 0 dropped frames. There are absolutely no debug warnings or errors in logcat when this janking is occurring.

When I looked at this through profiler, CPU wasn't very busy either.

I notice when I run VLC/android on this device it is subject to similar behavior on the same stream. The other weird thing is that other live streams - same type of encoding - seem much better and not prone to this issue. I have confirmed the streams themselves are fine - i don't see this behavior playing the same streams in Windows VLC.

So it sort of feels like a pause coming out of the hardware decoder to the renderer - maybe all the frames are there but just not being played on a smooth timeline? (Also I did enable media encoder async queuing but this made no difference).

I have the option of upping this device to Android 13, from 10, but is that really likely to make any difference?

Glad for any advice on how to narrow this down... is there a way to log what's going on frame-by-frame in the render queue?

Thanks SN

[FongMi]

Can you test media3 1.4.0-alpha01 ?
I found a commit that might be the problem.
This commit is include in 1.4.0-alpha02
029b8ba

[signumnova]

Interesting... I will try that

[signumnova]

@FongMi - no, not the same issue. I can still repro mine on 1.4.0-alpha01

[signumnova]

OK - So I do seen an interesting difference between initial run and the second time player object is created. (And I can't account for why this might be so yet). I have the debughelperview overlay running.

During the initial run, the playback state sits at a solid "ready", with the very occasional flash to "buffering". There is no jankiness during this time.

During the next run, (after player the player object is released, recreated and the source prepared again), its playback status regularly flashes to idle about once a second, and then back to ready. It seems that it's only when it's does this behavior that I see the semi-regular janking, so it seems more like some kind of buffering thing to me now.

If this is a difference in whats going on with the buffering between first and second instantiation this is hard to explain, as there are really no other media3 objects hanging around or being re-used, save for the player view itself.

(I did retest the app against exoplayer 1.4.0-alpha01 in reference to that issue that @FongMi mentioned but it had the same behavior, so whatever this is is not something that has crept in recently).

Playing a "live" udp stream is not the quite the same as playing from HLS or from a file, since it's not possible to buffer up beyond the 'current live' playback position (in effect), so there has to be an artificial delay between when the player starts pulling in data and when it actually starts playback. If the player is slower at rendering the frames than the stream rate, the network buffer will gradually fill up. Eventually some packets must get discarded which will cause a discontinuity/resync in the decoder. Conversely if the player is consuming and rendering the frames slightly faster than "real time" it will eventually 'catch up' and maybe run out of frames, which should lead to a re-buffering. So I'm really not quite sure what is the optimal set of settings for the LoadControl, nor whether this is enough by itself to produce the smoothest behavior for this type of live stream.

One other question I have is about audio. The problem I described above occurs with or without the audio track being decoded. If audio is selected though, it plays smoothly right through the video jank, so this suggests that there is no loss of input packets/frames going on. But I am curious about what algorithm exoplayer is using to keep the audio and video in sync. In our DVB-T signals the PTS for audio and video are about 1500mS apart, I guess because there is more latency in a typical video decoder pipeline to deal with. But I'm interested in how exoplayer manages this in terms of presenting both audio and video at the right time, since this would require buffering the audio and video frames in quite separate queues. Does it give audio more priority and try to sync the video with it, or does it attempt to sync both of them independently to a PTS clock?

[FongMi]

Maybe you need provide test url or m3u8 and ts files for test.

[signumnova]

I'd like to see if I can get to the bottom of why the second "run" behaves differently from the first, in particular why the player starts oscillating into "idle" state with the second state... according to the docs this is not a state that the player should regularly find itself in during normal playback. I can reproduce that particular behavior just using playback of a captured file of the same stream so it has nothing to do with the transport mechanism being network based. This may turn out to be some kind of red herring but then again it may not be.

If I get nowhere, I will modify the std Demo app to allow use of a udp stream URL (since I don't think it can out of the box) and see where that leads... then I have something more concrete to file a bug with if the glitching is readily reproduceable.

[signumnova]

Well I solved the weird behavior with the player status - it was simply that I can forgotten to stop the debughelper before releasing the player, so there were two status updaters clashing, one of which was the old player instance which was idle.

The jankiness issue remains...

[tonihei]

starts oscillating into "idle" state with the second state

If the player does into IDLE, it indicates you likely encountered an error and then tried to restart. It would probably help to attach an EventLogger (see ) and provide the output here to see more details about what's happening. https://developer.android.com/media/media3/exoplayer/debug-logging

[signumnova]

Nah, the status issue was a red herring. See previous comment.

The jankiness seems to be random.... i.e stream the same content repeatedly to the player and it isnt a repeatable effect... which I hate because it means its more of a timing thing... I need to do some further testing to try and produce a useful test case.