UpdateTexture slow

[saxofon]

Hi,

I'm running raylib and gstreamer, trying to get overlays on video.
Works brilliant on my x86 linux laptop and a raspberry pi4 but for some reason rendering the video frame slow on a AMD/Xilinx UltraScale+ ZynqMP SoC shown below here :

Jan 19 09:23:45  systemd[1]: Started AVP overlay daemon.
Jan 19 09:23:45 overlayd[869]: INFO: Initializing raylib 5.5
Jan 19 09:23:45 overlayd[869]: INFO: Platform backend: DESKTOP (GLFW)
Jan 19 09:23:45 overlayd[869]: INFO: Supported raylib modules:
Jan 19 09:23:45 overlayd[869]: INFO:     > rcore:..... loaded (mandatory)
Jan 19 09:23:45 overlayd[869]: INFO:     > rlgl:...... loaded (mandatory)
Jan 19 09:23:45 overlayd[869]: INFO:     > rshapes:... loaded (optional)
Jan 19 09:23:45 overlayd[869]: INFO:     > rtextures:. loaded (optional)
Jan 19 09:23:45 overlayd[869]: INFO:     > rtext:..... loaded (optional)
Jan 19 09:23:45 overlayd[869]: INFO:     > rmodels:... loaded (optional)
Jan 19 09:23:45 overlayd[869]: INFO:     > raudio:.... loaded (optional)
Jan 19 09:23:45 overlayd[869]: INFO: SYSTEM: Closest fullscreen videomode: 2560 x 1080
Jan 19 09:23:45 overlayd[869]: INFO: DISPLAY: Device initialized successfully
Jan 19 09:23:45 overlayd[869]: INFO:     > Display size: 2560 x 1080
Jan 19 09:23:45 overlayd[869]: INFO:     > Screen size:  2560 x 1080
Jan 19 09:23:45 overlayd[869]: INFO:     > Render size:  2560 x 1080
Jan 19 09:23:45 overlayd[869]: INFO:     > Viewport offsets: 0, 0
Jan 19 09:23:45 overlayd[869]: INFO: GL: Supported extensions count: 28
Jan 19 09:23:45 overlayd[869]: INFO: GL: OpenGL device information:
Jan 19 09:23:45 overlayd[869]: INFO:     > Vendor:   ARM
Jan 19 09:23:45 overlayd[869]: INFO:     > Renderer: Mali-400 MP
Jan 19 09:23:45 overlayd[869]: INFO:     > Version:  OpenGL ES 2.0 "fce4231"
Jan 19 09:23:45 overlayd[869]: INFO:     > GLSL:     OpenGL ES GLSL ES 1.00
Jan 19 09:23:45 overlayd[869]: INFO: GL: VAO extension detected, VAO functions loaded successfully
Jan 19 09:23:45 overlayd[869]: INFO: GL: NPOT textures extension detected, full NPOT textures supported
Jan 19 09:23:45 overlayd[869]: INFO: GL: ETC1 compressed textures supported
Jan 19 09:23:45 overlayd[869]: INFO: PLATFORM: DESKTOP (GLFW - Wayland): Initialized successfully
Jan 19 09:23:45 overlayd[869]: INFO: TEXTURE: [ID 1] Texture loaded successfully (1x1 | R8G8B8A8 | 1 mipmaps)
Jan 19 09:23:45 overlayd[869]: INFO: TEXTURE: [ID 1] Default texture loaded successfully
Jan 19 09:23:45 overlayd[869]: INFO: SHADER: [ID 1] Vertex shader compiled successfully
Jan 19 09:23:45 overlayd[869]: INFO: SHADER: [ID 2] Fragment shader compiled successfully
Jan 19 09:23:45 overlayd[869]: INFO: SHADER: [ID 3] Program shader loaded successfully
Jan 19 09:23:45 overlayd[869]: INFO: SHADER: [ID 3] Default shader loaded successfully
Jan 19 09:23:45 overlayd[869]: INFO: RLGL: Render batch vertex buffers loaded successfully in RAM (CPU)
Jan 19 09:23:45 overlayd[869]: INFO: RLGL: Render batch vertex buffers loaded successfully in VRAM (GPU)
Jan 19 09:23:45 overlayd[869]: INFO: RLGL: Default OpenGL state initialized successfully
Jan 19 09:23:45 overlayd[869]: INFO: TEXTURE: [ID 2] Texture loaded successfully (128x128 | GRAY_ALPHA | 1 mipmaps)
Jan 19 09:23:45 overlayd[869]: INFO: FONT: Default font loaded successfully (224 glyphs)
Jan 19 09:23:45 overlayd[869]: INFO: SYSTEM: Working Directory: /
Jan 19 09:23:45 overlayd[869]: INFO: TIMER: Target time per frame: 16.667 milliseconds
Jan 19 09:23:45 overlayd[869]: INFO: TEXTURE: [ID 3] Texture loaded successfully (1024x512 | GRAY_ALPHA | 1 mipmaps)
Jan 19 09:23:45 overlayd[869]: INFO: FONT: Data loaded successfully (96 pixel size | 95 glyphs)
Jan 19 09:23:45 overlayd[869]: INFO: TEXTURE: [ID 4] Texture loaded successfully (1024x512 | GRAY_ALPHA | 1 mipmaps)
Jan 19 09:23:45 overlayd[869]: INFO: FONT: Data loaded successfully (96 pixel size | 95 glyphs)
Jan 19 09:23:45 overlayd[869]: INFO: TEXTURE: [ID 5] Texture loaded successfully (1024x512 | GRAY_ALPHA | 1 mipmaps)
Jan 19 09:23:45 overlayd[869]: INFO: FONT: Data loaded successfully (96 pixel size | 95 glyphs)
Jan 19 09:23:45 overlayd[869]: INFO: IMAGE: Data loaded successfully (512x512 | R8G8B8A8 | 1 mipmaps)
Jan 19 09:23:45 overlayd[869]: INFO: TEXTURE: [ID 6] Texture loaded successfully (512x512 | R8G8B8A8 | 1 mipmaps)
Jan 19 09:23:45 overlayd[869]: INFO: IMAGE: Data loaded successfully (512x512 | R8G8B8A8 | 1 mipmaps)
Jan 19 09:23:45 overlayd[869]: INFO: TEXTURE: [ID 7] Texture loaded successfully (512x512 | R8G8B8A8 | 1 mipmaps)
Jan 19 09:23:45 overlayd[869]: INFO: IMAGE: Data loaded successfully (512x512 | R8G8B8A8 | 1 mipmaps)
Jan 19 09:23:45 overlayd[869]: INFO: TEXTURE: [ID 8] Texture loaded successfully (512x512 | R8G8B8A8 | 1 mipmaps)
Jan 19 09:23:45 overlayd[869]: INFO: IMAGE: Data loaded successfully (512x512 | R8G8B8A8 | 1 mipmaps)
Jan 19 09:23:45 overlayd[869]: INFO: TEXTURE: [ID 9] Texture loaded successfully (512x512 | R8G8B8A8 | 1 mipmaps)
Jan 19 09:23:45 overlayd[869]: INFO: IMAGE: Data loaded successfully (727x202 | R8G8B8A8 | 1 mipmaps)
Jan 19 09:23:45 overlayd[869]: INFO: TEXTURE: [ID 10] Texture loaded successfully (727x202 | R8G8B8A8 | 1 mipmaps)

In the loop I've measured up the calls which essentially is about all in the loop (some overlay drawing of course but
that is little) :

Jan 19 09:24:19 overlayd[869]: gst_app_sink_try_pull_sample 15 us
Jan 19 09:24:19 overlayd[869]: gst_sample_get_buffer 0 us
Jan 19 09:24:19 overlayd[869]: gst_buffer_map 2 us
Jan 19 09:24:19 overlayd[869]: UpdateTexture 20952 us
Jan 19 09:24:19 overlayd[869]: DrawTextureEx 6 us

Currently running raylib configured as desktop platform and system runs wayland/weston in all three cases.
Perhaps this is due to Mali-400 OpenGL setup?
Loop is similar to https://github.com/saxofon/video-testing/blob/9459bf093abbf422525ec564b928398c33e10711/src/v4l2-gstreamer-raylib.c#L284
Is there a more efficient way perhaps to do video frames with overlays?

[saxofon]

This can be closed.

The original problem here is that the video frame is formatted YUYV and is converted to RGB at this step.
Was still running together with weston (as the xilinx petalinux sets things up) so getting rid of that one then
made it possible for a redesign of the stack, meaning:

1. changing from libmali to lima
2. changing xilinx drivers zynqmp_disp.c and zynqmp_kms.c so that the video layer is primary and graphics is overlay.
   originally it is vice verse which is a bit strange since hw seems to map better to this new layout as well, but I guess
   it can make sense for the desktop usecases...
3. shoot gstreamer pipeline directly to a kmssink instead of via appsink as my testapp originally did.
4. let the kmssink for video be the video layer/plane and raylib use the overlay layer/plane.
   Now we can play videos at realtime, high resolution and cpu sits at around 2.1% load for the simple overlay graphics :)
   So far quite little changes to src/platforms/rcore_drm.c but there will be a bit more incoming to support multiple displays.
   Best would be to be able to support this in one process and multithread it per display (would make it coexist nicely with
   gstreamer integrations) but seems a quicker path right now would be to put raylib into a process for each display.
   As a quick hack I'm using window title that is passed to InitWindow, which is not really used in DRM platform, to code
   the DRM card devpath, bus-id, connector-id and plane-id.