dGPU not being used with libvirtio-vgpu

[jobs-git]

Host: Ubuntu 24.04
Guest: Win 10 (virtio-win-266-no gpu do driver + mvisor-win-vgpu driver installed)
Kernel: 6.8.0-51-generic
dGPU: Nvidia RTX 4070 - not connected to any monitor
iGPU: AMDGPU (Ryzen 7600) connected to monitor
Nvidia-driver: 550.127.05 (Vanilla) https://www.nvidia.com/en-us/drivers/details/233004/

Build step:

git clone https://gitlab.freedesktop.org/virgl/virglrenderer
git reset --hard 8df4cba170940dad9350a99900293adbcef39b6c
# apply patch for blob
cd virglrenderer
meson build --prefix=/usr && sudo ninja install -C build

git clone https://github.com/tenclass/mvisor 
cd mvisor
meson setup build -Dsdl=true -Dvgpu=true
meson compile -C build

I have an Nvidia 4070 RTX dGPU at /dev/dri/render129 on pcie slot. Host monitor is connected iGPU AMDGPU from Ryzen 7600. Virglrenderer has been patch and installed based on https://github.com/tenclass/mvisor-win-vgpu-driver readme.

Below is the configuration that I use:

name: Default configuration
base: q35.yaml

machine:
  memory: 4G
  vcpu: 4
  # Set vcpu thread priority value [-20, 19]
  # A higher value means a lower priority
  priority: 1
  # Turn on BIOS output and performance measurement
  debug: No
  # Turn on hypervisor to lower CPU usage (Hyper-V is used for Windows)
  hypervisor: Yes

objects:
  - name: cmos
    # gmtime for linux, localtime for windows
    rtc: localtime

  - class: qxl
  - class: spice-agent
  - class: usb-tablet

  - class: virtio-network
    mac: 00:50:00:11:22:33
    map: tcp:0.0.0.0:8022-:22

  - class: ata-cdrom 
    image: /home/vgpu-guest-driver-build-master.iso
  
  - class: ata-cdrom
    image: /home/virtio-win-0.1.266.iso

  - class: ata-disk
    image: /home/win10gl.qcow2
    snapshot: No

  - class: virtio-vgpu
    memory: 1G
    staging: yes
    blob: Yes
    node: /dev/dri/renderD129

I was suspecting it needs prime so I did:

DRI_PRIME=1 ./build/mvisor -c config/sample.yaml -vnc 5900

But I get this error:

failed to load driver: nouveau

[nooodles2023]

Try to delete " node: /dev/dri/renderD129"
Let mvisor choose its render node

[jobs-git]

Tried it, but no luck? Do we need vGPU unlock for consumer GPU?

[nooodles2023]

Strange, I haven't met your problem before.
If the host gpu support opengl well, it should work.
By the way, the vm need to get 12G memory at least to work properly.

[jobs-git]

Is there a possibility if you could also add in the Readme a table where it was tested? So far, I saw T4 which are enterprise GPUs.

[nooodles2023]

I have tested it on nvidia 1070ti/t4/3090, also tested on amd gpu but I forgot the gpu name

[nooodles2023]

glxinfo | grep OpenGL
post the output of this command here

[jobs-git]

glxinfo | grep OpenGL post the output of this command here

This lead me to a bit in right direction, now I mvisor is using the dGPU, but I need to do this:

__NV_PRIME_RENDER_OFFLOAD=1 __GLX_VENDOR_LIBRARY_NAME=nvidia ./build/mvisor -c config/sample.yaml -vnc 5900

this command:

__NV_PRIME_RENDER_OFFLOAD=1 __GLX_VENDOR_LIBRARY_NAME=nvidia glxinfo | grep OpenGL

shows this:

OpenGL vendor string: NVIDIA Corporation
OpenGL renderer string: NVIDIA GeForce RTX 4070 SUPER/PCIe/SSE2
OpenGL core profile version string: 4.6.0 NVIDIA 550.127.05
OpenGL core profile shading language version string: 4.60 NVIDIA
OpenGL core profile context flags: (none)
OpenGL core profile profile mask: core profile
OpenGL core profile extensions:
OpenGL version string: 4.6.0 NVIDIA 550.127.05
OpenGL shading language version string: 4.60 NVIDIA
OpenGL context flags: (none)
OpenGL profile mask: (none)
OpenGL extensions:
OpenGL ES profile version string: OpenGL ES 3.2 NVIDIA 550.127.05
OpenGL ES profile shading language version string: OpenGL ES GLSL ES 3.20
OpenGL ES profile extensions:

but Guest OS is still laggy, and GPU is not being shown in the task manager or device manager

[nooodles2023]

__NV_PRIME_RENDER_OFFLOAD

The VGPU should be seen in device manager, but it wouldn't display in task manager.