Skip to content

Conversation

@fepitre
Copy link
Member

@fepitre fepitre commented Nov 23, 2024

It prepares the work for RemoteVM.

@fepitre
Copy link
Member Author

fepitre commented Nov 23, 2024

@marmarek can you run openQA on it please?

@marmarek
Copy link
Member

Sure, I can, but I think you may want to first take care of unit tests, this one looks like a real issue:

  File "/home/user/builds/QubesOS/qubes-core-admin/qubes/vm/__init__.py", line 403, in load_extras
    for parent in self.xml.xpath("./devices"):
                  ^^^^^^^^^^^^^^
AttributeError: 'NoneType' object has no attribute 'xpath'

@fepitre fepitre force-pushed the localvm branch 3 times, most recently from 0e2e8d9 to 9b8407f Compare November 23, 2024 13:57
@codecov
Copy link

codecov bot commented Nov 25, 2024

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 70.24%. Comparing base (a16975e) to head (755e7cf).
Report is 5 commits behind head on main.

Additional details and impacted files
@@            Coverage Diff             @@
##             main     #637      +/-   ##
==========================================
+ Coverage   70.14%   70.24%   +0.09%     
==========================================
  Files          59       59              
  Lines       12726    12739      +13     
==========================================
+ Hits         8927     8948      +21     
+ Misses       3799     3791       -8     
Flag Coverage Δ
unittests 70.24% <100.00%> (+0.09%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@fepitre
Copy link
Member Author

fepitre commented Nov 25, 2024

Sure, I can, but I think you may want to first take care of unit tests, this one looks like a real issue:

  File "/home/user/builds/QubesOS/qubes-core-admin/qubes/vm/__init__.py", line 403, in load_extras
    for parent in self.xml.xpath("./devices"):
                  ^^^^^^^^^^^^^^
AttributeError: 'NoneType' object has no attribute 'xpath'

Fixed.

@qubesos-bot
Copy link

qubesos-bot commented Dec 3, 2024

OpenQA test summary

Complete test suite and dependencies: https://openqa.qubes-os.org/tests/overview?distri=qubesos&version=4.3&build=2025032603-4.3&flavor=pull-requests

Test run included the following:

New failures, excluding unstable

Compared to: https://openqa.qubes-os.org/tests/overview?distri=qubesos&version=4.3&build=2025031804-4.3&flavor=update

  • system_tests_dispvm

    • TC_20_DispVM_debian-12-xfce: test_011_dvm_run_preload (error)
      AttributeError: 'TC_20_DispVM_debian-12-xfce' object has no attribu...

    • TC_20_DispVM_debian-12-xfce: test_011_dvm_run_preload_next (error)
      AttributeError: 'TC_20_DispVM_debian-12-xfce' object has no attribu...

    • TC_20_DispVM_debian-12-xfce: test_011_dvm_run_preload_nogui (error)
      AttributeError: 'TC_20_DispVM_debian-12-xfce' object has no attribu...

    • TC_20_DispVM_fedora-41-xfce: test_011_dvm_run_preload (error)
      AttributeError: 'TC_20_DispVM_fedora-41-xfce' object has no attribu...

    • TC_20_DispVM_fedora-41-xfce: test_011_dvm_run_preload_next (error)
      AttributeError: 'TC_20_DispVM_fedora-41-xfce' object has no attribu...

    • TC_20_DispVM_fedora-41-xfce: test_011_dvm_run_preload_nogui (error)
      AttributeError: 'TC_20_DispVM_fedora-41-xfce' object has no attribu...

    • TC_20_DispVM_whonix-workstation-17: test_011_dvm_run_preload (error)
      AttributeError: 'TC_20_DispVM_whonix-workstation-17' object has no ...

    • TC_20_DispVM_whonix-workstation-17: test_011_dvm_run_preload_next (error)
      AttributeError: 'TC_20_DispVM_whonix-workstation-17' object has no ...

    • TC_20_DispVM_whonix-workstation-17: test_011_dvm_run_preload_nogui (error)
      AttributeError: 'TC_20_DispVM_whonix-workstation-17' object has no ...

  • system_tests_audio

  • system_tests_qwt_win10_seamless@hw13

    • windows_clipboard_and_filecopy: unnamed test (unknown)
    • windows_clipboard_and_filecopy: Failed (test died)
      # Test died: no candidate needle with tag(s) 'windows-Explorer-empt...
  • system_tests_qwt_win11@hw13

    • windows_install: wait_serial (wait serial expected)
      # wait_serial expected: qr/dcWzE-\d+-/...

    • windows_install: Failed (test died + timed out)
      # Test died: command 'script -e -c 'bash -x /usr/bin/qvm-create-win...

  • system_tests_basic_vm_qrexec_gui@hw7

  • system_tests_network_updates

  • system_tests_kde_gui_interactive

    • gui_keyboard_layout: wait_serial (wait serial expected)
      # wait_serial expected: "echo -e '[Layout]\nLayoutList=us,de' | sud...

    • gui_keyboard_layout: Failed (test died)
      # Test died: command 'test "$(cd ~user;ls e1*)" = "$(qvm-run -p wor...

Failed tests

22 failures
  • system_tests_dispvm

    • TC_20_DispVM_debian-12-xfce: test_011_dvm_run_preload (error)
      AttributeError: 'TC_20_DispVM_debian-12-xfce' object has no attribu...

    • TC_20_DispVM_debian-12-xfce: test_011_dvm_run_preload_next (error)
      AttributeError: 'TC_20_DispVM_debian-12-xfce' object has no attribu...

    • TC_20_DispVM_debian-12-xfce: test_011_dvm_run_preload_nogui (error)
      AttributeError: 'TC_20_DispVM_debian-12-xfce' object has no attribu...

    • TC_20_DispVM_fedora-41-xfce: test_011_dvm_run_preload (error)
      AttributeError: 'TC_20_DispVM_fedora-41-xfce' object has no attribu...

    • TC_20_DispVM_fedora-41-xfce: test_011_dvm_run_preload_next (error)
      AttributeError: 'TC_20_DispVM_fedora-41-xfce' object has no attribu...

    • TC_20_DispVM_fedora-41-xfce: test_011_dvm_run_preload_nogui (error)
      AttributeError: 'TC_20_DispVM_fedora-41-xfce' object has no attribu...

    • TC_20_DispVM_whonix-workstation-17: test_011_dvm_run_preload (error)
      AttributeError: 'TC_20_DispVM_whonix-workstation-17' object has no ...

    • TC_20_DispVM_whonix-workstation-17: test_011_dvm_run_preload_next (error)
      AttributeError: 'TC_20_DispVM_whonix-workstation-17' object has no ...

    • TC_20_DispVM_whonix-workstation-17: test_011_dvm_run_preload_nogui (error)
      AttributeError: 'TC_20_DispVM_whonix-workstation-17' object has no ...

  • system_tests_audio

  • system_tests_qwt_win10_seamless@hw13

    • windows_clipboard_and_filecopy: unnamed test (unknown)
    • windows_clipboard_and_filecopy: Failed (test died)
      # Test died: no candidate needle with tag(s) 'windows-Explorer-empt...
  • system_tests_qwt_win11@hw13

    • windows_install: wait_serial (wait serial expected)
      # wait_serial expected: qr/dcWzE-\d+-/...

    • windows_install: Failed (test died + timed out)
      # Test died: command 'script -e -c 'bash -x /usr/bin/qvm-create-win...

  • system_tests_basic_vm_qrexec_gui@hw7

  • system_tests_network_updates

  • system_tests_kde_gui_interactive

    • gui_keyboard_layout: wait_serial (wait serial expected)
      # wait_serial expected: "echo -e '[Layout]\nLayoutList=us,de' | sud...

    • gui_keyboard_layout: Failed (test died)
      # Test died: command 'test "$(cd ~user;ls e1*)" = "$(qvm-run -p wor...

Fixed failures

Compared to: https://openqa.qubes-os.org/tests/132953#dependencies

14 fixed
  • system_tests_whonix

    • whonixcheck: fail (unknown)
      Whonixcheck for sys-whonix failed...

    • whonixcheck: unnamed test (unknown)

  • system_tests_suspend

    • suspend: unnamed test (unknown)
    • suspend: Failed (test died)
      # Test died: no candidate needle with tag(s) 'SUSPEND-FAILED' match...
  • system_tests_basic_vm_qrexec_gui

  • system_tests_qrexec

  • system_tests_audio

  • system_tests_whonix@hw7

    • whonixcheck: fail (unknown)
      Whonixcheck for sys-whonix failed...

    • whonixcheck: unnamed test (unknown)

  • system_tests_guivm_vnc_gui_interactive

    • gui_filecopy: unnamed test (unknown)
    • gui_filecopy: Failed (test died)
      # Test died: no candidate needle with tag(s) 'files-work' matched...
  • system_tests_kde_gui_interactive

    • clipboard_and_web: unnamed test (unknown)

    • clipboard_and_web: Failed (test died)
      # Test died: no candidate needle with tag(s) 'qubes-website' matche...

    • clipboard_and_web: wait_serial (wait serial expected)
      # wait_serial expected: "lspci; echo 2E8vz-\$?-"...

Unstable tests

Performance Tests

Performance degradation:

13 performance degradations
  • whonix-gateway-17_socket: 8.80 :small_red_triangle_up: ( previous job: 7.24, degradation: 121.58%)
  • whonix-gateway-17_socket-root: 8.64 :small_red_triangle_up: ( previous job: 7.65, degradation: 112.91%)
  • whonix-gateway-17_exec-data-duplex-root: 99.31 :small_red_triangle_up: ( previous job: 88.52, degradation: 112.19%)
  • dom0_root_seq1m_q8t1_read 3:read_bandwidth_kb: 240499.00 :small_red_triangle_up: ( previous job: 446963.00, degradation: 53.81%)
  • dom0_root_seq1m_q8t1_write 3:write_bandwidth_kb: 79953.00 :small_red_triangle_up: ( previous job: 129298.00, degradation: 61.84%)
  • dom0_root_seq1m_q1t1_read 3:read_bandwidth_kb: 92757.00 :small_red_triangle_up: ( previous job: 294295.00, degradation: 31.52%)
  • dom0_root_seq1m_q1t1_write 3:write_bandwidth_kb: 46825.00 :small_red_triangle_up: ( previous job: 95454.00, degradation: 49.06%)
  • dom0_varlibqubes_rnd4k_q32t1_write 3:write_bandwidth_kb: 5673.00 :small_red_triangle_up: ( previous job: 6479.00, degradation: 87.56%)
  • dom0_varlibqubes_rnd4k_q1t1_write 3:write_bandwidth_kb: 3130.00 :small_red_triangle_up: ( previous job: 4903.00, degradation: 63.84%)
  • fedora-41-xfce_root_seq1m_q1t1_write 3:write_bandwidth_kb: 46591.00 :small_red_triangle_up: ( previous job: 87940.00, degradation: 52.98%)
  • fedora-41-xfce_root_rnd4k_q32t1_write 3:write_bandwidth_kb: 2973.00 :small_red_triangle_up: ( previous job: 3599.00, degradation: 82.61%)
  • fedora-41-xfce_volatile_seq1m_q8t1_write 3:write_bandwidth_kb: 88325.00 :small_red_triangle_up: ( previous job: 179949.00, degradation: 49.08%)
  • fedora-41-xfce_volatile_rnd4k_q32t1_write 3:write_bandwidth_kb: 3041.00 :small_red_triangle_up: ( previous job: 5672.00, degradation: 53.61%)

Remaining performance tests:

59 tests
  • debian-12-xfce_exec: 6.35 🟢 ( previous job: 7.12, improvement: 89.22%)
  • debian-12-xfce_exec-root: 28.85 :small_red_triangle_up: ( previous job: 28.65, degradation: 100.67%)
  • debian-12-xfce_socket: 8.17 🟢 ( previous job: 8.60, improvement: 95.04%)
  • debian-12-xfce_socket-root: 8.74 :small_red_triangle_up: ( previous job: 8.52, degradation: 102.58%)
  • debian-12-xfce_exec-data-simplex: 71.91 :small_red_triangle_up: ( previous job: 71.62, degradation: 100.41%)
  • debian-12-xfce_exec-data-duplex: 73.12 :small_red_triangle_up: ( previous job: 70.34, degradation: 103.95%)
  • debian-12-xfce_exec-data-duplex-root: 85.09 :small_red_triangle_up: ( previous job: 82.72, degradation: 102.87%)
  • debian-12-xfce_socket-data-duplex: 158.23 :small_red_triangle_up: ( previous job: 156.96, degradation: 100.81%)
  • fedora-41-xfce_exec: 9.18 🟢 ( previous job: 9.27, improvement: 99.11%)
  • fedora-41-xfce_exec-root: 61.04 🟢 ( previous job: 61.51, improvement: 99.24%)
  • fedora-41-xfce_socket: 8.50 🟢 ( previous job: 8.63, improvement: 98.52%)
  • fedora-41-xfce_socket-root: 8.59 🟢 ( previous job: 8.71, improvement: 98.70%)
  • fedora-41-xfce_exec-data-simplex: 61.92 🟢 ( previous job: 75.53, improvement: 81.98%)
  • fedora-41-xfce_exec-data-duplex: 65.16 🟢 ( previous job: 71.56, improvement: 91.06%)
  • fedora-41-xfce_exec-data-duplex-root: 112.09 :small_red_triangle_up: ( previous job: 109.13, degradation: 102.72%)
  • fedora-41-xfce_socket-data-duplex: 139.35 🟢 ( previous job: 150.61, improvement: 92.52%)
  • whonix-gateway-17_exec: 7.17 :small_red_triangle_up: ( previous job: 6.82, degradation: 105.15%)
  • whonix-gateway-17_exec-root: 41.69 :small_red_triangle_up: ( previous job: 40.43, degradation: 103.12%)
  • whonix-gateway-17_exec-data-simplex: 74.39 🟢 ( previous job: 78.32, improvement: 94.98%)
  • whonix-gateway-17_exec-data-duplex: 73.33 🟢 ( previous job: 76.65, improvement: 95.66%)
  • whonix-gateway-17_socket-data-duplex: 171.09 🟢 ( previous job: 171.76, improvement: 99.61%)
  • whonix-workstation-17_exec: 8.34 :small_red_triangle_up: ( previous job: 7.67, degradation: 108.79%)
  • whonix-workstation-17_exec-root: 57.67 🟢 ( previous job: 58.26, improvement: 98.97%)
  • whonix-workstation-17_socket: 7.52 🟢 ( previous job: 8.19, improvement: 91.81%)
  • whonix-workstation-17_socket-root: 8.12 🟢 ( previous job: 8.13, improvement: 99.94%)
  • whonix-workstation-17_exec-data-simplex: 75.93 :small_red_triangle_up: ( previous job: 74.99, degradation: 101.24%)
  • whonix-workstation-17_exec-data-duplex: 74.89 :small_red_triangle_up: ( previous job: 72.71, degradation: 103.00%)
  • whonix-workstation-17_exec-data-duplex-root: 102.71 :small_red_triangle_up: ( previous job: 99.82, degradation: 102.89%)
  • whonix-workstation-17_socket-data-duplex: 160.81 🟢 ( previous job: 169.50, improvement: 94.87%)
  • dom0_root_rnd4k_q32t1_read 3:read_bandwidth_kb: 98023.00 :green_circle: ( previous job: 79803.00, improvement: 122.83%)
  • dom0_root_rnd4k_q32t1_write 3:write_bandwidth_kb: 9664.00 :green_circle: ( previous job: 6149.00, improvement: 157.16%)
  • dom0_root_rnd4k_q1t1_read 3:read_bandwidth_kb: 11344.00 :green_circle: ( previous job: 10795.00, improvement: 105.09%)
  • dom0_root_rnd4k_q1t1_write 3:write_bandwidth_kb: 4746.00 :small_red_triangle_up: ( previous job: 4826.00, degradation: 98.34%)
  • dom0_varlibqubes_seq1m_q8t1_read 3:read_bandwidth_kb: 465619.00 :green_circle: ( previous job: 382273.00, improvement: 121.80%)
  • dom0_varlibqubes_seq1m_q8t1_write 3:write_bandwidth_kb: 285171.00 :green_circle: ( previous job: 250795.00, improvement: 113.71%)
  • dom0_varlibqubes_seq1m_q1t1_read 3:read_bandwidth_kb: 435998.00 :small_red_triangle_up: ( previous job: 437636.00, degradation: 99.63%)
  • dom0_varlibqubes_seq1m_q1t1_write 3:write_bandwidth_kb: 188206.00 :green_circle: ( previous job: 184752.00, improvement: 101.87%)
  • dom0_varlibqubes_rnd4k_q32t1_read 3:read_bandwidth_kb: 104030.00 :green_circle: ( previous job: 62195.00, improvement: 167.26%)
  • dom0_varlibqubes_rnd4k_q1t1_read 3:read_bandwidth_kb: 8278.00 :green_circle: ( previous job: 7669.00, improvement: 107.94%)
  • fedora-41-xfce_root_seq1m_q8t1_read 3:read_bandwidth_kb: 365357.00 :small_red_triangle_up: ( previous job: 368309.00, degradation: 99.20%)
  • fedora-41-xfce_root_seq1m_q8t1_write 3:write_bandwidth_kb: 163406.00 :green_circle: ( previous job: 162081.00, improvement: 100.82%)
  • fedora-41-xfce_root_seq1m_q1t1_read 3:read_bandwidth_kb: 299850.00 :small_red_triangle_up: ( previous job: 318716.00, degradation: 94.08%)
  • fedora-41-xfce_root_rnd4k_q32t1_read 3:read_bandwidth_kb: 87779.00 :green_circle: ( previous job: 82694.00, improvement: 106.15%)
  • fedora-41-xfce_root_rnd4k_q1t1_read 3:read_bandwidth_kb: 8827.00 :green_circle: ( previous job: 8485.00, improvement: 104.03%)
  • fedora-41-xfce_root_rnd4k_q1t1_write 3:write_bandwidth_kb: 1449.00 :green_circle: ( previous job: 542.00, improvement: 267.34%)
  • fedora-41-xfce_private_seq1m_q8t1_read 3:read_bandwidth_kb: 394943.00 :green_circle: ( previous job: 373957.00, improvement: 105.61%)
  • fedora-41-xfce_private_seq1m_q8t1_write 3:write_bandwidth_kb: 205730.00 :green_circle: ( previous job: 170062.00, improvement: 120.97%)
  • fedora-41-xfce_private_seq1m_q1t1_read 3:read_bandwidth_kb: 356537.00 :green_circle: ( previous job: 334687.00, improvement: 106.53%)
  • fedora-41-xfce_private_seq1m_q1t1_write 3:write_bandwidth_kb: 89114.00 :green_circle: ( previous job: 61534.00, improvement: 144.82%)
  • fedora-41-xfce_private_rnd4k_q32t1_read 3:read_bandwidth_kb: 74992.00 :small_red_triangle_up: ( previous job: 80283.00, degradation: 93.41%)
  • fedora-41-xfce_private_rnd4k_q32t1_write 3:write_bandwidth_kb: 3845.00 :green_circle: ( previous job: 2215.00, improvement: 173.59%)
  • fedora-41-xfce_private_rnd4k_q1t1_read 3:read_bandwidth_kb: 8208.00 :green_circle: ( previous job: 7540.00, improvement: 108.86%)
  • fedora-41-xfce_private_rnd4k_q1t1_write 3:write_bandwidth_kb: 1028.00 :small_red_triangle_up: ( previous job: 1130.00, degradation: 90.97%)
  • fedora-41-xfce_volatile_seq1m_q8t1_read 3:read_bandwidth_kb: 399305.00 :green_circle: ( previous job: 369868.00, improvement: 107.96%)
  • fedora-41-xfce_volatile_seq1m_q1t1_read 3:read_bandwidth_kb: 296040.00 :small_red_triangle_up: ( previous job: 324737.00, degradation: 91.16%)
  • fedora-41-xfce_volatile_seq1m_q1t1_write 3:write_bandwidth_kb: 72150.00 :green_circle: ( previous job: 17567.00, improvement: 410.71%)
  • fedora-41-xfce_volatile_rnd4k_q32t1_read 3:read_bandwidth_kb: 84891.00 :green_circle: ( previous job: 79021.00, improvement: 107.43%)
  • fedora-41-xfce_volatile_rnd4k_q1t1_read 3:read_bandwidth_kb: 8551.00 :green_circle: ( previous job: 7867.00, improvement: 108.69%)
  • fedora-41-xfce_volatile_rnd4k_q1t1_write 3:write_bandwidth_kb: 1758.00 :small_red_triangle_up: ( previous job: 1953.00, degradation: 90.02%)

`domain-init` and `domain-load` events.
"""

# pylint: disable=no-member
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why? it is defined in __init__

import qubesdb # pylint: disable=import-error

try:
# pylint: disable=no-member
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

same here

class.
"""
# cleanup old watch connection first, if any
# pylint: disable=no-member
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

and here

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't remember, I double checked, now it's good locally. I removed the three disable.

)
)
if not issubclass(vmclass, BaseVM):
if not issubclass(vmclass, LocalVM):
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

While default to LocalVM makes sense, should it really forbid using RemoteVM as some property value?


def __lt__(self, other: object):
if not isinstance(other, BaseVM):
if not isinstance(other, LocalVM):
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This should remain BaseVM, otherwise sorting VM objects will not work with RemoteVM

@marmarek
Copy link
Member

PipelineRetry

@marmarek marmarek merged commit 8a93ad3 into QubesOS:main Mar 27, 2025
4 of 5 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants