Skip to content

🧑‍🌾 multi_nested_message_instrospection_test failing consistently in nightly_linux_coverage #812

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Open
Crola1702 opened this issue Jun 14, 2024 · 11 comments
Open

🧑‍🌾 multi_nested_message_instrospection_test failing consistently in nightly_linux_coverage #812

Crola1702 opened this issue Jun 14, 2024 · 11 comments
Assignees
@ahcorde

Comments

@Crola1702
Copy link

Crola1702 commented Jun 14, 2024 •
edited
Loading

Bug report

Required Info:

  • Operating System:
    • Ubuntu Noble
  • Installation type:
    • Source
  • Version or commit hash:
    • Rolling and Jazzy

Steps to reproduce issue

  1. Run a build in nightly_linux_coverage
  2. See projectroot.test_multi_nested_message_introspection test fail

Additional information

Reference build: https://ci.ros2.org/job/nightly_linux_coverage/2426/

Test regressions:

Log output:

[ RUN      ] MultiNestedMessageIntrospectionTest/0.CanWriteTypeErasedMessage
/home/jenkins-agent/workspace/nightly_linux_coverage/ws/src/ros2/rosidl/rosidl_typesupport_introspection_tests/test/test_multi_nested_message_introspection.cpp:212: Failure
Expected equality of these values:
  item_reference
    Which is: 736-byte object <90-B1 3D-F2 2A-7F 00-00 90-00 00-00 2A-7F 00-00 F0-A7 01-BB 09-56 00-00 F0-A7 01-BB 09-56 00-00 00-00 00-01 0C-00 00-00 35-3C 2E-75 B6-AC 9F-D8 91-85 71-8B 00-00 A1-01 F0-FF FF-FF 00-00 00-01 ... 01-00 00-00 00-00 00-00 60-30 02-BB 09-56 00-00 09-00 00-00 00-00 00-00 0A-00 00-00 00-00 00-00 80-30 02-BB 09-56 00-00 09-00 00-00 00-00 00-00 0A-00 00-00 00-00 00-00 00-00 A1-01 00-00 00-00>
  message.array_of_arrays[i]
    Which is: 736-byte object <90-B1 3D-F2 2A-7F 00-00 90-B1 3D-F2 2A-7F 00-00 F0-A7 01-BB 09-56 00-00 F0-A7 01-BB 09-56 00-00 00-00 00-01 0C-00 00-00 35-3C 2E-75 B6-AC 9F-D8 91-85 71-8B 00-00 A1-01 F0-FF FF-FF 00-00 00-01 ... 01-00 00-00 00-00 00-00 00-C3 01-BB 09-56 00-00 09-00 00-00 00-00 00-00 0A-00 00-00 00-00 00-00 E0-C2 01-BB 09-56 00-00 09-00 00-00 00-00 00-00 0A-00 00-00 00-00 00-00 00-00 A1-01 10-00 00-00>

/home/jenkins-agent/workspace/nightly_linux_coverage/ws/src/ros2/rosidl/rosidl_typesupport_introspection_tests/test/test_multi_nested_message_introspection.cpp:256: Failure
Expected equality of these values:
  message
    Which is: 6864-byte object <90-B1 3D-F2 2A-7F 00-00 90-B1 3D-F2 2A-7F 00-00 F0-A7 01-BB 09-56 00-00 F0-A7 01-BB 09-56 00-00 00-00 00-01 0C-00 00-00 35-3C 2E-75 B6-AC 9F-D8 91-85 71-8B 00-00 A1-01 F0-FF FF-FF 00-00 00-01 ... 01-00 00-00 00-00 00-00 01-00 00-00 00-00 00-00 20-FA 01-BB 09-56 00-00 01-00 00-00 00-00 00-00 01-00 00-00 00-00 00-00 70-FF 01-BB 09-56 00-00 01-00 00-00 00-00 00-00 01-00 00-00 00-00 00-00>
  message_copy
    Which is: 6864-byte object <90-B1 3D-F2 2A-7F 00-00 90-00 00-00 2A-7F 00-00 F0-A7 01-BB 09-56 00-00 F0-A7 01-BB 09-56 00-00 00-00 00-01 0C-00 00-00 35-3C 2E-75 B6-AC 9F-D8 91-85 71-8B 00-00 A1-01 F0-FF FF-FF 00-00 00-01 ... 01-00 00-00 00-00 00-00 01-00 00-00 00-00 00-00 70-85 02-BB 09-56 00-00 01-00 00-00 00-00 00-00 01-00 00-00 00-00 00-00 10-8D 02-BB 09-56 00-00 01-00 00-00 00-00 00-00 01-00 00-00 00-00 00-00>

[  FAILED  ] MultiNestedMessageIntrospectionTest/0.CanWriteTypeErasedMessage, where TypeParam = rosidl_typesupport_introspection_tests__msg__MultiNested (1 ms)

Flakiness ratio:

job_name last_fail first_fail build_count failure_count failure_percentage
nightly_linux_coverage 2024-06-14 2024-05-30 15 15 100.0
nightly_win_rep 2024-06-13 2024-06-05 12 2 16.67
This was referenced Jun 14, 2024
Closed
Open
@fujitatomoya
Copy link
Contributor

/assign @ahcorde

@ahcorde
Copy link
Contributor

ahcorde commented Jul 17, 2024

I compiled ros2 rolling with the mixin coverage-gcc but I'm not able to reproduce this error locally

@Crola1702 @mjcarroll any ideas about how to reproduce this locally ?

@Blast545
Copy link
Contributor

Blast545 commented Aug 8, 2024

I don't think this is the most straightforward way to reproduce it, but I just installed the mixins and ran the same build and test commands that are run in CI, and was able to reproduce the error.

Those are the commands that I used, in case it helps:

colcon mixin add default https://raw.githubusercontent.com/colcon/colcon-mixin-repository/master/index.yaml
colcon mixin update default

colcon build --base-paths "src" --build-base "build" --install-base "install" --event-handlers console_cohesion+ console_package_list+ --cmake-args -DBUILD_TESTING=ON --no-warn-unused-cli -DCMAKE_BUILD_TYPE=Debug -DINSTALL_EXAMPLES=OFF -DSECURITY=ON -DAPPEND_PROJECT_NAME_TO_INCLUDEDIR=ON --packages-up-to action_msgs ament_index_cpp builtin_interfaces class_loader composition_interfaces console_bridge_vendor diagnostic_msgs fastcdr fastrtps foonathan_memory_vendor geometry_msgs libstatistics_collector libyaml_vendor lifecycle_msgs nav_msgs rcl rcl_action rcl_interfaces rcl_lifecycle rcl_logging_spdlog rcl_yaml_param_parser rclcpp rclcpp_action rclcpp_components rclcpp_lifecycle rcpputils rcutils rmw rmw_dds_common rmw_fastrtps_cpp rmw_fastrtps_shared_cpp rmw_implementation rosgraph_msgs rosidl_default_runtime rosidl_runtime_c rosidl_runtime_cpp rosidl_typesupport_c rosidl_typesupport_cpp rosidl_typesupport_fastrtps_c rosidl_typesupport_fastrtps_cpp rosidl_typesupport_interface rosidl_typesupport_introspection_c rosidl_typesupport_introspection_cpp spdlog_vendor statistics_msgs std_msgs std_srvs tracetools trajectory_msgs unique_identifier_msgs visualization_msgs interactive_markers launch_testing_ros message_filters ros2action ros2component ros2doctor ros2interface ros2lifecycle ros2lifecycle_test_fixtures ros2param ros2topic rosbag2_compression rosbag2_cpp rosbag2_storage rosbag2_storage_default_plugins rosbag2_test_common rosbag2_tests rosbag2_transport rosidl_generator_c rosidl_generator_cpp rosidl_generator_py rosidl_runtime_py rosidl_typesupport_introspection_tests test_cli test_cli_remapping test_communication test_launch_ros test_msgs test_quality_of_service test_rclcpp test_security test_tf2 test_tracetools tf2 tf2_bullet tf2_eigen tf2_geometry_msgs tf2_kdl tf2_msgs tf2_py tf2_ros tf2_sensor_msgs --ament-cmake-args -DCMAKE_CXX_FLAGS="${CMAKE_CXX_FLAGS} --coverage" -DCMAKE_C_FLAGS="${CMAKE_C_FLAGS} --coverage"

colcon test --base-paths "src" --build-base "build" --install-base "install" --pytest-with-coverage --event-handlers console_cohesion+ --retest-until-pass 2 --ctest-args -LE xfail --pytest-args -m "not xfail" --packages-up-to action_msgs ament_index_cpp builtin_interfaces class_loader composition_interfaces console_bridge_vendor diagnostic_msgs fastcdr fastrtps foonathan_memory_vendor geometry_msgs libstatistics_collector libyaml_vendor lifecycle_msgs nav_msgs rcl rcl_action rcl_interfaces rcl_lifecycle rcl_logging_spdlog rcl_yaml_param_parser rclcpp rclcpp_action rclcpp_components rclcpp_lifecycle rcpputils rcutils rmw rmw_dds_common rmw_fastrtps_cpp rmw_fastrtps_shared_cpp rmw_implementation rosgraph_msgs rosidl_default_runtime rosidl_runtime_c rosidl_runtime_cpp rosidl_typesupport_c rosidl_typesupport_cpp rosidl_typesupport_fastrtps_c rosidl_typesupport_fastrtps_cpp rosidl_typesupport_interface rosidl_typesupport_introspection_c rosidl_typesupport_introspection_cpp spdlog_vendor statistics_msgs std_msgs std_srvs tracetools trajectory_msgs unique_identifier_msgs visualization_msgs interactive_markers launch_testing_ros message_filters ros2action ros2component ros2doctor ros2interface ros2lifecycle ros2lifecycle_test_fixtures ros2param ros2topic rosbag2_compression rosbag2_cpp rosbag2_storage rosbag2_storage_default_plugins rosbag2_test_common rosbag2_tests rosbag2_transport rosidl_generator_c rosidl_generator_cpp rosidl_generator_py rosidl_runtime_py rosidl_typesupport_introspection_tests test_cli test_cli_remapping test_communication test_launch_ros test_msgs test_quality_of_service test_rclcpp test_security test_tf2 test_tracetools tf2 tf2_bullet tf2_eigen tf2_geometry_msgs tf2_kdl tf2_msgs tf2_py tf2_ros

@Blast545
Copy link
Contributor

Blast545 commented Aug 8, 2024

The errors is happening in rosidl_typesupport_introspection_tests, so I imagine this should suffice:

colcon mixin add default https://raw.githubusercontent.com/colcon/colcon-mixin-repository/master/index.yaml
colcon mixin update default

colcon build --base-paths "src" --build-base "build" --install-base "install" --event-handlers console_cohesion+ console_package_list+ --cmake-args -DBUILD_TESTING=ON --no-warn-unused-cli -DCMAKE_BUILD_TYPE=Debug -DINSTALL_EXAMPLES=OFF -DSECURITY=ON -DAPPEND_PROJECT_NAME_TO_INCLUDEDIR=ON --packages-up-to rosidl_typesupport_introspection_tests --ament-cmake-args -DCMAKE_CXX_FLAGS="${CMAKE_CXX_FLAGS} --coverage" -DCMAKE_C_FLAGS="${CMAKE_C_FLAGS} --coverage"

colcon test --base-paths "src" --build-base "build" --install-base "install" --pytest-with-coverage --event-handlers console_cohesion+ --retest-until-pass 2 --ctest-args -LE xfail --pytest-args -m "not xfail" --packages-up-to rosidl_typesupport_introspection_tests

@Blast545
Copy link
Contributor

🧑‍🌾 @ahcorde friendly ping

@Crola1702
Copy link
Author

Friendly ping @ahcorde

@Crola1702
Copy link
Author

Friendly ping @ahcorde 👀

@Blast545
Copy link
Contributor

👨‍🌾 I'm opening a PR to disable this test. It has been affecting two jobs for at least 10months now.

Blast545 added a commit that referenced this issue Mar 10, 2025
@Blast545
Disable test failing in coverage jobs, see #812
d8c80c0 
Signed-off-by: Jorge J. Perez <[email protected]>
@Blast545 Blast545 mentioned this issue Mar 10, 2025
Merged
@fujitatomoya
Copy link
Contributor

test failed on,

https://github.com/ros2/rosidl/blob/rolling/rosidl_typesupport_introspection_tests/include/rosidl_typesupport_introspection_tests/gtest/macros.hpp#L79

right after the deepcopy. maybe some data is not copied as expected? like shallow copy?

i am not familiar with those area, any thoughts?

@Crola1702
Copy link
Author

@ahcorde @mjcarroll any thoughts against #853?

@ros-discourse
Copy link

This issue has been mentioned on ROS Discourse. There might be relevant details there:

https://discourse.ros.org/t/ros-pmc-minutes-for-april-22-2025/43346/1

Blast545 added a commit that referenced this issue Apr 28, 2025
@Blast545
Disable test failing in coverage jobs, see #812 (#853)
f5bddb0 
Signed-off-by: Jorge J. Perez <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@ahcorde ahcorde

Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
5 participants
@ahcorde @Blast545 @fujitatomoya @ros-discourse @Crola1702