Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ROS Communication Testing: Add Tests for Topic Creation and Message Chain Verification #56

Merged
merged 18 commits into from
Dec 13, 2024

Conversation

Yunus-Selim
Copy link
Contributor

@Yunus-Selim Yunus-Selim commented Nov 19, 2024

closes #55

This code aims to test for the existence of topics and the existence of topic messages, to make sure there is communication between nodes. But it might have some bugs for the other config files. I pull request it for early analysis before the meeting.

And I have a few questions about the code:

  1. Do you want me to use something else like correctness of messages for the assertion instead of checking the existence of topic messages?
  2. Should I test for the RobotStatePublisher node for this task?
  3. Should I complete the TODOs in the test code? Do you think that they are necessary?

@luca-byte
Copy link
Member

  1. Yes, but in separate test files, also we may want to check the frequency of the messages on the topic. There is a Sampling frequency in the remote controller (remote_controller) 80ms and one for the telemetry/feedback 100Hz (PicoLowLevel)
  2. No
  3. Yes, by using the data from YAML files, harrdcoded.

Implement the Canbus in python to test feedback reception from the communication node. Also makes sense to test for failures

@Yunus-Selim Yunus-Selim self-assigned this Nov 26, 2024
@Yunus-Selim Yunus-Selim marked this pull request as ready for review November 29, 2024 11:58
Copy link
Member

@luca-byte luca-byte left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Excellent work!! 💯

There are some problems (namely the impossibility to run tests with colcon and the communication node tests) and I left some considerations about those.

Moreover I think that the tests/classes/functions could benefit from having some docstrings/comments that explain briefly the test objective and the method used to test it.

I also kindly ask you to write a more detailed description of the approach (values selected for the test, how are oracles derived, ...) on Outline, in the section I just created.

reseq_ros2/test/test_node_communication.py Outdated Show resolved Hide resolved
reseq_ros2/test/test_topic_correctness.py Outdated Show resolved Hide resolved
reseq_ros2/test/test_node_communication.py Outdated Show resolved Hide resolved
@luca-byte
Copy link
Member

I noticed that with the pytest mark the tests are visualized as only one test, thus making it difficult to see if one or more tests are failing.

Moreover I came to the conclusion that my suggestion to add the script was not a good idea for many reasons (simply because colcon does not allow to use the stdin, ...) so probably you will have to remove it (sorry 😁) however I still believe that we should skip tests when the interface is not available (and your approach to check the ip link output is the best onne in my opinion).

As per the first point however, I would put the test that require the CAN interface in separate files, so that it is clear that they were skipped even when merged in one test by launch_test in colcon

Regarding the socketcan shutdown I intended shutting it down at the end of the test, since now (with the new implementation, a failure to setup it blocks also tests which are independent from it. I meant something like

def test_with_canbus(...):
  # setup canbus
  ...
  canbus.shutdown(...)

or something similar, but if you agree on separating the tests depending on the CAN we can do as you did in these tests.

@luca-byte
Copy link
Member

Regarding documentation: 👌

Implement unittests for verifying topic creation and message chains
through various nodes (scaler, AGeVaR, communication, etc.). Ensures
that messages are transmitted and received correctly across the
topics and nodes. This includes subscriptions, publishing, and
feedback mechanisms.
- Implement tests to ensure the accuracy of messages published on various topics
- Validate message contents against expected values
- Check the correctness of the message frequency
- Add @unittest.skipUnless(stat[0], stat[1]) to relevant test functions
- Ensure tests are skipped if the CAN interface is not up
- Ensure CAN interface is set up correctly before running checks
- Ensure each problem is printed on a new line for clarity.
@Yunus-Selim Yunus-Selim force-pushed the 55-node-communication-test branch 2 times, most recently from 710af02 to 1d11543 Compare December 7, 2024 10:41
@Yunus-Selim Yunus-Selim force-pushed the 55-node-communication-test branch from 1d11543 to 94acf2f Compare December 7, 2024 18:37
Copy link
Member

@luca-byte luca-byte left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great job 💯

I noticed that between this PR and the ones testing the nodes, there is a lot of duplicated code (launch, ...). Would it be possible to merge together that code to improve readability and reduce duplicated code, maybe abstracting it in some classes/functions?

@luca-byte
Copy link
Member

I do not like much this approach because we are polluting the production code with code which is never executed by the robot (unlike launch utilities). Isn't it possible to include the file from the same folder (if flake complains in the inclusion you can silence it using noqa on the offending line)

After that I will approve the PR. Great job 🎯

@Yunus-Selim Yunus-Selim force-pushed the 55-node-communication-test branch 3 times, most recently from 1beb771 to 6a4d880 Compare December 11, 2024 20:05
Copy link
Member

@luca-byte luca-byte left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

In merging, the testing utilities of this PR can be merged with Yusuf's one?

@Yunus-Selim Yunus-Selim force-pushed the 55-node-communication-test branch from 6a4d880 to 8bcf777 Compare December 12, 2024 09:50
@Yunus-Selim
Copy link
Contributor Author

It can be merged without problem now.

Copy link
Member

@luca-byte luca-byte left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@luca-byte luca-byte merged commit 4071511 into main Dec 13, 2024
1 check passed
@luca-byte luca-byte deleted the 55-node-communication-test branch December 13, 2024 19:17
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
2 participants