High Frequency data is buffered at 100Hz

[ahmadSum1]

The high-frequency topics are being buffered/clustered at 100hz.
here is an example with an unmodified driver with the provided launch file:

• /imu/acceleration is about 55hz
• /imu/time_ref was fluctuating around 500~650hz on topic monitor but being clustered(buffered) instead of being evenly spread
  

Platform:

• Ubuntu 22.04
• Device: MTi-3-8A7G6

Bash dump

  ~/ros2_ws ❯ ros2 launch xsens_mti_ros2_driver xsens_mti_node.launch.py                                                                                                                          04:31:54 PM
[INFO] [launch]: All log files can be found below /home/cniuser/.ros/log/2024-08-16-16-32-45-400653-cnidesktop1-217246
[INFO] [launch]: Default logging verbosity is set to INFO
[INFO] [xsens_mti_node-1]: process started with pid [217249]
[xsens_mti_node-1] [INFO] [1723818765.470838952] [xsens_mti_node]: Creating XsControl object...
[xsens_mti_node-1] [INFO] [1723818765.478633880] [xsens_mti_node]: Scanning for devices...
[xsens_mti_node-1] [INFO] [1723818765.837507809] [xsens_mti_node]: Found a device with ID: 03887B6D @ port: /dev/ttyUSB0, baudrate: 921600
[xsens_mti_node-1] [INFO] [1723818765.837561029] [xsens_mti_node]: Opening port /dev/ttyUSB0 ...
[xsens_mti_node-1] [INFO] [1723818765.992350192] [xsens_mti_node]: Device: MTi-3-8A7G6, with ID: 03887B6D opened.
[xsens_mti_node-1] [INFO] [1723818766.045103005] [xsens_mti_node]: Measuring ...

[xsenssupport]

Hi,
We just updated the ROS2 driver to add the High Rate topic, so you could check the ros2 topic imu/acceleration_hr and imu/angular_velocity_hr for the 1000Hz high rate data, while for the 100Hz calibrated data, you could check the imu/data or imu/acceleration or imu/angular_velocity.
Note that if you had selected too many data output types, you need to increase your baudrate to a much higher value, for example 2M, otherwise there will be a lot of data loss.(You could see the warning messages in this ros2 driver, and also in the message window in the MT Manager: Data Overflow.)

Also considering direct serial connection for high data rate and real time performance, for example direct UART connection to the UART pins of a RaspberryPi/Jetson devices.

Also notice that there might be some bug in the ros2 topic hz, while calculating the data rate, since we checked that there is no data loss when only selecting output acceleration_hr and angular_velocity_hr, but the data rate given by the ros2 topic hz is incorrect for the 1000Hz or 2000Hz(MTI-680) data output.

If you only use USB connection, consider use:

setserial /dev/ttyUSB0 low_latency

[ahmadSum1]

Hi @xsenssupport , Thanks for the quick response. I have already tried most of what you said but in the end, I found that there is a hardcoded read buffer here:

Xsens_MTi_ROS_Driver_and_Ntrip_Client/src/xsens_mti_ros2_driver/lib/xspublic/xscontroller/serialcommunicator.cpp

Line 365 in f21f1ba

const int maxSz = 8192;

changed to

const int maxSz = XS_DEFAULT_READ_BUFFER_SIZE;

Then I had to significantly reduce the buffer size.

Xsens_MTi_ROS_Driver_and_Ntrip_Client/src/xsens_ros_mti_driver/lib/xspublic/xscontroller/iointerface.h

Line 48 in 7e2a47d

#define XS_DEFAULT_READ_BUFFER_SIZE (XS_MAXMSGLEN*8)

Here is what it looks like with a buffer size 50bytes:

I guess the missing frames are due to buffer size misalignment.!?

[ahmadSum1]

setserial /dev/ttyUSB0 low_latency

this essentially solves the issue upto 500hz without fiddling around with source code.

But at 1000hz, things are again buffered at around 500hz

[ahmadSum1]

As I understand, the large buffer size in the xsense driver is not an issue if the Linux serial port timer interrupt or timeout force triggers a data parsing in time. Seems like, the default is set at 10ms, hence the clustering behavior around 100hz.

Xsens_MTi_ROS_Driver_and_Ntrip_Client/src/xsens_ros_mti_driver/lib/xspublic/xscontroller/iointerface.h

Line 92 in 7e2a47d

\returns XRV_OK if all data was read successfully, XRV_TIMEOUT if some data was read, but not

However,

If you only use USB connection, consider use:

setserial /dev/ttyUSB0 low_latency

this seems to only set this Linux mysterious timeout to be ~2ms.

[xsenssupport]

As I understand, the large buffer size in the xsense driver is not an issue if the Linux serial port timer interrupt or timeout force triggers a data parsing in time. Seems like, the default is set at 10ms, hence the clustering behavior around 100hz.

Xsens_MTi_ROS_Driver_and_Ntrip_Client/src/xsens_ros_mti_driver/lib/xspublic/xscontroller/iointerface.h

Line 92 in 7e2a47d

\returns XRV_OK if all data was read successfully, XRV_TIMEOUT if some data was read, but not

However,

If you only use USB connection, consider use:

setserial /dev/ttyUSB0 low_latency

this seems to only set this Linux mysterious timeout to be ~2ms.

As mentioned, it is not a problem of Xsens ROS Driver or Xspublic library, it is a problem of ros2 topic hz, you could simply google ros2 topic hz issue, and you could find a lot of results, including this one.
You could verify the data rate by using the MT SDK's xda_public_cpp.

[ahmadSum1]

Yes and No.

it is a problem of ros2 topic hz

It is not an issue related to ROS2 topic hz.
It relates to Linux tick latency and how this ROS driver reads the data.

Solutions That I have tested so far (without having to patch the Linux kernel):

1. Using USB low latency mode (works great up to 500hz)

   setserial /dev/ttyUSB0 low_latency

2. • Fix the hardcoded buffer size and then
   • reduce XS_DEFAULT_READ_BUFFER_SIZE to 40~50 bytes,
     This forces data parsing and publishing to be faster. (tested with 1Khz, has more missing frames.)

[xsenssupport]

Hi,

I tested in with Raspberry Pi 5B, ubuntu 24.04, before using the set setserial command, the ros2 topic hz gives 683Hz, after using the setserial command, the data rate gives 1000Hz:

So there is no need to change any of the xspublic code, closing this issue.

[ahmadSum1]

Hello,
I am afraid, you have missed the real issue here.

I tested in with Raspberry Pi 5B, ubuntu 24.04, before using the set setserial command, the ros2 topic hz gives 683Hz, after using the setserial command, the data rate gives 1000Hz:

Could you please confirm if you checked the actual timing of the individual messages?
The messages are buffered. So, even though the average frequency is 1000hz, the real data is clustered around 500hz(with low latency) or 100hz(by default).