Point cloud frequency issue on realsense D435 #2923
Comments
|
Hi @TopRamen1 Depending on the capabilities of the particular computer, having depth, color and both infrared topics enabled simultaneously could be a processing burden on the computer's CPU. If you do not require the infra1 and infra2 topics that are being published (as the pointcloud is not using them) then you can reduce the CPU burden by disabling the infrared topics. Please try adding -p enable_infra1:=false -p enable_infra2:=false to your ros2 run launch instruction. |
|
Hi, thanks for a quick response. My actual desired setup is as follows: ros2 run realsense2_camera realsense2_camera_node --ros-args -p pointcloud.enable:=true -p enable_color:=false -p enable_infra1:=false -p enable_infra2:=false -p enable_infra:=false -p depth_module.profile:=848x480x6 In this case, I disabled both infra and color images since I'm interested only in pointcloud and depth. In this case, the outcome is even worse: It's strange for me because ROS 1 driver with the same cameras is able publish on steady 6 Hz on analogical settings. Is this a known issue for the driver to have such publishing frequency differences? |
|
There can be FPS drops or other issues sometimes when the pointcloud is enabled. Is the FPS 30 if you do not enable the pointcloud, please? |
|
I have chacked the frequency with just the ros run command without any arguments ros2 run realsense2_camera realsense2_camera_node Output ros2 topic hz /camera/camera/color/image_raw
The FPS is still not 30 |
|
Let's try disabling an option called auto_exposure_priority, as doing so can force the FPS to be constant for both depth and color instead of being permitted to vary.
|
|
The auto_exposure_priority did not improve the FPS when i ran with and without the point cloud enabled |
|
How does the FPS perform when the pointcloud is enabled with ros2 launch instead of ros2 run?
|
|
When i run with the ros2 launch command you have provided the fps is even more unstable. I managed to see 30 FPS shortly after the start of the driver but is declined sharply: average rate: 29.941 The other time I tried with the same command it started at barely 1 FPS and then climbed slowly up: average rate: 0.283 |
|
I'm running the driver from a ros humble container on an Intel NUC with ubuntu 20.04 installed with kernel: 5.13.0-28-generic Could this information help you? |
|
What happens to the FPS if you put your hand over the front of the camera and hold it there? If the FPS recovers to 30 then it could suggest that something in the environment that the camera is located in is causing the exposure to change incorrectly and therefore making the FPS change too (exposure value and FPS decrease / increase are connected). |
|
Yes indeed the rate becomes stable when I cover up the camera: I launched this command with the camera covered and then uncovered it during the test: Results: average rate: 29.511 As you can see I got stable 29 FPS at the start and then the frequency started to decrease. |
|
Thanks very much for the confirmation that covering the camera stabilizes FPS and uncovering it destabilizes it. Is the camera being used indoors or outdoors, please? |
|
Only indoors |
|
Thank you. Are there any strong light sources in the indoor location that could be casting light directly into the view of the camera? This can cause the infrared sensors to become saturated and negatively affect the camera's depth exposure value. FPS speed is tied to exposure, so if the exposure value enters certain ranges then the FPS can either fall or increase. This is described at IntelRealSense/librealsense#1957 When the IR sensors are saturated, if auto-exposure is enabled then covering the front of the camera with the hand can allow auto-exposure to auto-correct the problem. It will re-occur after removing the hand though if the strong light source is still directly in the view of the camera. An example of this would be if the camera was faced directly towards the sun. Intel introduced RealSense camera models with filters on the IR sensors to absorb light and improve depth quality. https://www.intelrealsense.com/stereo-depth-with-ir/ The CLAREX filter used in these camera models can be purchased separately too and retro-fitted over the front of a RealSense camera that is not equipped with the filter. If the camera is used indoors in an environment where the artificial lighting level is consistent then you could try disabling auto-exposure to see whether it prevents the FPS drop. In ROS2 you can test whether this will work by inputting the runtime command below after launch has completed.
|
|
Hi, changing this parameter does not make any noticeable difference. Regarding the light sources I have tested and there is no significant difference in performance. Additionally I have tested with a smaller window of 10 msgs and observed that the frequency is very volatile: average rate: 7.898 My main concern right now is that when I launch the ROS 1 driver the framerate is exactly as expected even when pointed at a direct light source. |
|
And what happens if auto_exposure_enabled is set to True after launch?
There have been some ROS2 cases where auto-exposure is initially true during launch but then gets disabled. This has been reported to cause an FPS drop. |
|
No it does not have any effect. I tried switching it back and forth without any noticeable change |
|
Continuing with the theory that it is an exposure related issue as the camera recovers when covered over and returns to the problem when uncovered: how is the FPS affected if you set the depth exposure manually to its default value of 33000
|
|
Unfortunately changing this param did not have any effect. The fps shows the same peaks and drops as in my last comment |
|
You mentioned in #2923 (comment) that you were using a ROS2 Humble container. Have you had the opportunity to test without a container to see whether it is a factor in the variable FPS that you are experiencing in the ROS2 wrapper, please? |
|
I do not have the option to test the camera with ros installed natively on the NUC. The strange issue I'm running into on the legion is that the framerate seems to be more stable with the setting on 30 FPS than when I tone it down to 6 FPS. It's strange to me that the FPS issue is not reported more often here. |
|
Historically, 6 FPS is more likely to have problems than the next highest value (15 FPS). Because new frames are delivered less often at 6 FPS, there is a higher risk of the frame delivery pipeline 'timing out' if the next new frame does not arrive quickly enough. Such errors tend to disappear at 15 FPS. |
|
Sadly for us, the 15 FPS is unstable as well. We have encountered at worst one to five-second gaps in the point cloud stream. |
|
If 30 FPS is more stable, you could try reducing the launch resolution from 848x480 to 640x480 at 30 FPS to see whether you get acceptable performance. |
|
Hi @TopRamen1 Do you require further assistance with this case, please? Thanks! |
|
Hi 30 FPS is not a viable option for us. The driver uses too much CPU in this mode. |
|
To the best of my knowledge there is no work being done in regard to optimization when using low FPS speeds such as 6 FPS. To a certain extent, performance will be dependent on the specification of the computer / computing device and the CPU chip that it uses, especially with single-board computers (like your SBC NUC). So an SBC may have lesser performance than the exact same launch on a desktop or laptop PC. |
|
I did some more digging and it turns out the ros2 topic hz tool is at fault for the lower frequency. This issue was mentioned in other threads: To sum up it seems like it's not an issue with the realsense-ros driver |
|
Thanks so much for the information! |
|
Hi @TopRamen1 Do you require further assistance with this case, please? Thanks! |
|
Case closed due to no further comments received. |
I'm running a basic rosrun on an
Platform: Intel NUC SBC
Realsense-ros: latest ros2-devel
Command: ros2 run realsense2_camera realsense2_camera_node --ros-args -p pointcloud.enable:=true
Outcome:
ros2 run logs:
ros2 topic hz:
The expected frequency is 30 hz but I'm getting on average: 11 Hz
Is this a known issue? And if so what could be a solution?
The text was updated successfully, but these errors were encountered: