unet node for image segmentation #27

gardeg · 2025-10-26T13:38:40Z

unet predict_ros made with launch file and increased readability

for more information, see https://pre-commit.ci

jorgenfj

You pass the yaml file to the node in the launch file, but the ros2 param api is never actually used in the node. Instead the node loads the yaml fiel into a python config and uses that instead. This setup is confusing. Also the config path is something that should be controlled by the launch file and not the node.

jorgenfj · 2025-10-26T18:01:56Z

unet_segmentation/launch/unet_segmentation.launch.py

+    input_topic = DeclareLaunchArgument("input_topic", default_value="/image_color")
+
+    node = Node(
+        package=LaunchConfiguration("package"),
+        executable="interface.py",
+        name="unet_segmentation_node",
+        parameters=[LaunchConfiguration("params_file")],
+        remappings=[
+            ("/image_color", LaunchConfiguration("input_topic")),
+            # Overlay and mask topics come from YAML; remap here only if needed:


Having the input_topic be set both by the config file and the launch config can be ambigous. We should only use 1 of these.

unet_segmentation/unet_segmentation/unet_segmentation_node.py

kluge7 · 2025-10-26T18:50:25Z

unet_segmentation/package.xml

+<package format="3">
+  <name>unet_segmentation</name>
+  <version>0.0.0</version>
+  <description>YOLO inference on images, publishing detections and annotated outputs.</description>


Update description

kluge7 · 2025-10-26T18:50:41Z

unet_segmentation/package.xml

+  <name>unet_segmentation</name>
+  <version>0.0.0</version>
+  <description>YOLO inference on images, publishing detections and annotated outputs.</description>
+  <maintainer email="[email protected]">kluge7</maintainer>


Replace this with your own email and github username

Copilot

Pull Request Overview

This PR introduces a UNet-based image segmentation ROS2 node with improved modularity and launch configuration. The implementation provides real-time segmentation with configurable inference parameters and publishing of both segmentation masks and visual overlays.

Key Changes:

UNet model implementation with configurable architecture (simple vs. full, bilinear vs. transposed convolution upsampling)
ROS2 node for real-time image segmentation with dynamic resizing and overlay generation
Launch file and YAML configuration for deployment flexibility

Reviewed Changes

Copilot reviewed 10 out of 14 changed files in this pull request and generated 2 comments.

Show a summary per file

File	Description
unet_segmentation/utils.py	Core utilities for model inference, image preprocessing, mask manipulation, and overlay generation
unet_segmentation/unet/unet_parts.py	UNet building blocks including double convolution, down/upsampling modules
unet_segmentation/unet/unet_model.py	Main UNet architecture with simple/full variants and gradient checkpointing support
unet_segmentation/unet/init.py	Package initialization exposing UNet class
interface.py	ROS2 node implementing subscription, inference pipeline, and publishing logic
config.py	Configuration dataclass with YAML loading and QoS profile setup
package.xml	ROS2 package metadata and dependencies
unet_segmentation.launch.py	Launch file with configurable parameters and topic remapping
unet_segmentation.yaml	Default configuration parameters for model and runtime settings
CMakeLists.txt	Build configuration for Python package and resource installation

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Copilot · 2025-10-26T18:54:45Z

unet_segmentation/package.xml

+<package format="3">
+  <name>unet_segmentation</name>
+  <version>0.0.0</version>
+  <description>YOLO inference on images, publishing detections and annotated outputs.</description>


The package description references 'YOLO inference' but this package is for UNet segmentation. The description should be updated to accurately reflect the package's purpose, e.g., 'UNet-based image segmentation node for ROS2, publishing segmentation masks and annotated overlays.'

Suggested change

<description>YOLO inference on images, publishing detections and annotated outputs.</description>

<description>UNet-based image segmentation node for ROS2, publishing segmentation masks and annotated overlays.</description>

unet_segmentation/unet_segmentation/unet/unet_model.py

for more information, see https://pre-commit.ci

Copilot

Pull Request Overview

Copilot reviewed 9 out of 12 changed files in this pull request and generated 7 comments.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Copilot · 2025-11-05T14:36:56Z

unet_segmentation/unet_segmentation/utils.py

+                logger.warn(f'Missing keys when loading: {missing}')
+            if unexpected:
+                logger.warn(f'Unexpected keys when loading: {unexpected}')


Use logger.warning() instead of logger.warn(). The warn() method is deprecated in Python's logging module and should be replaced with warning().

Suggested change

logger.warn(f'Missing keys when loading: {missing}')

if unexpected:

logger.warn(f'Unexpected keys when loading: {unexpected}')

logger.warning(f'Missing keys when loading: {missing}')

if unexpected:

logger.warning(f'Unexpected keys when loading: {unexpected}')

Copilot · 2025-11-05T14:36:56Z

unet_segmentation/unet_segmentation/utils.py

+                logger.warn(f'Missing keys when loading: {missing}')
+            if unexpected:
+                logger.warn(f'Unexpected keys when loading: {unexpected}')


Use logger.warning() instead of logger.warn(). The warn() method is deprecated in Python's logging module and should be replaced with warning().

Suggested change

logger.warn(f'Missing keys when loading: {missing}')

if unexpected:

logger.warn(f'Unexpected keys when loading: {unexpected}')

logger.warning(f'Missing keys when loading: {missing}')

if unexpected:

logger.warning(f'Unexpected keys when loading: {unexpected}')

Copilot · 2025-11-05T14:36:57Z

unet_segmentation/unet_segmentation/unet/unet_model.py

+            self.inc = torch.utils.checkpoint.checkpoint(self.inc)
+            self.down1 = torch.utils.checkpoint.checkpoint(self.down1)
+            self.down2 = torch.utils.checkpoint.checkpoint(self.down2)
+            self.down3 = torch.utils.checkpoint.checkpoint(self.down3)
+            self.down4 = torch.utils.checkpoint.checkpoint(self.down4)
+            self.up1 = torch.utils.checkpoint.checkpoint(self.up1)
+            self.up2 = torch.utils.checkpoint.checkpoint(self.up2)
+            self.up3 = torch.utils.checkpoint.checkpoint(self.up3)
+            self.up4 = torch.utils.checkpoint.checkpoint(self.up4)
+            self.outc = torch.utils.checkpoint.checkpoint(self.outc)


This implementation is incorrect. torch.utils.checkpoint.checkpoint is a function that performs checkpointing during forward pass, not a wrapper for modules. Assigning it to module attributes will break the model. Gradient checkpointing should be applied in the forward() method by wrapping module calls with torch.utils.checkpoint.checkpoint(module, inputs), or by using the module's .gradient_checkpointing_enable() method if available.

Copilot · 2025-11-05T14:36:57Z

unet_segmentation/unet_segmentation/unet_segmentation_node.py

+            resized_pil = ResizeIfLargerKeepAspect(
+                int(self.resize_width), int(self.resize_height)
+            )(pil_img)


Creating a new ResizeIfLargerKeepAspect instance on every image callback is inefficient. This transform should be created once during initialization (similar to self.image_transforms) and reused.

Copilot · 2025-11-05T14:36:57Z

unet_segmentation/unet_segmentation/unet_segmentation_node.py

+            )(pil_img)
+            resized_w, resized_h = resized_pil.size
+
+            image_tensor = self.image_transforms(resized_pil)


The image is being resized twice: once manually with ResizeIfLargerKeepAspect at lines 149-151, and again within self.image_transforms which also includes ResizeIfLargerKeepAspect (line 77 in utils.py). This redundant operation should be removed - either use only self.image_transforms(pil_img) or apply the resize once and use transforms without the resize step.

Copilot · 2025-11-05T14:36:57Z

unet_segmentation/unet_segmentation/unet_segmentation_node.py

+            resized_pil = ResizeIfLargerKeepAspect(
+                int(self.resize_width), int(self.resize_height)
+            )(pil_img)
+            resized_w, resized_h = resized_pil.size


Variable resized_w is not used.

Suggested change

resized_w, resized_h = resized_pil.size

# resized_w, resized_h = resized_pil.size # Removed unused variable assignment

Copilot · 2025-11-05T14:36:58Z

unet_segmentation/unet_segmentation/unet_segmentation_node.py

+            resized_pil = ResizeIfLargerKeepAspect(
+                int(self.resize_width), int(self.resize_height)
+            )(pil_img)
+            resized_w, resized_h = resized_pil.size


Variable resized_h is not used.

Suggested change

resized_w, resized_h = resized_pil.size

resized_w, _ = resized_pil.size

for more information, see https://pre-commit.ci

kluge7

unet_segmentation/launch/unet_segmentation.launch.py

unet_segmentation/model/unet-simple-320-240-l-5-e10-b16(1).pth

kluge7 · 2025-11-05T14:39:22Z

unet_segmentation/unet_segmentation/unet/__init__.py

+from .unet_model import UNet as UNet
+
+__all__ = ["UNet"]


Is this needed to run the code?

kluge7 · 2025-11-05T14:44:22Z

unet_segmentation/config/unet_segmentation.yaml

@@ -0,0 +1,17 @@
+/**:
+  ros__parameters:
+    model_path: "/home/gard/ros2_ws/src/vortex-deep-learning-pipelines/unet_segmentation/model/unet-simple-320-240-l-5-e10-b16(1).pth"


I want relative path

for more information, see https://pre-commit.ci

Copilot

Pull Request Overview

Copilot reviewed 9 out of 13 changed files in this pull request and generated 4 comments.

Comments suppressed due to low confidence (1)

unet_segmentation/unet_segmentation/unet_segmentation_node.py:63

Keyword argument 'model_file' is not a supported parameter name of function load_unet.

        self.net = load_unet(
            model_file=str(model_f),
            n_classes=self.classes,
            device=self.device,
            bilinear=bool(self.bilinear),
            simple=bool(self.simple),
            logger=self.get_logger(),
        )

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Copilot · 2025-11-05T17:27:57Z

unet_segmentation/unet_segmentation/unet_segmentation_node.py

+            self.declare_parameter(name, default)
+
+        # Bind as attributes
+        self.model_flie = self.get_parameter('model_file').value


Corrected spelling of 'flie' to 'file'.

Suggested change

self.model_flie = self.get_parameter('model_file').value

self.model_file = self.get_parameter('model_file').value

Copilot · 2025-11-05T17:27:57Z

unet_segmentation/unet_segmentation/unet_segmentation_node.py

+        self.bridge = CvBridge()
+
+        # Validate model path
+        model_f = Path(self.model_file).expanduser()


The attribute self.model_file is undefined due to the typo on line 110 (self.model_flie). This will cause an AttributeError at runtime when trying to access the model file path.

Copilot · 2025-11-05T17:27:57Z

unet_segmentation/unet_segmentation/unet_segmentation_node.py

+
+        # Load network
+        self.net = load_unet(
+            model_file=str(model_f),


The parameter name model_file does not match the function signature in utils.py which expects model_path. This will cause a TypeError at runtime.

Suggested change

model_file=str(model_f),

model_path=str(model_f),

Copilot · 2025-11-05T17:27:58Z

unet_segmentation/unet_segmentation/unet_segmentation_node.py

+            resized_pil = ResizeIfLargerKeepAspect(
+                int(self.resize_width), int(self.resize_height)
+            )(pil_img)
+            resized_w, resized_h = resized_pil.size


Variable resized_w is not used.

Suggested change

resized_w, resized_h = resized_pil.size

gardeg added 2 commits October 26, 2025 13:26

add unet segmentation node

d746a0f

add unet segmentation node finished

fc191f3

gardeg requested review from jorgenfj and kluge7 October 26, 2025 13:38

gardeg self-assigned this Oct 26, 2025

gardeg linked an issue Oct 26, 2025 that may be closed by this pull request

UNet node for image segmentation #23

Open

5 tasks

pre-commit-ci bot and others added 6 commits October 26, 2025 13:38

[pre-commit.ci] auto fixes from pre-commit.com hooks

55301df

for more information, see https://pre-commit.ci

Merge branch 'main' into 23-unet-node-for-image-segmentation

0a1cfc9

[pre-commit.ci] auto fixes from pre-commit.com hooks

3c34486

for more information, see https://pre-commit.ci

add unet segmentation node finished precomitt fix

9c690dc

[pre-commit.ci] auto fixes from pre-commit.com hooks

8ee5956

for more information, see https://pre-commit.ci

add unet segmentation node finished precomitt fixed

468884a

jorgenfj requested changes Oct 26, 2025

View reviewed changes

kluge7 requested changes Oct 26, 2025

View reviewed changes

kluge7 requested a review from Copilot October 26, 2025 18:54

Copilot AI reviewed Oct 26, 2025

View reviewed changes

gardeg and others added 4 commits November 5, 2025 15:09

fixed config setup and other small issues

3233797

[pre-commit.ci] auto fixes from pre-commit.com hooks

de618f8

for more information, see https://pre-commit.ci

Merge branch 'main' into 23-unet-node-for-image-segmentation

5f16759

fixed kluge email

c999220

kluge7 requested review from Copilot and kluge7 November 5, 2025 14:33

[pre-commit.ci] auto fixes from pre-commit.com hooks

cbe6f84

for more information, see https://pre-commit.ci

Copilot AI reviewed Nov 5, 2025

View reviewed changes

gardeg and others added 2 commits November 5, 2025 15:42

removed model

6c34ddd

[pre-commit.ci] auto fixes from pre-commit.com hooks

81b01b5

for more information, see https://pre-commit.ci

kluge7 requested changes Nov 5, 2025

View reviewed changes

gardeg and others added 2 commits November 5, 2025 15:58

fixed relativ path to model

beb0784

[pre-commit.ci] auto fixes from pre-commit.com hooks

b8cbe78

for more information, see https://pre-commit.ci

kluge7 self-requested a review November 5, 2025 17:25

kluge7 requested a review from Copilot November 5, 2025 17:25

Copilot AI reviewed Nov 5, 2025

View reviewed changes

	<description>YOLO inference on images, publishing detections and annotated outputs.</description>
	<description>UNet-based image segmentation node for ROS2, publishing segmentation masks and annotated overlays.</description>

	resized_w, resized_h = resized_pil.size
	# resized_w, resized_h = resized_pil.size # Removed unused variable assignment

	resized_w, resized_h = resized_pil.size
	resized_w, _ = resized_pil.size

	self.model_flie = self.get_parameter('model_file').value
	self.model_file = self.get_parameter('model_file').value

unet node for image segmentation #27

Are you sure you want to change the base?

unet node for image segmentation #27

Uh oh!

Conversation

gardeg commented Oct 26, 2025

Uh oh!

jorgenfj left a comment

Choose a reason for hiding this comment

Uh oh!

jorgenfj Oct 26, 2025

Choose a reason for hiding this comment

Uh oh!

Uh oh!

kluge7 Oct 26, 2025

Choose a reason for hiding this comment

Uh oh!

kluge7 Oct 26, 2025

Choose a reason for hiding this comment

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull Request Overview

Reviewed Changes

Uh oh!

Copilot AI Oct 26, 2025

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull Request Overview

Uh oh!

Copilot AI Nov 5, 2025

Choose a reason for hiding this comment

Uh oh!

Copilot AI Nov 5, 2025

Choose a reason for hiding this comment

Uh oh!

Copilot AI Nov 5, 2025

Choose a reason for hiding this comment

Uh oh!

Copilot AI Nov 5, 2025

Choose a reason for hiding this comment

Uh oh!

Copilot AI Nov 5, 2025

Choose a reason for hiding this comment

Uh oh!

Copilot AI Nov 5, 2025

Choose a reason for hiding this comment

Uh oh!

Copilot AI Nov 5, 2025

Choose a reason for hiding this comment

Uh oh!

kluge7 left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

kluge7 Nov 5, 2025

Choose a reason for hiding this comment

Uh oh!

kluge7 Nov 5, 2025

Choose a reason for hiding this comment

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull Request Overview

Uh oh!

Copilot AI Nov 5, 2025

Choose a reason for hiding this comment

Uh oh!

Copilot AI Nov 5, 2025

Choose a reason for hiding this comment

Uh oh!

Copilot AI Nov 5, 2025

Choose a reason for hiding this comment

Uh oh!

Copilot AI Nov 5, 2025

kluge7 left a comment •

edited

Loading