Skip to content

feat: support actual depth image with 32FC1 datatype #349

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 1 commit into from
Feb 22, 2025
Merged

feat: support actual depth image with 32FC1 datatype #349

merged 1 commit into from
Feb 22, 2025

Conversation

@bresilla
Copy link
Contributor

@bresilla bresilla commented Feb 15, 2025

  • Add separate processing for RGB and Depth camera types with specific capture source and image properties.
  • Introduce new data structures to handle RGB and Depth surface contexts.
  • Adjust pixel format and data handling based on camera type, including custom encoding and image data initialization.
  • Improve conditions for processing captured data by ensuring proper checks are in place for RGB and Depth data retrieval.
  • Add a Depth array to FRenderRequest struct to store depth information.

@bresilla
feat: support actual depth image with 32FC1 datatype
f4a24e5 
- Add separate processing for RGB and Depth camera types with specific capture source and image properties.
- Introduce new data structures to handle RGB and Depth surface contexts.
- Adjust pixel format and data handling based on camera type, including custom encoding and image data initialization.
- Improve conditions for processing captured data by ensuring proper checks are in place for RGB and Depth data retrieval.
- Add a `Depth` array to `FRenderRequest` struct to store depth information.
- Remove obsolete comments and redundant code sections to improve readability and maintainability.
@bresilla
Copy link
Contributor Author

bresilla commented Feb 15, 2025

Some context: This PR updates the camera component to output depth data in 32FC1, a commonly used format for depth images. In this format, depth values are stored as 32-bit floating-point numbers, representing the distance (in meters) from the camera.

Previously, the implementation was incorrect because the depth image was stored in RGB8 format. In this format, the pixel values—particularly the R channel, which typically holds depth data in Unreal—did not accurately represent depth. Instead, they were just a visual representation rather than meaningful depth values.

@yuokamoto
Copy link
Contributor

yuokamoto commented Feb 16, 2025

Thank you very much!
@james-yoo I added depth camera with your request before. Could you test this pr as well if you have time?
#225 (comment)

Thanks in advance.

@james-yoo
Copy link
Contributor

james-yoo commented Feb 17, 2025

@yuokamoto
I will check it.

@bresilla
Copy link
Contributor Author

bresilla commented Feb 20, 2025

Any update on this??

I am working to add camera segmentation based on Unreal Stencil thingy. Would be cool if this gets reviewed before.

@james-yoo
Copy link
Contributor

james-yoo commented Feb 21, 2025

@yuokamoto
OK, it seems that this PR output depth data in 32FC1 format. Great!

depth_test.mp4

@yuokamoto
Copy link
Contributor

yuokamoto commented Feb 22, 2025

@yuokamoto OK, it seems that this PR output depth data in 32FC1 format. Great!

depth_test.mp4

Cool thankyou!

@yuokamoto yuokamoto merged commit 3271e58 into rapyuta-robotics:devel Feb 22, 2025
james-yoo pushed a commit to james-yoo/RapyutaSimulationPlugins that referenced this pull request Feb 23, 2025
@bresilla @james-yoo
feat: support actual depth image with 32FC1 datatype (rapyuta-robotic…
bcb88e4 
…s#349)

- Add separate processing for RGB and Depth camera types with specific capture source and image properties.
- Introduce new data structures to handle RGB and Depth surface contexts.
- Adjust pixel format and data handling based on camera type, including custom encoding and image data initialization.
- Improve conditions for processing captured data by ensuring proper checks are in place for RGB and Depth data retrieval.
- Add a `Depth` array to `FRenderRequest` struct to store depth information.
- Remove obsolete comments and redundant code sections to improve readability and maintainability.
yuokamoto pushed a commit that referenced this pull request Feb 23, 2025
@james-yoo @bresilla
feat: support actual depth image with 32FC1 datatype (#349) (#351)
9405a07 
- Add separate processing for RGB and Depth camera types with specific capture source and image properties.
- Introduce new data structures to handle RGB and Depth surface contexts.
- Adjust pixel format and data handling based on camera type, including custom encoding and image data initialization.
- Improve conditions for processing captured data by ensuring proper checks are in place for RGB and Depth data retrieval.
- Add a `Depth` array to `FRenderRequest` struct to store depth information.
- Remove obsolete comments and redundant code sections to improve readability and maintainability.

Co-authored-by: Trim Bresilla <[email protected]>
yuokamoto pushed a commit that referenced this pull request Feb 24, 2025
@bresilla @yuokamoto
feat: support actual depth image with 32FC1 datatype (#349)
90c05d5 
- Add separate processing for RGB and Depth camera types with specific capture source and image properties.
- Introduce new data structures to handle RGB and Depth surface contexts.
- Adjust pixel format and data handling based on camera type, including custom encoding and image data initialization.
- Improve conditions for processing captured data by ensuring proper checks are in place for RGB and Depth data retrieval.
- Add a `Depth` array to `FRenderRequest` struct to store depth information.
- Remove obsolete comments and redundant code sections to improve readability and maintainability.
yuokamoto added a commit that referenced this pull request Feb 24, 2025
@yuokamoto @bresilla
feat: support actual depth image with 32FC1 datatype (#349) (#352)
3dea23f 
- Add separate processing for RGB and Depth camera types with specific capture source and image properties.
- Introduce new data structures to handle RGB and Depth surface contexts.
- Adjust pixel format and data handling based on camera type, including custom encoding and image data initialization.
- Improve conditions for processing captured data by ensuring proper checks are in place for RGB and Depth data retrieval.
- Add a `Depth` array to `FRenderRequest` struct to store depth information.
- Remove obsolete comments and redundant code sections to improve readability and maintainability.

Co-authored-by: Trim Bresilla <[email protected]>
yuokamoto pushed a commit that referenced this pull request Feb 24, 2025
@bresilla @yuokamoto
feat: support actual depth image with 32FC1 datatype (#349)
4ff58a9 
- Add separate processing for RGB and Depth camera types with specific capture source and image properties.
- Introduce new data structures to handle RGB and Depth surface contexts.
- Adjust pixel format and data handling based on camera type, including custom encoding and image data initialization.
- Improve conditions for processing captured data by ensuring proper checks are in place for RGB and Depth data retrieval.
- Add a `Depth` array to `FRenderRequest` struct to store depth information.
- Remove obsolete comments and redundant code sections to improve readability and maintainability.
yuokamoto added a commit that referenced this pull request Feb 24, 2025
@yuokamoto @bresilla
Ue5.3 action update (#354)
713bab7 
* feat: support actual depth image with 32FC1 datatype (#349)

- Add separate processing for RGB and Depth camera types with specific capture source and image properties.
- Introduce new data structures to handle RGB and Depth surface contexts.
- Adjust pixel format and data handling based on camera type, including custom encoding and image data initialization.
- Improve conditions for processing captured data by ensuring proper checks are in place for RGB and Depth data retrieval.
- Add a `Depth` array to `FRenderRequest` struct to store depth information.
- Remove obsolete comments and redundant code sections to improve readability and maintainability.

* update action

---------

Co-authored-by: Trim Bresilla <[email protected]>
github-actions bot pushed a commit that referenced this pull request Feb 24, 2025
@yuokamoto @bresilla @web-flow
Ue5.3 action update (#354)
6e46e55 
* feat: support actual depth image with 32FC1 datatype (#349)

- Add separate processing for RGB and Depth camera types with specific capture source and image properties.
- Introduce new data structures to handle RGB and Depth surface contexts.
- Adjust pixel format and data handling based on camera type, including custom encoding and image data initialization.
- Improve conditions for processing captured data by ensuring proper checks are in place for RGB and Depth data retrieval.
- Add a `Depth` array to `FRenderRequest` struct to store depth information.
- Remove obsolete comments and redundant code sections to improve readability and maintainability.

* update action

---------

Co-authored-by: Trim Bresilla <[email protected]>
github-actions bot pushed a commit that referenced this pull request Feb 24, 2025
@yuokamoto @bresilla @web-flow
Ue5.3 action update (#354)
e500750 
* feat: support actual depth image with 32FC1 datatype (#349)

- Add separate processing for RGB and Depth camera types with specific capture source and image properties.
- Introduce new data structures to handle RGB and Depth surface contexts.
- Adjust pixel format and data handling based on camera type, including custom encoding and image data initialization.
- Improve conditions for processing captured data by ensuring proper checks are in place for RGB and Depth data retrieval.
- Add a `Depth` array to `FRenderRequest` struct to store depth information.
- Remove obsolete comments and redundant code sections to improve readability and maintainability.

* update action

---------

Co-authored-by: Trim Bresilla <[email protected]>
github-actions bot pushed a commit that referenced this pull request Feb 24, 2025
@yuokamoto @bresilla @web-flow
Ue5.3 action update (#354)
b01be01 
* feat: support actual depth image with 32FC1 datatype (#349)

- Add separate processing for RGB and Depth camera types with specific capture source and image properties.
- Introduce new data structures to handle RGB and Depth surface contexts.
- Adjust pixel format and data handling based on camera type, including custom encoding and image data initialization.
- Improve conditions for processing captured data by ensuring proper checks are in place for RGB and Depth data retrieval.
- Add a `Depth` array to `FRenderRequest` struct to store depth information.
- Remove obsolete comments and redundant code sections to improve readability and maintainability.

* update action

---------

Co-authored-by: Trim Bresilla <[email protected]>
github-actions bot pushed a commit that referenced this pull request Feb 24, 2025
@yuokamoto @bresilla @web-flow
Ue5.3 action update (#354)
f22280f 
* feat: support actual depth image with 32FC1 datatype (#349)

- Add separate processing for RGB and Depth camera types with specific capture source and image properties.
- Introduce new data structures to handle RGB and Depth surface contexts.
- Adjust pixel format and data handling based on camera type, including custom encoding and image data initialization.
- Improve conditions for processing captured data by ensuring proper checks are in place for RGB and Depth data retrieval.
- Add a `Depth` array to `FRenderRequest` struct to store depth information.
- Remove obsolete comments and redundant code sections to improve readability and maintainability.

* update action

---------

Co-authored-by: Trim Bresilla <[email protected]>
yuokamoto added a commit that referenced this pull request Feb 24, 2025
@github-actions @yuokamoto @bresilla
Ue5.3 action update (#354) (#357)
830ff76 
* feat: support actual depth image with 32FC1 datatype (#349)

- Add separate processing for RGB and Depth camera types with specific capture source and image properties.
- Introduce new data structures to handle RGB and Depth surface contexts.
- Adjust pixel format and data handling based on camera type, including custom encoding and image data initialization.
- Improve conditions for processing captured data by ensuring proper checks are in place for RGB and Depth data retrieval.
- Add a `Depth` array to `FRenderRequest` struct to store depth information.
- Remove obsolete comments and redundant code sections to improve readability and maintainability.

* update action

---------

Co-authored-by: yuokamoto <[email protected]>
Co-authored-by: Trim Bresilla <[email protected]>
github-actions bot added a commit that referenced this pull request Feb 24, 2025
@github-actions @yuokamoto
@bresilla @web-flow
Ue5.3 action update (#354) (#357)
a8ac3cf 
* feat: support actual depth image with 32FC1 datatype (#349)

- Add separate processing for RGB and Depth camera types with specific capture source and image properties.
- Introduce new data structures to handle RGB and Depth surface contexts.
- Adjust pixel format and data handling based on camera type, including custom encoding and image data initialization.
- Improve conditions for processing captured data by ensuring proper checks are in place for RGB and Depth data retrieval.
- Add a `Depth` array to `FRenderRequest` struct to store depth information.
- Remove obsolete comments and redundant code sections to improve readability and maintainability.

* update action

---------

Co-authored-by: yuokamoto <[email protected]>
Co-authored-by: Trim Bresilla <[email protected]>
yuokamoto added a commit that referenced this pull request Feb 24, 2025
@github-actions @yuokamoto @bresilla
Ue5.3 action update (#354) (#356)
94e836a 
* feat: support actual depth image with 32FC1 datatype (#349)

- Add separate processing for RGB and Depth camera types with specific capture source and image properties.
- Introduce new data structures to handle RGB and Depth surface contexts.
- Adjust pixel format and data handling based on camera type, including custom encoding and image data initialization.
- Improve conditions for processing captured data by ensuring proper checks are in place for RGB and Depth data retrieval.
- Add a `Depth` array to `FRenderRequest` struct to store depth information.
- Remove obsolete comments and redundant code sections to improve readability and maintainability.

* update action

---------

Co-authored-by: yuokamoto <[email protected]>
Co-authored-by: Trim Bresilla <[email protected]>
yuokamoto added a commit that referenced this pull request Feb 24, 2025
@github-actions @yuokamoto @bresilla
Ue5.3 action update (#354) (#355)
52de862 
* feat: support actual depth image with 32FC1 datatype (#349)

- Add separate processing for RGB and Depth camera types with specific capture source and image properties.
- Introduce new data structures to handle RGB and Depth surface contexts.
- Adjust pixel format and data handling based on camera type, including custom encoding and image data initialization.
- Improve conditions for processing captured data by ensuring proper checks are in place for RGB and Depth data retrieval.
- Add a `Depth` array to `FRenderRequest` struct to store depth information.
- Remove obsolete comments and redundant code sections to improve readability and maintainability.

* update action

---------

Co-authored-by: yuokamoto <[email protected]>
Co-authored-by: Trim Bresilla <[email protected]>
yuokamoto added a commit that referenced this pull request Feb 24, 2025
@github-actions @yuokamoto @bresilla
Ue5.3 action update (#354) (#357) (#358)
9683598 
* feat: support actual depth image with 32FC1 datatype (#349)

- Add separate processing for RGB and Depth camera types with specific capture source and image properties.
- Introduce new data structures to handle RGB and Depth surface contexts.
- Adjust pixel format and data handling based on camera type, including custom encoding and image data initialization.
- Improve conditions for processing captured data by ensuring proper checks are in place for RGB and Depth data retrieval.
- Add a `Depth` array to `FRenderRequest` struct to store depth information.
- Remove obsolete comments and redundant code sections to improve readability and maintainability.

* update action

---------

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: yuokamoto <[email protected]>
Co-authored-by: Trim Bresilla <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@bresilla @yuokamoto @james-yoo