Allow passing an attachment loadOp and initialLayout to the Vulkan backend

[allkern]

Version/Branch of Dear ImGui:

Version 1.92, Branch: docking

Back-ends:

imgui_impl_vulkan.cpp + imgui_impl_sdl3.cpp

Compiler, OS:

Any

Full config/build information:

Dear ImGui 1.92.4 (19240)
--------------------------------
sizeof(size_t): 8, sizeof(ImDrawIdx): 2, sizeof(ImDrawVert): 20
define: __cplusplus=202002
define: _WIN32
define: _WIN64
define: __MINGW32__
define: __MINGW64__
define: __GNUC__=15
define: IMGUI_HAS_VIEWPORT
define: IMGUI_HAS_DOCK
--------------------------------
io.BackendPlatformName: imgui_impl_sdl3 (3.2.24; 3.2.24)
io.BackendRendererName: imgui_impl_vulkan
io.ConfigFlags: 0x00000083
 NavEnableKeyboard
 NavEnableGamepad
 DockingEnable
io.ConfigDpiScaleFonts
io.ConfigDpiScaleViewports
io.ConfigViewportsNoDecoration
io.ConfigViewportsNoDefaultParent
io.ConfigNavCaptureKeyboard
io.ConfigInputTextCursorBlink
io.ConfigWindowsResizeFromEdges
io.ConfigMemoryCompactTimer = 60.0
io.BackendFlags: 0x00002C1E
 HasMouseCursors
 HasSetMousePos
 PlatformHasViewports
 HasParentViewport
 RendererHasVtxOffset
 RendererHasTextures
 RendererHasViewports
--------------------------------
io.Fonts: 8 fonts, Flags: 0x00000000, TexSize: 512,128
io.Fonts->FontLoaderName: stb_truetype
io.DisplaySize: 960.00,746.00
io.DisplayFramebufferScale: 1.00,1.00
--------------------------------
style.WindowPadding: 8.00,8.00
style.WindowBorderSize: 0.00
style.FramePadding: 5.00,5.00
style.FrameRounding: 4.00
style.FrameBorderSize: 1.00
style.ItemSpacing: 8.00,4.00
style.ItemInnerSpacing: 4.00,4.00

Details:

My Issue/Question:

Some time ago I migrated my PlayStation 2 emulator from SDLGPU to Vulkan, the Vulkan backend utility functions work fine for me except for one small detail, I'm rendering something to the swapchain image before calling ImGui_ImplVulkan_RenderDrawData, this presents two issues:

• ImGui clears the framebuffer by default
• initialLayout is set to VK_IMAGE_LAYOUT_UNDEFINED causing garbage to appear on screen

I can set wd->ClearEnable to false and then it won't clear, but it instead sets the attachment load op to VK_ATTACHMENT_LOAD_OP_DONT_CARE, which causes issues with RenderDoc where the "background" of the frame is displayed as "LOAD DONT CARE" text. But then there's the initialLayout issue, this outright causes garbage to appear on screen on certain systems.
It would be pretty good if we could directly provide an attachment load op to the Vulkan backend instead of just a clear enable/disable flag, and an attachment initialLayout for cases where the user intends to overlay ImGui on top of other graphics.

Here's the relevant blob of code (specifically the loadOp and initialLayout lines)
https://github.com/ocornut/imgui/blob/docking/backends/imgui_impl_vulkan.cpp#L1737-L1745

Screenshots/Video:

Garbage appearing on screen because of initialLayout being set to VK_IMAGE_LAYOUT_UNDEFINED

Minimal, Complete and Verifiable Example code:

[ocornut]

I'm generally reluctant to perform changes because it would tend toward making a supposedly not formally supported API (ImGui_ImplVulkanH_CreateWindowSwapChain() via ImGui_ImplVulkanH_CreateOrResizeWindow()) more complex and increase my already too wide range of maintenance, but we can discuss those.

To be clear, you are using ImGui_ImplVulkanH_CreateOrResizeWindow() for the main window?

I agree on splitting wd->ClearEnable.
Fallbacking to VK_ATTACHMENT_LOAD_OP_DONT_CARE makes sense when imgui owned viewport sets ClearEnable=false knowing that we will rewrite all pixels.

It seems that we should provide configuration for the whole VkAttachmentDescription struct perhaps (ref #6515 (comment)) and this would also help solving #9139 / #4182 ?
I am currently investigating this.

PS:

Some time ago I migrated my PlayStation 2 emulator from SDLGPU to Vulkan,

Unless intended masochistic or wanting to specifically delve into low level GPU programming (perhaps needed for a PS2 emulator, I don't know) I would expect most people to want to take the exact opposite path.

[allkern]

To be clear, you are using ImGui_ImplVulkanH_CreateOrResizeWindow() for the main window?

Yes, I wrote my own versions of some of the helper functions but I chose to keep the lengthier ones like that.

I agree on splitting wd->ClearEnable.
Fallbacking to VK_ATTACHMENT_LOAD_OP_DONT_CARE makes sense when imgui owned viewport sets ClearEnable=false knowing that we will rewrite all pixels.

I just saw the commit to master, great work! 👏🏻 Would it be possible to merge that change into the docking branch?

Unless intended masochistic or wanting to specifically delve into low level GPU programming (perhaps needed for a PS2 emulator, I don't know) I would expect most people to want to take the exact opposite path.

No, I actually switched to Vulkan when I decided to integrate Parallel-GS (a Vulkan compute-based GS renderer), there was just no way around that other than Qt maybe? Either way, the GS (PS2 GPU) is a pretty demanding piece of hardware, writing a performant rasterizer does require more hands-on control of the GPU pipeline.

[ocornut]

Sorry I haven’t answered this as I intended to finish another thing before merging into docking, should do it tomorrow.

[allkern]

No problem, take your time.

[ocornut]

This is merged and amended for docking branch too.