Skip to content

HLSL: Added support for tessellation control/evaluation shaders#2604

Open
xen2 wants to merge 4 commits intoKhronosGroup:mainfrom
xen2:hlsl_tessel_shader
Open

HLSL: Added support for tessellation control/evaluation shaders#2604
xen2 wants to merge 4 commits intoKhronosGroup:mainfrom
xen2:hlsl_tessel_shader

Conversation

@xen2
Copy link
Contributor

@xen2 xen2 commented Feb 22, 2026

Adds hull shader (tesc) and domain shader (tese) code generation to the SPIRV-Cross HLSL backend.
This PR is built on top of #2603 which should be first merged in.
I also keep it as draft as I am still testing it in our project and refining it, but so far it worked well on several tessellation shaders our engine generates.
Please give me your thought regarding the general approach, would it be OK to merge it?

Hull shader (tesc) — function splitting

HLSL requires hull shader work split into a per-control-point entry point and a separate patch constant function, but SPIR-V expresses both as a single main(). The compiler splits them using a redirected emission approach: rather than writing to the main output stream, emit_instruction() writes to one of two capture buffers (tesc_per_cp_buffer or tesc_patch_buffer) based on a TescEmitPhase state machine:

  • SetupPerCP: triggered when an OpAccessChain using gl_InvocationID is seen (output writes begin).
  • PerCPPatch: triggered by OpFunctionCall to the patch constant function, or an OpStore to a tess level/patch variable.
  • PatchPostPatch: triggered at the merge block of the if (gl_InvocationID == 0) construct; the closing brace is stripped and subsequent output is discarded.
  • OpControlBarrier is suppressed entirely (hull shaders synchronize implicitly).

After emission, the two buffers are assembled into tesc_main() and tesc_main_patch(). A separate patch_constant() wrapper calls tesc_main_patch() and writes tess levels to the SPIRV_Cross_PatchConstant return struct.

Note: The function splitting relies on the GLSL-to-SPIR-V compiler wrapping patch constant work in an if (gl_InvocationID == 0) { ... } guard block. This is a limitation, but it is the pattern consistently emitted by both glslang and DXC when compiling GLSL/HLSL tessellation shaders to SPIR-V, so in practice it covers all real-world inputs.

Domain shader (tese)

Nothing special, entry point signature: [domain(...)] main(SPIRV_Cross_PatchConstant pc, const OutputPatch<SPIRV_Cross_Input, N> patch, float3/float2 domain : SV_DomainLocation).
Patch-decorated inputs copy from pc; per-CP inputs copy from patch[i] in a loop.

Other

  • SPIRV_Cross_PatchConstant: shared struct between tesc/tese with patch-decorated vars + tess level builtins (SV_TessFactor / SV_InsideTessFactor), sized by domain (tri: 3/1, quad: 4/2, isoline: 2/0).
  • get_tesc_tese_flags(): merges execution mode flags from the other tessellation stage, since SPIR-V allows domain/spacing/winding modes on either entry point.
  • input_cp_count: derived from BuiltInPosition input array size, with fallback to scanning other per-CP input arrays, then output_vertices. Maybe there is a better way?
  • Tess level builtins routed through SPIRV_Cross_PatchConstant
  • gl_TessCoord assigned from SV_DomainLocation at entry.

@xen2 xen2 marked this pull request as draft February 22, 2026 01:53
@xen2 xen2 force-pushed the hlsl_tessel_shader branch from 45525a9 to 86cba34 Compare February 22, 2026 02:09
@xen2 xen2 marked this pull request as ready for review March 12, 2026 08:01
@xen2
Copy link
Contributor Author

xen2 commented Mar 12, 2026

It seems to work well enough on my use cases (tested with D3D11 on various shaders incl flat & PN, with and without adjacent edge average displacement.
I will come back with additional fixes later if I encounter issues on more complex shaders.

@HansKristian-Work
Copy link
Contributor

HansKristian-Work commented Mar 13, 2026

I need to get to this at some later point when I actually have time ... 1ksloc PRs of new difficult features takes a ton of energy and time to digest.

@xen2
Copy link
Contributor Author

xen2 commented Mar 13, 2026

No problem, I understand it is a big feature and it might take time to get improved/merged.
Thanks for your help!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants