Unified and extensible post-processing pipeline #2719
Description
This issue is a feature proposal. Feel free to upvote (with 👍 ), comment and provide your use-cases if you're interested by this feature.
Context
This proposal comes from a hackathon focused on post-processing in iTowns.
We identified three main requirements:
- EDL needs to be applied on a subset of the scene (point clouds).
- Aerial perspective must be applied to the whole scene at the end of rendering.
- Users should be able to add their own effects on the whole scene.
After discussion, we concluded that linear post-processing applied on the whole scene is sufficient, without requiring more complex pipelines.
Today, post-processing is fragmented. Some logic lives in components like SkyManager, and there is no unified way to cover these needs.
Proposal overview
Introduce a unified post-processing system supporting:
- a global pipeline applied at the end of rendering,
- optional per-layer pipelines (initial focus: point clouds),
- user-defined effects inserted at well-defined stages.
The implementation should rely on the pmndrs/postprocessing library1, which offers better performance (by merging passes into unified shaders) and more control than Three.js’s default post-processing.
Global post-processing
Assignees: @HoloTheDrunk @Neptilo
Current work
Post-processing-related logic was extracted from SkyManager into an AtmosphereManager.
GlobeView is now responsible for initializing SkyManager, AtmosphereManager, and SunLightLayer (when enabled), and defines a time from which a sun direction is computed and passed to all three.
The code can be found here.
Identified issues
There is still a dependency between SkyManager and AtmosphereManager: SkyManager generates sky textures (an expensive operation) that are used by AtmosphereManager.
This change also moved a significant amount of realistic-lighting-specific code into GlobeView, cluttering the view. A possible improvement would be to introduce a RealisticLightingManager to centralize sky, atmosphere, and lighting logic.
More broadly, managers currently behave very similarly to layers (they initialize and update objects). This raises the question of how to decide whether something should be a manager or a layer, and why layers should be prevented from interacting with post-processing while managers are not.
Antialiasing
The classic rendering is better with antialiasing enabled on the renderer. However, post-processing pipelines require antialiasing to be disabled on the renderer for intermediate stages.
Since the renderer’s antialiasing property can only be set once, a final post-processing antialiasing pass is needed. FXAA is commonly used, while SSAA was suggested as a potentially higher-quality alternative, though its performance cost needs to be evaluated.
Pipeline requirements
The global pipeline should apply aerial perspective as a full-scene effect at the end of rendering, and allow users to insert custom passes after this effect and before final antialiasing without having to manage ordering manually.
Whether the composer should be managed by the engine rather than the view is still an open consideration.
Per-layer post-processing (EDL)
Assignees: @Desplandis @Nynjin
Goal
Implement EDL (Eye Dome Lighting) for point clouds using a dedicated pipeline.
The initial proof of concept uses a single composer with two passes: an EDL pass applied to point cloud points (based on the shader from the existing EDL PR, and a global pass for the rest of the geometries.
The code can be found here.
Current implementation
The POC is implemented directly inside c3DEngine with a PointCloudRenderer that contains the composer and the passes, using the pmndrs postprocessing library.
Design evolution
The PointCloudRenderer could be replaced by a generic PostProcessManager that stores passes and their associated layers or meshes. This would also require defining an ordering strategy to ensure effects render as expected.
A related question is whether post-processing should be applied per GeometryLayer or per primitive (e.g. THREE.Point), knowing that isolating them requires rendering those elements separately.
Technical considerations
A solution that works with two passes is to have one global pass that ignores point clouds, and an EDL pass that renders point clouds into an internal render target to use as texture and depth for the EDL shader, along with the input buffer’s texture.
The current stable version of postprocessing (v6.38.3) has an issue when using the same depth buffer for multiple renders, which is how EDL used to work. The framebuffer ends up being overwritten by the final pass. This issue should be fixed in version 7 (currently in alpha).
An alternative workaround involves swapping input and output buffers between passes and using a final compositing pass (sandbox example).
Limitations: VR compatibility
Post-processing does not work well with VR in Three.js. There are many known issues in the ecosystem. A simple search for “three.js postprocessing vr” returns many of those issues. Unless significant work is invested, it is preferable to disable post-processing effects in VR.
Open questions
- Should post-processing be applied per
GeometryLayeror per object primitive (e.g.THREE.Point), knowing this requires rendering those elements separately? - Should the engine manage the composer rather than the view?
- How should users be able to add passes after the aerial perspective effect and before final antialiasing without having to manage ordering themselves?
- How should passes be defined and organized in a final
EffectComposer, especially for effects like aerial perspective that may involve multiple passes? - Does SSAA provide better results than FXAA, and is its performance cost acceptable?
- How should managers vs layers be defined, given that managers behave similarly to layers (initialization and updates)? Why should layers be prevented from interacting with post-processing while managers are not?
Footnotes
-
pmndrs stands for Poimandres, which is not obvious unless you know the full name. ↩