Skip to content

Add basic support for voxel rendering and styling#1685

Open
j9liu wants to merge 78 commits intomainfrom
basic-voxel-support
Open

Add basic support for voxel rendering and styling#1685
j9liu wants to merge 78 commits intomainfrom
basic-voxel-support

Conversation

@j9liu
Copy link
Contributor

@j9liu j9liu commented May 29, 2025

Description

Depends on CesiumGS/cesium-native#1188.
Depends on #1694, so merge that first.
Depends on #1778, so merge that first.

This PR adds support for loading and rendering tilesets with 3DTILES_content_voxels, parsing glTFs with EXT_primitive_voxels payloads.

image

Data courtesy of Swisstopo

This is a meaty set of changes, so I'll add a written walkthrough of the changes in a follow-up comment.

Author checklist

  • I have submitted a Contributor License Agreement (only needed once).
  • I have done a full self-review of my code.
  • I have updated CHANGES.md with a short summary of my change (for user-facing changes).
    - [ ] I have added or updated unit tests to ensure consistent code coverage as necessary.
  • I have updated the documentation as necessary.

Remaining Tasks

  • Improve memory / lifetime management
  • Revisit voxel traversal mechanics. -- See changes in Add support for ellipsoid-based voxels #1711
    • I'm not happy with the way voxels are currently selected for rendering in VoxelResources. It's a weird compromise between CesiumJS's VoxelTraversal and the cesium-native traversal, and it falls apart for larger datasets.

Testing plan

Reach out to me offline for the pictured test data.

  • Load in a voxel tileset (same as any other Cesium3DTileset).
  • Use UCesiumVoxelMetadataComponent to gather its properties.
  • Test out applying custom shaders to the data.

Compatibility checks:

Unreal Version DX 11/12 Vulkan Metal Editor Built
5.4
5.5
5.6

@j9liu

This comment was marked as outdated.

@j9liu j9liu changed the base branch from main to property-attributes June 11, 2025 18:09
@j9liu j9liu changed the base branch from property-attributes to main June 17, 2025 19:54
@j9liu j9liu changed the base branch from main to property-attributes June 17, 2025 19:55
@j9liu
Copy link
Contributor Author

j9liu commented Mar 10, 2026

Finally getting around to @kring's comments...

I noticed that the tileset disappears when I get too close to it (inside its bounding volume, presumably). How realistic is it to avoid that? I was hoping to be able to fly through the voxel field. 😀

This should be possible if you check Two Sided on the resulting material! Here's a closeup from being inside the cube but still seeing the voxels:
image

This also allows you to fly into the voxel volume, but you just end up seeing a solid color of whatever voxel you're in. Maybe that's expected for this particular dataset??

image

I can get this weird lighting effect just by rotating the camera. I say it's lighting related because it goes away in the "Unlit" view. Any idea what this is?

I was still seeing this even after I switched to the Unlit lighting mode. It appears to be the result of the atmosphere's interaction with translucent materials. Here's the difference after I toggle CesiumSunSky:

Before After
image image

The sharp line across the back is the result of material being two-sided, but the light is still there when it's toggled off:

image

So I'm not sure how to get rid of this interaction. Maybe this is an artifact of using a cube mesh for rendering, rather than doing a separate post process / draw pass. I would be willing to explore the latter, but that will probably be another week or so of effort. Is it bad enough to hold up this PR?

TUniquePtr<FStaticMeshRenderData> RenderData =
MakeUnique<FStaticMeshRenderData>();
RenderData->AllocateLODResources(1);
TUniquePtr<FStaticMeshRenderData> pRenderData =
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This restores the loadPrimitive code so it only deals with typical 3D mesh data; the splats route has been moved to loadGaussianSplats

pCesiumPrimitive = pGaussianSplat;
createMesh = false;
} else if (meshPrimitive.mode == CesiumGltf::MeshPrimitive::Mode::POINTS) {
if (meshPrimitive.mode == CesiumGltf::MeshPrimitive::Mode::POINTS) {
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same story here; splats / voxels have their own load___GameThread function

Comment on lines +530 to +532
void UCesiumVoxelRendererComponent::UpdateTiles(
const std::vector<Cesium3DTilesSelection::Tile::ConstPointer>& VisibleTiles,
const std::vector<double>& VisibleTileScreenSpaceErrors) {
Copy link
Contributor Author

@j9liu j9liu Mar 11, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Context: This approach tries to match CesiumJS, which uses a custom traversal algorithm. Their algorithm uses a priority queue based on screen space error to retain parent detail more aggressively (avoiding LOD skips and holes).

I'm happy to take other suggestions or experiment with this, esp. because I feel like it's somewhat wasteful to do sort based on SSE every frame when Native is already handling tiles to us. At the same time, there needs to be a heuristic to determine when it's acceptable to kick a tile from its megatexture slot, since that visibility info is not sync'd 1-to-1 with Native's.

@j9liu j9liu requested a review from kring March 11, 2026 15:54
@j9liu j9liu assigned azrogers and unassigned azrogers Mar 11, 2026
@j9liu j9liu requested a review from azrogers March 11, 2026 15:54
@j9liu
Copy link
Contributor Author

j9liu commented Mar 11, 2026

Happy to mark this as ready for another review! I left a few comments that explain some of the decisions made, but please let me know if there's anything else I can clear up. I'm also happy to do a "pair programming" session to walk through some of the larger chunks of code.

@azrogers, would love to have your eyes on this, especially since you've dabbled in non-traditional rendering techniques (i.e. splats) and I figure you'll bring a fresh perspective
@kring, if you get the time to do another pass, I would greatly appreciate that too!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants