Support loading FBX/OBJ via assimp#133
Open
nmfisher wants to merge 49 commits into
Open
Conversation
This PR implements support for Linux/Desktop rendering in Flutter applications with EGL (Wayland + OpenGL) or DMA-BUF (Wayland + Vulkan or X11 + OpenGL). Windows/Linux/Vulkan namespaces are now standardized: - Base vulkan files: thermion::vulkan - Windows files: thermion::vulkan::windows - Linux files: thermion::vulkan::linux (changed from thermion::linux_platform::vulkan) - Wrap VulkanUtils and linux_vulkan_utils in appropriate namespaces Use block-linear DRM modifiers for VkImage creation when they support COLOR_ATTACHMENT (NVIDIA GPUs). The driver picks the optimal modifier from the list. If no COLOR_ATTACHMENT modifiers exist, fall back to a two-image blit approach (render to TILING_OPTIMAL, blit to LINEAR DMA-BUF export). - Add queryColorAttachmentModifiers() to filter DRM modifiers - createExportable() now uses block-linear modifiers with dedicated alloc - Add createWithBlit() factory for render+export image pair - Add BlitToExport() with command buffer on queue 1 - Pass actual DRM modifier to EGL (lo/hi split) - Fix double-free: pass VK_NULL_HANDLE for memory to Filament - Remove VK_KHR_swapchain (not needed for offscreen rendering) NOTES ON VSYNC: - drmWaitVBlank doesn't work on NVIDIA. - we can't use Gdk events to synchronize with Flutter, these are completely disconnected. - On Linux, we use a Flutter-synchronized render loop. Flutter's SchedulerBinding drives frame timing via addPersistentFrameCallback, which calls the new FrameScheduler_requestRender FFI to queue a non-blocking render on the native render thread. After each render, a post-render callback (FrameScheduler_setPostRenderCallback) marks textures as frame-available. refactor: previously we needed to duplicate Filament headers (along with EGL, GLES2, GLES3, etc) for earlier implementations. For the former, we can now use Dart build hooks to propagate headers through from the Dart package to the Flutter package. For the latter, none of these are used any more, so they are safe to delete refactor: use Dart build hooks to propagate headers through from the Dart package to the Flutter package
🤖 Generated with GitHub Actions
…is won't pass the second equality check
🤖 Generated with GitHub Actions
…t for tests on GitHub runners yet, so running tests with the Vulkan backend would only work with an actual Linux desktop
…s now extends NativeHandle<T> so we can use the T getNativeHandle() instead
Break the monolithic geometry.dart into individual files per shape (camera, cube, cylinder, pyramid, quad, sphere) with shared utils. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Replace old wireframe implementation with applyWireframeBarycentrics on GltfSceneAsset, which unwelds mesh vertices and assigns per-triangle barycentric coordinates to CUSTOM0 for edge detection in the shader. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Update FFI bindings, Dart interfaces and implementations for asset management, gizmo, view, and scene APIs. Add link hook and analysis_options. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Update existing tests for API changes and add new tests for cascade validation, gizmo rotation, input pipeline and quad rendering. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
🤖 Generated with GitHub Actions
…sset Instead of mutating existing glTF renderables in-place, extract mesh data from GltfSceneAsset (with world transforms), unweld vertices, assign barycentric coords, and create a new GeometrySceneAsset overlay entity. This allows toggling wireframe visibility by adding/removing from scene. Key changes: - GeometrySceneAsset: add TransformManager component, disable culling/shadows - GltfSceneAsset: add extractMeshData() and createWireframeOverlay() C++ methods - C API: SceneAsset_getMeshDataSize, SceneAsset_getMeshData, SceneAsset_createWireframeOverlayRenderThread - FFIWireframeAsset.createOverlayFromAsset() with color/width params - wireframe.mat: fix alpha premultiplication (hardcode alpha=1.0 in output) Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Disable face culling in silhouette material so the highlight outline renders from both sides. Also fix plane geometry winding order to match its declared normals, and add a test for both-sides visibility. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
BREAKING CHANGE: setGltfAnimationFrame(index, frame) is replaced by setGltfAnimationTime(index, timeInSeconds). The old implementation incorrectly computed the time offset passed to Filament's applyAnimation (60 * frame * 1000 instead of frame / frameRate). The new API accepts time in seconds, consistent with Filament's gltfio Animator and getGltfAnimationDuration. Callers must convert frames to seconds before calling. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
…ace axis line width Replaces fwidth(abs(dist)) with sqrt(dFdx²+dFdy²) on the raw signed offset to avoid the derivative discontinuity at the line center and fwidth's L1 overestimation at oblique camera angles. Adds a guard against near-zero division when the plane is viewed edge-on. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Create a separate solid-shaded overlay entity from a glTF asset using Lambertian diffuse lighting with configurable base color, light direction and intensity. Mirrors the wireframe overlay architecture but uses POSITION + TANGENTS (normals converted to tangent quaternions via SurfaceOrientation) instead of POSITION + CUSTOM0 (barycentrics). - Add solid.mat material (unlit with manual N·L in fragment shader) - Add extractMeshDataWithNormals() to extract positions + normals - Add createSolidOverlay() C++ method with SurfaceOrientation pipeline - Add FFISolidOverlayAsset Dart wrapper with createOverlayFromAsset() - Add solid to build.sh and hook/build.dart material sources - Fix wireframe.mat alpha premultiplication (hardcode alpha=1.0) - Improve wireframe anti-aliasing (Euclidean gradient + transition band) Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
…servation Replace the separate overlay entity approach (createWireframeOverlay, createSolidOverlay, applyWireframeBarycentrics) with a load-time preserveGeometry flag that rebuilds vertex buffers in-place with a superset of attributes (POSITION, TANGENTS, UV0, CUSTOM0 barycentrics, and optional BONE_INDICES/BONE_WEIGHTS). This preserves the full glTF feature set (animations, skeleton, instancing) while enabling free material swapping. Breaking changes: - createWireframeOverlay, createSolidOverlay, applyWireframeBarycentrics removed - extractMeshData, extractMeshDataWithNormals removed - FFIWireframeAsset, FFISolidOverlayAsset removed - loadGltf/loadGltfFromBuffer now accept preserveGeometry parameter Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Replace custom solid.mat with gltfio's built-in ubershader material (baseColorFactor, metallicFactor, roughnessFactor). This eliminates a compiled material and its C API surface while getting IBL, shadows, and proper specular for free. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
…nce wrappers Adds convenience methods (createUbershaderMaterial, createWireframeMaterial) on FilamentApp that return typed wrappers with named setters instead of raw uniform names. Also adds geometrySource parameter to stencil highlight for preserved-geometry instances, and implements the FFI wireframe material factory. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
…port - Pass UV coordinates to SurfaceOrientation for correct tangent generation (matching gltfio's TangentsJob Lengyel's method) - Add dummy COLOR attribute (FLOAT4, all 1.0) so original materials don't read black vertex colors after vertex buffer rebuild - Propagate material changes to all instances in setMaterialInstanceForAll - Apply preserved geometry to freshly-allocated instances in createInstance - Add preserved vertex/index buffer accessors for C API Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
…nstructor Move rebuildVertexBuffers() call into GltfSceneAsset constructor so geometry rebuild happens before instance creation. Rename preserveGeometry to rebuildVertices across C API, C++ implementation, Dart interface, and tests. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
…3 overflow Three bugs in rebuildVertexBuffers: 1. Instance update loop overwrote correctly-matched geometry with wrong buffers. The primary loop iterated getRenderableEntities() (order A) but the instance update loop iterated getAssetInstances()[0]->getEntities() (order B). Different orders caused each entity to get the wrong mesh's vertex data — correct shapes but garbled textures. Fix: skip instance 0 in the update loop (already handled by primary loop) and use name-based matching for instances 1+. 2. Original TANGENT vectors from glTF were ignored; tangent quaternions were recomputed from geometry, producing wrong tangent frames and broken normal mapping. Fix: read cgltf_attribute_type_tangent and pass to SurfaceOrientation::Builder::tangents(). 3. Triangle indices for SurfaceOrientation used ushort3 (uint16), which overflows for meshes with >65535 unwelded vertices (FlightHelmet's RubberWood_low has 131,574). Fix: use uint3. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Replace name-based entity matching with consistent use of getEntities() across primary and instance update loops. This fixes the nameless entity case and simplifies the code: - Primary loop now iterates getAssetInstances()[0]->getEntities() instead of getRenderableEntities(), matching the instance loops - Instance update uses sequential indexing (safe since all instances share the same scene graph and getEntities() order) - Remove _entityBufferMap and name-based matching - Fix dangling pointer: tris vector outlives orientBuilder.build() - Remove debug logging Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
…nstance Remove the separate createWireframeMaterialInstance (returning raw MaterialInstance) and createWireframeMaterial (returning typed wrapper). Now createWireframeMaterialInstance directly returns WireframeMaterialInstance. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- Delete SurfaceOrientation* after extracting tangent quaternions - Replace malloc/free with new uint8_t[]/delete[] for buffer data - Consolidate wireframe test cases Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
When flatShading is true (requires rebuildVertices: true), vertex normals are replaced with face normals computed from triangle edge cross products, giving a faceted/low-poly look with any material. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
…oggle Remove the flatShading parameter from loadGltf/loadGltfFromBuffer and rebuildVertexBuffers. Instead, always pre-compute both smooth and flat tangent quaternion BufferObjects during rebuildVertexBuffers, and add a setFlatShading(bool) method that swaps slot 1 (TANGENTS) on all preserved VertexBuffers. This is a GPU pointer swap with near-zero runtime cost. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Add single-channel (R) support to pixelBufferToPng/pixelBufferToBmp and thread numChannels through capture helpers so R32F depth textures render correctly as grayscale images. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Added obj_import.h C API header with FFI-compatible functions - Added obj_import.cpp wrapper implementation using Filament's Assimp - Copied Assimp headers from Filament third_party - Supports loading OBJ files from memory buffers - Extracts vertices, indices, normals, UVs, mesh names, and material names - Triangulates polygons and generates normals automatically - Uses Assimp post-processing: Triangulate, JoinIdenticalVertices, SortByPType, GenNormals, CalcTangentSpace, FlipUVs
- Added ffi_obj_importer.dart with ObjMesh and ObjImporter classes - Added ObjImporter FFI bindings to thermion_dart_ffi.g.dart - Exported ObjImporter and related classes from filament.dart - ObjMesh holds parsed mesh data (vertices, normals, UVs, indices, names) - ObjImporter.loadFromBuffer() loads OBJ files and returns list of meshes - Includes toGeometry() method to convert to Thermion Geometry - Handles UV flipping, dummy color/UV creation, and index type selection - Safe UTF-8 string conversion for mesh/material names
- Added GeometryUtils.parseObjFromBuffer() for loading OBJ files - Added ObjGeometryGroup class to hold mesh name, material name, and geometry - Fixed bug in Geometry constructor where dummy UVs were created but not assigned - Changed uvs handling to match pattern used for colors - Removed premature uvs assignment in constructor - Ensures dummy UVs are created when createDummyUvs is true and uvs is null - parseObjFromBuffer supports flipUvs parameter for coordinate system conversion
- Added abstract loadObj() and loadObjFromBuffer() methods to ThermionViewerBase - Implemented OBJ loading in ThermionViewerFFI - Supports optional addToScene and flipUvs parameters - loadObj() loads from URI (file path or asset) - loadObjFromBuffer() loads from byte buffer - Returns list of ThermionAsset objects (one per mesh in OBJ) - Follows existing pattern used for glTF loading
- Added assimp to libs array in hook/build.dart - Added Assimp include path to hook/build.dart includeDirs - Linked assimp library in web/CMakeLists.txt - Added Assimp header copying to build scripts (macOS, web) - Copies Assimp headers from Filament third_party during build - Enables OBJ loading across all platforms
- Added obj_import_tests.dart with 7 tests covering: - Basic OBJ parsing from buffer - Renderable asset creation - UV flipping (enabled/disabled) - OBJ files without normals/UVs (dummies created) - Multi-group OBJ files - Mesh and material name preservation - Added test_cube.obj asset for testing - All tests passing
- build_ios.sh: Added Assimp build and copy for iOS (arm64) - build_web.sh: Added Assimp build for Emscripten/WebAssembly (library copying already handled by existing find command) - build_linux.sh: Added Assimp build to build_third_party_libs function and copy steps for both release and debug - build_windows.bat: Added Assimp build in separate cmake directories (cmake-release-assimp and cmake-debug-assimp) and copy steps All platforms now build Assimp with minimal configuration: - ASSIMP_BUILD_ASSIMP_TOOLS=OFF - ASSIMP_BUILD_TESTS=OFF - ASSIMP_BUILD_SAMPLES=OFF - ASSIMP_WARNINGS_AS_ERRORS=OFF
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.