Physical-property assignment to 3D meshes
At NVIDIA Research, I developed a foundation-model framework for physical-property assignment to 3D meshes through part segmentation and multimodal material identification.
From geometry to material-aware parts
The framework links 2D foundation-model features across rendered views of a 3D asset, producing consistent parts that can be assigned semantic and material labels.
How it works
- Render the mesh from multiple views and extract image features and segmentation masks.
- Refine and aggregate features into a consistent 3D feature field, then cluster the mesh into parts.
- Use multimodal reasoning to label parts and estimate materials, grounding those labels back onto the 3D asset.
Outcome
Co-inventor on U.S. Patent Application 19/024,394, filed January 16, 2025 and published as US20260038283A1 on February 5, 2026.
Contributions
- Developed a framework connecting 3D part segmentation with material identification.
- Combined visual foundation models and multimodal reasoning for physical-property assignment.
System and method of semantic segmentation and material identification of 3D objects. Published patent application 2026.