Glossary term
Glossary term
Multimodal AI
Method representing a 3D scene as a continuous neural function mapping spatial coordinates and viewing direction to colour and density, enabling photorealistic novel-view synthesis.
NeRF (Mildenhall et al. 2020, Berkeley/Google) enables photorealistic novel-view synthesis of real scenes from 20-100 input photos. Luma AI's NeRF product is used by e-commerce retailers to generate interactive 360-degree product views from a handful of smartphone photos.
NVIDIA Instant NGP (2022) accelerates NeRF training from hours to seconds using hash-grid encodings, making real-time NeRF reconstruction feasible on a single RTX 3090. It is used in game development for rapid environment digitisation.
Nerfstudio (UC Berkeley) is an open-source NeRF framework used by robotics teams at Boston Dynamics and NASA JPL to reconstruct environments from robot-mounted camera footage for path planning and object manipulation.