In this age of AI, we’ve seen 3D visualization methods come a long way. Impressive visuals can make or break stakeholder buy-in. One of the biggest shifts in 3D rendering techniques is Gaussian splatting – a fast, efficient way to create lightweight, high-quality interactive 3D visualizations that support the development of diverse industries like real estate, manufacturing, eCommerce and retail, tourism and hospitality, VFX and gaming and infrastructure.
This article dives into when and why Gaussian splatting is the best choice to use, how it stacks up against alternate 3D visualization solutions like photogrammetry and NeRFs, and how to get started in Gaussian splatting while considering its future directions.
Key Takeaways:
- What is Gaussian splatting? A 3D visualization technique that creates high-res, interactive models using the efficient process of Gaussian functions, specific to Gaussian splats or G splats.
- Why Gaussian splats? Solves issues like large file sizes and long processing times while keeping 3D visualizations immersive and efficient.
- When does it make sense to use Gaussian splats? Perfect for visualizing complex environments, interiors, and landscapes, especially those requiring reflections, transparency, and interactivity—like large construction sites, architectural visualization, or detailed 3D product visualization.
- How it beats NeRFs & Photogrammetry: Gaussian splatting produces lightweight files, faster rendering without quality loss, and easier deployment on web, mobile or XR compared to photogrammetry or NeRFs.
What is Gaussian Splatting?
Imagine the Spatial Photos in Apple Vision Pro, but with real-time renders of complex structures, environments like construction sites, or detailed industrial machinery that you can interact with! While Apple Vision Pro focuses on spatial photos to create slightly immersive environments – it relies on curated content and structured environments.
Gaussian splats however, are a more versatile solution for industries to create dynamic, interactive models that are high-quality and stream in real-time. Think of Apple Vision Pro’s spatial photos—but instead of curated content, Gaussian splats let industries generate dynamic, interactive, real-time and complex 3D models of construction sites, different environments or detailed industrial machinery.
But what exactly is Gaussian splatting though? Gaussian splatting is an advanced, time-tested computer graphics technique built on a simple principle: projecting points (aka “splats“) from a point cloud in a 3D space, to combine textures from various such splats and obtain lifelike visuals and effects.
Each point, similar to a fuzzy blob, has its unique attribute to define its spatial position, color, opacity, and a shape. These splats once defined are considered 3D Gaussians that blend into the other points based on the view-angle to create an optimal projection. An alpha channel (opacity and transparency) is then added while rendering these projected splats to create a sharp and continuous visualization from all perspectives.
Image Source: arxiv.org
During rendering, the Gaussian functions (combination of singular bell curves with their own point clouds) are projected onto a 2D plane, combining their density and color values to merge and produce a final image. This process bypasses traditional meshing to result in lightweight, efficient visualizations.
Image Source: leeyngdo.github.io
These attributes allow each 3D Gaussian splat (each with its own Gaussian Function and bell curve) to blend with neighboring splats. Enabling a smooth, continuous visualization possible from all spatial perspectives, and a realistic representation of the objects or environment with accurate alpha channels.
The approach Gaussian splatting takes, replaces traditional methods such as polygons or raw point clouds, to provide a more efficient and visually cohesive system with:
- Lightweight files for seamless sharing of 3D models across mobile, AR, and XR platforms.
- Real-time rendering that eliminates unnecessary computation by directly projecting Gaussians onto an image plane when needed.
- Smooth visualization of surfaces as Gaussian splats are blended and without the pixelation or blurriness seen in older methods.
How to Implement Gaussian Splatting?
Implementing Gaussian splatting involves a series of carefully orchestrated steps that transform raw 3D data into high-quality, interactive visualizations. Each stage plays a crucial role in ensuring that the final product is lightweight, efficient, and visually stunning, whether it’s for web, mobile, or XR applications.
The Process from Data Capture to Interactive Experience
The process begins with data capture, where drones, mobile devices, or lidar scanners are used to gather 3D spatial data. These tools create a detailed point cloud that forms the foundation of the visualization. Depending on the complexity of the environment, such as construction sites or intricate architectural interiors, the data capture method may vary, but the goal remains the same: to obtain accurate, high-resolution input for further processing.
Once the data is captured, it moves through the Gaussian splatting pipeline. This stage involves processing the point cloud into a lightweight, high-fidelity 3D visualization. The Gaussian splatting algorithm assigns each point specific attributes, such as position, color, opacity, and density. By leveraging the Gaussian function, the pipeline ensures that these points blend seamlessly to create smooth, continuous surfaces that accurately represent the original scene.
Made with Luma AI
The processed data can then be brought into interactive applications during the Unity Gaussian splatting development phase. Here, developers bring the visualization to life by embedding it into platforms that support web, mobile, or XR experiences. Unity’s robust development tools allow for real-time interaction, enabling users to explore the 3D model from any angle, adjust lighting conditions, or even zoom in on intricate details.
Finally, the visualization undergoes immersive experience design, where branded UI/UX layers are added to enhance user engagement. This step involves creating controls for the interface, adding intuitive features and making sure everything is brand-aligned. Whether for architecture, product showcases, or stakeholder presentations, this final step ensures a polished, immersive experience.
Gaussian Splatting vs NeRFs vs Photogrammetry
When it comes to cutting-edge 3D visualization, Gaussian splatting, NeRFs (Neural Radiance Fields), and photogrammetry are frequently discussed technologies. Each has its own strengths and limitations, making it essential to understand how they compare and which use cases they serve best.
Gaussian splatting is a recently accelerating industry standard, setting the tones for all future 3D visualizations:
| Technique | Date Created or Popularized |
|---|---|
| Photogrammetry | 19th Century – modernized around 1970s |
| Point Clouds | 1960s – with LIDAR-based improvements in 1990s |
| 3D Data Visualization | 1970s – development with early computer graphics |
| NeRFs (Neural Radiance Fields) | 2020 – introduced by Google Research |
| Gaussian Splatting | 2023 – initial major breakthrough |
Photogrammetry: A Traditional Approach
Photogrammetry is a widely used method for creating 3D models by stitching together multiple images. Imagine a point cloud of images, which once combined, generates a mesh with textures mapped onto the model making it easier to adjust texture maps for modifications. Photogrammetry offers greater flexibility when altering the captured object compared to the radiance cloud data used by NeRFs or Gaussian splatting.
While photogrammetry is excellent for producing detailed static models, it often involves significant processing time and results in large file sizes, which can limit its usability for dynamic or interactive visualizations. On the other hand, NeRFs and Gaussian splatting don’t generate traditional polygonal 3D models, which means they aren’t ideal for projects like films or video games that require integration with “assets” from other software.
What are NeRFs?
NeRFs represent 3D scenes using a neural network rather than relying on traditional 3D meshes or point clouds. NeRF takes spatial location (x, y, z) and viewing direction (θ, ϕ) as inputs and produces outputs such as volume density (σ) and emitted radiance (R, G, B).
To create a new view, NeRFs sample points along camera rays and use volume rendering techniques to generate an image. While the results are visually impressive, the computational demands and slower rendering times often limit their practicality for real-time applications.
Photogrammetry vs Gaussian Splatting
Advantages: Photogrammetry delivers high detail for static models, making it ideal for environments where visual fidelity is paramount. Gaussian splatting, however, produces lightweight files and enables real-time rendering, providing greater flexibility for interactive applications.
Disadvantages: Photogrammetry requires time-consuming stitching and produces large file sizes, unlike the faster and more efficient Gaussian splatting.
Use Case: Static and detailed environments where time is not a constraint but visual fidelity is key.
Gaussian Splatting vs NeRFs
Advantages: Gaussian splatting excels in speed, lightweight file sizes, and its suitability for interactive environments. NeRFs shine in capturing dense radiance fields for static scenes with exceptional detail.
Disadvantages: NeRFs are computationally intensive and slower to render, making them less ideal for real-time applications.
Use Case: Choose Gaussian splatting for interactive 3D visualizations with reflections and transparency, while NeRFs are better suited for static, highly detailed scenes.
What’s Better: Gaussian Splatting, NeRFs, or Photogrammetry?
Advantages: Gaussian splatting offers the best combination of speed, interactivity, and efficiency. NeRFs provide unmatched detail for static scenes, while photogrammetry is perfect for detailed but static models.
Disadvantages: NeRFs require significant computational resources, and photogrammetry is time-intensive and produces heavy files.
Use Case: Opt for Gaussian splatting for dynamic, interactive projects; NeRFs for static and dense radiance fields; and photogrammetry for photorealistic but non-interactive models.
Made with Niantic’s Scaniverse by Sam Field
Industrial Applications of Gaussian Splatting
Industries can make use of Gaussian splatting to create photorealistic 3D visualizations of physical sites for various purposes, such as marketing, interactive displays, and project reviews. This technique bridges the gap between data-heavy 3D modeling and lightweight, accessible visualization, making it ideal for showcasing projects in a visually compelling and interactive manner.
Make use of the magic and find the real world application in:
- Media & Entertainment
- Real Estate & Construction
- Industrial & Manufacturing
- eCommerce & Retail
- Tourism & Hospitality
- Food & Beverage
- Energy & Utilities
Media & Entertainment: The Future of Spatial Streaming
Some of the most valuable content that will drive demand for XR headsets is streaming and broadcast rights for live concerts and sporting or other events. This is already happening today with 360 videos providing a courtside seat that you can look around from a fixed POV.
As heard in CES 2025, Meta’s content team also seems to have set streaming live events using 6DoF (6 Degrees of Freedom: up/down, forward/back, left/right, pitch, roll, yaw) for a 360 view and interactivity. Gaussian splats just might be the solution to stream 6DoF environments that you can fully move around in.
Real Estate & Construction: 3D Interior or Architecture Visualization
Gaussian splatting enables developers and architects to present interactive 3D walkthroughs, giving stakeholders a complete sense of scale and design. Reflections from glass buildings or metallic surfaces are rendered seamlessly, enhancing realism in 3D architectural visualization.
Industrial & Manufacturing: 3D Data or Product Visualization
Site managers and engineers can use lightweight 3D data visualizations for planning and logistics. Gaussian splatting provides high-resolution details for complex machinery or layouts, ensuring efficient communication.
eCommerce & Retail: 3D Product Visualization
For industries like eCommerce and retail, Gaussian splatting enhances 3D product visualization, enabling customers to explore products from every angle. The lightweight models ensure smooth integration into web or XR platforms.
Made with Luma AI
Tourism & Hospitality: 3D Interior or Architecture Visualization
Gaussian splatting simplifies 3D interior visualization by efficiently handling complex environments with reflections and transparency, such as glass and metallic finishes in architectural visualization.
Food & Beverage: Interactive 3D Menus
The Food & Beverage industry is experientially driven, impacting all of the senses, but often falls flat in marketing and user acquisition. Innovative brands have tried to enhance 3D food digitization with photogrammetry solutions, but the output was never good enough. Glass, ceramics, and liquids are impossible to capture using photogrammetry techniques, resulting in holes in meshes or unwanted artifacts on the materials. Now, with Gaussian splatting, all of these surfaces can be captured with high resolution, giving food models a realistic subsurface appearance.
One solution we’ve been impressed by is CS Connect’s use of Gaussian splatting to create immersive, interactive digital menus. CS Connect has developed a tailored solution for the food industry that enables restaurants to capture high-quality food models with a digital footprint of less than 1MB that can be easily integrated into a website. This allows users to view the food in 3D and rotate the models, enhancing the user experience and buyer confidence.
Made with AI-Powered 360 Menu App by CS Connect
Energy & Utilities: 3D Architecture Visualization
Visualize expansive projects like wind farms or utility grids in 3D, providing clear status updates to stakeholders and regulators without lengthy processing delays. The combination of Gaussian splatting and Unity creates streamlined and immersive solutions.
Challenges and Future Directions for Gaussian Splatting:
Gaussian splatting represents a groundbreaking evolution in 3D visualization, but like any cutting-edge tech, Gaussian splatting has hurdles to overcome.
To accurately represent scenes, 3D Gaussian splatting requires a large number of 3D Gaussians to maintain the high fidelity of rendered images. It demands substantial GPU memory for rendering and training, as well as gigabytes of disk space to store point cloud data for individual scenes, depending on the scene size and image resolution. Larger scale projects have higher memory requirement and there’s limited integration with existing workflows so extracting a surface mesh from millions of tiny, unstructured 3D Gaussians after splatting is not supported and hence doesn’t allow for mesh-based editing, sculpting, animating, or relighting.
Implementing Advanced Tools and Expertise
This is the stage where software, tools and experienced Unity developers step in to make the Gaussian splats usable – from scanning to interactive deployment. To carry out this technique and lift the pain points of heavy production and GPU requirements, Rock Paper Reality uses tools like Unity or Unreal Engine to bring in the captures or recordings, create 3D splats out of them, render and clean up the 3D splat – as well as create an interactive, user-friendly experience out of the 3D visualization.
The adoption of Gaussian splatting is only expected to grow as industries increasingly demand lightweight, high-resolution 3D visualizations. RPR has met with teams like Evercoast who do 4D Spatial Captures or otherwise known as Volumetric Capture videos, and have now chosen to swap out their pipeline for GSplats. Many such Volumetric Capture pipelines are already swapping out their current approach for GSplats because it’s proven to be far more efficient.
The continued advancements in GPU technologies and rendering pipelines will also likely make Gaussian splatting more accessible in the future, for smaller businesses as well. Currently, there are several tools like Luma AI that also make capturing and visualizing 3D objects and scenes much more straightforward than manually conducting the process.
Everything You Need to Get Started with Gaussian Splatting:
Gaussian splatting is the future of industrial visualization. Lightweight, efficient, and stunningly realistic, it solves key pain points for industries like construction, manufacturing, and utilities.
Rock Paper Reality’s vendor agnostic approach helps you find the right pipeline based on specific objectives. Whether it’s creating immersive 3D product visualization, architectural visualization, or 3D interior visualization services, we provide end-to-end services for Gaussian splatting to empower businesses to communicate effectively and secure stakeholder confidence.
We have designers and Unity developers to put together the entire experience. Additionally creating an immersive UX/UI to layer over the 3D visualization for custom branded experiences. The entire experience is built and taken care of by Rock Paper Reality, where all the customer has to do is hit “play” and enjoy the experience, instead of having to understand the core technology to run it on mobile, XR or desktop.
FAQs about Gaussian Splatting
What is Gaussian splatting?
Gaussian splatting is a 3D visualization technique that uses Gaussian functions to create lightweight, high-quality models for real-time rendering. It’s ideal for applications like architectural visualization, 3D product visualization, 3D data visualization and more.
How exactly does Gaussian splatting work?
3D Gaussian splatting uses a mathematical function called the Gaussian- a formula that represents data in a bell-shaped symmetrical curve. In 3D visualization, this function defines each splat’s 3D position (x, y, z), size, and density, creating a smooth gradient of intensity from the center outward. Each splat is anchored to a specific coordinate in 3D space, determined by the input data, such as point clouds captured via techniques like NeRF, lidar or photogrammetry.
Image Source: arxiv.org
The shape and orientation of each splat are determined by a covariance matrix that is defined to give the object or environment its shape. This matrix controls how the Gaussian spreads across different axes and enables anisotropic adjustments for complex surfaces, meaning adjustable properties, even for material that vary with different orientations, as compared to the otherwise familiar isotropic materials, where the properties are the same in all directions.
The color information mapped to each splat often incorporates view-dependent effects for realistic shading. In this case, transparency is also adjusted based on the density and blending requirements of the product visualization.
What hardware is required to use Gaussian splatting?
Gaussian splatting is lightweight enough to run on mobile devices and XR platforms, making it highly accessible for various applications, including Unity Gaussian splatting.
How does Gaussian splatting improve stakeholder engagement?
By providing immersive, interactive 3D visualizations, Gaussian splatting helps stakeholders explore projects from any angle, enhancing understanding and buy-in. Applications like 3D architectural visualization and 3D product visualization showcase its impact.
What’s the difference between NeRFs and Gaussian splatting?
While NeRFs focus on dense radiance fields for static scenes, Gaussian splatting is faster and more suitable for interactive, real-time applications, such as Unity Gaussian splatting and 3D visualization services.
References
- https://www.researchgate.net/figure/Photogrammetry-workflow-from-photographs-acquisition-to-generating-texture-meshes_fig3_367392156
- https://arxiv.org/html/2402.00752v2
- https://arxiv.org/html/2009.10862v5
- https://leeyngdo.github.io/blog/computer-graphics/2024-04-09-gaussian-splatting/
- https://www.sciencedirect.com/science/article/abs/pii/S0262885624004098
- https://ckoziol.com/blog/2024/radiance_methods/
- https://www.linkedin.com/pulse/3d-scene-worth-1000-splats-doru-arfire/
- https://www.magnopus.com/blog/the-rise-of-3d-gaussian-splatting
- https://www.heliosvisions.com/2024/11/07/why-drone-footage-and-gaussian-splats-are-the-future-of-3d-visualizations-in-aec/
- https://innoarea.com/en/noticias/gaussian-splatting/
- https://www.geoweeknews.com/blogs/gaussian-splatting-3d-scene-rendering-nerfs
- https://www.8thwall.com/blog/post/176532471715/introducing-gaussian-splats-niantic-studio
