Results

GRay: Ray Tracing 3D Gaussians Near the Speed of Splats

Project Page

Scene Viewer

While Dense Initialization (DI) slows down Gaussian Splatting (3DGS), our experiments show it instead speeds up Gaussian Ray Tracing (3DGRT).We attribute this effect to their fundamental difference in algorithmic complexity, which allows ray tracing to better exploit small Gaussian sizes.Our ray tracer called GRay, designed to leverage this effect, optimizes and renders near 3DGS's speed while staying near 3DGRT's quality.

Dataset

Scene

Baseline Initialization

Method

3DGS (SI, 30K Iters)
08:17OPT 171FPS 25.10PSNR/0.263LPIPS
3DGRT (SI, 30K Iters)
56:30OPT 63FPS 24.61PSNR/0.260LPIPS
GRay (Always DI, 15K Iters)
05:38OPT 263FPS 24.38PSNR/0.233LPIPS

Note that dense initialization is prone to floaters; floaters are masked with scene-specific near clipping planes for all methods.

Full Quantitative Results

We compare our ray tracer GRay to 3D Gaussian Splatting (3DGS) and 3D Gaussian Ray Tracing (3DGRT) under different configurations, namely the default SfM sparse init (SI) and dense init (DI). We also compare to EDGS and RayGaussX. Results are averaged over all 13 standard 3DGS benchmarking scenes (Mip-NeRF 360, Deep Blending, and Tanks & Temples) and measured on an RTX 4090.

Method PSNR SSIM LPIPS Init Time Opt Time FPS Init #G Final #G #Iterations
3DGSSI 27.10 0.831 0.262 00:00 06:18 253 0.11M 2.25M 30000
3DGSDI 26.97 0.827 0.226 01:58 10:09 241 3.97M 3.28M 30000
3DGSSI + 15K iters 26.55 0.814 0.296 00:00 02:47 299 0.11M 1.87M 15000
3DGSDI + 15K iters 26.68 0.830 0.231 01:58 05:37 235 3.97M 3.39M 15000
3DGSSI + RegularAdam 27.28 0.836 0.253 00:00 18:13 190 0.11M 2.49M 30000
3DGSDI + RegularAdam 27.29 0.838 0.211 01:58 23:56 157 3.97M 3.50M 30000
EDGSRegularAdam 27.55 0.850 0.204 00:00 30:38 173 3.98M 2.26M 30000
EDGSRegularAdam + 15K iters 27.26 0.851 0.212 00:00 20:07 167 3.98M 2.16M 15000
RayGaussXSI 28.14 0.856 0.221 00:00 56:18 39 0.11M 3.28M 30000
3DGRTSI 26.77 0.828 0.258 00:00 55:01 68 0.11M 3.24M 30000
3DGRTDI 26.46 0.824 0.229 01:58 40:05 131 3.97M 2.64M 30000
3DGRTSI + 15K iters 26.05 0.818 0.268 00:00 24:04 50 0.11M 3.39M 15000
3DGRTDI + 15K iters 26.28 0.830 0.230 01:58 20:13 120 3.97M 2.62M 15000
3DGRTSI + SparseAdam 26.61 0.825 0.264 00:00 53:37 52 0.11M 3.32M 30000
3DGRTDI + SparseAdam 26.16 0.815 0.239 01:58 33:43 136 3.97M 2.69M 30000
GRay
 26.47 0.819 0.236 01:58 05:40 248 3.27M 1.52M 15000

Conclusions

First, dense initialization speeds up ray tracing directly (3DGRT), while it slows down splatting (3DGS). Second, compared to default 3DGRT (SI), our overall method optimizes 9.7× faster and reaches 3.6× the frame rate at minimal reduction in quality.

Limitations

Our method inherits dense initialization's quality profile, making some scenes significantly better, and others significantly worse. In particular, scenes with fine detail like vegetation (e.g. Garden) typically improve considerably, while scenes with reflections (e.g. Counter) are artifact-prone. Dense initialization is also prone to floaters, an existing limitation which we do not address.