Franke L, Fink L, Stamminger M (2025)
Publication Type: Journal article
Publication year: 2025
Book Volume: 8
Article Number: 18
Journal Issue: 1
DOI: 10.1145/3728302
Open Access Link: https://arxiv.org/abs/2410.17932
Recent advances in novel view synthesis have demonstrated impressive
results in fast photorealistic scene rendering through differentiable
point rendering, either via Gaussian Splatting (3DGS) [Kerbl and Kopanas
et al. 2023] or neural point rendering [Aliev et al. 2020].
Unfortunately, these directions require either a large number of small
Gaussians or expensive per-pixel post-processing for reconstructing fine
details, which negatively impacts rendering performance. To meet the
high performance demands of virtual reality (VR) systems, primitive or
pixel counts therefore must be kept low, affecting visual quality.
In this paper, we propose a novel hybrid approach based on foveated
rendering as a promising solution that combines the strengths of both
point rendering directions regarding performance sweet spots. Analyzing
the compatibility with the human visual system, we find that using a
low-detailed, few primitive smooth Gaussian representation for the
periphery is cheap to compute and meets the perceptual demands of
peripheral vision. For the fovea only, we use neural points with a
convolutional neural network for the small pixel footprint, which
provides sharp, detailed output within the rendering budget. This
combination also allows for synergistic method accelerations with point
occlusion culling and reducing the demands on the neural network.
Our evaluation confirms that our approach increases sharpness and
details compared to a standard VR-ready 3DGS configuration, and
participants of a user study overwhelmingly preferred our method. Our
system meets the necessary performance requirements for real-time VR
interactions, ultimately enhancing the user's immersive experience.
The project page can be found at: https://lfranke.github.io/vr_splatting
APA:
Franke, L., Fink, L., & Stamminger, M. (2025). VR-Splatting: Foveated Radiance Field Rendering via 3D Gaussian Splatting and Neural Points. Proceedings of the ACM on Computer Graphics and Interactive Techniques, 8(1). https://doi.org/10.1145/3728302
MLA:
Franke, Linus, Laura Fink, and Marc Stamminger. "VR-Splatting: Foveated Radiance Field Rendering via 3D Gaussian Splatting and Neural Points." Proceedings of the ACM on Computer Graphics and Interactive Techniques 8.1 (2025).
BibTeX: Download