NeRFtrinsic Four: An end-to-end trainable NeRF jointly optimizing diverse intrinsic and extrinsic camera parameters

Schieber H, Deuser F, Egger B, Oswald N, Roth D (2024)

Publication Type: Journal article

Publication year: 2024

Journal

Computer Vision and Image Understanding Elsevier

Book Volume: 249

Article Number: 104206

DOI: 10.1016/j.cviu.2024.104206

Abstract

Novel view synthesis using neural radiance fields (NeRF) is the state-of-the-art technique for generating high-quality images from novel viewpoints. Existing methods require a priori knowledge about extrinsic and intrinsic camera parameters. This limits their applicability to synthetic scenes, or real-world scenarios with the necessity of a preprocessing step. Current research on the joint optimization of camera parameters and NeRF focuses on refining noisy extrinsic camera parameters and often relies on the preprocessing of intrinsic camera parameters. Further approaches are limited to cover only one single camera intrinsic. To address these limitations, we propose a novel end-to-end trainable approach called NeRFtrinsic Four. We utilize Gaussian Fourier features to estimate extrinsic camera parameters and dynamically predict varying intrinsic camera parameters through the supervision of the projection error. Our approach outperforms existing joint optimization methods on LLFF and BLEFF. In addition to these existing datasets, we introduce a new dataset called iFF with varying intrinsic camera parameters. NeRFtrinsic Four is a step forward in joint optimization NeRF-based view synthesis and enables more realistic and flexible rendering in real-world scenarios with varying camera parameters.

Authors with CRIS profile

Bernhard Egger Professur für Cognitive Computer Vision (Stiftungsprofessur)

Involved external institutions

Technische Universität München (TUM) DE Germany (DE) Universität der Bundeswehr München DE Germany (DE)

How to cite

APA:

Schieber, H., Deuser, F., Egger, B., Oswald, N., & Roth, D. (2024). NeRFtrinsic Four: An end-to-end trainable NeRF jointly optimizing diverse intrinsic and extrinsic camera parameters. Computer Vision and Image Understanding, 249. https://doi.org/10.1016/j.cviu.2024.104206

MLA:

Schieber, Hannah, et al. "NeRFtrinsic Four: An end-to-end trainable NeRF jointly optimizing diverse intrinsic and extrinsic camera parameters." Computer Vision and Image Understanding 249 (2024).

BibTeX: Download