Rotation-Equivariant Conditional Spherical Neural Fields for Learning a Natural Illumination Prior

Gardner JA, Egger B, Smith WA (2022)

Publication Type: Conference contribution

Publication year: 2022

Publisher: Neural information processing systems foundation

Book Volume: 35

Conference Proceedings Title: Advances in Neural Information Processing Systems

Event location: New Orleans, LA, USA

ISBN: 9781713871088

Abstract

Inverse rendering is an ill-posed problem. Previous work has sought to resolve this by focussing on priors for object or scene shape or appearance. In this work, we instead focus on a prior for natural illuminations. Current methods rely on spherical harmonic lighting or other generic representations and, at best, a simplistic prior on the parameters. We propose a conditional neural field representation based on a variational auto-decoder with a SIREN network and, extending Vector Neurons, build equivariance directly into the network. Using this, we develop a rotation-equivariant, high dynamic range (HDR) neural illumination model that is compact and able to express complex, high-frequency features of natural environment maps. Training our model on a curated dataset of 1.6K HDR environment maps of natural scenes, we compare it against traditional representations, demonstrate its applicability for an inverse rendering task and show environment map completion from partial observations. A PyTorch implementation, our dataset and trained models can be found at jadgardner.github.io/RENI.

Authors with CRIS profile

Bernhard Egger Professur für Cognitive Computer Vision (Stiftungsprofessur)

Involved external institutions

University of York GB United Kingdom (GB)

How to cite

APA:

Gardner, J.A., Egger, B., & Smith, W.A. (2022). Rotation-Equivariant Conditional Spherical Neural Fields for Learning a Natural Illumination Prior. In S. Koyejo, S. Mohamed, A. Agarwal, D. Belgrave, K. Cho, A. Oh (Eds.), Advances in Neural Information Processing Systems. New Orleans, LA, USA: Neural information processing systems foundation.

MLA:

Gardner, James A.D., Bernhard Egger, and William A.P. Smith. "Rotation-Equivariant Conditional Spherical Neural Fields for Learning a Natural Illumination Prior." Proceedings of the 36th Conference on Neural Information Processing Systems, NeurIPS 2022, New Orleans, LA, USA Ed. S. Koyejo, S. Mohamed, A. Agarwal, D. Belgrave, K. Cho, A. Oh, Neural information processing systems foundation, 2022.

BibTeX: Download