The Sky’s the Limit: Relightable Outdoor Scenes via a Sky-Pixel Constrained Illumination Prior and Outside-In Visibility
Gardner JA, Kashin E, Egger B, Smith WA (2025)
Publication Type: Conference contribution
Publication year: 2025
Journal
Publisher: Springer Science and Business Media Deutschland GmbH
Book Volume: 15112 LNCS
Pages Range: 126-143
Conference Proceedings Title: Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Event location: Milan, ITA
ISBN: 9783031729485
DOI: 10.1007/978-3-031-72949-2_8
Abstract
Inverse rendering of outdoor scenes from unconstrained image collections is a challenging task, particularly illumination/albedo ambiguities and occlusion of the illumination environment (shadowing) caused by geometry. However, there are many cues in an image that can aid in the disentanglement of geometry, albedo and shadows. Whilst sky is frequently masked out in state-of-the-art methods, we exploit the fact that any sky pixel provides a direct observation of distant lighting in the corresponding direction and, via a neural illumination prior, a statistical cue to derive the remaining illumination environment. The incorporation of our illumination prior is enabled by a novel ‘outside-in’ method for computing differentiable sky visibility based on a neural directional distance function. This is highly efficient and can be trained in parallel with the neural scene representation, allowing gradients from appearance loss to flow from shadows to influence the estimation of illumination and geometry. Our method estimates high-quality albedo, geometry, illumination and sky visibility, achieving state-of-the-art results on the NeRF-OSR relighting benchmark. Our code and models can be found at https://github.com/JADGardner/neusky.
Authors with CRIS profile
Involved external institutions
United Kingdom (GB)How to cite
APA:
Gardner, J.A., Kashin, E., Egger, B., & Smith, W.A. (2025). The Sky’s the Limit: Relightable Outdoor Scenes via a Sky-Pixel Constrained Illumination Prior and Outside-In Visibility. In Aleš Leonardis, Elisa Ricci, Stefan Roth, Olga Russakovsky, Torsten Sattler, Gül Varol (Eds.), Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics) (pp. 126-143). Milan, ITA: Springer Science and Business Media Deutschland GmbH.
MLA:
Gardner, James A.D., et al. "The Sky’s the Limit: Relightable Outdoor Scenes via a Sky-Pixel Constrained Illumination Prior and Outside-In Visibility." Proceedings of the 18th European Conference on Computer Vision, ECCV 2024, Milan, ITA Ed. Aleš Leonardis, Elisa Ricci, Stefan Roth, Olga Russakovsky, Torsten Sattler, Gül Varol, Springer Science and Business Media Deutschland GmbH, 2025. 126-143.
BibTeX: Download