Object-Aware Monocular Depth Prediction with Instance Convolutions

Simsar E, Ornek EP, Manhardt F, Dhamo H, Navab N, Tombari F (2022)

Publication Type: Journal article

Publication year: 2022

Journal

IEEE Robotics and Automation Letters IEEE

Book Volume: 7

Pages Range: 5389-5396

Journal Issue: 2

DOI: 10.1109/LRA.2022.3155823

Abstract

With the advent of deep learning, estimating depth from a single RGB image has recently received a lot of attention, being capable of empowering many different applications ranging from path planning for robotics to computational cinematography. Nevertheless,while the depth maps are in their entirety fairly reliable, the estimates around object discontinuities are still far from satisfactory. This can beattributed to the fact that the convolutional operator naturally aggregates features across object discontinuities, resulting in smooth transitions rather than clear boundaries. Therefore, in order to circumvent this issue, we propose a novel convolutional operator which is explicitly tailored to avoid feature aggregation of different object parts. In particular, our method is based on estimating per-part depth values by means of super-pixels. The proposed convolutional operator, which we dub 'Instance Convolution,' then only considers each object part individually on the basis of the estimated super-pixels. Our evaluation with respect to the NYUv2, iBims and KITTI datasets demonstrate the advantages of Instance Convolutions over the classical convolution at estimating depth around occlusion boundaries, while producing comparable results elsewhere. Our code is available at github.com/enisimsar/instance-conv.

Involved external institutions

Technische Universität München (TUM) DE Germany (DE) Google Inc. US United States (USA) (US)

How to cite

APA:

Simsar, E., Ornek, E.P., Manhardt, F., Dhamo, H., Navab, N., & Tombari, F. (2022). Object-Aware Monocular Depth Prediction with Instance Convolutions. IEEE Robotics and Automation Letters, 7(2), 5389-5396. https://doi.org/10.1109/LRA.2022.3155823

MLA:

Simsar, Enis, et al. "Object-Aware Monocular Depth Prediction with Instance Convolutions." IEEE Robotics and Automation Letters 7.2 (2022): 5389-5396.

BibTeX: Download