Variational Object-Aware 3-D Hand Pose from a Single RGB Image
Gao Y, Wang Y, Falco P, Navab N, Tombari F (2019)
Publication Type: Journal article
Publication year: 2019
Journal
Book Volume: 4
Pages Range: 4239-4246
Article Number: 2930425
Journal Issue: 4
Abstract
We propose an approach to estimate the 3D pose of a human hand while grasping objects from a single RGB image. Our approach is based on a probabilistic model implemented with deep architectures, which is used for regressing, respectively, the 2D hand joints heat maps and the 3D hand joints coordinates. We train our networks so to make our approach robust to large objectand self-occlusions, as commonly occurring with the task at hand. Using specialized latent variables, the deep architecture internally infers the category of the grasped object so to enhance the 3D reconstruction, based on the underlying assumption that objects of a similar category, i.e., with similar shape and size, are grasped in a similar way. Moreover, given the scarcity of 3D hand-object manipulation benchmarks with joint annotations, we propose a new annotated synthetic dataset with realistic images, hand masks, joint masks and 3D joints coordinates. Our approach is flexible as it does not require depth information, sensor calibration, data gloves, or finger markers.We quantitatively evaluate it on synthetic datasets achieving state-of-the-art accuracy, as well as qualitatively on real sequences.
Involved external institutions
Germany (DE)
Sweden (SE)How to cite
APA:
Gao, Y., Wang, Y., Falco, P., Navab, N., & Tombari, F. (2019). Variational Object-Aware 3-D Hand Pose from a Single RGB Image. IEEE Robotics and Automation Letters, 4(4), 4239-4246. https://doi.org/10.1109/LRA.2019.2930425
MLA:
Gao, Yafei, et al. "Variational Object-Aware 3-D Hand Pose from a Single RGB Image." IEEE Robotics and Automation Letters 4.4 (2019): 4239-4246.
BibTeX: Download