Self-supervised 3D Human Pose Estimation in Static Video via Neural Rendering
Schmidtke L, Hou B, Vlontzos A, Kainz B (2023)
Publication Type: Conference contribution
Publication year: 2023
Journal
Publisher: Springer Science and Business Media Deutschland GmbH
Book Volume: 13803 LNCS
Pages Range: 704-713
Conference Proceedings Title: Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Event location: Tel Aviv, ISR
ISBN: 9783031250651
DOI: 10.1007/978-3-031-25066-8_42
Abstract
Inferring 3D human pose from 2D images is a challenging and long-standing problem in the field of computer vision with many applications including motion capture, virtual reality, surveillance or gait analysis for sports and medicine. We present preliminary results for a method to estimate 3D pose from 2D video containing a single person and a static background without the need for any manual landmark annotations. We achieve this by formulating a simple yet effective self-supervision task: our model is required to reconstruct a random frame of a video given a frame from another timepoint and a rendered image of a transformed human shape template. Crucially for optimisation, our ray casting based rendering pipeline is fully differentiable, enabling end to end training solely based on the reconstruction task.
Authors with CRIS profile
Involved external institutions
United Kingdom (GB)How to cite
APA:
Schmidtke, L., Hou, B., Vlontzos, A., & Kainz, B. (2023). Self-supervised 3D Human Pose Estimation in Static Video via Neural Rendering. In Leonid Karlinsky, Tomer Michaeli, Ko Nishino (Eds.), Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics) (pp. 704-713). Tel Aviv, ISR: Springer Science and Business Media Deutschland GmbH.
MLA:
Schmidtke, Luca, et al. "Self-supervised 3D Human Pose Estimation in Static Video via Neural Rendering." Proceedings of the 17th European Conference on Computer Vision, ECCV 2022, Tel Aviv, ISR Ed. Leonid Karlinsky, Tomer Michaeli, Ko Nishino, Springer Science and Business Media Deutschland GmbH, 2023. 704-713.
BibTeX: Download