Forecasting People Trajectories and Head Poses by Jointly Reasoning on Tracklets and Vislets

Hasan I, Setti F, Tsesmelis T, Amin S, Del Bue A, Cristani M, Galasso F, Belagiannis V (2021)

Publication Type: Journal article

Publication year: 2021

Journal

IEEE Transactions on Pattern Analysis and Machine Intelligence Institute of Electrical and Electronics Engineers (IEEE)

Book Volume: 43

Pages Range: 1267-1278

Article Number: 8883081

Journal Issue: 4

DOI: 10.1109/TPAMI.2019.2949414

Abstract

In this article, we explore the correlation between people trajectories and their head orientations. We argue that people trajectory and head pose forecasting can be modelled as a joint problem. Recent approaches on trajectory forecasting leverage short-term trajectories (aka tracklets) of pedestrians to predict their future paths. In addition, sociological cues, such as expected destination or pedestrian interaction, are often combined with tracklets. In this article, we propose MiXing-LSTM (MX-LSTM) to capture the interplay between positions and head orientations (vislets) thanks to a joint unconstrained optimization of full covariance matrices during the LSTM backpropagation. We additionally exploit the head orientations as a proxy for the visual attention, when modeling social interactions. MX-LSTM predicts future pedestrians location and head pose, increasing the standard capabilities of the current approaches on long-term trajectory forecasting. Compared to the state-of-the-art, our approach shows better performances on an extensive set of public benchmarks. MX-LSTM is particularly effective when people move slowly, i.e., the most challenging scenario for all other models. The proposed approach also allows for accurate predictions on a longer time horizon.

Authors with CRIS profile

Vasileios Belagiannis

Involved external institutions

Inception Institute of Artificial Intelligence (IIAI) AE United Arab Emirates (AE) University of Verona / Università degli Studi di Verona IT Italy (IT) Italian Institute of Technology / Istituto Italiano di Tecnologia (IIT) IT Italy (IT) OSRAM GmbH DE Germany (DE) Universität Ulm DE Germany (DE)

How to cite

APA:

Hasan, I., Setti, F., Tsesmelis, T., Amin, S., Del Bue, A., Cristani, M.,... Belagiannis, V. (2021). Forecasting People Trajectories and Head Poses by Jointly Reasoning on Tracklets and Vislets. IEEE Transactions on Pattern Analysis and Machine Intelligence, 43(4), 1267-1278. https://dx.doi.org/10.1109/TPAMI.2019.2949414

MLA:

Hasan, Irtiza, et al. "Forecasting People Trajectories and Head Poses by Jointly Reasoning on Tracklets and Vislets." IEEE Transactions on Pattern Analysis and Machine Intelligence 43.4 (2021): 1267-1278.

BibTeX: Download