Studying Time Conceptualisation via Speech, Prosody, and Hand Gesture: Interweaving Manual and Computational Methods of Analysis
Uhrig P, Payne E, Pavlova I, Burenko I, Dykes N, Baltazani M, Burrows E, Hale S, Torr P, Wilson A (2023)
Publication Language: English
Publication Type: Conference contribution, Conference Contribution
Publication year: 2023
Edited Volumes: Gesture and Speech in Interaction (GeSpIn) Conference
Conference Proceedings Title: Gesture and Speech in Interaction
Event location: Nijmegen
URI: https://hdl.handle.net/21.11116/0000-000D-A250-1
DOI: 10.17617/2.3527220
Open Access Link: https://pure.mpg.de/rest/items/item_3527220_5/component/file_3527221/content
Abstract
This paper presents a new interdisciplinary methodology for the analysis of future conceptualisations in big messy media data. More specifically, it focuses on the depictions of post-Covid futures by RT during the pandemic, i.e. on data which are of interest not just from the perspective of academic research but also of policy engagement. The methodology has been developed to support the scaling up of fine-grained data-driven analysis of discourse utterances larger than individual lexical units which are centred around ‘will’ + the infinitive. It relies on the true integration of manual analytical and computational methods and tools in researching three modalities – textual, prosodic1, and gestural. The paper describes the process of building a computational infrastructure for the collection and processing of video data, which aims to empower the manual analysis. It also shows how manual analysis can motivate the development of computational tools. The paper presents individual computational tools to demonstrate how the combination of human and machine approaches to analysis can reveal new manifestations of cohesion between gesture and prosody. To illustrate the latter, the paper shows how the boundaries of prosodic units can work to help determine the boundaries of gestural units for future conceptualisations.
Authors with CRIS profile
Involved external institutions
United Kingdom (GB)
Germany (DE)How to cite
APA:
Uhrig, P., Payne, E., Pavlova, I., Burenko, I., Dykes, N., Baltazani, M.,... Wilson, A. (2023). Studying Time Conceptualisation via Speech, Prosody, and Hand Gesture: Interweaving Manual and Computational Methods of Analysis. In Wim Pouw, James Trujillo, Hans Rutger Bosker, Linda Drijvers, Marieke Hoetjes, Judith Holler, Sarka Kadava, Lieke Van Maastricht, Ezgi Mamus, Asli Ozyurek (Eds.), Gesture and Speech in Interaction. Nijmegen, NL.
MLA:
Uhrig, Peter, et al. "Studying Time Conceptualisation via Speech, Prosody, and Hand Gesture: Interweaving Manual and Computational Methods of Analysis." Proceedings of the Gesture and Speech in Interaction (GeSpIn) Conference, Nijmegen Ed. Wim Pouw, James Trujillo, Hans Rutger Bosker, Linda Drijvers, Marieke Hoetjes, Judith Holler, Sarka Kadava, Lieke Van Maastricht, Ezgi Mamus, Asli Ozyurek, 2023.
BibTeX: Download