Third party funded individual grant
Acronym: MU 2686/12-1
Start date : 01.11.2017
End date : 31.10.2020
Website: https://www.audiolabs-erlangen.de/fau/professor/mueller/projects/anchor
The increasing digitization results in extensive music collections, which, in addition to audio and video recordings, also contain symbolically or graphically encoded sheet music. One such example is the multimedia music catalog offered by the Carus publishing house, a leading international music publisher for religious and secular choral music. In addition to its extensive catalog of carefully edited sheet music comprising more than 30,000 choir pieces, Carus also produces reference recordings and video material for teaching and practice purposes. Generally speaking, the main goal of Music Information Retrieval (MIR) is to develop efficient search and navigation systems that allow users to analyze and search through complex music collections concerning different music-related aspects. In this DFG-funded transfer project, we adapted and improved MIR methods while testing them in practice.
As a first central contribution, we developed web-based user interfaces for interactive and synchronous access to different music representations and analysis results. Not only are these interfaces of interest for possible commercial applications, but they also serve to communicate research results across different disciplines.
One of the project's important research activities was to develop and implement a content-based retrieval system, where a user can make a search request in the form of a short audio excerpt or a YouTube link (e.g., a choir recording sung by an amateur choir). The goal is then to automatically identify, based purely on the acoustic waveform, all other performances or versions of the piece of music in a given collection (e.g., a high-quality recording by a professional choir in the Carus catalog). For this task, we could achieve considerable improvements in terms of the retrieval system's runtime and storage requirements by combining embedding techniques based on neural networks with graph-based indexing techniques.
Another research focus was on automated methods for measuring, analyzing, and adjusting intonation fluctuations in unaccompanied, polyphonic vocal music.
Finally, based on the experience gained with our project partner, we developed music datasets along with annotations, which we made freely available for research purposes. These datasets are suitable for developing and evaluating algorithmic approaches for various MIR tasks, including music transcription, music synchronization, chord recognition, fundamental frequency estimation, and intonation analysis.
Through the collaboration with the Carus publishing house, it was also possible to establish cross-connections to music education and musicology and initiate interdisciplinary collaborations.
Processing musical signals automatically is a major challenge because of the variety of music in form and content. The Metrum project, funded by the DFG from 2012 to 2015, dealt with the development of fundamental computer-assisted methods for analyzing and structuring musical signals. In the transfer project we propose, these automated methods are to be tested in practice, in cooperation with the Carus publishing house—a leading music publisher for religious and secular choral music with headquarters in Stuttgart. We plan to develop web-based prototypes for the interactive navigation and analysis of choral music recordings, which may be used to support music lessons in schools or to train choir directors. Besides realizing the prototypes, the following fundamental questions are in the focus of the project. First, we want to examine the suitability of novel navigation, visualization and sonification methods for musical structures by integrating them into user interfaces. Second, we want to investigate how and to what extent (partially) automated procedures can help simplify the process of linking and analyzing multimedia content — tasks that are often done in a purely manual fashion. In particular, we want to evaluate and adapt automated synchronization and structuring methods considering specific application scenarios. Third, we want to develop personalization strategies embedded into interactive tools that allow users to structure and analyze music recordings according to their specific needs, expectations, and requirements. Such strategies are not only of great practical relevance, but raise new scientific questions on how to develop adaptive strategies for analyzing and modifying musical signals. Through the close collaboration with the Carus publishing house, the project offers numerous cross-connections to fields such as music education and musicology, which will stimulate further interdisciplinary cooperations.