Third party funded individual grant
Acronym: MU 2686/10-2 (No. 328416299)
Start date : 01.01.2021
End date : 31.12.2023
Extension date: 30.09.2024
Website: https://www.audiolabs-erlangen.de/fau/professor/mueller/projects/sereco2
This is a follow-up project, which continues the previous DFG-funded project "Source Separation and Restoration of Drum Sound Components in Music Recordings" [MU 2686/10-1] aiming at the development of techniques for separating and restoring sound events as occurring in complex music recordings. In the first phase ([MU 2686/10-1]), we focused on percussive sound sources, where we decomposed a drum recording into individual drum sound events. Using Non-Negative Matrix Factor Deconvolution (NMFD) as our central methodology, we studied how to generate and integrate audio- and score-based side information to guide the decomposition. We tested our approaches within concrete application scenarios, including audio remixing (redrumming) and swing ratio analysis of jazz music. In the second phase of the project ([MU 2686/10-2]), our goals are significantly extended. First, we want to go beyond the drum scenario by considering other challenging music scenarios, including piano music (e.g., Beethoven Sonatas, Chopin Mazurkas), piano songs (e.g., Klavierlieder by Schubert), and string music (e.g., Beethoven String Quartets). In these scenarios, our goal is to decompose a music recording into individual note-related sound events. As our central methodology, we develop a unifying audio decomposition framework that combines classical signal processing and machine learning with recent deep learning (DL) approaches. Furthermore, we adopt generative DL techniques for improving the perceptual quality of restored sound events. As a general goal, we investigate how prior knowledge, such as score information can be integrated into DL-based learning to improve the interpretability of the trained models.