M-HOF-Opt: Multi-Objective Hierarchical Output Feedback Optimization via Multiplier Induced Loss Landscape Scheduling

Sun X, Chen N, Gossmann A, Wohlrapp M, Xing Y, Dorigatti E, Feistner C, Drost F, Scarcella D, Beer LH, Marr C (2025)

Publication Type: Conference contribution

Publication year: 2025

Publisher: ML Research Press

Book Volume: 258

Pages Range: 5149-5157

Conference Proceedings Title: Proceedings of Machine Learning Research

Event location: Mai Khao, THA

Abstract

A probabilistic graphical model is proposed, modeling the joint model parameter and multiplier evolution, with a hypervolume based likelihood, promoting multi-objective descent in structural risk minimization. We address multi-objective model parameter optimization via a surrogate single objective penalty loss with time-varying multipliers, equivalent to online scheduling of loss landscape. The multiobjective descent goal is dispatched hierarchically into a series of constraint optimization sub-problems with shrinking bounds according to Pareto dominance. The bound serves as setpoint for the low-level multiplier controller to schedule loss landscapes via output feedback of each loss term. Our method forms closed loop of model parameter dynamic, circumvents excessive memory requirements and extra computational burden of existing multiobjective deep learning methods, and is robust against controller hyperparameter variation, demonstrated on domain generalization tasks.

Authors with CRIS profile

Carla Feistner Professur für Angewandte Geologie und Modellierung von Umweltsystemen

Involved external institutions

Center for Devices and Radiological Health, Food and Drug Administration (CDRH) US United States (USA) (US) Technische Universität München (TUM) DE Germany (DE) Royal Institute of Technology / Kungliga Tekniska Högskolan (KTH) SE Sweden (SE) Helmholtz Zentrum München - Deutsches Forschungszentrum für Gesundheit und Umwelt (HMGU) / Helmholtz Munich DE Germany (DE) Volkswagen AG DE Germany (DE)

How to cite

APA:

Sun, X., Chen, N., Gossmann, A., Wohlrapp, M., Xing, Y., Dorigatti, E.,... Marr, C. (2025). M-HOF-Opt: Multi-Objective Hierarchical Output Feedback Optimization via Multiplier Induced Loss Landscape Scheduling. In Yingzhen Li, Stephan Mandt, Shipra Agrawal, Emtiyaz Khan (Eds.), Proceedings of Machine Learning Research (pp. 5149-5157). Mai Khao, THA: ML Research Press.

MLA:

Sun, Xudong, et al. "M-HOF-Opt: Multi-Objective Hierarchical Output Feedback Optimization via Multiplier Induced Loss Landscape Scheduling." Proceedings of the 28th International Conference on Artificial Intelligence and Statistics, AISTATS 2025, Mai Khao, THA Ed. Yingzhen Li, Stephan Mandt, Shipra Agrawal, Emtiyaz Khan, ML Research Press, 2025. 5149-5157.

BibTeX: Download