Interpolation and approximation via Momentum ResNets and Neural ODEs

Ruiz-Balet D, Affili E, Zuazua Iriondo E (2022)

Publication Language: English

Publication Status: Submitted

Publication Type: Journal article

Future Publication Type: Journal article

Publication year: 2022

Journal

IEEE Control Systems Letters Institute of Electrical and Electronics Engineers (IEEE)

Book Volume: 162

URI: https://dcn.nat.fau.eu/wp-content/uploads/InterpolationResNets.pdf

DOI: 10.1016/j.sysconle.2022.105182

Open Access Link: https://dcn.nat.fau.eu/wp-content/uploads/InterpolationResNets.pdf

Abstract

In this article, we explore the effects of memory terms in continuous-layer Deep Residual Networks by studying Neural ODEs (NODEs). We investigate two types of models. On one side, we consider the case of Residual Neural Networks with dependence on multiple layers, more precisely Momentum ResNets. On the other side, we analyse a Neural ODE with auxiliary states playing the role of memory states. We examine the interpolation and universal approximation properties for both architectures through a simultaneous control perspective. We also prove the ability of the second model to represent sophisticated maps, such as parametrizations of time-dependent functions. Numerical simulations complement our study.

Authors with CRIS profile

Enrique Zuazua Iriondo Lehrstuhl für Dynamics, Control, Machine Learning and Numerics (Alexander von Humboldt-Professur)

Additional Organisation(s)

Lehrstuhl für Dynamics, Control, Machine Learning and Numerics (Alexander von Humboldt-Professur)

Involved external institutions

University of Deusto / Universidad de Deusto ES Spain (ES) Universidad Autónoma de Madrid (UAM) ES Spain (ES)

How to cite

APA:

Ruiz-Balet, D., Affili, E., & Zuazua Iriondo, E. (2022). Interpolation and approximation via Momentum ResNets and Neural ODEs. IEEE Control Systems Letters, 162. https://doi.org/10.1016/j.sysconle.2022.105182

MLA:

Ruiz-Balet, Domènec, Elisa Affili, and Enrique Zuazua Iriondo. "Interpolation and approximation via Momentum ResNets and Neural ODEs." IEEE Control Systems Letters 162 (2022).

BibTeX: Download