Adversarial network compression
Belagiannis V, Farshad A, Galasso F (2019)
Publication Type: Conference contribution
Publication year: 2019
Journal
Publisher: Springer Verlag
Book Volume: 11132 LNCS
Pages Range: 431-449
Conference Proceedings Title: Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Event location: Munich
ISBN: 9783030110178
DOI: 10.1007/978-3-030-11018-5_37
Abstract
Neural network compression has recently received much attention due to the computational requirements of modern deep models. In this work, our objective is to transfer knowledge from a deep and accurate model to a smaller one. Our contributions are threefold: (i) we propose an adversarial network compression approach to train the small student network to mimic the large teacher, without the need for labels during training; (ii) we introduce a regularization scheme to prevent a trivially-strong discriminator without reducing the network capacity and (iii) our approach generalizes on different teacher-student models. In an extensive evaluation on five standard datasets, we show that our student has small accuracy drop, achieves better performance than other knowledge transfer approaches and it surpasses the performance of the same network trained with labels. In addition, we demonstrate state-of-the-art results compared to other compression strategies.
Authors with CRIS profile
Involved external institutions
How to cite
APA:
Belagiannis, V., Farshad, A., & Galasso, F. (2019). Adversarial network compression. In Laura Leal-Taixé, Stefan Roth (Eds.), Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics) (pp. 431-449). Munich, DE: Springer Verlag.
MLA:
Belagiannis, Vasileios, Azade Farshad, and Fabio Galasso. "Adversarial network compression." Proceedings of the 15th European Conference on Computer Vision, ECCV 2018, Munich Ed. Laura Leal-Taixé, Stefan Roth, Springer Verlag, 2019. 431-449.
BibTeX: Download