Sequence Alignment-based Similarity Metric in Evolutionary Neural Architecture Search

Pava MÁ, Groh R, Kist A (2024)

Publication Type: Conference contribution

Publication year: 2024

Publisher: ML Research Press

Book Volume: 256

Conference Proceedings Title: Proceedings of Machine Learning Research

Event location: Paris FR

Abstract

Neural Architecture Search (NAS) has emerged as a powerful method for automating the design of deep neural networks across diverse applications, with evolutionary optimization showing particular promise in addressing its intricate demands. However, the effectiveness of this approach highly depends on balancing exploration and exploitation, ensuring that the search does not prematurely converge to suboptimal solutions while still achieving near-optimal outcomes. This paper addresses this challenge by proposing a novel similarity metric inspired by global sequence alignment from biology. Unlike most of the existing methods that require trained model weights for comparison, our metric operates directly on neural network architectures within the defined search space, eliminating the need for model training when comparing two architectures. We outline the computation of the normalized similarity metric and demonstrate its application in quantifying diversity within populations in evolutionary NAS. Experimental results conducted on popular datasets for image classification, such as CIFAR-10, CIFAR-100, and ImageNet16-120, show the effectiveness of our approach in guiding diversity based on our suggested control function. Additionally, we highlight the usefulness of our similarity metric in comparing individuals to identify advantageous or disadvantageous architectural design choices. The code is available at https://github.com/ankilab/evonas_similarity_metric.

Authors with CRIS profile

René Groh Juniorprofessur für Artificial Intelligence in Communication Disorders Andreas Kist Juniorprofessur für Artificial Intelligence in Communication Disorders

How to cite

APA:

Pava, M.Á., Groh, R., & Kist, A. (2024). Sequence Alignment-based Similarity Metric in Evolutionary Neural Architecture Search. In Katharina Eggensperger, Roman Garnett, Joaquin Vanschoren, Marius Lindauer, Jacob R. Gardner (Eds.), Proceedings of Machine Learning Research. Paris, FR: ML Research Press.

MLA:

Pava, Mateo Ávila, René Groh, and Andreas Kist. "Sequence Alignment-based Similarity Metric in Evolutionary Neural Architecture Search." Proceedings of the 3rd International Conference on Automated Machine Learning, AutoML 2024, Paris Ed. Katharina Eggensperger, Roman Garnett, Joaquin Vanschoren, Marius Lindauer, Jacob R. Gardner, ML Research Press, 2024.

BibTeX: Download