Open Access

ARTICLE

Deep Neural Network Architecture Search via Decomposition-Based Multi-Objective Stochastic Fractal Search

by Hongshang Xu¹, Bei Dong^1,2,*, Xiaochang Liu¹, Xiaojun Wu^1,2

1 School of Computer Science, Shaanxi Normal University, Xi’an, 710119, China
2 Key Laboratory of Modern Teaching Technology, Ministry of Education, Xi’an, 710062, China

* Corresponding Author: Bei Dong. Email:

(This article belongs to the Special Issue: Artificial Intelligence Algorithm for Industrial Operation Application)

Intelligent Automation & Soft Computing 2023, 38(2), 185-202. https://doi.org/10.32604/iasc.2023.041177

Received 13 April 2023; Accepted 14 July 2023; Issue published 05 February 2024

Abstract

Deep neural networks often outperform classical machine learning algorithms in solving real-world problems. However, designing better networks usually requires domain expertise and consumes significant time and computing resources. Moreover, when the task changes, the original network architecture becomes outdated and requires redesigning. Thus, Neural Architecture Search (NAS) has gained attention as an effective approach to automatically generate optimal network architectures. Most NAS methods mainly focus on achieving high performance while ignoring architectural complexity. A myriad of research has revealed that network performance and structural complexity are often positively correlated. Nevertheless, complex network structures will bring enormous computing resources. To cope with this, we formulate the neural architecture search task as a multi-objective optimization problem, where an optimal architecture is learned by minimizing the classification error rate and the number of network parameters simultaneously. And then a decomposition-based multi-objective stochastic fractal search method is proposed to solve it. In view of the discrete property of the NAS problem, we discretize the stochastic fractal search step size so that the network architecture can be optimized more effectively. Additionally, two distinct update methods are employed in step size update stage to enhance the global and local search abilities adaptively. Furthermore, an information exchange mechanism between architectures is raised to accelerate the convergence process and improve the efficiency of the algorithm. Experimental studies show that the proposed algorithm has competitive performance comparable to many existing manual and automatic deep neural network generation approaches, which achieved a parameter-less and high-precision architecture with low-cost on each of the six benchmark datasets.

---

Keywords

---