Open Access

ARTICLE

Guided Dropout: Improving Deep Networks Without Increased Computation

Yifeng Liu¹, Yangyang Li^1,*, Zhongxiong Xu¹, Xiaohan Liu¹, Haiyong Xie², Huacheng Zeng³

1 National Engineering Research Center for Risk Perception and Prevention (NERC-RPP), CAEIT, Beijing, 100041, China
2 University of Science and Technology of China, Hefei, Anhui, 230026, China
3 Department of Computer Science and Engineering, Michigan State University, East Lansing, MI, 48824, USA

* Corresponding Author: Yangyang Li. Email:

Intelligent Automation & Soft Computing 2023, 36(3), 2519-2528. https://doi.org/10.32604/iasc.2023.033286

Received 13 June 2022; Accepted 26 October 2022; Issue published 15 March 2023

Abstract

Deep convolution neural networks are going deeper and deeper. However, the complexity of models is prone to overfitting in training. Dropout, one of the crucial tricks, prevents units from co-adapting too much by randomly dropping neurons during training. It effectively improves the performance of deep networks but ignores the importance of the differences between neurons. To optimize this issue, this paper presents a new dropout method called guided dropout, which selects the neurons to switch off according to the differences between the convolution kernel and preserves the informative neurons. It uses an unsupervised clustering algorithm to cluster similar neurons in each hidden layer, and dropout uses a certain probability within each cluster. Thereby this would preserve the hidden layer neurons with different roles while maintaining the model’s scarcity and generalization, which effectively improves the role of the hidden layer neurons in learning the features. We evaluated our approach compared with two standard dropout networks on three well-established public object detection datasets. Experimental results on multiple datasets show that the method proposed in this paper has been improved on false positives, precision-recall curve and average precision without increasing the amount of computation. It can be seen that the increased performance of guided dropout is thanks to shallow learning in the networks. The concept of guided dropout would be beneficial to the other vision tasks.

---

Keywords

---