Lirong Yin¹, Lei Wang¹, Zhuohang Cai², Siyu Lu^2,*, Ruiyang Wang², Ahmed AlSanad³, Salman A. AlQahtani³, Xiaobing Chen⁴, Zhengtong Yin⁵, Xiaolu Li⁶, Wenfeng Zheng^2,3,*

CMES-Computer Modeling in Engineering & Sciences, Vol.141, No.1, pp. 771-786, 2024, DOI:10.32604/cmes.2024.052622 - 20 August 2024

Abstract Recent advancements in natural language processing have given rise to numerous pre-training language models in question-answering systems. However, with the constant evolution of algorithms, data, and computing power, the increasing size and complexity of these models have led to increased training costs and reduced efficiency. This study aims to minimize the inference time of such models while maintaining computational performance. It also proposes a novel Distillation model for PAL-BERT (DPAL-BERT), specifically, employs knowledge distillation, using the PAL-BERT model as the teacher model to train two student models: DPAL-BERT-Bi and DPAL-BERT-C. This research enhances the dataset More >

Fangzhou He^1,3, Ke Ding^1,2, Dingjiang Yan³, Jie Li^3,*, Jiajun Wang^1,2, Mingzhe Chen^1,2

CMC-Computers, Materials & Continua, Vol.80, No.2, pp. 3021-3045, 2024, DOI:10.32604/cmc.2024.053632 - 15 August 2024

Abstract Massive computational complexity and memory requirement of artificial intelligence models impede their deployability on edge computing devices of the Internet of Things (IoT). While Power-of-Two (PoT) quantization is proposed to improve the efficiency for edge inference of Deep Neural Networks (DNNs), existing PoT schemes require a huge amount of bit-wise manipulation and have large memory overhead, and their efficiency is bounded by the bottleneck of computation latency and memory footprint. To tackle this challenge, we present an efficient inference approach on the basis of PoT quantization and model compression. An integer-only scalar PoT quantization (IOS-PoT)… More >

Amna Khatoon¹, Weixing Wang^1,*, Asad Ullah², Limin Li^3,*, Mengfei Wang¹

CMC-Computers, Materials & Continua, Vol.80, No.1, pp. 527-546, 2024, DOI:10.32604/cmc.2024.051147 - 18 July 2024

Abstract Integrating Tiny Machine Learning (TinyML) with edge computing in remotely sensed images enhances the capabilities of road anomaly detection on a broader level. Constrained devices efficiently implement a Binary Neural Network (BNN) for road feature extraction, utilizing quantization and compression through a pruning strategy. The modifications resulted in a 28-fold decrease in memory usage and a 25% enhancement in inference speed while only experiencing a 2.5% decrease in accuracy. It showcases its superiority over conventional detection algorithms in different road image scenarios. Although constrained by computer resources and training datasets, our results indicate opportunities for More >

Hassen Louati^1,2, Ali Louati^3,*, Elham Kariri³, Slim Bechikh²

CMES-Computer Modeling in Engineering & Sciences, Vol.138, No.3, pp. 2519-2547, 2024, DOI:10.32604/cmes.2023.030806 - 15 December 2023

Abstract Recent developments in Computer Vision have presented novel opportunities to tackle complex healthcare issues, particularly in the field of lung disease diagnosis. One promising avenue involves the use of chest X-Rays, which are commonly utilized in radiology. To fully exploit their potential, researchers have suggested utilizing deep learning methods to construct computer-aided diagnostic systems. However, constructing and compressing these systems presents a significant challenge, as it relies heavily on the expertise of data scientists. To tackle this issue, we propose an automated approach that utilizes an evolutionary algorithm (EA) to optimize the design and compression More >

Zerui Zhen¹, Zihao Wu², Lei Feng^1,*, Wenjing Li¹, Feng Qi¹, Shixuan Guo¹

CMC-Computers, Materials & Continua, Vol.75, No.2, pp. 2939-2955, 2023, DOI:10.32604/cmc.2023.036505 - 31 March 2023

Abstract Asynchronous federated learning (AsynFL) can effectively mitigate the impact of heterogeneity of edge nodes on joint training while satisfying participant user privacy protection and data security. However, the frequent exchange of massive data can lead to excess communication overhead between edge and central nodes regardless of whether the federated learning (FL) algorithm uses synchronous or asynchronous aggregation. Therefore, there is an urgent need for a method that can simultaneously take into account device heterogeneity and edge node energy consumption reduction. This paper proposes a novel Fixed-point Asynchronous Federated Learning (FixedAsynFL) algorithm, which could mitigate the… More >