Md Hasibur Rahman, Mohammed Arif Uddin, Zinnat Fowzia Ria, Rashedur M. Rahman^*

CMES-Computer Modeling in Engineering & Sciences, Vol.142, No.2, pp. 1637-1666, 2025, DOI:10.32604/cmes.2024.058329 - 27 January 2025

Abstract The rapid growth of digital data necessitates advanced natural language processing (NLP) models like BERT (Bidirectional Encoder Representations from Transformers), known for its superior performance in text classification. However, BERT’s size and computational demands limit its practicality, especially in resource-constrained settings. This research compresses the BERT base model for Bengali emotion classification through knowledge distillation (KD), pruning, and quantization techniques. Despite Bengali being the sixth most spoken language globally, NLP research in this area is limited. Our approach addresses this gap by creating an efficient BERT-based model for Bengali text. We have explored 20 combinations… More > Graphic Abstract

Ao Shen¹, Zhiquan Lai^1,*, Dongsheng Li^1,*, Xiaoyu Hu²

CMC-Computers, Materials & Continua, Vol.82, No.1, pp. 307-325, 2025, DOI:10.32604/cmc.2024.057491 - 03 January 2025

Abstract Large-scale Language Models (LLMs) have achieved significant breakthroughs in Natural Language Processing (NLP), driven by the pre-training and fine-tuning paradigm. While this approach allows models to specialize in specific tasks with reduced training costs, the substantial memory requirements during fine-tuning present a barrier to broader deployment. Parameter-Efficient Fine-Tuning (PEFT) techniques, such as Low-Rank Adaptation (LoRA), and parameter quantization methods have emerged as solutions to address these challenges by optimizing memory usage and computational efficiency. Among these, QLoRA, which combines PEFT and quantization, has demonstrated notable success in reducing memory footprints during fine-tuning, prompting the development… More >

Da-Chun Wu^*, Zong-Nan Shih

CMES-Computer Modeling in Engineering & Sciences, Vol.141, No.1, pp. 353-383, 2024, DOI:10.32604/cmes.2024.050813 - 20 August 2024

Abstract A new steganographic method by pixel-value differencing (PVD) using general quantization ranges of pixel pairs’ difference values is proposed. The objective of this method is to provide a data embedding technique with a range table with range widths not limited to powers of 2, extending PVD-based methods to enhance their flexibility and data-embedding rates without changing their capabilities to resist security attacks. Specifically, the conventional PVD technique partitions a grayscale image into 1 × 2 non-overlapping blocks. The entire range [0, 255] of all possible absolute values of the pixel pairs’ grayscale differences in the… More >

Fangzhou He^1,3, Ke Ding^1,2, Dingjiang Yan³, Jie Li^3,*, Jiajun Wang^1,2, Mingzhe Chen^1,2

CMC-Computers, Materials & Continua, Vol.80, No.2, pp. 3021-3045, 2024, DOI:10.32604/cmc.2024.053632 - 15 August 2024

Abstract Massive computational complexity and memory requirement of artificial intelligence models impede their deployability on edge computing devices of the Internet of Things (IoT). While Power-of-Two (PoT) quantization is proposed to improve the efficiency for edge inference of Deep Neural Networks (DNNs), existing PoT schemes require a huge amount of bit-wise manipulation and have large memory overhead, and their efficiency is bounded by the bottleneck of computation latency and memory footprint. To tackle this challenge, we present an efficient inference approach on the basis of PoT quantization and model compression. An integer-only scalar PoT quantization (IOS-PoT)… More >

Amna Khatoon¹, Weixing Wang^1,*, Asad Ullah², Limin Li^3,*, Mengfei Wang¹

CMC-Computers, Materials & Continua, Vol.80, No.1, pp. 527-546, 2024, DOI:10.32604/cmc.2024.051147 - 18 July 2024

Abstract Integrating Tiny Machine Learning (TinyML) with edge computing in remotely sensed images enhances the capabilities of road anomaly detection on a broader level. Constrained devices efficiently implement a Binary Neural Network (BNN) for road feature extraction, utilizing quantization and compression through a pruning strategy. The modifications resulted in a 28-fold decrease in memory usage and a 25% enhancement in inference speed while only experiencing a 2.5% decrease in accuracy. It showcases its superiority over conventional detection algorithms in different road image scenarios. Although constrained by computer resources and training datasets, our results indicate opportunities for More >

Jiqiang Chen, Ranran Han, Dongqing Zhang, Litao Ma^*

CMC-Computers, Materials & Continua, Vol.79, No.3, pp. 5067-5082, 2024, DOI:10.32604/cmc.2024.051494 - 20 June 2024

Abstract Imbalanced datasets are common in practical applications, and oversampling methods using fuzzy rules have been shown to enhance the classification performance of imbalanced data by taking into account the relationship between data attributes. However, the creation of fuzzy rules typically depends on expert knowledge, which may not fully leverage the label information in training data and may be subjective. To address this issue, a novel fuzzy rule oversampling approach is developed based on the learning vector quantization (LVQ) algorithm. In this method, the label information of the training data is utilized to determine the antecedent… More >

Yuejiao Wang, Zhong Ma^*, Chaojie Yang, Yu Yang, Lu Wei

CMC-Computers, Materials & Continua, Vol.79, No.1, pp. 819-836, 2024, DOI:10.32604/cmc.2024.047108 - 25 April 2024

Abstract The quantization algorithm compresses the original network by reducing the numerical bit width of the model, which improves the computation speed. Because different layers have different redundancy and sensitivity to data bit width. Reducing the data bit width will result in a loss of accuracy. Therefore, it is difficult to determine the optimal bit width for different parts of the network with guaranteed accuracy. Mixed precision quantization can effectively reduce the amount of computation while keeping the model accuracy basically unchanged. In this paper, a hardware-aware mixed precision quantization strategy optimal assignment algorithm adapted to… More >

Lu Wei, Zhong Ma^*, Chaojie Yang

CMES-Computer Modeling in Engineering & Sciences, Vol.138, No.1, pp. 981-1000, 2024, DOI:10.32604/cmes.2023.027085 - 22 September 2023

Abstract The demand for adopting neural networks in resource-constrained embedded devices is continuously increasing. Quantization is one of the most promising solutions to reduce computational cost and memory storage on embedded devices. In order to reduce the complexity and overhead of deploying neural networks on Integer-only hardware, most current quantization methods use a symmetric quantization mapping strategy to quantize a floating-point neural network into an integer network. However, although symmetric quantization has the advantage of easier implementation, it is sub-optimal for cases where the range could be skewed and not symmetric. This often comes at the… More > Graphic Abstract

Tuo Ma, Zhiwei Li, Qingjiang Li^*, Haijun Liu, Zhongjin Zhao, Yinan Wang

CMC-Computers, Materials & Continua, Vol.77, No.3, pp. 3237-3263, 2023, DOI:10.32604/cmc.2023.045948 - 26 December 2023

Abstract With the continuous development of deep learning, Deep Convolutional Neural Network (DCNN) has attracted wide attention in the industry due to its high accuracy in image classification. Compared with other DCNN hardware deployment platforms, Field Programmable Gate Array (FPGA) has the advantages of being programmable, low power consumption, parallelism, and low cost. However, the enormous amount of calculation of DCNN and the limited logic capacity of FPGA restrict the energy efficiency of the DCNN accelerator. The traditional sequential sliding window method can improve the throughput of the DCNN accelerator by data multiplexing, but this method’s… More >

Zerui Zhen¹, Zihao Wu², Lei Feng^1,*, Wenjing Li¹, Feng Qi¹, Shixuan Guo¹

CMC-Computers, Materials & Continua, Vol.75, No.2, pp. 2939-2955, 2023, DOI:10.32604/cmc.2023.036505 - 31 March 2023

Abstract Asynchronous federated learning (AsynFL) can effectively mitigate the impact of heterogeneity of edge nodes on joint training while satisfying participant user privacy protection and data security. However, the frequent exchange of massive data can lead to excess communication overhead between edge and central nodes regardless of whether the federated learning (FL) algorithm uses synchronous or asynchronous aggregation. Therefore, there is an urgent need for a method that can simultaneously take into account device heterogeneity and edge node energy consumption reduction. This paper proposes a novel Fixed-point Asynchronous Federated Learning (FixedAsynFL) algorithm, which could mitigate the… More >