AdaptForever: Elastic and Mutual Learning for Continuous NLP Task Mastery
Ke Chen1,2, Cheng Peng1,2, Xinyang He1,2, Jiakang Sun1,2, Xu Liu1,2, Xiaolin Qin1,2,*, Yong Zhong1,2,*
CMC-Computers, Materials & Continua, Vol.82, No.3, pp. 4003-4019, 2025, DOI:10.32604/cmc.2025.057443
- 06 March 2025
(This article belongs to the Special Issue: Advancements in Natural Language Processing (NLP) and Fuzzy Logic)
Abstract In natural language processing (NLP), managing multiple downstream tasks through fine-tuning pre-trained models often requires maintaining separate task-specific models, leading to practical inefficiencies. To address this challenge, we introduce AdaptForever, a novel approach that enables continuous mastery of NLP tasks through the integration of elastic and mutual learning strategies with a stochastic expert mechanism. Our method freezes the pre-trained model weights while incorporating adapters enhanced with mutual learning capabilities, facilitating effective knowledge transfer from previous tasks to new ones. By combining Elastic Weight Consolidation (EWC) for knowledge preservation with specialized regularization terms, AdaptForever successfully maintains More >