Lirong Yin¹, Lei Wang¹, Zhuohang Cai², Siyu Lu^2,*, Ruiyang Wang², Ahmed AlSanad³, Salman A. AlQahtani³, Xiaobing Chen⁴, Zhengtong Yin⁵, Xiaolu Li⁶, Wenfeng Zheng^2,3,*

CMES-Computer Modeling in Engineering & Sciences, Vol.141, No.1, pp. 771-786, 2024, DOI:10.32604/cmes.2024.052622 - 20 August 2024

Abstract Recent advancements in natural language processing have given rise to numerous pre-training language models in question-answering systems. However, with the constant evolution of algorithms, data, and computing power, the increasing size and complexity of these models have led to increased training costs and reduced efficiency. This study aims to minimize the inference time of such models while maintaining computational performance. It also proposes a novel Distillation model for PAL-BERT (DPAL-BERT), specifically, employs knowledge distillation, using the PAL-BERT model as the teacher model to train two student models: DPAL-BERT-Bi and DPAL-BERT-C. This research enhances the dataset More >

Yaru Song^*, Huahu Xu, Dikai Fang

Intelligent Automation & Soft Computing, Vol.39, No.3, pp. 397-416, 2024, DOI:10.32604/iasc.2023.040521 - 11 July 2024

Abstract Visual Question Answering (VQA) has sparked widespread interest as a crucial task in integrating vision and language. VQA primarily uses attention mechanisms to effectively answer questions to associate relevant visual regions with input questions. The detection-based features extracted by the object detection network aim to acquire the visual attention distribution on a predetermined detection frame and provide object-level insights to answer questions about foreground objects more effectively. However, it cannot answer the question about the background forms without detection boxes due to the lack of fine-grained details, which is the advantage of grid-based features. In… More >

Wenfeng Zheng¹, Siyu Lu¹, Zhuohang Cai¹, Ruiyang Wang¹, Lei Wang², Lirong Yin^2,*

CMES-Computer Modeling in Engineering & Sciences, Vol.139, No.3, pp. 2729-2745, 2024, DOI:10.32604/cmes.2023.046692 - 11 March 2024

Abstract In the field of natural language processing (NLP), there have been various pre-training language models in recent years, with question answering systems gaining significant attention. However, as algorithms, data, and computing power advance, the issue of increasingly larger models and a growing number of parameters has surfaced. Consequently, model training has become more costly and less efficient. To enhance the efficiency and accuracy of the training process while reducing the model volume, this paper proposes a first-order pruning model PAL-BERT based on the ALBERT model according to the characteristics of question-answering (QA) system and language More >

Feng Yan¹, Wushouer Silamu², Yanbing Li^1,*

CMC-Computers, Materials & Continua, Vol.76, No.1, pp. 65-80, 2023, DOI:10.32604/cmc.2023.038177 - 08 June 2023

Abstract Visual question answering (VQA) requires a deep understanding of images and their corresponding textual questions to answer questions about images more accurately. However, existing models tend to ignore the implicit knowledge in the images and focus only on the visual information in the images, which limits the understanding depth of the image content. The images contain more than just visual objects, some images contain textual information about the scene, and slightly more complex images contain relationships between individual visual objects. Firstly, this paper proposes a model using image description for feature enhancement. This model encodes… More >

Siyu Lu¹, Yueming Ding¹, Zhengtong Yin², Mingzhe Liu^3,*, Xuan Liu⁴, Wenfeng Zheng^1,*, Lirong Yin⁵

Computer Systems Science and Engineering, Vol.47, No.1, pp. 1149-1161, 2023, DOI:10.32604/csse.2023.038598 - 26 May 2023

Abstract Visual question answering (VQA) has attracted more and more attention in computer vision and natural language processing. Scholars are committed to studying how to better integrate image features and text features to achieve better results in VQA tasks. Analysis of all features may cause information redundancy and heavy computational burden. Attention mechanism is a wise way to solve this problem. However, using single attention mechanism may cause incomplete concern of features. This paper improves the attention mechanism method and proposes a hybrid attention mechanism that combines the spatial attention mechanism method and the channel attention More >

Hong Li^1,*, Jianjun Li¹, Guohui Li¹, Rong Gao², Lingyu Yan²

CMC-Computers, Materials & Continua, Vol.75, No.1, pp. 1687-1709, 2023, DOI:10.32604/cmc.2023.035239 - 06 February 2023

Abstract Expert Recommendation (ER) aims to identify domain experts with high expertise and willingness to provide answers to questions in Community Question Answering (CQA) web services. How to model questions and users in the heterogeneous content network is critical to this task. Most traditional methods focus on modeling questions and users based on the textual content left in the community while ignoring the structural properties of heterogeneous CQA networks and always suffering from textual data sparsity issues. Recent approaches take advantage of structural proximities between nodes and attempt to fuse the textual content of nodes for… More >

Seongung Jo¹, Heung-Seon Oh^1,*, Sanghun Im¹, Gibaeg Kim¹, Seonho Kim²

CMC-Computers, Materials & Continua, Vol.74, No.2, pp. 2967-2980, 2023, DOI:10.32604/cmc.2023.031983 - 31 October 2022

Abstract Analyzing Research and Development (R&D) trends is important because it can influence future decisions regarding R&D direction. In typical trend analysis, topic or technology taxonomies are employed to compute the popularities of the topics or codes over time. Although it is simple and effective, the taxonomies are difficult to manage because new technologies are introduced rapidly. Therefore, recent studies exploit deep learning to extract pre-defined targets such as problems and solutions. Based on the recent advances in question answering (QA) using deep learning, we adopt a multi-turn QA model to extract problems and solutions from… More >

Byeongmin Choi¹, YongHyun Lee¹, Yeunwoong Kyung², Eunchan Kim^3,*

Intelligent Automation & Soft Computing, Vol.36, No.1, pp. 71-82, 2023, DOI:10.32604/iasc.2023.032783 - 29 September 2022

Abstract Recently, pre-trained language representation models such as bidirectional encoder representations from transformers (BERT) have been performing well in commonsense question answering (CSQA). However, there is a problem that the models do not directly use explicit information of knowledge sources existing outside. To augment this, additional methods such as knowledge-aware graph network (KagNet) and multi-hop graph relation network (MHGRN) have been proposed. In this study, we propose to use the latest pre-trained language model a lite bidirectional encoder representations from transformers (ALBERT) with knowledge graph information extraction technique. We also propose to applying the novel method, More >

Thi Thanh Sang Nguyen^*, Dang Huu Trong Ho, Ngoc Tram Anh Nguyen

Intelligent Automation & Soft Computing, Vol.36, No.1, pp. 601-616, 2023, DOI:10.32604/iasc.2023.032080 - 29 September 2022

Abstract Question-Answer systems are now very popular and crucial to support human in automatically responding frequent questions in many fields. However, these systems depend on learning methods and training data. Therefore, it is necessary to prepare such a good dataset, but it is not an easy job. An ontology-based domain knowledge base is able to help to reason semantic information and make effective answers given user questions. This study proposes a novel chatbot model involving ontology to generate efficient responses automatically. A case study of admissions advising at the International University–VNU HCMC is taken into account… More >

Rachid Karra^*, Abdelali Lasfar

Intelligent Automation & Soft Computing, Vol.35, No.1, pp. 335-349, 2023, DOI:10.32604/iasc.2023.026695 - 06 June 2022

Abstract In contrast with the research of new models, little attention has been paid to the impact of low or high-quality data feeding a dialogue system. The present paper makes the first attempt to fill this gap by extending our previous work on question-answering (QA) systems by investigating the effect of misspelling on QA agents and how context changes can enhance the responses. Instead of using large language models trained on huge datasets, we propose a method that enhances the model's score by modifying only the quality and structure of the data feed to the model.… More >