Open Access

ARTICLE

A Semantic Supervision Method for Abstractive Summarization

by Sunqiang Hu¹, Xiaoyu Li¹, Yu Deng^1,*, Yu Peng¹, Bin Lin², Shan Yang³

1 School of Information and Software Engineering, University of Electronic Science and Technology of China, Chengdu, 610054, China
2 School of Engineering, Sichuan Normal University, Chengdu, 610066, China
3 Department of Chemistry, Physics, and Atmospheric Sciences, Jackson State University, Jackson, MS, 39217, USA

* Corresponding Author: Yu Deng. Email:

Computers, Materials & Continua 2021, 69(1), 145-158. https://doi.org/10.32604/cmc.2021.017441

Received 30 January 2021; Accepted 17 March 2021; Issue published 04 June 2021

Abstract

In recent years, many text summarization models based on pre-training methods have achieved very good results. However, in these text summarization models, semantic deviations are easy to occur between the original input representation and the representation that passed multi-layer encoder, which may result in inconsistencies between the generated summary and the source text content. The Bidirectional Encoder Representations from Transformers (BERT) improves the performance of many tasks in Natural Language Processing (NLP). Although BERT has a strong capability to encode context, it lacks the fine-grained semantic representation. To solve these two problems, we proposed a semantic supervision method based on Capsule Network. Firstly, we extracted the fine-grained semantic representation of the input and encoded result in BERT by Capsule Network. Secondly, we used the fine-grained semantic representation of the input to supervise the fine-grained semantic representation of the encoded result. Then we evaluated our model on a popular Chinese social media dataset (LCSTS), and the result showed that our model achieved higher ROUGE scores (including R-1, R-2), and our model outperformed baseline systems. Finally, we conducted a comparative study on the stability of the model, and the experimental results showed that our model was more stable.

---

Keywords

---