SISM: A SELF-INTERACTIVE APPROACH TO WEB NEWS SUMMARIZATION USING DEEP LEARNING
Keywords:
Web news summarization, Deep learning, Self-Interactive summarization model, Natural language processingAbstract
The exponential growth of online information has intensified the need for efficient web news summarization techniques. This paper introduces the Self-Interactive Summarization Model (SISM), a novel approach that combines advanced deep learning methods with an innovative refined tuning process. SISM employs a two-stage strategy: extensive pre-training on a diverse dataset, followed by a specialized fine-tuning phase. We present a new, carefully curated dataset that reflects the varied nature of web news articles, enabling comprehensive model evaluation. Our experiments demonstrate SISM's superiority over existing state-of-the-art models, with significant improvements in ROUGE scores across multiple test sets. The study highlights the critical role of our refined tuning process in enhancing summarization quality and adaptability to diverse news content. SISM's performance underscores its potential to advance the field of automated web news summarization, offering more accurate and contextually relevant summaries.References
[1] Nenkova, A., McKeown, K. A survey of text summarization techniques. In Mining text data. Springer, Boston, MA. 2013: 43-76.
[2] Liu, M., Ma, Z., Li, J., Wu, YC., Wang, X. Deep-Learning-Based Pre-training and Refined Tuning for Web Summarization Software. IEEE Access, 2024, 12: 92120-92129.
[3] Sutskever, I., Vinyals, O., Le, QV. Sequence to sequence learning with neural networks. In Advances in neural information processing systems, 2024: 3104-3112.
[4] Cho, K., Van Merri?nboer, B., Gulcehre, C., Bahdanau, D., Bougares, F., Schwenk, H., Bengio, Y. Learning phrase representations using RNN encoder-decoder for statistical machine translation. arXiv preprint arXiv:1406.1078. 2014.
[5] Bahdanau, D., Cho, K., Bengio, Y. Neural machine translation by jointly learning to align and translate. arXiv preprint arXiv:1409.0473. 2014.
[6] Luong, M. T., Pham, H., Manning, CD. Effective approaches to attention-based neural machine translation. arXiv preprint arXiv:1508.04025. 2015.
[7] Cheng, J., Lapata, M. Neural summarization by extracting sentences and words. arXiv preprint arXiv:1603.07252. 2016.
[8] Nallapati, R., Zhai, F., Zhou, B. Summarunner: A recurrent neural network based sequence model for extractive summarization of documents. In Thirty-First AAAI Conference on Artificial Intelligence. 2017.
[9] See, A., Liu, P. J., Manning, CD. Get to the point: Summarization with pointer-generator networks. arXiv preprint arXiv:1704.04368. 2017.
[10] Lin, Z., Feng, M., Santos, C. N. D., Yu, M., Xiang, B., Zhou, B., Bengio, Y. A structured self-attentive sentence embedding. arXiv preprint arXiv:1703.03130. 2017.
[11] Erkan, G., Radev, DR. LexRank: Graph-based lexical centrality as salience in text summarization. Journal of artificial intelligence research, 2004, 22: 457-479.
[12] Nallapati, R., Zhai, F., Zhou, B. Summarunner: A recurrent neural network based sequence model for extractive summarization of documents. In Thirty-First AAAI Conference on Artificial Intelligence. 2017.
[13] Zhou, Q., Yang, N., Wei, F., Huang, S., Zhou, M., Zhao, T. Neural document summarization by jointly learning to score and select sentences. arXiv preprint arXiv:1807.02305. 2018.
[14] Liu, Y., Lapata, M. Text summarization with pretrained encoders. arXiv preprint arXiv:1908.08345. 2019.
[15] Rush, A. M., Chopra, S., Weston, J. A neural attention model for abstractive sentence summarization. arXiv preprint arXiv:1509.00685. 2015.
[16] Nallapati, R., Zhou, B., Gulcehre, C., Xiang, B. Abstractive text summarization using sequence-to-sequence rnns and beyond. arXiv preprint arXiv:1602.06023. 2016.
[17] Fan, A., Grangier, D., Auli, M. Controllable abstractive summarization. arXiv preprint arXiv:1711.05217. 2018.
[18] Xiao, W., Carenini, G. Extractive summarization of long documents by combining global and local context. arXiv preprint arXiv:1909.08089. 2019.
[19] Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Polosukhin, I. Attention is all you need. In Advances in neural information processing systems, 2017: 5998-6008.
[20] Zhang, X., Wei, F., Zhou, M. HIBERT: Document level pre-training of hierarchical bidirectional transformers for document summarization. arXiv preprint arXiv:1905.06566. 2019.
[21] Zhong, M., Liu, P., Chen, Y., Wang, D., Qiu, X., Huang, X. Extractive summarization as text matching. arXiv preprint arXiv:2004.08795. 2009.
[22] Wang, H., Liu, Q., Gao, Z., Nie, Y., Liu, Z., Zheng, H. Neural multi-task learning for aspect-based sentiment analysis. In 2019 International Joint Conference on Neural Networks (IJCNN). IEEE. 2019: 1-8.
[23] Narayan, S., Cohen, S. B., Lapata, M. Don't give me the details, just the summary! topic-aware convolutional neural networks for extreme summarization. arXiv preprint arXiv:1808.08745. 2018.
[24] Xu, J., Gan, Z., Cheng, Y., Liu, J. Discourse-aware neural extractive text summarization. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, 2020: 5021-5031.
[25] Jia, Y., Ye, Y., Feng, Y., Lai, Y., Yan, R., Zhao, D. Modeling discourse cohesion for discourse parsing via memory network. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics2018, 2: 429-434.
[26] Dong, Y., Shen, Y., Crawford, E., van Hoof, H., Cheung, JCK. BanditSum: Extractive summarization as a contextual bandit. arXiv preprint arXiv:1809.09672. 2018.
[27] Narayan, S., Cohen, S. B., Lapata, M. Don't give me the details, just the summary! topic-aware convolutional neural networks for extreme summarization. arXiv preprint arXiv:1808.08745. 2018.
[28] Fabbri, A. R., Li, I., She, T., Li, S., Radev, D. R. Multi-news: A large-scale multi-document summarization dataset and abstractive hierarchical model. arXiv preprint arXiv:1906.01749. 2019.
[29] Wang, X., Wu, Y. C., Ji, X., Fu, H. Algorithmic discrimination: examining its types and regulatory measures with emphasis on US legal practices. Frontiers in Artificial Intelligence, 2024, 7: 1320277.
[30] Zhang, T., Kishore, V., Wu, F., Weinberger, K. Q., Artzi, Y. Bertscore: Evaluating text generation with bert. arXiv preprint arXiv:1904.09675. 2019.