Author(s)

Aleksy Nowaks

Affiliation(s)

Faculty of Mathematics, Informatics and Mechanics, The University of Warsaw, Poland. 

Corresponding Author

Aleksy Nowaks

ABSTRACT

The exponential growth of online information has intensified the need for efficient web news summarization techniques. This paper introduces the Self-Interactive Summarization Model (SISM), a novel approach that combines advanced deep learning methods with an innovative refined tuning process. SISM employs a two-stage strategy: extensive pre-training on a diverse dataset, followed by a specialized fine-tuning phase. We present a new, carefully curated dataset that reflects the varied nature of web news articles, enabling comprehensive model evaluation. Our experiments demonstrate SISM's superiority over existing state-of-the-art models, with significant improvements in ROUGE scores across multiple test sets. The study highlights the critical role of our refined tuning process in enhancing summarization quality and adaptability to diverse news content. SISM's performance underscores its potential to advance the field of automated web news summarization, offering more accurate and contextually relevant summaries.

KEYWORDS

Web news summarization; Deep learning; Self-Interactive summarization model; Natural language processing

CITE THIS PAPER

Aleksy Nowaks. SISM: A self-interactive approach to web news summarization using deep learning. Multidisciplinary Journal of Engineering and Technology. 2024, 1(1): 9-16. DOI: 10.61784/mjet3004.

REFERENCES

[1] Nenkova, A., McKeown, K. A survey of text summarization techniques. In Mining text data. Springer, Boston, MA. 2013: 43-76.

[2] Liu, M., Ma, Z., Li, J., Wu, YC., Wang, X. Deep-Learning-Based Pre-training and Refined Tuning for Web Summarization Software. IEEE Access, 2024, 12: 92120-92129.

[3] Sutskever, I., Vinyals, O., Le, QV. Sequence to sequence learning with neural networks. In Advances in neural information processing systems, 2024: 3104-3112.

[4] Cho, K., Van Merri?nboer, B., Gulcehre, C., Bahdanau, D., Bougares, F., Schwenk, H., Bengio, Y. Learning phrase representations using RNN encoder-decoder for statistical machine translation. arXiv preprint arXiv:1406.1078. 2014.

[5] Bahdanau, D., Cho, K., Bengio, Y. Neural machine translation by jointly learning to align and translate. arXiv preprint arXiv:1409.0473. 2014.

[6] Luong, M. T., Pham, H., Manning, CD. Effective approaches to attention-based neural machine translation. arXiv preprint arXiv:1508.04025. 2015.

[7] Cheng, J., Lapata, M. Neural summarization by extracting sentences and words. arXiv preprint arXiv:1603.07252. 2016.

[8] Nallapati, R., Zhai, F., Zhou, B. Summarunner: A recurrent neural network based sequence model for extractive summarization of documents. In Thirty-First AAAI Conference on Artificial Intelligence. 2017.

[9] See, A., Liu, P. J., Manning, CD. Get to the point: Summarization with pointer-generator networks. arXiv preprint arXiv:1704.04368. 2017.

[10] Lin, Z., Feng, M., Santos, C. N. D., Yu, M., Xiang, B., Zhou, B., Bengio, Y. A structured self-attentive sentence embedding. arXiv preprint arXiv:1703.03130. 2017.

[11] Erkan, G., Radev, DR. LexRank: Graph-based lexical centrality as salience in text summarization. Journal of artificial intelligence research, 2004, 22: 457-479.

[12] Nallapati, R., Zhai, F., Zhou, B. Summarunner: A recurrent neural network based sequence model for extractive summarization of documents. In Thirty-First AAAI Conference on Artificial Intelligence. 2017.

[13] Zhou, Q., Yang, N., Wei, F., Huang, S., Zhou, M., Zhao, T. Neural document summarization by jointly learning to score and select sentences. arXiv preprint arXiv:1807.02305. 2018.

[14] Liu, Y., Lapata, M. Text summarization with pretrained encoders. arXiv preprint arXiv:1908.08345. 2019.

[15] Rush, A. M., Chopra, S., Weston, J. A neural attention model for abstractive sentence summarization. arXiv preprint arXiv:1509.00685. 2015.

[16] Nallapati, R., Zhou, B., Gulcehre, C., Xiang, B. Abstractive text summarization using sequence-to-sequence rnns and beyond. arXiv preprint arXiv:1602.06023. 2016.

[17] Fan, A., Grangier, D., Auli, M. Controllable abstractive summarization. arXiv preprint arXiv:1711.05217. 2018.

[18] Xiao, W., Carenini, G. Extractive summarization of long documents by combining global and local context. arXiv preprint arXiv:1909.08089. 2019.

[19] Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N.,  Polosukhin, I. Attention is all you need. In Advances in neural information processing systems, 2017: 5998-6008.

[20] Zhang, X., Wei, F., Zhou, M. HIBERT: Document level pre-training of hierarchical bidirectional transformers for document summarization. arXiv preprint arXiv:1905.06566. 2019.

[21] Zhong, M., Liu, P., Chen, Y., Wang, D., Qiu, X., Huang, X. Extractive summarization as text matching. arXiv preprint arXiv:2004.08795. 2009.

[22] Wang, H., Liu, Q., Gao, Z., Nie, Y., Liu, Z., Zheng, H. Neural multi-task learning for aspect-based sentiment analysis. In 2019 International Joint Conference on Neural Networks (IJCNN). IEEE. 2019: 1-8.

[23] Narayan, S., Cohen, S. B., Lapata, M. Don't give me the details, just the summary! topic-aware convolutional neural networks for extreme summarization. arXiv preprint arXiv:1808.08745. 2018.

[24] Xu, J., Gan, Z., Cheng, Y., Liu, J. Discourse-aware neural extractive text summarization. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, 2020: 5021-5031.

[25] Jia, Y., Ye, Y., Feng, Y., Lai, Y., Yan, R., Zhao, D. Modeling discourse cohesion for discourse parsing via memory network. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics2018, 2: 429-434.

[26] Dong, Y., Shen, Y., Crawford, E., van Hoof, H., Cheung, JCK. BanditSum: Extractive summarization as a contextual bandit. arXiv preprint arXiv:1809.09672. 2018.

[27] Narayan, S., Cohen, S. B., Lapata, M. Don't give me the details, just the summary! topic-aware convolutional neural networks for extreme summarization. arXiv preprint arXiv:1808.08745. 2018.

[28] Fabbri, A. R., Li, I., She, T., Li, S., Radev, D. R. Multi-news: A large-scale multi-document summarization dataset and abstractive hierarchical model. arXiv preprint arXiv:1906.01749. 2019.

[29] Wang, X., Wu, Y. C., Ji, X., Fu, H. Algorithmic discrimination: examining its types and regulatory measures with emphasis on US legal practices. Frontiers in Artificial Intelligence, 2024, 7: 1320277.

[30] Zhang, T., Kishore, V., Wu, F., Weinberger, K. Q., Artzi, Y. Bertscore: Evaluating text generation with bert. arXiv preprint arXiv:1904.09675. 2019.

[31] Scialom, T., Dray, P. A., Lamprier, S., Piwowarski, B., Staiano, J. QuestEval: Summarization asks for fact-based evaluation. arXiv preprint arXiv:2103.12693. 2021.

[32] Wasson, M. Using leading text for news summaries: Evaluation results and implications for commercial summarization applications. In 36th Annual Meeting of the Association for Computational Linguistics and 17th International Conference on Computational Linguistics, 1998: 2.

[33] Brandow, R., Mitze, K., Rau, LF. Automatic condensation of electronic publications by sentence selection. Information Processing & Management, 1995, 31(5): 675-685.

[34] Devlin, J., Chang, M. W., Lee, K., Toutanova, K. Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:1810.04805. 2018.

[35] Radford, A., Wu, J., Child, R., Luan, D., Amodei, D., Sutskever, I. Language models are unsupervised multitask learners. OpenAI blog, 2019, 1(8): 9.

[36] Liu, P. J., Saleh, M., Pot, E., Goodrich, B., Sepassi, R., Kaiser, L., Shazeer, N. Generating wikipedia by summarizing long sequences. arXiv preprint arXiv:1801.10198. 2018.

[37] Chen, X., Liu, M., Niu, Y., Wang, X., Wu, YC. Deep-Learning-Based Lithium Battery Defect Detection via Cross-Domain Generalization. IEEE Access, 2024, 12: 78505-78514

[38] Wang, X., Wu, Y. C., Ma, Z. Blockchain in the courtroom: exploring its evidentiary significance and procedural implications in US judicial processes. Frontiers in Blockchain, 2024, 7, 1306058.

[39] Ma, Z., Chen, X., Sun, T., Wang, X., Wu, Y. C., Zhou, M. Blockchain-Based Zero-Trust Supply Chain Security Integrated with Deep Reinforcement Learning for Inventory Optimization. Future Internet, 2024, 16(5): 163.

[40] Wang, X., Wu, Y. C., Zhou, M., Fu, H. Beyond Surveillance: Privacy, Ethics, and Regulations in Face Recognition Technology. Frontiers in Big Data, 2024, 7, 1337465.

[41] Kedzie, C., McKeown, K., Daume III, H. Content selection in deep learning models of summarization. arXiv preprint arXiv:1810.12343. 2018.