Author(s)

Matthew Walkman, Lindsay McGregory^*

Affiliation(s)

School of Computer Science, University of Waterloo, Waterloo, Canada.

Corresponding Author

Lindsay McGregory

ABSTRACT

This paper introduces SummaGAN, a novel application of Generative Adversarial Networks (GANs) for text summarization. Unlike traditional summarization methods that rely on extractive techniques, SummaGAN uses adversarial learning to generate coherent and contextually accurate summaries. The model includes a transformer-based generator that creates summaries and a discriminator that evaluates their quality, guiding the generator to produce outputs that closely mimic human-written summaries. A large, diverse dataset of over 100,000 articles from domains such as news, scientific literature, and blogs was used to train and fine-tune the model. Experimental results show that SummaGAN significantly outperforms existing baseline models, including traditional extractive summarizers and advanced abstractive models, across multiple evaluation metrics such as ROUGE, BLEU, METEOR, and the newly introduced Coherence and Consistency Score (CCS). SummaGAN achieved a 15% improvement in ROUGE-1 scores and a 20% enhancement in BLEU scores, indicating better summary relevance and fluency. The CCS metric highlights SummaGAN's superior ability to maintain the logical flow and factual accuracy of the source text. This research demonstrates the potential of GANs to address challenges in text summarization, such as redundancy and loss of meaning, through dynamic adversarial learning. The integration of GANs with transformer architectures presents a robust framework for future NLP advancements. Future research will explore scaling the model for larger datasets, applying it in multilingual contexts, and refining the adversarial training process for improved efficiency and performance.

KEYWORDS

SummaGAN; Generative Adversarial Networks (GANs); Natural language processing; Text summarization technology

CITE THIS PAPER

Matthew Walkman, Lindsay McGregory. SummaGAN: enhancing web news summarization through generative adversarial networks. World Journal of Information Technology. 2024, 2(2): 22-27. DOI: 10.61784/wjit3001.

REFERENCES

[1] Jones, K. S.  Automatic summarizing: The state of the art. Information Processing & Management. 2007, 43(6): 1449-1481.

[2] Ma, Z., Chen, X., Sun, T., Wang, X., Wu, Y. C., & Zhou, M. Blockchain-Based Zero-Trust Supply Chain Security Integrated with Deep Reinforcement Learning for Inventory Optimization. Future Internet. 2024, 16(5): 163.

[3] Dorr, B. J., Zajic, D., & Schwartz, R. Hedge Trimmer: A parse-and-trim approach to headline generation. In Proceedings of the HLT-NAACL 03 on Text summarization Workshop. Association for Computational Linguistics. 2003, 5: 1-8.   

[4] Goodfellow, I. J., et al. Generative adversarial nets. Advances in neural information processing systems. 2014, 2672-2680.

[5] Yu, L., Zhang, W., Wang, J., & Yu, Y. SeqGAN: Sequence Generative Adversarial Nets with Policy Gradient. Proceedings of the AAAI Conference on Artificial Intelligence. 2017, 2852-2858.

[6] Liu, Y., Ott, M., Goyal, N., Du, J., Joshi, M., Chen, D., ... & Stoyanov, V. RoBERTa: A robustly optimized BERT pretraining approach. arXiv preprint arXiv: 1907.11692. 2019.

[7] WANG, X., WU, Y. C., Zhou, M., & Fu, H. Beyond Surveillance: Privacy, Ethics, and Regulations in Face Recognition Technology. Frontiers in Big Data, 7, 1337465.

[8] Radford, A., Narasimhan, K., Salimans, T., & Sutskever, I. Improving language understanding by generative pre-training. OpenAI Blog. 2018.

[9] Zhang, Y., Gan, Z., Fan, K., Chen, Z., Henao, R., Shen, D., & Carin, L. Adversarial feature matching for text generation. ICML. 2017.

[10] Devlin, J., Chang, M. W., Lee, K., & Toutanova, K. BERT: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. 2019, 1: 4171-4186.

[11] Strubell, E., Ganesh, A., & McCallum, A. Energy and policy considerations for modern deep learning research. In Proceedings of the AAAI conference on artificial intelligence. 2020, 34(9): 13693-13696.

[12] Dushyant, K., Muskan, G., Annu, Gupta, A., & Pramanik, S. Utilizing machine learning and deep learning in cybesecurity: an innovative approach. Cyber security and digital forensics. 2022, 271-293.

[13] Sun, Y., Wang, X., & Tang, X. Deep learning face representation from predicting 10,000 classes. In Proceedings of the IEEE conference on computer vision and pattern recognition. 2014, 1891-1898.

[14] Yan, W. Computational methods for deep learning. Springer. 2021, 10: 978-3.

[15] Liu, M., Ma, Z., Li, J., Wu, Y. C., & Wang, X. Deep-Learning-Based Pre-training and Refined Tuning for Web Summarization Software. IEEE Access. 2024, 12: 92120-92129.

[16] Lewis, M., Liu, Y., Goyal, N., Ghazvininejad, M., Mohamed, A., Levy, O., ... & Zettlemoyer, L. BART: Denoising sequence-to-sequence pre-training for natural language generation, translation, and comprehension. arXiv preprint arXiv: 1910.13461. 2019.

[17] Zhou, Y., Geng, X., Shen, T., Long, G., & Jiang, D. EventBERT: A pre-trained model for event correlation reasoning. In ACM Web Conference 2022. 2022, 850-859.

[18] Chung, H. W., Hou, L., Longpre, S., Zoph, B., Tay, Y., Fedus, W., ... & Brahma, S. Scaling instruction-finetuned language models. arXiv preprint arXiv: 2210.11416. 2022.

[19] Nguyen, D. Q., Vu, T., & Nguyen, A. T. BERTweet: A pre-trained language model for English tweets. EMNLP 2020.

[20] Hwang, S. J., & Jeong, C. S. Integrating pre-trained language model into neural machine translation. arXiv preprint arXiv: 2310.19680. 2023.

[21] Yu, Y., Chung, J., Yun, H., Hessel, J., Park, J. S., Lu, X., ... & Kim, G. Fusing pre-trained language models with multimodal prompts through reinforcement learning. In IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2023, 10845-10856.

[22] Tan, J., Wan, X., & Xiao, J. Abstractive document summarization with a graph-based attentional neural model. In 55th Annual Meeting of the Association for Computational Linguistics. 2017, 1: 1171-1181.

[23] Liu, Y., & Lapata, M. Text summarization with pretrained encoders. In 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP). 2019, 3721-3731.

[24] Chen, X. Using Big Data Analysis Technology to Analyze the Impact of Household Leverage Ratio on House Price Bubble. In International Conference On Signal And Information Processing, Networking And Computers. Singapore: Springer Nature Singapore. 2021, 900-909.

[25] Li, J., Tang, T., Zhao, W. X., Nie, J. Y., & Wen, J. R. Pretrained language models for text generation: A survey. arXiv preprint arXiv: 2201.05273. 2022.

[26] Paulus, R., Xiong, C., & Socher, R. A deep reinforced model for abstractive summarization. arXiv preprint arXiv: 1705.04304. 2017.

[27] Narayan, S., Cohen, S., & Lapata, M. Ranking sentences for extractive summarization with reinforcement learning. In North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT). 2018, 1747-1759.

[28] Shi, T., Keneshloo, Y., Ramakrishnan, N., & Reddy, C. K. Neural abstractive text summarization with sequence-to-sequence models. ACM Transactions on Data Science. 2021, 2(1): 1-37.

[29] Liu, M. Machine Learning Based Graph Mining of Large-scale Network and Optimization. In 2021 2nd International Conference on Artificial Intelligence and Information Systems. 2021, 1-5.

[30] Wang, X., Wu, Y. C., Ji, X., & Fu, H. Algorithmic discrimination: examining its types and regulatory measures with emphasis on US legal practices. Frontiers in Artificial Intelligence. 2024, 7, 1320277.

[31] Rush, A. M., Chopra, S., & Weston, J. A neural attention model for abstractive sentence summarization. In Conference on Empirical Methods in Natural Language Processing (EMNLP). 2015, 379-389.

[32] Zhang, J., Zhao, M., Liu, Y., & Wang, L. PEGASUS: Pre-training with extracted gap-sentences for abstractive summarization. In Proceedings of the 37th International Conference on Machine Learning. 2020, 11328-11339.

[33] WANG, X., WU, Y. C., Zhou, M., & Fu, H. Beyond Surveillance: Privacy, Ethics, and Regulations in Face Recognition Technology. Frontiers in Big Data. 2024, 7: 1337465.

[34] Chen, X., Liu, M., Niu, Y., Wang, X., & Wu, Y. C. Deep-Learning-Based Lithium Battery Defect Detection via Cross-Domain Generalization. IEEE Access. 2024, 12: 78505-78514.

[35] Wang, X., Wu, Y. C., & Ma, Z. Blockchain in the courtroom: exploring its evidentiary significance and procedural implications in US judicial processes. Frontiers in Blockchain. 2024, 7: 1306058.