Author(s)

JiTao Ma^*, HongWei Huang, Jun Du

Affiliation(s)

Network Management Center, Yunnan Science and Technology Information Research Institute, Kunming 650051, Yunnan, China.

Corresponding Author

JiTao Ma

ABSTRACT

With the rapid growth of scientific research projects and increasing complexity in interdisciplinary collaboration, traditional expert assignment methods—such as manual screening and keyword matching—are becoming inadequate. This study proposes an intelligent reviewer assignment system based on Retrieval-Augmented Generation (RAG), which enhances semantic understanding and improves the accuracy of matching scientific projects with suitable experts. The system constructs detailed knowledge profiles for both projects and experts across four dimensions: research questions, methods, results, and conclusions. Domain-specific prompts guide large language models (LLMs) to extract structured knowledge from unstructured textual inputs. These profiles are then transformed into semantic vectors using BERT-based embeddings and matched using cosine similarity. Experimental results show that the proposed method significantly outperforms baseline approaches in terms of precision, recall, and F1-score. Specifically, the model achieves 79% precision, 75% recall, and 77% F1-score at Top-5 recommendations. This work contributes to the development of more intelligent, accurate, and scalable systems for scientific peer review and expert assignment.

KEYWORDS

Intelligent reviewer assignment; Retrieval-Augmented Generation (RAG); Semantic profiling; Expert matching

CITE THIS PAPER

JiTao Ma, HongWei Huang, Jun Du. A Retrieval-Augmented Generation (RAG)-based intelligent reviewer assignment system for scientific project evaluation. Journal of Computer Science and Electrical Engineering. 2025, 7(5): 40-45. DOI: https://doi.org/10.61784/jcsee3073.

REFERENCES

[1] Sedaghat A R, Bernal-Sprekelsen M, Fokkens W J, et al. How to be a good reviewer: A step-by-step guide for approaching peer review of a scientific manuscript. Laryngoscope Investigative Otolaryngology, 2024, 9(3): e1266.

[2] Aksoy M, Yanik S, Amasyali M F. Reviewer assignment problem: A systematic review of the literature. Journal of Artificial Intelligence Research, 2023, 76: 761-827.

[3] Bornhorst J, Rokke D, Day P, et al. B-122 Estimated improvement of sigma error metrics associated with manual secondary result review, and subsequent artificial intelligence driven quality assurance. Clinical Chemistry, 2023, 69(Supplement_1): hvad097-456.

[4] Horbach S S, Halffman W W. The changing forms and expectations of peer review. Research Integrity and Peer Review, 2018, 3: 1–15.

[5] Liang D, Xu P, Shakeri S, et al. Embedding-based zero-shot retrieval through query generation. arXiv preprint arXiv:2009.10270, 2020. 

[6] Al Hashimy A S H, Kulathuramaiyer N. An automated learner for extracting new ontology relations. In: Proceedings of the 2012 International Conference on Advanced Computer Science Applications and Technologies (ACSAT), IEEE, 2012: 19-24.

[7] Jiménez-Ruiz E, Cuenca Grau B. LogMap: Logic-based and scalable ontology matching. In: Proceedings of the International Semantic Web Conference. Berlin, Heidelberg: Springer, 2011: 273–288.

[8] Devlin J, Chang M W, Lee K, et al. BERT: Pre-training of deep bidirectional transformers for language understanding. In: Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), 2019: 4171–4186. 

[9] Lewis P, Perez E, Piktus A, et al. Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks. arXiv preprint arXiv:2005.11401, 2020.

[10] Johnson J, Douze M, Jégou H. Billion-scale similarity search with GPUs. IEEE Transactions on Big Data, 2021, 7(3): 535–547.

[11] Wang H, Zhang D, Li J, et al. Entropy-optimized dynamic text segmentation and RAG-enhanced LLMs for construction engineering knowledge base. Applied Sciences, 2025, 15(6): 3134.