Optimizing Reviewer Assignment with Recommender Systems: Models, Related Work, and Evaluation
Main Article Content
Abstract
Peer reviewer assignment to academic articles is important in ensuring the quality and originality of academic publications. Traditional methods of selecting reviewers are generally plagued by inefficiency, reviewer burnout, and inconsistency between the subject of the manuscript and the reviewer area of expertise. In attempting to avoid such drawbacks, recommender systems have been explored as a means of solving the reviewer assignment problem. This article reviews the recommender system techniques in detail by reviewing their application in peer reviewer selection. Additionally, related works shall be examined for how different methods work, their strength and limitations, the dataset used by them, and evaluation metrics used in measuring system performance.
Manuscript received:11 Mar 2025 | Revised: 30 Apr 2025 | Accepted: 13 May 2025 | Published: 30 Jul 2025
Article Details

This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License.
References
R. Muscarella and L. Poorter, “Ten simple rules for managing communications with a large number of coauthors,” PLoS Computational Biology, vol. 18, no. 6, p. e1010185, 2022.
DOI: https://doi.org/10.1371/journal.pcbi.1010185
R. Smith, “Peer review: A flawed process at the heart of science and journals,” Journal of the Royal Society of Medicine, vol. 99, no. 4, pp. 178–182, 2006.
DOI: https://doi.org/10.1258/jrsm.99.4.178
A. O. Ogunde et al., “The design of a hybrid model-based journal recommendation system,” Advances in Science, Technology and Engineering Systems Journal, vol. 5, no. 6, pp. 35–45, 2020.
DOI: https://doi.org/10.25046/aj0506139
F.T. Abdul Hussien, A.M.S. Rahma and H.B. Abdul Wahab, “Recommendation Systems for E-commerce Systems: An Overview,” Journal of Physics: Conference Series, vol. 1897, no. 1, p. 012024, 2021.
DOI: https://doi.org/10.1088/1742-6596/1897/1/012024
T.-E. Tai, S.-C. Haw, W.-E. Kong and K.-W. Ng, “Performance Evaluation of Machine Learning Techniques on Resolution Time Prediction in Helpdesk Support System,” International Journal on Robotics, Automation and Sciences, vol. 6, no. 2, pp. 59–68, 2024.
DOI: https://doi.org/10.33093/ijoras.2024.6.2.9
S.-C. Haw, J. Jayaram, E.A. Anaam and H.A. Santoso, “Exploring Recommender Systems in Healthcare: A Review on Methods, Applications and Evaluations,” International Journal on Robotics, Automation and Sciences, vol. 6, no. 2, pp. 6–15, 2024.
DOI: https://doi.org/10.33093/ijoras.2024.6.2.2
Y. Deldjoo, M. Schedl, P. Cremonesi and G. Pasi, “Recommender Systems Leveraging Multimedia Content,” ACM Computing Surveys, vol. 53, no. 5, p. 110, 2020.
DOI: https://doi.org/10.1145/3407190
S. Wang et al., “Graph Learning based Recommender Systems: A Review,” Proceedings of the International Joint Conference on Artificial Intelligence (IJCAI), pp. 4493–4500, 2021.
DOI: https://doi.org/10.24963/ijcai.2021/630
D. Roy and M. Dutta, “A systematic review and research perspective on recommender systems,” Journal of Big Data, vol. 9, no. 1, p. 59, 2022.
DOI: https://doi.org/10.1186/s40537-022-00592-5
M.F. Aljunid and M. Dh, “An Efficient Deep Learning Approach for Collaborative Filtering Recommender System,” Procedia Computer Science, vol. 171, pp. 1054–1061, 2020.
DOI: https://doi.org/10.1016/j.procs.2020.04.090
P. Gonzalez, G.S. Wilson and A.J. Purvis, “Peer review in academic publishing: Challenges in achieving the gold standard,” Journal of University Teaching and Learning Practice, vol. 19, no. 5, p. 1, 2022.
DOI: https://doi.org/10.53761/1.19.5.1
Z. Gu et al., “Adversarial attacks on content-based filtering journal recommender systems,” Computers, Materials & Continua, vol. 64, no. 3, pp. 1755–1770, 2020.
DOI: https://doi.org/10.32604/cmc.2020.010739
R.J. DiDomenico, W.L. Baker and S.T. Haines, “Improving peer review: What reviewers can do,” American Journal of Health-System Pharmacy, vol. 74, no. 24, pp. 2080–2084, 2017.
DOI: https://doi.org/10.2146/ajhp170190
S.T. Haines, W.L. Baker and R.J. DiDomenico, “Improving peer review: What journals can do,” American Journal of Health-System Pharmacy, vol. 74, no. 24, pp. 2086–2090, 2017.
DOI: https://doi.org/10.2146/ajhp170188
A. Kumar and S.N. Ahmed, “Challenges faced in the peer review system in open access journals,” Asian Journal of Medical Sciences, vol. 13, no. 12, pp. 1–5, 2022.
DOI: https://doi.org/10.3126/ajms.v13i12.49361
T.C.M. Bakker and J.F.A. Traniello, “Peer-review reciprocity and commitment to manuscript evaluation,” Behavioral Ecology and Sociobiology, vol. 73, no. 3, p. 38, 2019.
DOI: https://doi.org/10.1007/s00265-019-2647-2
A. Day, “Exploratory analysis of text duplication in peer-review reveals peer-review fraud and paper mills,” Scientometrics, vol. 127, no. 10, pp. 6183–6200, 2022.
DOI: https://doi.org/10.1007/s11192-022-04504-5
B. Tabatadze, “Technological Aspects of Open Journal Systems (OJS),” Journal of Technical Science and Technologies, vol. 8, no. 1, pp. 23–29, 2024.
DOI: https://doi.org/10.31578/jtst.v8i1.151
W.H.W. Ishak, F.M. Yamin, M.F.M. Mohsin and M.F. Mansor, “A Comparative Review of Conference Management Systems,” Journal of Technology and Operations Management, vol. 18, no. 2, pp. 87–93, 2023.
DOI: https://doi.org/10.32890/jtom2023.18.2.7
S. Zhang, L. Yao, A. Sun and Y. Tay, “Deep learning based recommender system: A survey and new perspectives,” ACM Computing Surveys, vol. 52, no. 1, p. 5, 2019.
DOI: https://doi.org/10.1145/3285029
S. Dwivedi and V.S.K. Roshni, “Recommender system for big data in education,” Proceedings of the 5th National Conference on E-Learning and E-Learning Technologies (ELELTECH 2017), pp. 1–5, 2017.
DOI: https://doi.org/10.1109/ELELTECH.2017.8074993
G. Geetha, M. Safa, C. Fancy and D. Saranya, “A Hybrid Approach using Collaborative Filtering and Content Based Filtering for Recommender System,” Journal of Physics: Conference Series, vol. 1000, no. 1, p. 012101, 2018.
DOI: https://doi.org/10.1088/1742-6596/1000/1/012101
M. Srifi, A. Oussous, A.A. Lahcen and S. Mouline, “Recommender systems based on collaborative filtering using review texts—A survey,” Information, vol. 11, no. 6, p. 317, 2020.
DOI: https://doi.org/10.3390/INFO11060317
R. Widayanti, M.H.R. Chakim, C. Lukita, U. Rahardja and N. Lutfiani, “Improving Recommender Systems using Hybrid Techniques of Collaborative Filtering and Content-Based Filtering,” Journal of Applied Data Sciences, vol. 4, no. 3, pp. 196–207, 2023.
DOI: https://doi.org/10.47738/jads.v4i3.115
Y. Afoudi, M. Lazaar and M. Al Achhab, “Hybrid recommendation system combined content-based filtering and collaborative prediction using artificial neural network,” Simulation Modelling Practice and Theory, vol. 113, p. 102375, 2021.
DOI: https://doi.org/10.1016/j.simpat.2021.102375
J. Wei, J. He, K. Chen, Y. Zhou and Z. Tang, “Collaborative filtering and deep learning based recommendation system for cold start items,” Expert Systems with Applications, vol. 69, pp. 29–39, 2017.
DOI: https://doi.org/10.1016/j.eswa.2016.09.040
J. Chen, H. Zhang, X. He, L. Nie, W. Liu and T.S. Chua, “Attentive collaborative filtering: Multimedia recommendation with item- and component-level attention,” Proceedings of the 40th International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR 2017), pp. 335–344, 2017.
DOI: https://doi.org/10.1145/3077136.3080797
Z. Zheng, X. Li, M. Tang, F. Xie and M. R. Lyu, “Web Service QoS Prediction via Collaborative Filtering: A Survey,” IEEE Transactions on Services Computing, vol. 15, no. 4, pp. 2094–2109, 2022.
DOI: https://doi.org/10.1109/TSC.2020.2995571
G. George and A.M. Lal, “Hy-MOM: Hybrid Recommender System Framework Using Memory-Based and Model-Based Collaborative Filtering Framework,” Cybernetics and Information Technologies, vol. 22, no. 1, pp. 100–115, 2022.
DOI: https://doi.org/10.2478/cait-2022-0009
A. Pujahari and D.S. Sisodia, “Model-based collaborative filtering for recommender systems: An empirical survey,” Proceedings of the 1st International Conference on Power, Control and Computing Technologies (ICPC2T 2020), pp. 251–255, 2020.
DOI:https://doi.org/10.1109/ICPC2T48082.2020.9071454
G. Chornous, I. Nikolskyi, M. Wyszynski, G. Kharlamova and P. Stolarczyk, “A hybrid user-item-based collaborative filtering model for e-commerce recommendations,” Journal of International Studies, vol. 14, no. 4, pp. 157–170, 2021.
DOI: https://doi.org/10.14254/2071-8330.2021/14-4/11
P.B. Thorat, R.M. Goudar and S. Barve, “Survey on Collaborative Filtering, Content-based Filtering and Hybrid Recommendation System,” International Journal of Computer Applications, vol. 110, no. 4, pp. 31–36, 2015.
DOI: https://doi.org/10.5120/19308-0760
Z. Yang, B. Wu, K. Zheng, X. Wang and L. Lei, “A survey of collaborative filtering-based recommender systems for mobile internet applications,” IEEE Access, vol. 4, pp. 3273–3287, 2016.
DOI: https://doi.org/10.1109/ACCESS.2016.2573314
M. Riyahi and M.K. Sohrabi, “Providing effective recommendations in discussion groups using a new hybrid recommender system based on implicit ratings and semantic similarity,” Electronic Commerce Research and Applications, vol. 40, p. 100938, 2020.
DOI: https://doi.org/10.1016/j.elerap.2020.100938
S. Sharma, V. Rana, and M. Malhotra, “Automatic recommendation system based on hybrid filtering algorithm,” Education and Information Technologies, vol. 27, no. 2, pp. 1523–1538, 2022.
DOI: https://doi.org/10.1007/s10639-021-10643-8
F.S. Gohari, H. Haghighi and F.S. Aliee, “A semantic-enhanced trust based recommender system using ant colony optimization,” Applied Intelligence, vol. 46, no. 2, pp. 380–398, 2017.
DOI: https://doi.org/10.1007/s10489-016-0830-y
M. Al-Hassan, H. Lu and J. Lu, “A semantic enhanced hybrid recommendation approach: A case study of e-Government tourism service recommendation system,” Decision Support Systems, vol. 72, pp. 97–109, 2015.
DOI: https://doi.org/10.1016/j.dss.2015.02.001
N. Saidani, K. Adi and M.S. Allili, “A semantic-based classification approach for an enhanced spam detection,” Computers & Security, vol. 94, p. 101716, 2020.
DOI: https://doi.org/10.1016/j.cose.2020.101716
N.A.N.B.M. Romzi, S.-C. Haw, W.-E. Kong, H.A. Santoso and G.-K. Tong, “Generative AI Recommender System in E-Commerce,” International Journal on Advanced Science, Engineering and Information Technology, vol. 14, no. 6, pp. 1823–1835, 2024.
DOI: https://doi.org/10.18517/ijaseit.14.6.10509
M.O. Ayemowa, R. Ibrahim and M.M. Khan, “Analysis of Recommender System Using Generative Artificial Intelligence: A Systematic Literature Review,” IEEE Access, vol. 12, pp. 87742–87766, 2024.
DOI: https://doi.org/10.1109/ACCESS.2024.3416962
Y. Deldjoo et al., “A Review of Modern Recommender Systems Using Generative Models (Gen-RecSys),” Proceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, pp. 6448–6458, 2024.
DOI: https://doi.org/10.1145/3637528.3671474
C.P. Ling Tung, S.-C. Haw, W.-E. Kong and P. Naveen, “Movie Recommender System Based on Generative AI,” Proceedings of the 2024 International Symposium on Parallel Computing and Distributed Systems (PCDS), pp. 1–7, 2024.
DOI:https://doi.org/10.1109/PCDS61776.2024.10743897
M.J. Mokarrama, S. Khatun and M.S. Arefin, “A content-based recommender system for choosing universities,” Turkish Journal of Electrical Engineering and Computer Sciences, vol. 28, no. 4, pp. 2070–2083, 2020.
DOI: https://doi.org/10.3906/ELK-1911-37
S. Sridhar, D. Dhanasekaran and G.C. P. Latha, “Content-Based Movie Recommendation System Using MBO with DBN,” Intelligent Automation and Soft Computing, vol. 35, no. 3, pp. 3031–3043, 2023.
DOI: https://doi.org/10.32604/iasc.2023.030361
R. Mishra and S. Rathi, “Efficient and Scalable Job Recommender System Using Collaborative Filtering,” Lecture Notes in Electrical Engineering, vol. 635, pp. 931–939, 2020.
DOI: https://doi.org/10.1007/978-981-15-1420-3_91
G. Chalkiadakis, I. Ziogas, M. Koutsmanis, E. Streviniotis, C. Panagiotakis and H. Papadakis, "A Novel Hybrid Recommender System for the Tourism Domain," Algorithms, vol. 16, no. 4, p. 215, 2023.
DOI: https://doi.org/10.3390/a16040215
M.V. Farashah, A. Etebarian, R. Azmi and R.E. Dastjerdi, "A hybrid recommender system based-on link prediction for movie baskets analysis," Journal of Big Data, vol. 8, no. 1, p. 10, 2021.
DOI: https://doi.org/10.1186/s40537-021-00422-0
E. Asani, H. Vahdat-Nejad and J. Sadri, "Restaurant recommender system based on sentiment analysis," Machine Learning with Applications, vol. 6, p. 100114, 2021.
DOI: https://doi.org/10.1016/j.mlwa.2021.100114
I. Karabila, N. Darraz, A. El-Ansari, N. Alami and M. El Mallahi, "Enhancing Collaborative Filtering-Based Recommender System Using Sentiment Analysis," Future Internet, vol. 15, no. 7, p. 235, 2023.
DOI: https://doi.org/10.3390/fi15070235
A. Zhang, Y. Chen, L. Sheng, X. Wang and T.-S. Chua, "On Generative Agents in Recommendation," Proceedings of the 47th International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR 2024), pp. 1807–1817, 2024.
DOI: https://doi.org/10.1145/3626772.3657844
J. Protasiewicz et al., "A recommender system of reviewers and experts in reviewing problems," Knowledge-Based Systems, vol. 106, pp. 164–178, 2016.
DOI: https://doi.org/10.1016/j.knosys.2016.05.041
Y. Kalmukov, "An algorithm for automatic assignment of reviewers to papers," Scientometrics, vol. 124, no. 3, pp. 1957–1974, 2020.
DOI: https://doi.org/10.1007/s11192-020-03519-0
D.T. Hoang, N.T. Nguyen, B. Collins and D. Hwang, "Decision Support System for Solving Reviewer Assignment Problem," Cybernetics and Systems, vol. 52, no. 5, pp. 393–411, 2021.
DOI: https://doi.org/10.1080/01969722.2020.1871227
S. Tan, Z. Duan, S. Zhao, J. Chen and Y. Zhang, "Improved reviewer assignment based on both word and semantic features," Information Retrieval Journal, vol. 24, no. 3, pp. 323–349, 2021.
DOI: https://doi.org/10.1007/s10791-021-09390-8
T. Pradhan, S. Sahoo, U. Singh and S. Pal, "A proactive decision support system for reviewer recommendation in academia," Expert Systems with Applications, vol. 169, p. 114331, 2021.
DOI: https://doi.org/10.1016/j.eswa.2020.114331
C.K. Kreutz and R. Schenkel, "RevASIDE: Assignment of Suitable Reviewer Sets for Publications from Fixed Candidate Pools," Proceedings of the 30th ACM International Conference on Information and Knowledge Management (CIKM 2021), 2021.
DOI: https://doi.org/10.1145/3487664.3487673
X. Liu, X. Wang and D. Zhu, "Reviewer recommendation method for scientific research proposals: a case for NSFC," Scientometrics, vol. 127, no. 6, pp. 3285–3305, 2022.
DOI: https://doi.org/10.1007/s11192-022-04389-4
Y. Yong, Z. Yao and Y. Zhao, "Beyond Accuracy: A Feature Crossing Method for Chinese Thesis Reviewer Recommendation," Proceedings of the 2021 IEEE International Conference on Systems, Man and Cybernetics (SMC), pp. 1609–1614, 2021.
DOI: https://doi.org/10.1109/SMC52423.2021.9658668
S. Azad et al., "A Reviewer Recommender System for Scientific Articles Using a New Similarity Threshold Discovery Technique," Lecture Notes in Networks and Systems, vol. 658, pp. 481–494, 2023.
DOI: https://doi.org/10.1007/978-981-19-9483-8_42
Y. Huang, S. Xu, S. Cai and L. Lü, "A multilayer network diffusion-based model for reviewer recommendation," Chinese Physics B, vol. 33, no. 3, p. 038901, 2024.
DOI: https://doi.org/10.1088/1674-1056/ad181d
W. Liao, Y. Zhu, Y. Li, Q. Zhang, Z. Ou, and X. Li, "RevGNN: Negative Sampling Enhanced Contrastive Graph Learning for Academic Reviewer Recommendation," Proceedings of the 2024 ACM SIGKDD Conference on Knowledge Discovery and Data Mining, 2024.
DOI: https://doi.org/10.1145/3679200
J.A. Drozdz and M.R. Ladomery, "The Peer Review Process: Past, Present, and Future," British Journal of Biomedical Science, vol. 81, p. 12054, 2024.
DOI: https://doi.org/10.3389/bjbs.2024.12054
B. Aczel et al., "The present and future of peer review: Ideas, interventions, and evidence," Proceedings of the National Academy of Sciences (PNAS), vol. 122, no. 5, p. e2401232121, 2025.
DOI: https://doi.org/10.1073/pnas.2401232121
S.-A. Teh, S.-C. Haw and H. A. Santoso, "Hybrid-based Research Article Recommender System," International Journal of Membrane Science and Technology, vol. 10, no. 2, pp. 1587–1606, 2023.
DOI: https://doi.org/10.15379/ijmst.v10i2.1830
Z.-T. Yap, S.-C. Haw and N.E.B. Ruslan, "Hybrid-based food recommender system utilizing KNN and SVD approaches," Cogent Engineering, vol. 11, no. 1, p. 2436125, 2024.
DOI: https://doi.org/10.1080/23311916.2024.2436125
C.H. Morales-Alarcón, E. Bodero-Poveda, H.M. Villa-Yánez, and P.A. Buñay-Guisñan, "Blockchain and Its Application in the Peer Review of Scientific Works: A Systematic Review," Publications, vol. 12, no. 4, p. 40, 2024.
DOI: https://doi.org/10.3390/publications12040040
P. Shrestha, S. Bhunia, A. Carvalho, C. Anderson and G. Lee, "A Case Study on Blockchain-based Anonymous Reviewer Incentive Token (BARIT)," Proceedings of the 2024 IEEE International Conference on Systems, Man, and Cybernetics (SMC), pp. 1564–1571, 2024.
DOI:https://doi.org/10.1109/SMC54092.2024.10831118
A.C. Ribeiro, A. Sizo and L.P. Reis, "Investigating the reviewer assignment problem: A systematic literature review," Journal of Information Science, 2023.
DOI: https://doi.org/10.1177/01655515231176668
K. Kousha and M. Thelwall, "Artificial intelligence to support publishing and peer review: A summary and review," Learned Publishing, vol. 37, no. 1, pp. 4–12, 2024.