Hello! I’m Jiho Shin, an incoming Postdoctoral Researcher at SAIL/MCIS Lab in the School of Computing at Queen's University, Kingston.
My research focuses on applying deep learning and natural language processing techniques to automated software engineering (AI4SE). In particular, I investigate the reliability and trustworthiness of foundation models and large language models in tasks such as text-to-source-code generation, unit test and test-oracle (assertion) generation, automatic program repair, code summarization, and code translation.
8. Pre-trained Models for Bytecode Instructions
Donggyu Kim, Taemin Kim, Jiho Shin, Song Wang, Heeyoul Choi, and Jaechang Nam
18th IEEE International Conference on Software Testing, Verification and Validation (ICST-Short 2025)
7. Prompt Engineering or Fine-Tuning: An Empirical Assessment of LLMs for Code
Jiho Shin, Clark Tang, Tahmineh Mohati, Maleknaz Nayebi, Song Wang, Hadi Hemmati
22nd IEEE/ACM International Conference on Mining Software Repositories (MSR 2025)
6. Domain Adaptation for Code Model-Based Unit Test Case Generation
Jiho Shin, Sepehr Hashtroudi, Hadi Hemmati, Song Wang
33rd ACM SIGSOFT International Symposium on Software Testing and Analysis (ISSTA 2024)
5. An Empirical Study on the Stability of Explainable Software Defect Prediction
Jiho Shin, Reem Aleithan, Jaechang Nam, Junjie Wang, Nima Shiri Harzevili, Song Wang
30th Asia-Pacific Software Engineering Conference (APSEC 2023)
🏆 Distinguished Paper Award
4. Automatic Static Vulnerability Detection for Machine Learning Libraries: Are We There Yet?
Nima Shiri Harzevili, Jiho Shin, Junjie Wang, Song Wang, Nachiappan Nagappan
34th IEEE International Symposium on Software Reliability Engineering (ISSRE 2023)
3. Characterizing and Understanding Software Security Vulnerabilities in Machine Learning Libraries
Nima Shiri Harzevili, Jiho Shin, Junjie Wang, Song Wang, Nachiappan Nagappan
20th IEEE/ACM International Conference on Mining Software Repositories (MSR 2023)
2. API Recommendation for Machine Learning Libraries: How Far Are We?
Moshi Wei, Yuchao Huang, Junjie Wang, Jiho Shin, Nima Shiri Harzevili, Song Wang
30th ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering (ESEC/FSE 2022)
1. Similar Patch Recommendation for Actionable Defect Prediction
Jiho Shin and Jaechang Nam
22nd Korea Conference on Software Engineering (KCSE 2020)
🏆 Distinguished Paper Award
3. Assessing Evaluation Metrics for Neural Test Oracle Generation
Jiho Shin, Hadi Hemmati, Moshi Wei, Song Wang
IEEE Transactions on Software Engineering (TSE 2024 and ICSE-JF 2025)
2. The Good, the Bad, and the Missing: Neural Code Generation for Machine Learning Tasks
Jiho Shin, Moshi Wei, Junjie Wang, Lin Shi, Song Wang
ACM Transactions on Software Engineering and Methodology (TOSEM 2023)
1. A Survey of Automatic Code Generation from Natural Language
Jiho Shin and Jaechang Nam
Journal of Information Processing Systems (JIPS 2021)
6. Toward Automated Validation of Language Model Synthesized Test Cases using Semantic Entropy
Hamed Taherkhani, Jiho Shin, Muhammad Ammar Tahir, Md Rakib Hossain Misu, Vineet Sunil Gattani, Hadi Hemmati
arXiv preprint 2025, (Under Review)
5. Surveying the Benchmarking Landscape of Large Language Models in Code Intelligence
Mohammad Abdollahi, Ruixin Zhang, Nima Shiri Harzevili, Jiho Shin, Song Wang, Hadi Hemmati
HAL Open Science 2025, (Under Review)
4. StaAgent: An Agentic Framework for Testing Static Analyzers
Elijah Nnorom, Md Basim Uddin Ahmed, Jiho Shin, Hung Viet Pham, Song Wang
arXiv preprint 2025, (Under Review)
3. SecVulEval: Benchmarking LLMs for Real-World C/C++ Vulnerability Detection
Md Basim Uddin Ahmed, Nima Shiri Harzevili, Jiho Shin, Hung Viet Pham, Song Wang
arXiv preprint 2025, (Under Review)
2. Retrieval-Augmented Test Generation: How Far Are We?
Jiho Shin, Reem Aleithan, Hadi Hemmati, Song Wang
arXiv preprint 2024, (Under Review)
1. Checker Bug Detection and Repair in Deep Learning Libraries
Nima Shiri Harzevili, Mohammad Mahdi Mohajer, Jiho Shin, Moshi Wei, Gias Uddin, Jinqiu Yang, Junjie Wang, Song Wang, Zhen Ming (Jack) Jiang, Nachiappan Nagappan
arXiv preprint 2024, (Under Review)
York University, Toronto 2021 ~ Present
Dept. of EECS Ph.D.
Research Field: AI/ML for Software Engineering, Code Generation, Automated Software Testing, LLM for SE
Supervisors: Dr. Song Wang and Dr. Hadi Hemmati
Handong Global University, Pohang 2019 ~ 2021
Dept. of CSEE M.Sc.
Research Field: Code Generation, Actionable and Explainable Defect Prediction
Supervisor: Dr. Jaechang Nam
Thesis: Actionable Defect Prediction
Handong Global University, Pohang 2012 ~ 2019
Dept. of CSEE B.Sc.
Specialization: Web Application, Human-Computer Interaction, Computer Vision
Turing
RLEF/CodeGenAgent Lead (Remote) Jan. 2024 ~ Present
HGS (Hinduja Group Companies)
Applied AI Engineering Intern (Remote) Oct. 2024 ~ Feb. 2025
York University
Graduate Research Assistant Sep. 2021 ~ Present
Teaching Assistant Sep. 2021 ~ Present
Handong Global University
Graduate Research Assistant Mar. 2019 ~ Feb. 2021
Teaching Assistant Sep. 2018 ~ Dec. 2020
Republic of Korea Navy
Military Interpreter (KOR/ENG) Feb. 2014 ~ Jan. 2016
Organizing Committee
[ICST’24] Web Chair, 17th IEEE International Conference on Software Testing, Verification and Validation.
Student Volunteer
[ICSE’25] SV, 50th IEEE/ACM International Conference on Software Engineering.
[ICST’24] SV, 17th IEEE International Conference on Software Testing, Verification and Validation.
Reviewer/PC
[JSS’25] Reviewer - Served as a reviewer for the Journal of Systems and Software.
[EMSE’25] Reviewer - Served as a reviewer for the Empirical Software Engineering.
[COMPSAC-SETA’25] PC - Served as a PC for the Symposium on Software Engineering Technologies & Applications.
[STVR'25] Reviewer - Served as a reviewer for the Software Testing, Verification and Reliability.
[TOSEM’25] Reviewer - Served as a reviewer for the ACM Transactions on Software Engineering and Methodology.
[MSR’25] Junior PC - Served as a Junior PC for MSR’25 (acceptance rate: 111/262 = 42.4%).
[ICSE’25] Shadow PC - Served as a Shadow PC for ICSE’25 (acceptance rate: 50/299=16.7%).
[TSE’24-25] Reviewer - Served as a reviewer for the IEEE Transactions on Software Engineering.
MSR'25 - attended the conference to present a Technical Track paper.
ICSE'25 - attended the conference to present a Journal First paper.
AIware'24 - attended the AIware Leadership Bootcamp
ISSTA'24 - presented about Unit Test Generation with Domain Adaptation for the technical track.
ICST'24 - organized and attended the conference as a web chair and a student volunteer.
APSEC'23 - presented the XDP (eXplainable Defect Prediction) technical paper 🏆.
CSER'22-24 - attended the Consortium for Software Engineering Research for multiple presentations and posters.
KCSE'20 - presented the idea of change suggestions for actionable defect prediction 🏆.