Result

IRSE 2024 Results of Task 2:

We are thrilled to announce the official results for Task 2 in our shared challenge on quality estimation of LLM-generated code. This task received a total of three submissions from IIT-KGP. For this task, our evaluation framework centred around two crucial metrics: local nDCG (normalized Discounted Cumulative Gain) and global nDCG. These metrics serve to measure ranking quality, providing insights at both local and global levels, which ensures a balanced and robust evaluation of each approach. This focus on multi-level ranking quality highlights those submissions that best capture the nuanced aspects of code quality assessment.

We are grateful to all participants for their valuable contributions, as every submission plays an essential role in advancing this evolving field. The detailed results are presented below in tabular form, offering an easy reference to the rankings and achievements across the board. Congratulations to all teams on their hard work and dedication!