17 submissions in total! We are enjoying this track with you all.
This track is mostly an analysis-based track where we come forward as a community to understand the impact of AI and IR in Software Engineering tasks. Contending and evaluating Software artefacts can be tricky and we love this challenge. In the previous year, we got amazing papers from the teams and it helped to analyse the paper better.
Nevertheless, we still will announce the top 3 teams this year for Task 1.
This is evaluated based on the % increase in F1 score from baseline and the quality of data generated. It was a difficult task and achieving even a 1% increase is commendable
Tripti Kumari, IIT Dhanbad (4% increase in F1-Score, 300 new data)
Hanna Abi Akl, DSTI France (1.5% increase, 421 new data)
Seetharam Killivalavan, SSN College of Engineering (5% increase only in Precision, 1239 new data)
Congratulations to all. We have got some results wherein the performance has decreased slightly and this is very interesting for us to analyse noise. Their datasets can serve as a good test case to understand spurious labels that are generated from LLM models
Noise Identification Teams are
Vishesh Aggarwal, Microsoft Research
Priyam Dalmia, American Express
Shakti Papdeja, Amazon
Raj Shah, IIT Goa
Aritra Mitra, IIT Kharagpur
Lisa Sarkar, IIT Kharagpur
Invited and extended working notes (all teams eligible) will be published in an edited book on Generative AI for Code Processing in the Transactions on Computer Systems and Networks, Springer
The IRSE Team
Let's Gear up for a smashing presentation in Goa!