SUBMISSION FORMAT

Task1: Generative AI based Software Metadata Classification

Participants will need to provide the two artifacts as part of their results:

These can be submitted through google form - https://forms.gle/akGZ5fjV156ZrsPY7

B) Excel containing the generated dataset that would contain three columns (Code, Comment and label (Useful and not Useful) written in C). There is no restriction on the data volume, however using the same the participants must be able to improve the accuracy 

Task2: Translation of Programming Language to  Natural Language

 Generated Summary for the test codes -- Results from the top three experiments conducted by the participant 

EVALUATION

For Task 1, evaluation will be performed based on the F1 score and Accuracy metrics and the amount of improvement by adding the generated data to the seed dataset that has been provided by the challenge. The top F1 scores will be published on the leader board.

For Task 2, evaluation will be based on the bleu score and technology-assisted review scores obtained from the generated summary and the one obtained from a LLM. The top scores will be published on the leader board.