Evaluators Summary

The number of unique evaluators per language: (#Task 1 + #Task 2)

• •  Hindi: 29 (17 +17)

  • Marathi: 20 (17 +17)

  • Telugu: 26 (17 +17)

*Task 1: Naturalness Evaluation, Task 2: Speaker Similarity Evaluation

Note:

1) Consistency check is done by taking the difference of the responses in the repeated files and its mean and standard deviation (STD) score is reported. 

2) STD of all responses is also provided to check whether the evaluator is clicking on only one score/not.