FSR Challenge 2023 Rules

Read these first

• First, please read and agree to the following rules before participating

• You should only register for the challenge if you actually plan to submit an entry to the challenge

Task

• Build an ASR that is suitable for recognizing Hakka Speech (reading or spantaneous speech).

Evaluation Metric

• Track 1: Character error rate (CER)

• Track 2: Syllable error rate (SER)

Material Provided in the Challenge

• This challenge is based on the HAT-Vol1 corpus collected in 2022.

• HAT-Vol1 is reading speech and consists of about 100 speakers, in total about 80 hours (training, evaluation, and test sets).

Instructions

• Listening to the evaluation test data, or any other human interaction with the test data, is not allowed.

• Participants should return both the recognition hypotheses (unit sequences) and the corresponding likelihoods of the hypotheses.

• The output hypotheses should only be UTF-8 encoded characters.

• Participants need to complete a form giving the general technical specification of their developed speech recognition system to facilitate easy cross-system comparisons (e.g. is it a transformer-based system? What features and training procedures are adopted? etc.).

• Each participant needs to submit a paper (4-6 pages, double column) to ROCLING 2023 and is expected to present their work at the conference.

Use of External Data

• You are allowed to use external data in any way you wish, subject to any exclusions given in these rules, and clear descriptions/statements are provided about what and how external databases are adopted (for cross-system comparisons).

Retention of Submitted Recognition Hypotheses

• Any recognition hypotheses that you submit for evaluation will be retained by the FSR Challenge organizers for future use.

• You must include in your submission a statement of whether you give the organizers permission to publicly distribute your results in anonymized form. We really appreciate it if all participants approve this consent agreement!

How are these rules enforced?

• This is a challenge, which is designed to answer scientific questions, and not a competition. Therefore, we rely on your honesty in preparing your entry.

• If you have any doubt about how to apply these rules, please contact the organizers (sarc@nycu.edu.tw) immediately.