Participate

Participation & shared task procedure

The WMT shared task on sign language translation follows the procedures established by the shared task on machine translation that has been running since 2007. Here, we are outlining the participation process in order to make it more clear for new participants. 


Please note: The participation is about exchanging ideas, not about winning the competition! We encourage broad participation -- if you feel that your method is interesting but not state-of-the-art, then please participate in order to disseminate it and measure progress. The community may use your idea in a better manner and cite your paper, or even learn something useful from the negative results. 


Step 1: System training


Step 2: Processing of the test data


Step 3: Submission of the system outputs 


Furthermore, submissions should use the WMT XML format. We provide the test sources in this XML format, see the data tab. The exact submission format we require depends on the translation direction:


DSGS-to-German translation


In this case system outputs are German text. We require that these outputs are combined with the XML file containing the test sources, see here for an example.


German-to-DSGS translation


DSGS outputs must be one mp4 video for each line of German input text. Participants are free to submit any content they deem suitable as a translation. Examples: avatar animations, videos featuring photo-realistic signers or a pose estimation video.


Output videos should be wrapped in XML as well, but instead of text the <segment> elements in the XML should contain public links.

<?xml version='1.0' encoding='utf-8'?>

<dataset id="slttest2022.de-dsgs">

  <doc origlang="de" id="srf.0">

    <src lang="de">

      <p>

        <seg id="0">Guten Abend meine Damen und Herren - willkommen zur "Tagesschau".</seg>

      </p>

    </src>

    <hyp system="YOUR SYSTEM NAME" language="dsgs">

      <p>

        <seg id="0">https://www.your_hosting.com/your_url_for_this_segment</seg>

      </p>

    </hyp>

  </doc>

</dataset>

Example: public URL of a DSGS system output video wrapped in the WMT XML format

Important: we will attempt to retrieve public links on August 6. On this day videos must be available to us.


Step 4: Human evaluation


Step 5: Paper submission