Submission & Evaluation

Submission

For submitting your trained model(s) and source code (or executable file)

While enrolling please give the following details:

Name(s) of the participant(s)
Affiliation(s)
Description your approach for each task
Attachment of your source code (or executable file). If your submission is for multiple tasks, please put the source code (or executable file) in separate folders.
Description how to compile source code or execute submitted file.
Link(s) to your trained model(s) (if your train model(s) are too large to send via email, you should place the model file(s) on Dropbox, Google Drive or any file sharing site and send us the link to download your model file(s)).

Note that, when you submit your system for multiple tasks, please put the system of each task in an individual folder.

To compare the submitted systems, we will evaluate them using our private test set.

This private test set is unpublished and collected by unseen writers from the same VieTreeBank corpus.

The evaluation measurements are the Character Error Rate (CER) and the Word Error Rate (WER).

For each task, both CER and WER on the private test set are used to rank the submitted systems.

Every submitted system should be able to accept a single input file name and a single output file name (InkML) as follows.

run.exe input.inkml output.inkml  (for Windows)

OR

./run input.inkml output.inkml  (for Linux)

The system processes the input InkML file whose format is shown as follows.

Then, it writes down an output InkML file which contains the transcription result for each "traceGroup" as shown in the following part.

Input file contains the annotationXML node with basic information and several traceGroup nodes.

All "Tg_Truth" nodes of "traceGroup" are cleaned.

<ink>

  <annotationXML>

    <Description>Cursive online handwriting</Description>

    <Content_Category>Text</Content_Category>

    <Language>Vietnamese</Language>

    <Native_Language>Vietnamese</Native_Language>

    <Start_Time>2018-04-13T11:13:30</Start_Time>

    <DevName>FujitsuTabletPC</DevName>

    <SamplingRate>120</SamplingRate>

    <MaxNormalPressure>255</MaxNormalPressure>

  </annotationXML>

  <traceGroup id="tg_0_0_0">

    <annotationXML>

      <Tg_Truth></Tg_Truth>

    </annotationXML>

    <trace id="tr_0_0"> x1 y1, x2 y2, x3 y3, ....</trace>

    <trace id="tr_0_1"> x11 y11, x12 y12, x13 y13, ....</trace>

  </traceGroup>

  <traceGroup id="tg_0_0_1">

    <annotationXML>

      <Tg_Truth></Tg_Truth>

    </annotationXML>

    <trace id="tr_0_2"> x21 y21, x22 y22, x23 y23, ....</trace>

    <trace id="tr_0_3"> x31 y31, x32 y32, x33 y33, ....</trace>

    <trace id="tr_0_4"> x41 y41, x42 y42, x43 y43, ....</trace>

    <trace id="tr_0_5"> x51 y51, x52 y52, x53 y53, ....</trace>

  </traceGroup>

...

</ink>

Output file should write the transcription result of each traceGroup to the "Tg_Truth" as follows:

<ink>

  <annotationXML>

    <Description>Cursive online handwriting</Description>

    <Content_Category>Text</Content_Category>

    <Language>Vietnamese</Language>

    <Native_Language>Vietnamese</Native_Language>

    <Start_Time>2018-04-13T11:13:30</Start_Time>

    <DevName>FujitsuTabletPC</DevName>

    <SamplingRate>120</SamplingRate>

    <MaxNormalPressure>255</MaxNormalPressure>

  </annotationXML>

  <traceGroup id="tg_0_0_0">

    <annotationXML>

      <Tg_Truth>Chúng</Tg_Truth>

    </annotationXML>

    <trace id="tr_0_0"> x1 y1, x2 y2, x3 y3, ....</trace>

    <trace id="tr_0_1"> x11 y11, x12 y12, x13 y13, ....</trace>

  </traceGroup>

  <traceGroup id="tg_0_0_1">

    <annotationXML>

      <Tg_Truth>tôi</Tg_Truth>

    </annotationXML>

    <trace id="tr_0_2"> x21 y21, x22 y22, x23 y23, ....</trace>

    <trace id="tr_0_3"> x31 y31, x32 y32, x33 y33, ....</trace>

    <trace id="tr_0_4"> x41 y41, x42 y42, x43 y43, ....</trace>

    <trace id="tr_0_5"> x51 y51, x52 y52, x53 y53, ....</trace>

  </traceGroup>

...

</ink>

Google Sites

Report abuse