Manually Uploading Files

Before getting started with the Rapid Annotator, you need first to sign up. Then, you will be assigned either an experimenter or an annotator profile. Note that only experimenter profiles can both start new projects and add annotators to them.

Create an Experiment

  1. After logging in, click on the “Add Experiment” button to start a new project.


The Red Hen Rapid Annotator will process data of the following types: Image, Audio, Text, and Video. Formats allowed for these categories are as the following:

  • Image: jpg, png, gif.
  • Audio: mp3, wma, mp2, wav.
  • Text: txt, html.
  • Video: mov, mp4, wmv, webm.

2. Add an experiment name, description, and type of data in the pop-up dialogue. Select "manual" and click on “Add Experiment”.


Hint: For audio and video experiments you will be asked to provide time frame boundaries of the file(s) regarding the original source. This info is not mandatory, although to avoid any malfunction set by default one (1) in both cells. [For manual uploads, this dialogue should actually not be there - we will get this fixed.]

3. Now click on "Choose files" and go to the directory where the files are located. Once files are uploaded you can further modify either name or description info by double clicking on them.

Note: Uploaded files must match type of data and allowed formats (see step 1). For projects with a large number of files you can navigate through them with the "Previous" and "Next" buttons.

4. After pressing “Start Upload”, wait till the status change and a check mark is displayed. Otherwise files will be not collected.


Annotation

Labels and Levels of Annotation

1. Click on “Add Labels” in the options bar. Here you will find three different buttons.


2. To create a new category system for annotation click on “Add Annotation Level” to set a new category (e.g., age, gender) and press “Submit”. On the following pop-up window, you can also decide the order in which the level of annotation is displayed for the item annotation.

3. Below the level name, press “Add Label” and assign the values for this level of annotation. For rapid annotation, you can set a Key Binding for every value belonging to an annotation level. These will be further displayed on the annotation screen. If no key is bound to any label, a key binding will be automatically assigned.

You can add several labels without leaving the pop-up by clicking on "Add Another" button. When you are finished with the values press "submit". Here you can either add another level of annotation or leave this page by pressing “Done”.

4. Back to the Labels page, you can either export or import from other projects levels of annotation.

  • "Make Global" button export all the levels of annotation available in the current project to a pool whereof they can be imported and used in another annotation. Assign a name and click on "Submit".


  • Click on "Import Annotation Levels" to get access to public annotation levels made by the "Make Public" option. Click on "Import" and add the annotation system to your current project.

Getting to Annotate

1. Before beginning to annotate, you will need to select at least one an annotator (yourself or a collaborator) from the users who have already signed up. Here, you can also add new owners who will have the same experimenter privileges to add/delete data and annotators.

Hint: You can partionate the total number of files among the annotators assigned to it. Click on "Settings".

And here click on "Data partition". Here you can add the annotators among whom you want to split the total of files.

2. Press "Done" and return to the Homepage. Here click on "Experiment to Annotate".

Now, on the Home Screen, go to “Experiments to Annotate”. You will have access to experiments you have been assigned to. By clicking on any experiment, you will see the valid labels and their key shortcuts.

3. Here you can find experiments have been assigned to your account under the privileges of "Annotator". Now you can click on the name of the project you want to work on and start the annotation.

4. On the experiment screen you will carry out the annotation. In the example below, either by clicking on the “adult” button or pressing the “a” key, the value for this level of annotation is saved to the server. As soon as you select any label, the next annotation level (if any) is loaded. After all the levels have been codified for a file, the next one is rendered for annotation. You can see the progress both as a percentage and the number of the file out of the total.

Hints: The “Undo” button resets any annotation you may wish to. After all files are annotated you can either go back to a former stage by clicking "Undo" or "Done" to finish the annotation process.

The “Turn off/on looping” feature is only available for video and audio files.


Results

1. Go back to "Home" page and click on the experiment. Here press “Settings” on options bar.

2. Click on “View Results”.

3. Here you can see the results of the current project. Finally, in this screen you can either discard all annotations or export them onto your computer as an .xls spreadsheet.