Submission

Results Format

We accept submissions in JSON with the following format. The format example is only for one image with the unique "image_id". When you put results of all testing images together, you should have a comma to separate them, e.g., {"image_classification": [{Result_Format_for_Image1}, {Result_Format_for_Image2}, ..., {Result_Format_for_ImageN}]}.

Note: You are very welcomed to submit the results on trainval (both JSON and evalution results) as well when you submit the results on testing set. Therefore, we can check whether our evaluation matrix is consistent with your own evaluation results on the trainval set.

1 Image Classification

[{ "image_id" : int, "category_id" : [59 booleans] "category_score": [59 float]}]

Note: Each booleanfloat indicates the confidence this image belongs to a specific category. The category_ids follow the following category_id's ordering, i.e., [2, 23, 25, 31, 34, 45, 59, 65, 72, 98, 397, 113, 207, 258, 284, 308, 347, 368, 416, 427, 9, 18, 22, 33, 44, 46, 68, 80, 85, 104, 115, 144, 158, 159, 162, 187, 189, 220, 232, 259, 260, 105, 296, 355, 295, 324, 326, 349, 354, 360, 366, 19, 415, 420, 424, 440, 445, 454, 458], which is the same as the order they appear in the "categories" filed of trainval JSON.

2 Object detection

[{ "image_id" : int, "category_id" : int, "bbox" : [x, y, width, height], "score" : float}]

3 Semantic Segmentation

[{ "image_id" : int, "category_id" : int, "segmentation" : RLE}]

4 Instance Segmentation

[{ "image_id" : int, "category_id" : int, "segmentation" : RLE, "score" : float}]

5 Objectness

[{ "image_id" : int, "bbox" : [x, y, width, height], "score" : float}]

6 Part Segmentation

[{ "image_id" : int, "category_id" : int, "parts" : [{"part_id": int, "segmentation" : RLE}, ..., {"part_id": int, "segmentation" : RLE}]}]

Note: For part segmentation, you don't need to take into consideration the specific category the part belongs to. You can treat each part , which is globally defined in the "parts" of trainval_*.json, as one individual class.

7 Boundary detection

[{ "image_id" : int, "segmentation" : RLE}]

9 Keypoint detection

[{ "image_id" : int, "category_id" : 284, "keypoints" : [x1, y1, v1, ..., xk, yk, vk], "score" : float}]

Note: We evaluate the keypoint detection based on multiple persons. You don't need to provide us with the bounding box of person detection before the keypoint detection.

Our automatic submission system is on the way - stay tuned. We will be using Codalab to automatically process and rank submissions.

In the meantime, please email your submissions as zip files to pascalindetail@gmail.com.

Timeline

• Deploy Codalab server - May 30, 2017
• Announce Codalab competition settings
• Add evaluation scripts to Detail API
• Publish Codalab server, start accepting early submissions
• Workshop