Labeling platform

An easy-to-use and accessible labeling platform. The platform can be used to assess human observer labels for image quality assessment or to perform a blinded reading or a qualitative evaluation.

The platform is built up as a client-server architecture with a lightweight client (HTML web page frontend) and processing and encrypted storing on server side, as illustrated in the Figure below.

Users can register to the platform and provides some background information to allow the assignment for a specific test-case. Afterwards they automatically receive an invitation to participate in a labeling when a new labeling test-case is generated. The users login to the platform via a web browser.

Overview

Test-case

consists of:

    • associated images
    • specific test-case/labeling question
    • test-case description
    • 1 to M discrete or continuous labels
    • [optional] guidance page which users can inspect anytime
    • [optional] global best and worst control image

and the displayed web page front end contains

(a) one displayed image, i.e. independent labeling

(b) 1 to N images in parallel, i.e. in relation to each other

(c) a reference image and 1 to N images in parallel

User types

- admin: generate and manage test-cases, manage users, analyze and export labels, maintain image database

- user: access labeling test-cases

Images

- images can be in DICOM, MHD or GIPL format

- standard viewing capabilities: zooming, rotating, brightness and contrast adjustment

- viewing is realized via SimpleITK: https://github.com/SimpleITK/SimpleITK

Download

The labeling platform is publicly available under Apache 2.0 license: https://github.com/thomaskuestner/LabelingPlatform

Please cite the paper, if you use it in a scientific publication.

Getting Started

    1. Set/modify the parameters (Email, database structure, ...) in
      1. Utility/Constant.cs
    2. specify your MySQL connection (SERVERNAME, SQL_ADMIN, SQL_ADMIN_PWD) in
      1. Web.config (line 9): <add name="MySqlConnection" connectionString="Server=SERVERNAME;Database=labelingframework;Uid=SQL_ADMIN;Pwd=SQL_ADMIN_PWD;" />
    3. [optional] modify CSS style sheet in
      1. Styles/Site.css
    4. Compile the labeling framework library
    5. Windows: Install Microsoft Visual Studio, open LabelingFramework.csproj and compile
    6. Unix/OSX: Install Mono (http://www.mono-project.com/) and compile via
      1. xbuild /p:Configuration=Release LabelingFramework.csproj
    7. Deploy to your web server
    8. Windows/Unix/OSX: Apache via mod_mono (http://www.mono-project.com/docs/web/mod_mono/)
    9. Windows: Internet Information Services (IIS; https://technet.microsoft.com/en-us/library/ee692294(v=ws.10).aspx)
    10. Import MySQL procedures and tables from
      1. MySQL/default_database.sql
    11. with default administrator (needed for successfull initial login, please change password after first login)
    12. user: admin
    13. password: 123456
    14. (default user: 1, password: 1)

Documentation

Structure of labeling web page frontend

More information on the usage and the structure can be viewed directly in the labeling platform: https://github.com/thomaskuestner/LabelingPlatform/usage_help.html

or

1: Testcase description box

2: Control buttons

3: Display

4: Navigation bar

Control buttons

close testcase

help

tutorial page (additional testcase information)

report a problem

reset image (zoom, brightness, contrast)

link images

Display

Navigation bar

References