Labeling platform

An easy-to-use and accessible labeling platform. The platform can be used to assess human observer labels for image quality assessment or to perform a blinded reading or a qualitative evaluation.

The platform is built up as a client-server architecture with a lightweight client (HTML web page frontend) and processing and encrypted storing on server side, as illustrated in the Figure below.

Users can register to the platform and provides some background information to allow the assignment for a specific test-case. Afterwards they automatically receive an invitation to participate in a labeling when a new labeling test-case is generated. The users login to the platform via a web browser.

Overview

Test-case

consists of:

associated images
specific test-case/labeling question
test-case description
1 to M discrete or continuous labels
[optional] guidance page which users can inspect anytime
[optional] global best and worst control image

and the displayed web page front end contains

(a) one displayed image, i.e. independent labeling

(b) 1 to N images in parallel, i.e. in relation to each other

User types

- admin: generate and manage test-cases, manage users, analyze and export labels, maintain image database

- user: access labeling test-cases

Images

- images can be in DICOM, MHD or GIPL format

- standard viewing capabilities: zooming, rotating, brightness and contrast adjustment

- viewing is realized via SimpleITK: https://github.com/SimpleITK/SimpleITK

Download

The labeling platform is publicly available under Apache 2.0 license: https://github.com/thomaskuestner/LabelingPlatform

Please cite the paper, if you use it in a scientific publication.

Getting Started

Set/modify the parameters (Email, database structure, ...) in
1. Utility/Constant.cs
specify your MySQL connection (SERVERNAME, SQL_ADMIN, SQL_ADMIN_PWD) in
1. Web.config (line 9): <add name="MySqlConnection" connectionString="Server=SERVERNAME;Database=labelingframework;Uid=SQL_ADMIN;Pwd=SQL_ADMIN_PWD;" />
[optional] modify CSS style sheet in
1. Styles/Site.css
Compile the labeling framework library
Windows: Install Microsoft Visual Studio, open LabelingFramework.csproj and compile
Unix/OSX: Install Mono (http://www.mono-project.com/) and compile via
1. xbuild /p:Configuration=Release LabelingFramework.csproj
Deploy to your web server
Windows/Unix/OSX: Apache via mod_mono (http://www.mono-project.com/docs/web/mod_mono/)
Windows: Internet Information Services (IIS; https://technet.microsoft.com/en-us/library/ee692294(v=ws.10).aspx)
Import MySQL procedures and tables from
1. MySQL/default_database.sql
with default administrator (needed for successfull initial login, please change password after first login)
user: admin
password: 123456
(default user: 1, password: 1)