Labeling platform
An easy-to-use and accessible labeling platform. The platform can be used to assess human observer labels for image quality assessment or to perform a blinded reading or a qualitative evaluation.
The platform is built up as a client-server architecture with a lightweight client (HTML web page frontend) and processing and encrypted storing on server side, as illustrated in the Figure below.
Users can register to the platform and provides some background information to allow the assignment for a specific test-case. Afterwards they automatically receive an invitation to participate in a labeling when a new labeling test-case is generated. The users login to the platform via a web browser.
Overview
Test-case
consists of:
- associated images
- specific test-case/labeling question
- test-case description
- 1 to M discrete or continuous labels
- [optional] guidance page which users can inspect anytime
- [optional] global best and worst control image
and the displayed web page front end contains
(a) one displayed image, i.e. independent labeling
(b) 1 to N images in parallel, i.e. in relation to each other
(c) a reference image and 1 to N images in parallel
User types
- admin: generate and manage test-cases, manage users, analyze and export labels, maintain image database
- user: access labeling test-cases
Images
- images can be in DICOM, MHD or GIPL format
- standard viewing capabilities: zooming, rotating, brightness and contrast adjustment
- viewing is realized via SimpleITK: https://github.com/SimpleITK/SimpleITK
Download
The labeling platform is publicly available under Apache 2.0 license: https://github.com/thomaskuestner/LabelingPlatform
Please cite the paper, if you use it in a scientific publication.
Getting Started
- Set/modify the parameters (Email, database structure, ...) in
Utility/Constant.cs
- specify your MySQL connection (SERVERNAME, SQL_ADMIN, SQL_ADMIN_PWD) in
Web.config (line 9): <add name="MySqlConnection" connectionString="Server=SERVERNAME;Database=labelingframework;Uid=SQL_ADMIN;Pwd=SQL_ADMIN_PWD;" />
- [optional] modify CSS style sheet in
Styles/Site.css
- Compile the labeling framework library
- Windows: Install Microsoft Visual Studio, open
LabelingFramework.csproj
and compile - Unix/OSX: Install Mono (http://www.mono-project.com/) and compile via
xbuild /p:Configuration=Release LabelingFramework.csproj
- Deploy to your web server
- Windows/Unix/OSX: Apache via mod_mono (http://www.mono-project.com/docs/web/mod_mono/)
- Windows: Internet Information Services (IIS; https://technet.microsoft.com/en-us/library/ee692294(v=ws.10).aspx)
- Import MySQL procedures and tables from
MySQL/default_database.sql
- with default administrator (needed for successfull initial login, please change password after first login)
- user: admin
- password: 123456
- (default user: 1, password: 1)
Documentation
Structure of labeling web page frontend
More information on the usage and the structure can be viewed directly in the labeling platform: https://github.com/thomaskuestner/LabelingPlatform/usage_help.html
or
close testcase
help
tutorial page (additional testcase information)
report a problem
reset image (zoom, brightness, contrast)
link images