Pre-deployment Tests

Pre-deployment Tests platform enables clinician, product manager, marketing executive to validate AI algorithms independently. Pre-deployment / Validation project is a merger of three essential components of any AI validation project i.e. Data, AI Inference, Ground Truth.

Data (Dataset) refers to the actual image data, either CT, MR, CR, DX, MG, US etc, on which an AI algorithm is to be tested.
AI inference refers to output created by AI algorithms when run on a specific dataset.
GT refers to the ‘actual’ labels for each of the cases / images in the dataset.

Once a Pre-deployment project is created by merging the three components - i.e. data, AI output and GT - there are multiple different validation techniques / pipelines that can be used to evaluate the performance of the AI, and share feedback with the developers of the AI:

Classification

View pre-deployment test projects

Click on "Pre-deployment -> Classification" to view all pre-deployment projects created by the user
Each snapshot of the project summary describes project id, project name, project description, total number of studies, algorithm detail, findings of the algorithm, status of ground truth and inference result.
Use “Edit project” icon to edit the project detail like project name and description
Use “Delete project” to remove a project
Click on "View Analysis" to view pre-deployment analysis report

Create Pre-deployment test project

Provide "Project Name" , "Project Description" , "Inferencing Detail" and dataset to create project.
All the fields are mandatory
In case of "Inferencing" there are three options available (a) Run algorithm (b) Upload inferencing result (c) Import from inferencing Project
"Run Algorithm" allows for automated running of AI algorithms on a given dataset from pre-deployment platform.
In case the user (AI developer) cannot or does not want to integrate the algorithm onto CARPL, the user still has the ability to define model outputs through "Upload Inference result" in the form of a CSV file. This allows the AI developer to gain access to all of CARPL’s pre-deployment features without integrating the algorithm into the platform.

In case of "Upload Inference result" user need to add findings of the inference similar to the csv file heading.

Use "Import from Inference project" to import the AI and GT from an existing pre-deployment project

Pre-deployment Projects detailed analysis

This section has two interfaces 1) Model Inference 2) Validation
"Update Inference Results" allows user to update AI result (csv file).This will give the user a default template to download. And user can update the inference result in that template and upload again.
"Update GT" allows the user to upload the GT result (csv file). This will give the user a default template to download. And user can update the result in that template and upload again.
Update GT can be updated by importing result from validation project. and uploading csv file.

Page updated

Report abuse