Team UCL Paediatrics

Team Members

Main Project Team:

Milan Kapur – Academic Clinical Fellow in Paediatric Intensive Care, UCL

Tangid Mohammad – 3rd Year Medical Student, UCL Medical School

Alex Brown – Academic Clinical Lecturer in Paediatric Intensive Care / Acute Paediatrics, UCL

Advisory Team Members:

Prof. Mark Peters – Professor of Paediatric Intensive Care, UCL & Consultant, Great Ormond Street PICU

Dr Samiran Ray – Consultant, Great Ormond Street PICU

Prof Padmanabhan Ramnarayan - Professor in Paediatric Critical Care, Imperial College London & Consultant, St Mary’s PICU

About Your Team

Our team comprises clinicians who combine work in paediatric intensive care with academic interests in data science and machine learning. We were drawn to the competition as a way to explore applying data science techniques to relatively real-world data, and to try out some new approaches!

Alex & Milan did most of the coding and experiments for the project. Tangid looked in to applying the Phoenix Sepsis Score as a synthetic feature, and whether this improved performance of the overall model. Our senior team members acted in an advisory capacity – our hope had been to integrate clinical acumen along with machine learning to improve performance..

Summary of Approach

We created a flexible and extensible experimental pipeline (built using Weights & Biases), which could be used with a variety of different model types. Our initial hope had been to incorporate synthetic features based on domain knowledge alongside simple MLP or tree-based models to improve performance. However, we found that these approaches could not improve over a well-tuned tree-boosted classifier.

Data Preprocessing

Missing values were imputed by their mean (for numerical values) or mode (for categorical values). Categorical variables were one-hot encoded. We trialed encoding categorical variables as ordinal values where possible, but did not find that this improved performance. We did not filter for outlying values or normalise / transform the input data. We trialled feature engineering by constructing the Phoenix Sepsis Score and including it as an input feature, but this led to a decrease in model performance. 

Data Augmentation

No data augmentation. We trialled use of positive class upweighting but found this did not improve performance; resulting models overestimated risk significantly.

Modeling Approach

We experimented with multiple different approaches, trialing MLPs and gradient boosted trees. Experiments demonstrated relatively simple feature selection using Shapley values, alongside well-tuned XGBoost models resulted in robust performance that was not improved by the addition of synthetic features. 

MLP classifiers performed substantially worse than XGBoost, as expected. 

We experimented with using novel advanced tabular data classifiers such at TabPFN and TabDPT, however, we found that they performed similarly to a well-tuned XGBoost model but were computationally much more expensive to implement. 

Model Tuning & Optimization

To determine the optimal hyperparameters for the final selected XGBoost model, we used a 5-fold cross-validation hyperparameter grid sweep on the training data. Results were logged to Weights and Biases allowing us to visualise and select the most appropriate value for each hyperparameter. 

Links

GitHub Respository: https://github.com/ucl-paediatrics/sepsis

Team website: https://profiles.ucl.ac.uk/31142-alex-brown