Research

My primary interests involve decision making in low-data and low-resource settings. I am most motivated by operations research problems that carry significant public impact.

Dissertation work at Northwestern has centered on the critical issue of substandard and falsified medical products, which afflict millions globally—most pointedly in low- and middle-income countries. This work has featured edifying collaborations with a variety of institutions, including the University of Notre Dame, the Liberia Medicines and Health Products Regulatory Authority, and United States Pharmacopeia. These collaborations have produced interesting theoretical and practical tools for combatting poor-quality medicines, which are described in the drop-down sections below.

Prior roles in global public health also yielded some implementation and observational research regarding community health and the Ebola crisis. This research is referenced in Teaching/Collaboration on the Data during Ebola page.

Substandard and Falsified Pharmaceuticals and Post-Market Surveillance: An Overview

Substandard and falsified pharmaceuticals (SFPs) have a significant impact on global public health--especially in low-resource settings: WHO estimates hundreds of thousands of excess deaths annually in sub-Saharan Africa due to substandard and falsified anti-malarials alone. Post-market surveillance (PMS) is a crucial and widespread tool for medical products regulators in low-resource settings who aim to reduce the amount of SFPs consumed by the public. In PMS, testing of pharmaceutical samples from consumer-facing locations provides estimates of SFP prevalence at these locations, and regulators use analysis of PMS testing results to take enforcement actions such as investigations, warnings or recalls against causes of high SFP prevalence. The crucial problem is that once an SFP is detected, it is not obvious if the reason the product became an SFP was due to conditions at the procurement point, or elsewhere upstream in the supply chain. Poor manufacture, transportation, or storage, as well as the infiltration of falsified substitutes, can all drive SFP occurrence.

Research shows that supply-chain environments are principal drivers of the SFP generation. Additionally, supply-chain information is frequently available in the form of manufacturer or distributor labels on tested samples. However, current PMS methodology does not fully utilize supply-chain information in identifying the sources of SFPs in supply chains. My dissertation work explores methods for integrating available supply-chain information with PMS testing data to identify SFP sources in supply chains. These methods should flexibly apply to the variety of regulatory environments faced by regulators in low-resource settings and must account for modest operational and computational budgets. An improved ability to identify SFP sources means a more effective and efficient deployment of enforcement actions by regulators, reducing the amount of SFPs reaching the public.

Supply-Chain PMS

Considering PMS with available supply-chain information is called supply-chain PMS. From an industrial engineering perspective, the two most crucial decisions in supply-chain PMS are:

Which locations should be tested?
How should results be analyzed?

The first chapter of my dissertation focuses on the second question. "Inferring sources of substandard and falsified products in pharmaceutical supply chains" (available on arXiv) examines if rates of SFP generation at different supply-chain locations can be inferred using PMS testing data and supply-chain information at two echelons of a pharmaceutical supply chain. Even when only considering two supply-chain echelons, this work establishes the presence of unidentifiability of SFP rates: using testing results and supply-chain information alone, it is not possible to find a set of SFP rates that "best" explains these data. This work then proposes a Bayesian method for mitigating this unidentifiability that uses regulator prior assessments of SFP risk to distinguish among reasonable explanations. A case study of real PMS data from a low-resource settings shows the usefulness of this approach: consideration of supply-chain information permits disentangling many possible explanations for detected SFPs.

This work was featured in the Spring 2022 issue of Northwestern Engineering, found here.

Eugene (on the left) meeting the Hon. Senator Saah Joseph of Montserrado County, Liberia, for a discussion on the impact of substandard and falsified medical products.

INFORMS 2021 Presentation - Eugene Wickett - Inferring poor-quality products in pharmaceutical supply chains.mp4

Presentation on supply-chain PMS recorded for the 2021 INFORMS conference.

Example logistigate output of SFP-rate inference for PMS data from a low-resource setting.

Logistigate

I have incorporated the methodology of supply-chain PMS into a software package, logistigate, in Python. This package is available in alpha release on Github, and a walkthrough with examples is available here.

logistigate uses regulator assessments of SFP risk, testing data, and available supply-chain information to provide inference of SFP rates at each supply-chain location. This software meets a few crucial needs of contemporary pharmaceutical regulation in low-resource settings:

No sample-size requirements: Current PMS practice uses conservative sample sizes under conventional confidence intervals for proportions to estimate SFP rates. logistigate has no minimum sample size: inference can be produced for the supply chain even if some locations are represented in only one sample. It is left to the modeler to determine whether the resulting inference is sufficient to take enforcement action, or if more testing data are required.
Flexible designation of supply-chain locations: The literature on SFPs describes a multitude of SFP drivers: neglect by individual outlets, distributors, or manufacturers; geographic distinctions like rivers or mountains that cause different economic conditions; or regulatory capacity at different levels of the supply chain. logistigate allows for multiple definitions of each supply-chain echelon that can capture these different drivers. For instance, a modeler can group all manufacturers by country, and inspect if elevated SFP rates are associated with different source countries. Similarly, if the modeler suspects that SFP rates will markedly differ by national region, testing locations can be grouped by the parent national region, and analysis conducted. This flexibility expands the power of PMS when data collection is frequently constrained by limited sampling budgets.
Speed: A typical supply chain in a low-resource setting contains a few hundred locations; logistigate generates inference for the SFP rates of a supply chain of this size within a few minutes. This speed for conducting inference grants modelers the ability to try different definitions for each echelon and re-run analyses.

Choosing samples

My first dissertation chapter establishes how PMS data can be considered in a supply-chain context. The next chapters study how this data can be assembled in a manner that further regulatory objectives. The key decisions are:

Which consumer-facing locations should be tested?
How many tests should be collected before conducting analysis?

Usual PMS testing takes place in batches: regulators visit some collection of outlets according to a sampling plan, collect samples, and test if these samples meet registration specifications. A variety of transportation, testing, and other operational considerations influence the size of a given batch of PMS data.

The next iteration of my research aims to give regulators a means of comparing different proposed sampling plans. The first step is quantifying regulator objectives: what actions will regulators take under different inferences? Once regulator objectives are quantified, Bayesian experiment design provides tools for analyzing sampling plans and identifying plans that best further these objectives. Bayesian experiment analysis is computationally expensive, however; ongoing work looks to speed up analysis though approximation algorithms.