Substandard and falsified pharmaceuticals (SFPs) have a significant impact on global public health--especially in low-resource settings: WHO estimates hundreds of thousands of excess deaths annually in sub-Saharan Africa due to substandard and falsified anti-malarials alone. Post-market surveillance (PMS) is a crucial and widespread tool for medical products regulators in low-resource settings who aim to reduce the amount of SFPs consumed by the public. In PMS, testing of pharmaceutical samples from consumer-facing locations provides estimates of SFP prevalence at these locations, and regulators use analysis of PMS testing results to take enforcement actions such as investigations, warnings or recalls against causes of high SFP prevalence. The crucial problem is that once an SFP is detected, it is not obvious if the reason the product became an SFP was due to conditions at the procurement point, or elsewhere upstream in the supply chain. Poor manufacture, transportation, or storage, as well as the infiltration of falsified substitutes, can all drive SFP occurrence.
Research shows that supply-chain environments are principal drivers of SFP generation. Additionally, supply-chain information is frequently available in the form of manufacturer or distributor labels on tested samples. However, prior PMS methodology does not fully utilize supply-chain information in identifying the sources of SFPs in supply chains. My dissertation work explores methods for integrating available supply-chain information with PMS testing data to identify SFP sources in supply chains. These methods flexibly apply to the variety of regulatory environments faced by regulators in low-resource settings and account for modest operational and computational budgets. An improved ability to identify SFP sources means a more effective and efficient deployment of enforcement actions by regulators, reducing the amount of SFPs reaching the public.
Considering PMS with available supply-chain information is called supply-chain PMS. From an operational perspective, the two most crucial decisions in supply-chain PMS are:
Which locations should be tested?
How should results be analyzed?
The first chapter of my dissertation focuses on the second question. "Inferring sources of substandard and falsified products in pharmaceutical supply chains" (available in IISE Transactions) examines if rates of SFP generation at different supply-chain locations can be inferred using PMS testing data and supply-chain information at two echelons of a pharmaceutical supply chain. Even when only considering two supply-chain echelons, this work establishes the presence of unidentifiability of SFP rates: using testing results and supply-chain information alone, it is not possible to find a set of SFP rates that "best" explains these data. This work then proposes a Bayesian method for mitigating this unidentifiability that uses regulator prior assessments of SFP risk to distinguish among reasonable explanations. A case study of real PMS data from a low-resource settings shows the usefulness of this approach: consideration of supply-chain information permits disentangling many possible explanations for detected SFPs. Strikingly, a few supply-chain locations of the case study that would be classified as high-SFP locations when ignoring supply-chain connections switch to low-SFP locations under this approach. (See sub-page "Logisticgate Software" for an example.)
This work was highlighted as a feature article in the IISE Transactions Special Issue on Analytical Methods for Detecting, Disrupting, and Dismantling Illicit Operation (see IISE's post on LinkedIn and X) and was also featured in the Spring 2022 issue of Northwestern Engineering, found here.
Eugene meeting the Hon. Senator Saah Joseph of Montserrado County, Liberia, for a discussion on the impact of substandard and falsified medical products.
Presentation on supply-chain PMS recorded for the 2021 INFORMS conference.
Testing of antibiotic samples.
My first dissertation chapter establishes how to anaylze a set of supply-chain PMS data. The next two chapters study a given PMS dataset can help decide where to test next. The key decisions are:
Which consumer-facing locations should be tested?
How many tests should be collected at each location?
PMS testing typically takes place in batches: regulators visit some collection of outlets according to a sampling plan, collect samples, and test if these samples meet registration specifications. A variety of transportation, testing, and other operational considerations influence the size of a given batch of PMS data. The frequency of testing batches relies on government and/or non-government partner funding.
Before integrating operational constraints into sampling plan development, the value of any given plan must be estimated. We use utility to denote the value of a proposed sampling plan. Once a plan's utility can be estimated, plans meeting different operational constraints can be compared; the plan with the highest utility under these constraints can then be chosen. My second dissertation chapter, "Measuring sampling plan utility in post-marketing surveillance of medical products" (available in the Journal of Quality Technology) provides a method for estimating a plan's utility: prior data, regulatory assessments of risk, available supply-chain information, and the regulator's goals for PMS are integrated into a Bayesian function yielding the utility. Standard calculation techniques for typical settings require hours of compute time; this paper provides efficient estimation methods that approximate the utility in a few minutes.
The last dissertation chapter seeks the utility-maximizing plan satisfying operational constraints. This problem is an instance of "orienteering": a route must be chosen through a subset of nodes in a network, where the visiting of a node yields some utility. Orienteering in supply-chain PMS is complicated by two factors. First, the utility of visited nodes is not independent: because nodes share upstream supply-chain connections, the utility gained from visiting Node A depends on whether Node B is visited. Second, a plan must specify not only whether a node is visited, but also how many testing resources should be allocated to each node.