PACMan: Mitigating Bias In Proposal Reviews Utilizing Machine Learning

Keith Ceruti II

Mentor: Dr. Lou Strolger

Department of Instruments, Space Telescope Science Institute

It is often hard to allocate and distribute resources, most processes can be biased and arduous, requiring months to plan and accomplish with marginal success. In context, astronomers from around the world require access to the unique capabilities of the Hubble Space Telescope (HST) and the James Webb Space Telescope (JWST), as there are no other telescopes powerful enough to execute their research goals. The Proposal Auto-Categorizer and Manager, also known as PACMan, is a tool being developed to help with time allocation for the Space Telescope Science Institute (STScI) in Baltimore MD. STScI creates a Time Allocation Committee (TAC) to review thousands of scientific proposals received each year, making recommendations to assign time to a few hundred for HST and JWST. PACMan will assist the TAC in mitigating expertise bias in reviewer assignments, leading to more efficient and reliable selection. Through a combination of a naive Bayesian classifier, vector cosine similarity, and other machine learning techniques, PACMan makes proposal-to-panelists assignments with relatively good accuracy. However, PACMan still needs validation tests and metrics of its performance. Various testing will include analyzing raw scores against “experts”, e.g., isolating scores for experts in the subject matter of each proposal, comparing PIs to their proposals, and abstracting only the top scores for a proposal. Through testing we should be able to identify the range of scores that define the “experts” of any given proposal.