Informing Policy by Integrating Optimization & Causal Inference


A permeating theme in the problems I encounter through my partners is the need to allocate scarce resources needed to satisfy basic needs, such as housing or ventilators, more efficiently and more fairly. Because of ethical concerns or logistical constraints, data available to inform improved allocation is observational, implying that treatment groups may be systematically different from one another and making it challenging to draw the inferences needed to make better decisions.

I am leading a research direction that integrates causal inference and optimization to design optimal, efficient, and fair allocation policies from data collected in deployment to improve policy-making at LAHSA, LAC+USC Medical Center, and Chestnut Health Systems.

My research in this stream is supported in part by my NSF CAREER award, one of my METRANS grants, my two USC Zumberge grants, and my CESTTRR grant.

Learning Optimal, Fair, and Interpretable Policies for Allocating Scarce Resources

My work in this area includes methods integrating optimization and causal inference for learning optimal interpretable policies, led by visiting PhD student Javad Azizi [8] and by my undergraduate student Nathan Jo [12], and policies that take the form of a bipartite matching, led by my recent PhD graduate Aida Rahmattalabi [16] and my PhD student Bill Tang [18]. My first steps were taken in the papers [8, 16] based on data from the HMIS from 16 communities across the US and showed that optimization based approaches yield policies that are more fair and more efficient. For example, in [16], we obtain wait times as low as a first come, first served policy while improving the rate of exits from homelessness overall and for vulnerable groups (by 7% for Black individuals and by 15% for those under 17 years old). More recently, my research group created methods that are provably asymptotically optimal. Notably, we devised an approach based on MIO for learning optimal prescriptive trees that can incorporate arbitrary capacity and fairness constraints [12]. This research earned Nathan Jo a finalist position for the INFORMS Undergraduate Operations Research Prize Award 2021, the USC University-wide Discovery Scholar Prize, and the USC Discovery Scholar Distinction. We also devised a method for learning a provably optimal policy (among all policies that map covariates to treatment assignments) [18]. Our works in [16] and [18] were highlighted as "Committee's Choice" presentation at the INFORMS Annual Meeting 2021. On data from the HMIS in LA, our policies result in an increase of up to 9pp in efficiency. Together with visiting PhD student Han Kyul Kim and Master's student John Dryden, we are applying these tools to design personalized substance use prevention policies based on the GAIN (Global Appraisal of Individual Needs) dataset at Chestnut Health Systems.

Figure. The Coordinated Entry System stores the information of all waitlisted youth to allocate housing resources based on a score based priority rule; The current policy prioritizes youth based on six key experiences that increase their chances of experiencing 5 or more years of homelessness.

Figure. Results from our paper [8]. Success probability across all youth (left) and by race (right) when fairness across races is enforced. The current policy and classical machine learning approaches are unfair, with “Whites” having higher success probability than “Blacks” and “Hispanics.” In contrast, our proposed policies, in particular Linear EF outperform significantly the state of the art at marginal cost to overall efficiency.

Designing Policies that Align with the Preferences of Policy-Makers

Designing an allocation policy involves making moral trade-offs such as choosing between maximizing efficiency and equity. I have led a research direction that advances robust optimization to actively learn the preferences of policy-makers over such conflicting goals [26]. Visiting PhD student Patrick Vossler designed a GUI based on my algorithm which we will deploy at LAHSA. Motivated by the fact that e.g., at LAC+USC Medical Center, decisions are made by committees of stakeholders, with my PhD student Caroline Johnston, we have generalized this method to the multi-stakeholder setting [14].

Figure. Preference elicitation at LAHSA: we propose to learn the policy-maker preferences over policy characteristics (e.g., fairness/efficiency trade-off) by asking them pairwise comparisons over policy outcomes or asking us to tell us how much they like a particular policy.


Figure. We have devised algorithms that strategically select, out of a potentially huge number of questions to ask the policy-makers, questions to ask to be able to offer them their preferred policy.

Fairness in Resource Allocation: Metrics and Impossibility Results

Natural questions when designing "fair" allocation policies are: How to define fairness? Are different fairness constraints compatible with one another? To answer these, in work led by my undergraduate student Nathan Jo, we propose a framework for evaluating fairness of both historical and counterfactual policies [13]. Our work culminates with a set of impossibility results that investigate the interplay between the different metrics. This work is guiding our discussion with policy-makers at LAHSA.

Partners

Related Grants

CAREER: Robust, Interpretable, and Fair Allocation of Scarce Resources in Socially Sensitive Settings

National Science Foundation, Operations Engineering

Role: Sole PI

Total Award Period Covered: 05/01/2021-04/30/2026 (5 Years)

Total Award Amount: $519,682

CES Triage Tool Redesign and Implementation

Conrad N. Hilton Foundation and United Way of Greater Los Angeles Home for Good

Role: Co-PI (PI: Eric Rice)

Total Award Period Covered: 01/01/2020-12/31/2022

Total Award Amount: $1,450,000

Own Share: $334,000

Designing Fair, Efficient, and Interpretable Policies for Allocating Scarce Resources

USC James H. Zumberge Faculty Research & Innovation Fund Diversity & Inclusion Grant Program

Role: Sole PI

Funded for the period: 07/2018-07/2019

Total award amount: $30,000

Own Share: $30,000

Addressing Ethical Dilemmas during the COVID-19 Pandemic through AI

USC Zumberge Special Solicitation – Epidemic & Virus Related Research and Development award

Role: Sole PI

Funded for the period: 07/20-02/21

Total award amount: $28,047

Socially Optimal Personalized Routing with Preference Learning

U.S. Department of Transportation, METRANS University Transportation Center and National Center for Sustainable Transportation

Role: PI (co-PI: Maged Dessouky)

Award ID: DTRT13-G-UTC57

Funded for the period: 08/2017-07/2018

Total award amount: $99,998

Own Share: $79,096

References

  1. Facts About Suicide. URL: https://www.cdc.gov/suicide/facts/index.html.

  2. Grand Challenges for Social Work. URL: https://grandchallengesforsocialwork.org.

  3. LAHSA CES Triage Tool Research & Refinement webpage. URL: https://www.lahsa.org/ documents?id=4370-ces-triage-tool-research-refinement.pdf.

  4. Landslides 101. URL: https://www.usgs.gov/programs/landslide-hazards/landslides-101.

  5. Too Many Donor Kidneys Are Discarded in U.S. Before Transplantation. URL:https://www.pennmedicine.org/news/news-releases/2020/december/ too-many-donor-kidneys-are-discarded-in-us-before-transplantation.

  6. S Aghaei, M J Azizi, and P Vayanos. Learning optimal and fair decision trees for non- discriminative decision-making.In Proceedings of the 33rd AAAI Conference on Artificial Intelligence, 2019.

  7. S Aghaei, A Gómez, and P Vayanos. Strong optimal classification trees. Major revision at Operations Research, 2022. URL: https://arxiv.org/abs/2002.09142.

  8. M J Azizi, P Vayanos, B Wilder, E Rice, and M Tambe. Designing fair, efficient, and in- terpretable policies for prioritizing homeless youth for housing resources. In Proceedings of the 15th International Conference on the Integration of Constraint Programming, Artificial Intelligence, and Operations Research, 2018.

  9. C Bandi, N Trichakis, and P Vayanos. Robust multiclass queuing theory for wait time esti- mation in resource allocation systems. Management Science, 65(1):152–187, 2018.

  10. Q Jin, A Georghiou, P Vayanos, and G Hanasusanto. Distributionally robust optimization with decision-dependentinformation discovery. In preparation for submission to INFORMS Journal on Computing, 2022.

  11. N Jo, S Aghaei, J Benson, A Gómez, and P Vayanos. Learning optimal fair classication trees. Under review at second ACM conference on Equity and Access in Algorithms, Mechanisms, and Optimization (EAAMO’22), 2022. URL: https://arxiv.org/pdf/2201.09932.pdf.

  12. N Jo, S Aghaei, A Gómez, and P Vayanos. Learning optimal prescriptive trees from observa- tional data. Major Revision at Management Science, short version appeared at 2022 AAAI Workshop on AI and Behavior Change, 2021. URL:https://arxiv.org/pdf/2108.13628. pdf.

  13. N Jo, B Tang, K Dullerud, S Aghaei, E Rice, and P Vayanos. Evaluating fairness of contextual resource allocationsystems: metrics and impossibility results. In preparation for submission to 37th AAAI Conference on Artificial Intelligence, 2022.

  14. C Johnston, S Blessenhohl, and P Vayanos. Preference Elicitation and Aggregation to Aid with Patient Triage during the COVID-19 Pandemic. In preparation for submission to Operations Research; short version appeared in International Conference on Machine Learning (ICML) Workshop on Participatory Approaches to Machine Learning, 2020.

  15. N Justin, S Aghaei, A G´omez, and P Vayanos. Optimal robust classification trees. In prepa- ration for submission to Operations Research; short version appeared in AAAI Workshop on Adversarial Machine Learning and Beyond, 2021.

  16. A Rahmattalabi, P Vayanos, K Dullerud, and E Rice. Learning resource allocation policies from observational data with an application to homeless services delivery. In 2022 ACM Conference on Fairness, Accountability, and Transparency (FAccT ’22), 2022.

  17. A Rahmattalabi, P Vayanos, A Fulginiti, E Rice, B Wilder, A Yadav, and M Tambe. Exploring algorithmic fairness in robust graph covering problems. In Proceedings of the 33rd Conference on Neural Information Processing Systems (NeurIPS), 2019.

  18. B Tang, C Koçyigit, and P Vayanos. Learning optimal policies for allocating housing to people experiencinghomelessness from data collected in deployment. In preparation for submission to Management Science, 2022.

  19. United-Nations. Resolution adopted by the General Assembly on 6 July 2017, Work of the Statistical Commissionpertaining to the 2030 Agenda for Sustainable Development. 2017.

  20. P Vayanos, A Georghiou, and H Yu. Robust optimization with decision-dependent information discovery. Major revisionat Management Science, 2019. URL: https://arxiv.org/pdf/ 2004.08490.pdf.

  21. P Vayanos, Q Jin, and G Elissaios. ROC++: Robust Optimization in C++. Forthcoming at INFORMS Journal on Computing, 2022. URL: https://arxiv.org/pdf/2006.08741.pdf.

  22. P Vayanos, Q Jin, and G Elissaios. ROCPP Version v2020.0140. INFORMS Journal on Computing, 2022.doi:10.5281/zenodo.6360996.

  23. P Vayanos, D Kuhn, and B Rustem. Decision rules for information discovery in multi-stage stochastic programming. InProceedings of the 50th IEEE Conference on Decision and Control, pages 7368–7373, 2011.

  24. P Vayanos, D Kuhn, and B Rustem. A constraint sampling approach for multi-stage robust optimization. Automatica, 48(3):459–471, 2012.

  25. P Vayanos, W Wiesemann, and D Kuhn. Hedging electricity swing options in incomplete markets. In Proceedings of the 18th IFAC Wold Congress, pages 846–853, 2011.

  26. P Vayanos, Y Ye, D McElfresh, J Dickerson, and E Rice. Robust active preference elicitation. Under second round ofreview at Management Science, 2021. URL: https://arxiv.org/pdf/ 2003.01899.pdf.

  27. P Vossler, S Aghaei, N Justin, N Jo, A Gómez, and P Vayanos. ODTlearn package v0.1. URL: https://github.com/D3M-Research-Group/odtlearn.

  28. P Vossler, S Aghaei, N Justin, N Jo, A Gómez, and P Vayanos. ODTlearn: a Python package for learning optimal decision trees. In preparation for submission to Journal of Machine Learning Research, 2022.

  29. Y Ye, C Doehring, A Georghiou, P Vayanos, and H Robinson. Conserving biodiversity via ad- justable robust optimization. In preparation for submission to Management Science; short version appeared in Proc. of the 21st International Conference on Autonomous Agents and Mul- tiagent Systems (AAMAS 2022), Workshop on Autonomous Agents for Social Good (AASG), 2022.