Research on Human–AI Collaboration

To Rehumanize A Dehumanizing Process

From Human to Data to Algorithms

1. Human-Algorithmic Bias: Source, Evolution, and Impact (with Xiyang Hu, Yan Huang, Beibei Li). Management Science, 2026, 72(1): 96-118. [Paper link] [Amazon AWS AI Research Grant 2023; Finalist CIST 2021 Best Student Paper Award]

In this study, leveraging a unique repeat decision-making setting in a high-stakes micro-lending context, we aim to uncover the underlying source, evolution dynamics, and associated impacts of bias. We first develop and estimate a structural econometric model of the decision dynamics to understand the source and evolution of potential bias in human evaluators in microloan granting. We find that both preference-based bias and belief-based bias are present in human evaluators' decisions and are in favor of female applicants. Through counterfactual simulations, we quantify the effects of the two types of bias on both fairness and profits. The results show that the elimination of either of the two biases improves the fairness in financial resource allocation, as well as the platform profits. Furthermore, to examine how human biases evolve when being inherited by machine learning (ML) algorithms, we then train a set of state-of-the-art ML algorithms for default risk prediction on both real-world datasets with human biases encoded within and counterfactual datasets with human biases partially or fully removed. By comparing the decision outcomes in different counterfactual settings, we find that even fairness-unaware ML algorithms can reduce bias present in human loan-granting decisions. Interestingly, while removing both types of human biases from the training data can further improve ML fairness, the fairness-enhancing effects vary significantly between new and repeat applicants. Based on our findings, we discuss how to reduce decision bias most effectively in a human-machine learning pipeline.  

Prompt Human–AI Collaborative Value in Real-world Applications

Opening the Interactive Box: Human–Algorithm Interaction Mechanisms 

2. 1+1>2? Information, Humans, and Machines (with Yingjie Zhang). Information Systems Research, 2025, 36(1): 394-418. [Paper link]

Drawing upon studies in dual-process theories of reasoning that propose different conditions necessary to arouse humans' active information processing and systematic thinking, we tailor the experimental treatments to vary the level of information complexity, the presence of collaboration, and the availability of machine explanations. We observe that, with large volumes of information and with machine explanations alone, human evaluators cannot add extra value to the final collaborative outcomes. However, when extensive information is coupled with machine explanations, human involvement significantly reduces the financial default rate compared with machine-only decisions. We disentangle the underlying mechanisms with three-step empirical analyses. We reveal that the coexistence of large-scale information and machine explanations can invoke humans'active rethinking, which, in turn, shrinks gender gaps and increases prediction accuracy. In particular, we demonstrate that humans can spontaneously associate newly emerging features with others that have been overlooked but have the potential to correct the machine's mistakes.  We also examine cases where humans tend to either follow or overrule AI, along with the corresponding outcomes.   

3. The Power of Disagreement: A Field Experiment to Investigate Human-Algorithm Collaboration in Loan Evaluations (with Hongchang Wang, Yingjie Zhang). Management Science, 2026, 72(1): 96-118. [Paper link] 

What is collaborative value in human-algorithm collaboration, and how can it be achieved? We study these questions by conducting a field experiment where human evaluators and algorithms worked together to evaluate loan applications. We apply a two-by-two design to represent four collaboration scenarios, i.e., limited/rich information and with/without disclosure of algorithm rationale. In the experiment, human evaluators are asked to first make an initial independent decision and then make the final decision after receiving an algorithmic recommendation. We find that the final decisions are better (as measured by both right approval and right denial) than human-only decisions or algorithm-only decisions, indicating the presence of collaborative value. We measure this value with a novel concept, "decision augmentation," and find that disclosing algorithm rationale decreases collaborative value under the limited-information scenario but increases collaborative value under the rich-information scenario. To further understand the path to collaborative value, we propose a framework for mechanism examination that centers on collaborative disagreement, which occurs when human evaluators reject algorithmic recommendations. We then examine several vital, rationality-based factors within this framework and come to the following conclusions: (1) collaborative disagreement exhibit sizeable predictive power on collaborative value; (2) the differences between human evaluators and algorithms in decision-making contribute to disagreement but not to collaborative value; (3) the algorithm self-contradiction level increases disagreement and helps human evaluators disagree with the algorithms at the right time.  

Reclaiming Human Agency: Cognitive Safeguards in AI-Assisted Decisions

4. Empathic Algorithm Collaboration: Decision Augmentation for Socially Consequential Decision-Making (with Thomas Ware, T. S. Raghu, Benjamin Shao). Under review. [Paper link] 

The integration of artificial intelligence (AI) into decision-making systems presents new challenges for maintaining human intentionality, empathy, and accountability, particularly in socially consequential contexts where outcomes materially affect individuals’ lives. This study introduces a re-humanizing approach to AI-assisted decision-making by framing algorithmic collaboration through the lens of structural embeddedness, where AI is not simply a tool for optimization, but a socially situated actor in decision ecosystems. We propose Empathic Algorithm Collaboration (EAC) as a mechanism to restore individualized, ethically grounded consideration in human-AI decision environments. Drawing on theories of social empathy and person-situation interactionism, EAC consists of two key components – Interpersonal Perspective-Taking (IPT) and Contextual Understanding (CU) – which guide human evaluators to engage more reflectively with AI recommendations. Through a field experiment in a micro-lending context, we examine the impact of EAC on deliberative intensity and decision quality. Results show that EAC not only fosters more conscientious deliberation but also improves decision accuracy. Furthermore, differences in evaluator responses across borrower conditions suggest that person-situation interactionism plays a critical role in shaping the cognitive and ethical engagement of decision-makers. These findings highlight the potential of EAC to support human-centered, context-sensitive decision augmentation and contribute to a broader understanding of how AI can be aligned with social values in structurally embedded domains of socially consequential decision-making.  

5. Responsible Engagement: Human–AI Work System Design for Counteracting Agency Reversal (with Thomas Ware, T. S. Raghu, Benjamin Shao). Under review. [Paper link] 

A central challenge in AI-supported decision-making is designing workflows in which humans and algorithms complement rather than displace one another. Leveraging a microlending experimental setting, we introduce responsible engagement (RE), an interaction design intervention that imputes accountability to the human decision-maker to preserve agency without compromising the efficiency advantages of AI. Drawing on Work System Theory, we examine how RE restructures relational dynamics within AI-assisted workflows by embedding responsibility directly into the decision process. We find that when responsibility is enacted through work system design, evaluators experience greater agency and accountability, engage in more deliberative information processing, detect algorithmic errors at higher rates, and maintain stronger alignment with organizational standards. Collectively, producing more complementary human–AI performance. These effects are particularly consequential in judgment tasks, that involves context-sensitive evaluation under uncertainty and the risk of uncritical reliance on AI is particularly consequential. By demonstrating how interaction design can re-anchor human agency within AI-supported workflows, this research contributes to the literature on human–AI collaboration and offers practical guidance for designing work systems that preserve human intentionality while maintaining AI’s instrumental value.

Institutional Design: Incentive Alignment for Human–AI Value Maximization

6. Incentive Contract Design to Maximize Human–AI Collaborative Value (with Dingwei Gu, Yingjie Zhang). In progress. [Paper link] 

This ongoing study develops analytical models to examine incentive contract design that maximizes human–AI collaborative value, accounting for dynamic gaming interactions among the incremental value AI generates for employees, employees' risk attitudes, employers' ability to learn hidden information, and the resulting payoffs for both sides.

7. When AI Stays with Managers: Field Evidence on Employee Effort and Outcomes in Hierarchical Decision-Making (with Yinglin Ruan, Yingjie Zhang, Peijian Song). In manuscript drating. [Paper link] 

Hierarchical decision-making settings, where frontline employees gather and evaluate information while managers make final decisions, are pervasive in organizations, yet remain underexplored in the human–AI interaction literature, which has largely focused on individual-level decision support. In such settings, we introduce gatekeeping AI, where algorithmic recommendations are provided exclusively to managers. While this design may improve monitoring and reduce information asymmetry, it also introduces tensions regarding employee effort, incentives, and decision efficiency that require empirical examination. We investigate these issues using a mixed-method approach, combining a field intervention in a procurement context (2,051 cases) with qualitative interviews, grounded in agency theory and accountability theory. We find that gatekeeping AI increases employee effort and improves cost savings. Although greater effort extends employees’ task completion time, gatekeeping AI simultaneously accelerates managerial decision-making, resulting in an overall reduction in procurement time. Our findings show that these effects arise because gatekeeping AI reshapes information and monitoring, triggering anticipated justification, moral hazard mitigation, and competition pressure. This study highlights role-contingent AI access as a critical design dimension and advances understanding of how AI creates value in hierarchical organizations.

An Evolutionary Perspective on Human–AI Interaction

From Static Interaction To Dynamic Learning

8. Augmented Algorithms, Adaptive Humans? Evidence from a Natural Experiment (with Xianghua Lu, Yiyu Huang, Hai Wang). Under major revision. [Paper link]

AI capability continuously evolves through interactions with accumulated data and domain experts. This is achieved through ongoing learning to enhance specific algorithms for decision-making from human individuals in diverse industries. Would individuals be more inclined to follow AI advice if they understood that AI systems acquire wisdom from humans? Testing our hypotheses in the on-demand food delivery domain, we find the following: (1) High-experienced human riders show increased compliance with more human-like AI augmentation. (2) Their short-term performance becomes more balanced, with improved hourly delivery productivity but decreased on-time delivery ratios. (3) Mechanism analysis reveals their proactive shifts from prioritizing personal preferences to a balanced approach recommended by AI. (4) Over the long term, high-experienced riders recover on-time delivery ratios through self-regulated learning. (5) Low-experienced riders who consistently adhere to AI suggestions also benefit from AI capability augmentation in food delivery performance. Our findings delineate a dynamic cycle of mutual learning and reinforcement to demonstrate reciprocal wisdom between AI and humans, which underscores the critical role of high-experienced humans in achieving superior collaborative task outcomes in human-AI system evolution.  

9. Algorithmic Advancements and Workforce Dynamics in the Gig Economy: Evidence from the On-Demand Delivery Market (with Yiyu Huang, Chunhua Wu, Xianghua Lu). Under review. [Paper link] 

This paper examines how algorithmic advancements shape worker productivity and labor market dynamics in the gig economy. While prior research has emphasized technology's effects in traditional labor markets, the implications of continuous algorithmic advancement in the gig economy remain insufficiently understood. Using four years of comprehensive rider-level data from a major on-demand food delivery platform, we develop and estimate a structural model that links heterogeneous productivity responses, dynamic participation choices, and market-level demand–supply conditions. We find that improvements in dispatch and routing algorithms compress skill gaps by disproportionately raising the productivity of low-skilled riders, thereby reducing the marginal value of specialized domain skills. This skill-equalizing effect triggers asymmetric participation responses in a flexible labor environment: low-skilled riders increase engagement, whereas high-skilled riders reduce participation and are more likely to exit over time. Despite the resulting compositional decline in traditional skills, overall delivery performance improves and outcome dispersion widens, driven by emerging heterogeneity in workers’ ability to leverage algorithmic support. These findings reveal a new mechanism through which iterative algorithmic systems reshape labor markets: they equalize traditional skill premiums while generating new productivity stratification plausibly rooted in algorithm literacy. We discuss implications for platform design and labor governance in technology-mediated work.

From Traditional AI To Generative/Agentic AI

10. Visioning Human-Agentic AI Teaming: Continuity, Tension, and Future Research (with Bowen Lou, T. S. Raghu, Yingjie Zhang). Under 2nd round review. [Paper link]

Artificial intelligence is undergoing a structural transformation marked by the rise of agentic systems capable of open-ended action trajectories, generative representations and outputs, and evolving objectives. These properties introduce structural uncertainty into human–AI teaming (HAT), including uncertainty about behavior trajectories, epistemic grounding, and the stability of governing logics over time. Under such conditions, alignment cannot be secured through agreement on bounded outputs; it must be continuously sustained as plans unfold and priorities shift. We advance Team Situation Awareness (Team SA) theory, grounded in shared perception, comprehension, and projection, as an integrative anchor for this transition. While Team SA remains analytically foundational, its stabilizing logic presumes that shared awareness, once achieved, will support coordinated action through iterative updating. Agentic AI challenges this presumption. Our argument unfolds in two stages: first, we extend Team SA to reconceptualize both human and AI awareness under open-ended agency, including the sensemaking of projection congruence across heterogeneous systems. Second, we interrogate whether the dynamic processes traditionally assumed to stabilize teaming in relational interaction, cognitive learning, and coordination and control continue to function under adaptive autonomy. By distinguishing continuity from tension, we clarify where foundational insights hold and where structural uncertainty introduces strain, and articulate a forward-looking research agenda for HAT. We find that under open-ended agency, Team SA processes can produce the opposite of their intended effects: relational legitimacy may rest on epistemic fragility, iterative updating may amplify rather than correct divergence, and shared awareness may coexist with substantive loss of oversight. The central challenge of HAT is not whether humans and AI can agree in the moment, but whether they can remain aligned as futures are continuously generated, revised, enacted, and governed over time. 

From Individual-Level To Population-Level Impact

11. From Signal to Noise: How Widespread LLM Usage Transforms Evaluator Effort and Credit Screening Outcomes (with Paramveer Dhillon, Yi Gao, Yingjie Zhang). Under major revision. [Paper link] [INFORMS 2025 eBusiness Section Best Paper Award]

Large language models (LLMs) have transformed how applicants present themselves in screening processes and have created a fundamental tension: while AI-assisted writing enables better communication of applicant quality, widespread usage may erode the informational content that evaluators rely upon for decision-making. We examine this trade-off through a randomized field experiment where 59 professional evaluators assessed 1,000 micro-loan applications, exogenously varying LLM usage rates from 0% to 75% across treatment groups. Our results reveal a non-monotonic relationship between crowd-level LLM usage and screening performance. Moderate usage rates (15-30%) improve approval outcomes for qualified borrowers without affecting default rates, while widespread usage (60-75%) generates "signal dilution," a systematic degradation in diagnostic value as stylistic homogenization reduces variance in quality indicators. Drawing on Effort-Accuracy Tradeoff Theory and Signal Detection Theory, we show that high usage rates diminish evaluators' perceived discriminatory power, prompting reduced cognitive effort and increased approval conservatism. These behavioral adaptations prove counterproductive, increasing Type I errors while failing to reduce Type II errors, ultimately worsening portfolio performance and constraining credit access. We complement our empirical findings with an analytical model that extends the analysis beyond experimental constraints, deriving optimal usage thresholds and revealing that evaluator uncertainty about LLM prevalence can paradoxically worsen screening outcomes. Our analysis establishes signal dilution and evaluator effort adjustment as key mechanisms through which AI democratization undermines decision quality in information-intensive markets, with implications for recruitment, admissions, and other high-stakes screening environments.