Abstract

We evaluate a pilot in an Inspections Department to test the returns to a pair of algorithms that varied in their sophistication. We find that both algorithms provided substantial prediction gains over inspectors, suggesting that even simple data may be helpful. However, these gains did not result in improved decisions. Inspectors used their decision authority to override algorithmic recommendations, in part in consideration of other organizational objectives. Interviews with 55 departments find that while many ran similar pilots, all provided considerable decision authority to inspectors, and those with sophisticated pilots transitioned to simpler approaches. These findings suggest that for algorithms to improve managerial decisions, organizations must consider the returns to algorithmic sophistication in each context, and carefully manage how decision authority is allocated and used.