Design and analysis of near-optimal index policies for dynamic resource allocation in Markov decision process models of a variety of systems

Theory and algorithms for restless bandit indexation: I introduced the first general sufficient indexability conditions for finite-state restless bandits, along with an adaptive-greedy index-computing algorithm for the Whittle index, and the framework of partial conservation laws (PCLs), in 2001; I extended such results to finite-state restless bandits fed by a general resource, casting the PCL framework into a polyhedral linear programming framework, in 2002; I extended such results in 2006 to countable-state semi-Markov restless bandits, and to real-state restless bandits in 2020; see also the survey 2007. For an introductory treatment, see 2010

Design of new dynamic index policies for a variety of models via restless bandit indexation: control of admission and routing to parallel queues (2002, 2007); scheduling a multiclass make-to-order/make-to-stock M/G/1 queue (NM-2006); scheduling a multiclass finite-buffer delay-/loss-sensitive queue (2006)

Design of efficient algorithms for index computation: for the Gittins index (2007); for the Asawa and Tekenektzis index for bandits with switching costs (2008); for the classic index of Bradt, Johnson, and Karlin (1956) for finite-horizon bandits (2011)

Stochastic scheduling; see J. Niño-Mora (2009). Stochastic scheduling. In Encyclopedia of Optimization, 2nd edition, C.A. Floudas and P.M. Pardalos, eds., pp. 3818--3824. Springer, New York. [pdf]

Multiclass queueing networks: scheduling and control; see J. Niño-Mora (2011). Klimov's model. In Wiley Encyclopedia of Operations Research and Management Science.

Conservation laws; see J. Niño-Mora (2011). Conservation laws and related applications. In Wiley Encyclopedia of Operations Research and Management Science.

Mathematical programming / achievable performance region approach to Markov decision process models