Classic bandits
Bandits with switching costs/times
Restless bandits
Multi-gear bandits
Multiclass queueing networks
Poisson's equation