Related Courses
Algorithm configuration, empirical:
Hutter, F., Hoos, H. H., Leyton-Brown, K., & Stützle, T. (2009). ParamILS: an automatic algorithm configuration framework. Journal of Artificial Intelligence Research, 36, 267-306.
Hutter, F., Hoos, H. H., & Leyton-Brown, K. (2011, January). Sequential model-based optimization for general algorithm configuration. In International conference on learning and intelligent optimization (pp. 507-523). Springer, Berlin, Heidelberg.
Ansótegui, C., Malitsky, Y., Samulowitz, H., Sellmann, M., & Tierney, K. (2015, June). Model-based genetic algorithms for algorithm configuration. In Twenty-Fourth International Joint Conference on Artificial Intelligence.
Li, Jamieson, DeSalvo, Rostamizadeh, Talwalker. (2018). Hyberband: A Novel Bandit-Based Approach to Hyperparameter Optimization. In JMLR 2018.
Snoek, J., Larochelle, H., Adams, R. (2012). Practical Bayesian Optimization of Machine Learning Algorithms. In NeurIPS 2012.
Algorithm configuration, theoretical:
Kleinberg, R., Leyton-Brown, K., & Lucier, B. (2017, August). Efficiency Through Procrastination: Approximately Optimal Algorithm Configuration with Runtime Guarantees. In IJCAI (Vol. 3, p. 1).
Weisz, G., Gyorgy, A., & Szepesvari, C. (2018, July). LeapsAndBounds: A Method for Approximately Optimal Algorithm Configuration. In International Conference on Machine Learning (pp. 5257-5265).
Weisz, G., Gyorgy, A., & Szepesvári, C. (2019, May). CAPSANDRUNS: An improved method for approximately optimal algorithm configuration. In International Conference on Machine Learning (pp. 6707-6715).
Kleinberg, R., Leyton-Brown, K., Lucier, B., & Graham, D. (2019). Procrastinating with Confidence: Near-Optimal, Anytime, Adaptive Algorithm Configuration. In Advances in Neural Information Processing Systems (pp. 8881-8891).
Kirschner, Mutny, Hiller, Ischebeck, Krause. (2019) Adaptive and Safe Bayesian Optimization in High Dimensions via One-Dimensional Subspaces. In ICML 2019.
Algorithm portfolio:
Gomes, C. P., & Selman, B. (2001). Algorithm portfolios. Artificial Intelligence, 126(1-2), 43-62.
Xu, L., Hutter, F., Hoos, H. H., & Leyton-Brown, K. (2008). SATzilla: portfolio-based algorithm selection for SAT. Journal of artificial intelligence research, 32, 565-606.
KhudaBukhsh, A. R., Xu, L., Hoos, H. H., & Leyton-Brown, K. (2009, June). SATenstein: Automatically building local search SAT solvers from components. In Twenty-First International Joint Conference on Artificial Intelligence.
Xu, L., Hoos, H., & Leyton-Brown, K. (2010, July). Hydra: Automatically configuring algorithms for portfolio-based selection. In Twenty-Fourth AAAI Conference on Artificial Intelligence.
Hurley, B., Kotthoff, L., Malitsky, Y., & O’Sullivan, B. (2014, May). Proteus: A hierarchical portfolio of solvers and transformations. In International Conference on AI and OR Techniques in Constriant Programming for Combinatorial Optimization Problems (pp. 301-317). Springer, Cham.
Khalil, E. B., Dilkina, B., Nemhauser, G. L., Ahmed, S., & Shao, Y. (2017, August). Learning to Run Heuristics in Tree Search. In IJCAI (pp. 659-666).
SAT solvers:
Haim, S., & Walsh, T. (2009, June). Restart strategy selection using machine learning techniques. In International Conference on Theory and Applications of Satisfiability Testing (pp. 312-325). Springer, Berlin, Heidelberg.
Flint, A., & Blaschko, M. (2012). Perceptron learning of SAT. In Advances in Neural Information Processing Systems (pp. 2771-2779).
Selsam, D., Lamm, M., Bünz, B., Liang, P., de Moura, L., & Dill, D. L. (2018). Learning a SAT solver from single-bit supervision. arXiv preprint arXiv:1802.03685.
Selsam, D., & Bjørner, N. (2019, July). Guiding high-performance SAT solvers with unsat-core predictions. In International Conference on Theory and Applications of Satisfiability Testing (pp. 336-353). Springer, Cham.
Differentiable Optimization:
Amos, B., Kolter, Z. (2017). OptNet: Differentiable Optimization as a Layer in Neural Networks. In ICML 2017.
Wang, P.-W., Donti, P., Wilder, B., Kolter, Z. (2019). SATNet: Bridging deep learning and logical reasoning using a differentiable satisfiability solver. In ICML 2019.
Amos, B., Jimenez Rodriguez, I., Sacks, J., Boots, B., Kolter, Z. (2018). Differentiable MPC for End-to-end Planning and Control. In NeurIPS 2018.
Wilder, B., Ewing, W., Dilkina, B., Tambe, M. (2019). End to end learning and optimization on graphs. In NeurIPS 2019.
Akshay Agrawal, Brandon Amos, Shane Barratt, Stephen Boyd, Steven Diamond, Zico Kolter. (2019) Differentiable Convex Optimization Layers. In NeurIPS 2019.
Integer program solvers:
He, H., Daume III, H., & Eisner, J. M. (2014). Learning to search in branch and bound algorithms. In Advances in neural information processing systems (pp. 3293-3301).
Khalil, E. B., Le Bodic, P., Song, L., Nemhauser, G., & Dilkina, B. (2016, February). Learning to branch in mixed integer programming. In Thirtieth AAAI Conference on Artificial Intelligence.
Khalil, E. B., Dilkina, B., Nemhauser, G. L., Ahmed, S., & Shao, Y. (2017, August). Learning to Run Heuristics in Tree Search. In IJCAI (pp. 659-666).
Song, J., Lanka, R., Zhao, A., Bhatnagar, A., Yue, Y., & Ono, M. (2018). Learning to search via retrospective imitation. arXiv preprint arXiv:1804.00846.
Balcan, M. F., Dick, T., Sandholm, T., & Vitercik, E. (2018, July). Learning to Branch. In International Conference on Machine Learning (pp. 344-353).
Song, J., Lanka, R., Yue, Y., & Ono, M. (2019). Co-training for Policy Learning. In UAI 2019.
Gasse, M., Chételat, D., Ferroni, N., Charlin, L., & Lodi, A. (2019). Exact combinatorial optimization with graph convolutional neural networks. In Advances in Neural Information Processing Systems (pp. 15554-15566).
Tang, Y., Agrawal, S., & Faenza, Y. (2019). Reinforcement learning for integer programming: Learning to cut. arXiv preprint arXiv:1906.04859.
General combinatorial optimization:
Khalil, E., Dai, H., Zhang, Y., Dilkina, B., & Song, L. (2017). Learning combinatorial optimization algorithms over graphs. In Advances in Neural Information Processing Systems (pp. 6348-6358).
Li, Z., Chen, Q., & Koltun, V. (2018). Combinatorial optimization with graph convolutional networks and guided tree search. In Advances in Neural Information Processing Systems (pp. 539-548).
Mao, H., Schwarzkopf, M., Venkatakrishnan, S. B., Meng, Z., & Alizadeh, M. (2019). Learning scheduling algorithms for data processing clusters. In Proceedings of the ACM Special Interest Group on Data Communication (pp. 270-288).
Chen, X., & Tian, Y. (2019). Learning to perform local rewriting for combinatorial optimization. In Advances in Neural Information Processing Systems (pp. 6278-6289).
Wilder, B., Ewing, E., Dilkina, B., & Tambe, M. (2019). End to end learning and optimization on graphs. In Advances in Neural Information Processing Systems (pp. 4674-4685).
Wilder, B., Dilkina, B., & Tambe, M. (2019, July). Melding the data-decisions pipeline: Decision-focused learning for combinatorial optimization. In Proceedings of the AAAI Conference on Artificial Intelligence (Vol. 33, pp. 1658-1665).
Theorem proving & SMT:
Wang, M., Tang, Y., Wang, J., & Deng, J. (2017). Premise selection for theorem proving by deep graph embedding. In Advances in Neural Information Processing Systems (pp. 2786-2796).
Kaliszyk, C., Urban, J., Michalewski, H., & Olšák, M. (2018). Reinforcement learning of theorem proving. In Advances in Neural Information Processing Systems (pp. 8822-8833).
Huang, D., Dhariwal, P., Song, D., & Sutskever, I. (2018). Gamepad: A learning environment for theorem proving. arXiv preprint arXiv:1806.00608.
Lederman, G., Rabe, M. N., Lee, E. A., & Seshia, S. A. (2018). Learning heuristics for automated reasoning through deep reinforcement learning. arXiv preprint arXiv:1807.08058.
Balunovic, M., Bielik, P., & Vechev, M. (2018). Learning to solve SMT formulas. In Advances in Neural Information Processing Systems (pp. 10317-10328).
Yang, K., & Deng, J. (2019). Learning to prove theorems via interacting with proof assistants. arXiv preprint arXiv:1905.09381.
Wang, M., & Deng, J. (2020). Learning to Prove Theorems by Learning to Generate Theorems. arXiv preprint arXiv:2002.07019.
Neural program synthesis:
Si, X., Dai, H., Raghothaman, M., Naik, M., & Song, L. (2018). Learning loop invariants for program verification. In Advances in Neural Information Processing Systems (pp. 7751-7762).
Si, X., Yang, Y., Dai, H., Naik, M., & Song, L. (2018). Learning a meta-solver for syntax-guided program synthesis.
Classical algorithm + ML:
Kraska, T., Beutel, A., Chi, E. H., Dean, J., & Polyzotis, N. (2018, May). The case for learned index structures. In Proceedings of the 2018 International Conference on Management of Data (pp. 489-504).
Mitzenmacher, M. (2018). A model for learned bloom filters and optimizing by sandwiching. In Advances in Neural Information Processing Systems (pp. 464-473).
Purohit, M., Svitkina, Z., & Kumar, R. (2018). Improving online algorithms via ml predictions. In Advances in Neural Information Processing Systems (pp. 9661-9670).
Hsu, C. Y., Indyk, P., Katabi, D., & Vakilian, A. (2019, January). Learning-Based Frequency Estimation Algorithms. In International Conference on Learning Representations.
Indyk, P., Vakilian, A., & Yuan, Y. (2019). Learning-Based Low-Rank Approximations. In Advances in Neural Information Processing Systems (pp. 7400-7410).
Mitzenmacher, M. (2019). Scheduling with Predictions and the Price of Misprediction. arXiv preprint arXiv:1902.00732.
Mao, H., Schwarzkopf, M., Venkatakrishnan, S. B., Meng, Z., & Alizadeh, M. (2019). Learning scheduling algorithms for data processing clusters. In Proceedings of the ACM Special Interest Group on Data Communication (pp. 270-288).
Rohatgi, D. (2020). Near-Optimal Bounds for Online Caching with Machine Learned Advice. In Proceedings of the Fourteenth Annual ACM-SIAM Symposium on Discrete Algorithms (pp. 1834-1845). Society for Industrial and Applied Mathematics.
Yihe Dong, Piotr Indyk, Ilya Razenshteyn, Tal Wagner. (2020). Learning Space Partitions for Nearest Neighbor Search. In ICLR 2020.
Continuous Optimization:
Andrychowicz, M., Denil, M., Gomez, S., Hoffman, M. W., Pfau, D., Schaul, T., ... & De Freitas, N. (2016). Learning to learn by gradient descent by gradient descent. In Advances in neural information processing systems (pp. 3981-3989).
Xu, H., Zhang, H., Hu, Z., Liang, X., Salakhutdinov, R., & Xing, E. (2018). Autoloss: Learning discrete schedules for alternate optimization. arXiv preprint arXiv:1810.02442.
Li & Malik. (2017). Learning to Optimize. In ICLR 2017.
Kim, Wiseman, Miller, Sontag, Rush. (2018). Semi-Amortized Variational Autoencoders, In ICML 2018.
Marino, Yue, Mandt. (2018). Iterative Amortized Inference, In ICML 2018.
Marino, Cvitkovic, Yue. (2018). A General Method for Amortizing Variational Filtering, In NeurIPS 2018.
Metz, Maheswaranathan, Nixon, Freeman, Sohl-Dickstein. (2019). Understanding and correcting pathologies in the training of learned optimizers, In ICML 2019.
Chen, Hoffman, Colmenarejo, Denil, Lillicrap, Botvinick, de Freitas. (2017). Learning to Learn without Gradient Descent by Gradient Descent, In ICML 2017.
Hsieh, J.-T., Zhao, S., Eismann, S., Mirabella, L., Ermon, S. (2019). Learning Neural PDE Solvers with Convergence Guarantees. In ICLR 2019.
Finn, C., Abbeel, P., Levine, S., (2017). Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks. In ICML 2017.
Bello, I., Zoph, B., Vasudevan, V., Le., Q. (2017). Neural Optimizer Search with Reinforcement Learning. In ICML 2017.
Wichrowska et al. (2017). Learned Optimizers that Scale and Generalize. In ICML 2017.
Neural Architecture Search:
Li & Le. (2017) Neural Architecture Search with Reinforcement Learning, In ICLR 2017.
Elsken, Metzen, Hutter. (2018). Neural Architecture Search: A Survey.
He, Gong, Marino, Mori, Lehrmann. (2019). Variational Autoencoders with Jointly Optimized Dependency Structure, In ICLR 2019.
Liu, Simonyan, Yang. (2019). DARTS: Differentiable Architecture Search, In ICLR 2019.
ML for System Design:
Mirhoseini, A., Pham, H., Le, Q. V., Steiner, B., Larsen, R., Zhou, Y., ... & Dean, J. (2017, August). Device placement optimization with reinforcement learning. In Proceedings of the 34th International Conference on Machine Learning-Volume 70 (pp. 2430-2439). JMLR. org.
Mirhoseini, A., Goldie, A., Pham, H., Steiner, B., Le, Q. V., & Dean, J. (2018). A hierarchical model for device placement.
Chen, T., Zheng, L., Yan, E., Jiang, Z., Moreau, T., Ceze, L., ... & Krishnamurthy, A. (2018). Learning to optimize tensor programs. In Advances in Neural Information Processing Systems (pp. 3389-3400).
Chen, T., Moreau, T., Jiang, Z., Zheng, L., Yan, E., Shen, H., ... & Guestrin, C. (2018). {TVM}: An automated end-to-end optimizing compiler for deep learning. In 13th {USENIX} Symposium on Operating Systems Design and Implementation ({OSDI} 18) (pp. 578-594).
Krishnan, S., Yang, Z., Goldberg, K., Hellerstein, J., & Stoica, I. (2018). Learning to optimize join queries with deep reinforcement learning. arXiv preprint arXiv:1808.03196.
Paliwal, A., Gimeno, F., Nair, V., Li, Y., Lubin, M., Kohli, P., & Vinyals, O. (2019). Regal: Transfer learning for fast optimization of computation graphs. arXiv preprint arXiv:1905.02494.
Zhou, Y., Roy, S., Abdolrashidi, A., Wong, D., Ma, P. C., Xu, Q., ... & Laudon, J. (2019). GDP: Generalized Device Placement for Dataflow Graphs. arXiv preprint arXiv:1910.01
Ipek, Mutlu, Martinez, Caruana. (2008). Self-Optimizing Memory Controllers: A Reinforcement Learning Approach. In ISCA 2008.
ML for Engineering Design:
Berkenkamp, Schoellig, Krause. (2016). Safe Controller Optimization for Quadrotors with Gaussian Processes. In ICRA 2016.
Song, Chen, Yue. (2019). A General Framework for Multi-Fidelity Bayesian Optimization with Gaussian Processes. In AISTATS 2019.
Yang, Wu, Arnold. (2019). Machine Learning-guided directed evolution for protein engineering. In Nature Methods 2019.
Brookes et al. (2019). Conditioning by adaptive sampling for robust design. In ICML 2019.
Hany Abdelrahman, Felix Berkenkamp, Jan Poland, and Andreas Krause. (2016). Bayesian Optimization for Maximum Power Point Tracking in Photovoltaic Power Plants. In ECC 2016.
Miscellaneous:
Kruber, M., Lübbecke, M. E., & Parmentier, A. (2017, June). Learning when to use a decomposition. In International Conference on AI and OR Techniques in Constraint Programming for Combinatorial Optimization Problems (pp. 202-210). Springer, Cham.
Djolonga, J., & Krause, A. (2017). Differentiable learning of submodular models. In Advances in Neural Information Processing Systems (pp. 1013-1023).
Nair, V., Dvijotham, D., Dunning, I., & Vinyals, O. (2018). Learning Fast Optimizers for Contextual Stochastic Integer Programs. In UAI (pp. 591-600).
Alabi, D., Kalai, A. T., Liggett, K., Musco, C., Tzamos, C., & Vitercik, E. (2019, June). Learning to Prune: Speeding up Repeated Computations. In Conference on Learning Theory (pp. 30-33).
Bertsimas, D., & Stellato, B. (2019). Online mixed-integer optimization in milliseconds. arXiv preprint arXiv:1907.02206.
Ross, S., Zhou, J., Yue, Y., Dey, D., Bagnell, D. (2013). Learning Policies for Contextual Submodular Prediction. In ICML 2013.