Meta-Learning: Automated Model Construction and Adaptation

Automated Design of Network Architectures (Off-line learning)

Renato Negrinho, Geoff GordonDeepArchitect: Automatically Designing and Training Deep Architectures , Code

Bowen Baker et al, Designing Neural Network Architectures using Reinforcement Learning, ICLR 2017

Barret Zoph, Quoc Le. Neural Architecture Search with Reinforcement Learning, ICLR 2017

Zelda Mariet, Suvrit Sra, Diversity Networks, ICLR 2016

Cortes, C., et al. AdaNet: Adaptive Structural Learning of Artificial Neural Networks, ICML 2017 

George Philipp, Jaime G. Carbonell. Nonparametric Neural Networks, ICLR 2017

Online Architecture Adaptation (On-line, incremental learning)

Sahil Garg, Irina Rish, Guillermo A. Cecchi, Aurelie Lozano. Neurogenesis-Inspired Dictionary Learning: Online Model Adaption in a Changing World , IJCAI  2017

Neurogenesis Deep Learning,  Timothy J. Draelos, Nadine E. Miner, Christopher C. Lamb, Craig M. Vineyard, Kristofor D. Carlson, Conrad D. James, James B. AimoneICLR-16 version

J Lee, J Yun, S Hwang, E Yang. Lifelong learning with dynamically expandable networks.  arXiv Oct 2017


"Routing", Internal Attention, Selective Execution (On-line, Lifelong Learning) 

Ben Poole*, Friedemann Zenke*, Surya Ganguli. Intelligent synapses for multi-task and transfer learning

Jacobs, Robert A and Jordan, Michael I and Nowlan, Steven J and Hinton, Geoffrey E.  Adaptive mixtures of local experts, 1991.

Lanlan Liu, Jia Deng. Dynamic Deep Neural Networks: Optimizing Accuracy-Efficiency Trade-offs by Selective Execution.  Jan 2017

Chrisantha Fernando, et al, PathNet: Evolution Channels Gradient Descent in Super Neural Networks

Embedding Choice in NLP:


ICLR-2017  A Simple but Tough-to-Beat Baseline for Sentence Embeddings

Learning to Learn

Ke Li, Jitendra Malik Learning to Optimize

Jane X Wang et al. Learning to reinforcement learn

Hyperband: Bandit-Based Configuration Evaluation for Hyperparameter Optimization
Lisha Li, Kevin Jamieson, Giulia DeSalvo, Afshin Rostamizadeh, Ameet Talwalkar

Shahriari et al. Taking the Human Out of the Loop: A review of Bayesian Optimization

Bergstra et al. Making a Science of Model Search: Hyperparameter Optimization in Hundreds of Dimensions for Vision Architectures