Abstract: We present three results related to gradient descent for large neural networks. First, we present work showing how modified gradient descent algorithms can effectively handle non-convex inversion for deep neural networks. Our "surfing" algorithm carries out gradient descent over an evolving optimization surface obtained from the flow of parameters during training. Next, we introduce a new robust estimation framework where the goal is to recover a model if it has been corrupted after being estimated on data. Our results show how gradient descent enables repair of over-parameterized statistical models, including neural networks. Finally, we present results that prove convergence of a randomized backpropagation algorithm that is motivated from biological considerations. Joint work with Zhou Fan, Chao Gao, Ganlin Song, and Chris Xu.

Bio: John Lafferty is the John C. Malone Professor of Statistics and Data Science at Yale, with a secondary appointment in Computer Science. His main areas of research are machine learning, computational and statistical aspects of learning algorithms and high-dimensional data, modeling human language, and computational neuroscience. Lafferty received his doctoral degree in mathematics from Princeton University, where he was a member of the Program in Applied and Computational Mathematics. Prior to joining the Yale faculty in 2017, he was Louis Block Professor of Statistics and of Computer Science at the University of Chicago, where he helped launch a new Ph.D. program, the Committee on Computational and Applied Mathematics. He was also a faculty member in the Computer Science Department and the Machine Learning Department at Carnegie Mellon University, where he played a role in founding and directing the Machine Learning Ph.D. program. He initially began his work in machine learning at the IBM Watson Center in Yorktown Heights, N.Y. Lafferty serves on the advisory board of the Neural Information Processing Systems Foundation, and he is a founding Co-Editor-in-Chief of the ACM/IMS Journal of Data Science, a new journal being designed to bridge the computer science, machine learning, and statistics research communities. He is currently Associate Director of the Wu Tsai Institute at Yale University, and the Director of the Center for Neurocomputation and Machine Intelligence within the Institute.