Coding Theory for Large-Scale Machine Learning
Program
The workshop will be held in Room 202.
MORNING SESSION
08:45 - 09:00 Opening Remarks
09:00 - 09:30 Invited Talk by Salman Avestimehr (USC): Lagrange Coded Computing: Optimal Design for Resilient, Secure, and Private Distributed Learning
09:30 - 10:00 Invited Talk by Vivienne Sze (MIT): Exploiting redundancy for efficient processing of DNNs and beyond
10:00 - 10:10 Spotlight: "Locality Driven Coded Computation," Michael Rudow, Rashmi Vinayak and Venkat Guruswami
10:10 - 10:20 Spotlight: "CodeNet: Training Large-Scale Neural Networks in Presence of Soft-Errors," Sanghamitra Dutta, Ziqian Bai, Tze Meng Low and Pulkit Grover
10:20 - 10:30 Spotlight: "Reliable Clustering with Redundant Data Assignment," Venkat Gandikota, Arya Mazumdar and Ankit Singh Rawat
10:30 - 11:30 Poster Session
11:30 - 12:00 Invited Talk by Rashmi Vinayak (CMU): Resilient ML inference via coded computation: A learning-based approach
12:00 - 13:30 Lunch
AFTERNOON SESSION
13:30 - 14:00 Invited Talk by Markus Weimer (Microsoft): A case for coded computing on elastic compute
14:00 - 14:30 Invited Talk by Alex Dimakis (UT Austin): Coding theory for Distributed Learning
14:30 - 15:00 Invited Talk by Wei Zhang (IBM Research): Distributed deep learning system building at IBM: Scale-up and Scale-out case studies
15:00 - 15:30 Coffee Break
15:30 - 15:40 Spotlight: "OverSketched Newton: Fast Convex Optimization for Serverless Systems," Vipul Gupta, Swanand Kadhe, Thomas Courtade, Michael Mahoney and Kannan Ramchandran
15:40 - 15:50 Spotlight: "Cooperative SGD: A Unified Framework for the Design and Analysis of Communication-Efficient SGD Algorithms", Jianyu Wang and Gauri Joshi
15:50 - 16:00 Spotlight: "Secure Coded Multi-Party Computation for Massive Matrices with Adversarial Nodes," Seyed Reza, Mohammad Ali Maddah-Ali and Mohammad Reza Aref
16:00 - 17:15 Poster Session
Our invited speakers are:
- Salman Avestimehr (USC):
Salman Avestimehr is a Professor and co-director of the Communication Sciences Institute at the Electrical and Computer Engineering Department of University of Southern California. He received his Ph.D. in 2008 and M.S. degree in 2005 in Electrical Engineering and Computer Science, both from the University of California, Berkeley. Prior to that, he obtained his B.S. in Electrical Engineering from Sharif University of Technology in 2003. His research interests include information theory, coding theory, and large-scale distributed computing and machine learning. Dr. Avestimehr has received a number of awards for his research, including an Information Theory Society and Communication Society Joint Paper Award, a Presidential Early Career Award for Scientists and Engineers (PECASE) from the White House, a Young Investigator Program (YIP) award from the U. S. Air Force Office of Scientific Research, a National Science Foundation CAREER award, the David J. Sakrison Memorial Prize, and several Best Paper Awards at Conferences. He has been an Associate Editor for IEEE Transactions on Information Theory. He is currently a general Co-Chair of the 2020 International Symposium on Information Theory (ISIT).
- Alex Dimakis (UT Austin):
Alex Dimakis is an Associate Professor at the Electrical and Computer Engineering department, University of Texas at Austin. From 2009 until 2012 he was with the Viterbi School of Engineering, University of Southern California. He received his Ph.D. in 2008 and M.S. degree in 2005 in electrical engineering and computer sciences from UC Berkeley and the Diploma degree from the National Technical University of Athens in 2003. During 2009 he was a CMI postdoctoral scholar at Caltech.
He received an ARO young investigator award in 2014, the NSF Career award in 2011, a Google faculty research award in 2012 and the Eli Jury dissertation award in 2008. He is the co-recipient of several best paper awards including the joint Information Theory and Communications Society Best Paper Award in 2012. He served two terms as an associate editor for IEEE Signal Processing letters and is currently serving as an associate editor for IEEE Transactions on Information Theory. His research interests include information theory, coding theory and machine learning.
- Vivienne Sze (MIT)
Vivienne Sze received the B.A.Sc. (Hons) degree in electrical engineering from the University of Toronto, Toronto, ON, Canada, in 2004, and the S.M. and Ph.D. degree in electrical engineering from the Massachusetts Institute of Technology (MIT), Cambridge, MA, in 2006 and 2010 respectively. She received the Jin-Au Kong Outstanding Doctoral Thesis Prize in electrical engineering at MIT in 2011.
She is currently an Associate Professor in the Electrical Engineering and Computer Science Department at MIT. Her research interests include energy-efficient algorithms and architectures for portable multimedia applications. From September 2010 to July 2013, she was a Member of Technical Staff in the Systems and Applications R&D Center at Texas Instruments (TI), Dallas, TX, where she designed low-power algorithms and architectures for video coding. She also represented TI in the JCT-VC committee of ITU-T and ISO/IEC standards body during the development of High Efficiency Video Coding (HEVC), which received a Primetime Emmy Engineering Award. Within the committee, she was the primary coordinator of the core experiment on coefficient scanning and coding.
She is a recipient of the 2017 Qualcomm Faculty Award, the 2016 Google Faculty Research Award, the 2016 AFOSR Young Investigator Research Program (YIP) Award, the 2016 3M Non-Tenured Faculty Award, the 2014 DARPA Young Faculty Award, the 2007 DAC/ISSCC Student Design Contest Award, and a co-recipient of the 2017 CICC Outstanding Invited Paper Award, the 2016 IEEE Micro Top Picks Award and the 2008 A-SSCC Outstanding Design Award.
- Rashmi Vinayak (CMU)
Rashmi K. Vinayak is an assistant professor in the Computer Science department at Carnegie Mellon University. She received her PhD in the EECS department at UC Berkeley in 2016, and was a postdoctoral researcher at AMPLab/RISELab and BLISS at UC Berkeley. Her dissertation received the Eli Jury Award 2016 from the EECS department at UC Berkeley for "outstanding achievement in the area of systems, communications, control, or signal processing".
Rashmi is a recipient of the Facebook Communications and Networking Research Award 2018, Google Faculty Research Award 2018, IEEE Data Storage Best Paper and Best Student Paper Awards for the years 2011/2012. She was also a recipient of the Facebook Fellowship 2012-13, the Microsoft Research PhD Fellowship 2013-15, and the Google Anita Borg Memorial Scholarship 2015-16. Her research interests lie broadly in the areas of computer and networked systems and coding theory.
- Markus Weimer (Microsoft)
My career goal is to make machine learning more useful to more people.
To that end, I am an architect in Microsoft’s Cloud and AI division. There, I work with the Database and Developer Tools groups to bring machine learning to app developers, using properly and responsibly managed data. Prior to my current role, I led the team which launched ML.NET into Open Source. ML.NET is Microsoft’s machine learning toolkit. I also started and open sourced what is now known as Apache REEF.
Besides my role at Microsoft, I am also a member of the Apache Software Foundation and was the inaugural PMC chair (VP) of Apache REEF.
- Wei Zhang (IBM Research)
Dr. Wei Zhang (B.Eng’05, Beijing Unveristy of Technology; MSc’08, Technical University of Denmark; PhD’13, University of Wisconsin, Madison) is a research staff member at IBM T.J.Watson Research Center. Currently, he works in the machine learning acceleration department. His research interests include systems and large-scale optimization. His recent works in distributed deep learning are published in ICDM(2016,2017), IJCAI(2016,2017), MASCOT(2017), DAC(2017), AAAI (2018), NeurIPS (2017,2018), ICML (2018), and ICASSP(2019). His work won the ICDM’16 best paper award runner-up and MASCOT’17 best paper nominee. His NeurIPS’17 paper are ICML’18 papers were both invited to present orally at a 20-min length in the conference. Prior to his IBM career, he studied under Prof. Shan Lu at UW-Madison, with a focus on concurrent software system reliability. While at Wisconsin, he published papers in ASPLOS (2010,2011,2013), PLDI(2011), OSDI(2012) and OOPSLA(2013). His PLDI’11 paper won the SIGPLAN Research Highlights Award.