Ehsan Amid

Senior Research Scientist at Google Brain

About Me

I finished my PhD in Computer Science with a focus on Machine Learning Theory at the University of California, Santa Cruz in June 2020. My research focuses on learning theory, divergence measures, optimization, and their applications in ML. I was formerly a Student Researcher at Google Research, Brain Team. I had the pleasure of working in the Federated Assistant team at Google as a Research Scientist, where I worked on federated optimization. I joined Google Brain as a Research Scientist in Oct 2021.

News


Education

PhD in Computer Science, University of California, Santa Cruz, CA (2015 - 2020)

      • Thesis Advisor: Prof. Manfred K. Warmuth

      • Thesis Title: Tempered Bregman Divergence for Continuous and Discrete Time Mirror Descent and Robust Classification (Best Thesis Award!) pdf

MSc in Machine Learning and Data Mining (with Distinction), Aalto University, Espoo, Finland (2012 - 2014)

      • Thesis Advisor: Prof. Erkki Oja

      • Thesis Title: Application of alpha-Divergence for Stochastic Neighbor Embedding in Data Visualization pdf

BSc in Telecommunication, Tehran Polytechnic, Tehran, Iran (2007 - 2012)

  • Thesis Advisor: Prof. S. M. Ahadi

  • Thesis Title: Musical Instrument Classification Using Statistical Models

Work Experience

  • Research Scientist, Google Research, Brain Team, Mountain View (Since October 2021)

  • Research Scientist, Google Mountain View (Aug 2020 - September 2021)

  • Research Scientist Intern, Google Brain Mountain View (March 2019 - June 2020)

  • Software Engineering Intern, Google Cloud (June 2018 - September 2018)

  • Data Scientist Intern, Microsoft (June 2017 - September 2017)

  • Data Scientist Intern, Adobe (June 2016 - November 2016)

  • Research Assistant, Helsinki Institute for Information Technology (September 2014 - July 2015)

Publications

2023

  • Ehsan Amid, Richard Nock, Manfred Warmuth. "Clustering above Exponential Families with Tempered Exponential Measures", International Conference on Artificial Intelligence and Statistics (AISTATS), 2023. pdf

2022

  • Yatong Chen, Abhishek Kumar, Yang Liu, Ehsan Amid. "Fast Implicit Constrained Optimization of Non-decomposable Objectives for Deep Networks", Has it Trained Yet? NeurIPS 2022 Workshop, 2022. pdf

  • Abel L. Peirson*, Ehsan Amid*, Yatong Chen, Vlad Feinberg, Manfred K. Warmuth, Rohan Anil. "Fishy: Layerwise Fisher Approximation for Higher-order Neural Network Optimization", Has it Trained Yet? NeurIPS 2022 Workshop, 2022 (*Equal Contribution). pdf

  • Ehsan Amid, Rohan Anil, Christopher Fifty, and Manfred K. Warmuth. "Layerwise Bregman Representation Learning with Applications to Knowledge Distillation", arXiv preprint arXiv:2209.07080, 2022. pdf

  • Jiaheng Wei, Zhaowei Zhu, Tianyi Luo, Ehsan Amid, Abhishek Kumar, Yang Liu. "To Aggregate or Not? Learning with Separate Noisy Labels", arXiv preprint arXiv:2206.07181, 2022. pdf

  • Ehsan Amid*, Om Thakkar*, Arun Narayanan, Rajiv Mathews, Françoise Beaufays. "Extracting Targeted Training Data from ASR Models, and How to Mitigate It", INTERSPEECH, 2022 (*Equal Contribution). Oral - pdf

  • Ehsan Amid, Rohan Anil, Wojciech Kotłowski, Manfred K Warmuth. "Learning from Randomly Initialized Neural Network Features", arXiv preprint arXiv:2202.06438, 2022. pdf

  • Ehsan Amid*, Rohan Anil*, and Manfred K. Warmuth. "LocoProp: Enhancing BackProp via Local Loss Optimization", International Conference on Artificial Intelligence and Statistics (AISTATS), 2022 (*Equal Contribution). pdf

  • Ehsan Amid, Arun Ganesh*, Rajiv Mathews, Swaroop Ramaswamy, Shuang Song, Thomas Steinke, Vinith M Suriyakumar, Om Thakkar, Abhradeep Thakurta. "Public Data-Assisted Mirror Descent for Private Model Training", International Conference on Machine Learning (ICML), 2022 (*Corresponding Author). Oral - pdf

2021

  • Abhishek Kumar and Ehsan Amid. "Constrained Instance and Class Reweighting for Robust Learning under Label Noise", arXiv preprint arXiv:2111.05428, 2021. pdf

  • Christopher Fifty, Ehsan Amid, Zhe Zhao, Tianhe Yu, Rohan Anil, Chelsea Finn. "Efficiently Identifying Task Groupings for Multi-Task Learning", NeurIPS, 2021. Spotlight - pdf

  • Negin Majidi, Ehsan Amid, Hossein Talebi, and Manfred K. Warmuth. "Exponentiated Gradient Reweighting for Robust Training Under Label Noise and Beyond", arXiv preprint arXiv:2104.01493, 2021. pdf

  • Sina Rezaei Aghdam, Ehsan Amid, Marija Furdek, and Alexandre Graell i Amat. "Privacy-Preserving Wireless Federated Learning Exploiting Inherent Hardware Impairments", International Workshop on Computer Aided Modeling and Design of Communication Links and Networks (CAMAD), 2021. pdf

  • Manfred K. Warmuth, Wojciech Kotłowski, and Ehsan Amid. "A case where a spindly two-layer linear network whips any neural network with a fully connected input layer", ALT, 2021. pdf

2020

  • Christopher Fifty, Ehsan Amid, Zhe Zhao, Tianhe Yu, Rohan Anil, and Chelsea Finn. "Measuring and Harnessing Transference in Multi-Task Learning", arXiv preprint arXiv:2010.15413, 2020. pdf

  • Ehsan Amid, Rohan Anil, Christopher Fifty, and Manfred K. Warmuth. "Step-size Adaptation Using Exponentiated Gradient Updates", Workshop on "Beyond first-order methods in ML systems" at the 37th International Conference on Machine Learning (ICML), 2020. pdf

  • Ehsan Amid and Manfred K. Warmuth. "Winnowing with Gradient Descent", Conference on Learning Theory (COLT), 2020. pdf

  • Ehsan Amid and Manfred K. Warmuth. "Reparameterizing Mirror Descent as Gradient Descent", NeurIPS, 2020. pdf

  • Hossein Talebi, Ehsan Amid, Peyman Milanfar, and Manfred K. Warmuth. "Rank-smoothed Pairwise Learning in Perceptual Quality Assessment", IEEE International Conference on Image Processing (ICIP), 2020. pdf

  • Ehsan Amid and Manfred K. Warmuth. "Divergence-based motivation for online EM and combining hidden variable models", UAI, 2020. pdf

  • Ehsan Amid and Manfred K. Warmuth. "An Implicit Form of Krasulina's k-PCA Update without the Orthonormality Constraint", AAAI, 2020. pdf

2019

  • Ehsan Amid, Manfred K. Warmuth, Rohan Anil, and Tomer Koren. "Robust Bi-Tempered Logistic Loss Based on Bregman Divergences", Neurips, 2019. pdf, code, demo

  • Ehsan Amid, Manfred K. Warmuth, and Sriram Srinivasan. "Two-temperature logistic regression based on the Tsallis divergence", AISTATS, 2019. pdf

  • Ehsan Amid and Manfred K. Warmuth. "TriMap: Large-scale Dimensionality Reduction Using Triplets", arXiv preprint arXiv:1910.00204, 2019. pdf, code (incorporated into the scanpy package, FlowJo implementation by Ian Taylor)

2018 and older

  • Ehsan Amid and Manfred K. Warmuth. "A more globally accurate dimensionality reduction method using triplets", arXiv preprint arXiv:1803.00854, 2018. pdf

  • Ehsan Amid, Nikos Vlassis, Manfred K. Warmuth. "Low-dimensional data embedding via robust ranking", arXiv preprint arXiv:1611.09957, 2016. pdf, code

  • Ehsan Amid, Aristides Gionis, and Antti Ukkonen. "Semi-supervised kernel metric learning using relative comparisons." arXiv preprint arXiv:1612.00086, 2016. pdf

  • Ehsan Amid and Antti Ukkonen. "Multiview triplet embedding: Learning attributes in multiple maps." International Conference on Machine Learning (ICML), 2015. pdf, code

  • Ehsan Amid, Aristides Gionis, and Antti Ukkonen. "A kernel-learning approach to semi-supervised clustering with relative distance comparisons." Joint European Conference on Machine Learning and Knowledge Discovery in Databases (ECML PKDD), 2015. pdf, code

  • Ehsan Amid, Onur Dikmen, and Erkki Oja. "Optimizing the Information Retrieval Trade-off in Data Visualization Using alpha-Divergence." arXiv preprint arXiv:1505.05821, 2015. pdf, code

  • Ehsan Amid, et al. "Unsupervised feature extraction for multimedia event detection and ranking using audio content." 2014 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). IEEE, 2014. pdf

  • Ehsan Amid. "Bayesian Non-parametric Image Segmentation with Markov Random Field Prior", Scandinavian Conference on Image Analysis, 2013. pdf

  • S. Ishikawa, M. Koskela, M. Sjöberg, J. Laaksonen, E. Oja, E. Amid, K. Palomaki, A. Mesaros, and M. Kurimo. "Picsom experiments in TRECVID 2013", in Proc. of the TRECVID 2013 Workshop, 2013. pdf

  • Sina R. Aghdam and Ehsan Amid. "A Fast Method of Steel Surface Defect Detection Using Decision Trees Applied to LBP based Features", IEEE Conference on Industrial Electronics and Applications, 2012.

  • Ehsan Amid, Sina R. Aghdam and Hamidreza Amindavar. "Enhanced Performance for Support Vector Machines as Multiclass Classifiers in Steel Surface Defect Detection", International Conference on Computer Vision and Image Processing, 2012.

  • Ehsan Amid and Sina R. Aghdam. "Musical Instrument Classification Using Embedded Hidden Markov Models", International Conference on Computer Vision and Image Processing, 2012.

Patent

  • Nikos Vlassis and Ehsan Amid. “Enhanced triplet embedding and triplet creation for high-dimensional data visualizations”, US Patent 10127694, 2018. pdf

Invited Talks

  • TriMap: Large-scale Dimensionality Reduction Using Triplets

      • Feb, 2018 - Google Brain, Mountain View, CA

      • Nov 2019 - UC Santa Cruz Genomics Institute

  • Robust Bi-tempered Logistic Loss

      • July 2019 - Google Mountain View, CA

      • Nov 2019 - Google New York City, NY

      • April 2020 - PARC

Reviewing Contributions

Journal

  • JMLR

  • TMLR

  • Data Mining and Knowledge Discovery Journal (DAMI)

Conference

  • NeurIPS

  • COLT

  • ICML

  • AISTATS

  • ALT

  • ICLR

  • AAAI

  • IJCAI

Open Source Projects

TriMap: Large-scale Dimensionality Reduction Using Triplets

Robust Bi-Tempered Logistic Loss Based on Bregman Divergences

Tutorial

  • Instructor for UCSC Tools of the Trade Bootcamp Series (topics: editors, git, LaTeX, Make, Shell, etc.)

Website: https://sites.google.com/a/ucsc.edu/bootcamps/home

Software Knowledge

Computing

  • Python, TensorFlow, Maple, MATLAB, R

Programming

  • C, Java, Bash Scripting

Parallel Computing

  • Numba, Familiar with Apache Spark and JAX

Other

  • SQL, LATEX , git, Unix, Linux

Honors & Awards

  • Fall 2015 - UCSC Regent’s Fellowship University of California, Santa Cruz

  • Summer 2014 - Awarded Master’s Degree with Distinction in Machine Learning and Data Mining, Aalto University

  • 2012-2014 - Honours Programme Grant in the Department of Information and Computer Science, Aalto University (ics.aalto.fi/en/studies/honours_programme)

  • Summer 2007 - National Physics Olympiad: Silver Medal (www.ysc.ac.ir)

References

Manfred Warmuth

Professor Emeritus

Department of Computer Science, University of California, Santa Cruz

Email: manfred@soe.ucsc.edu

Rohan Anil

Software Engineer

Google Brain, Mountain View, CA

Email: rohananil@google.com

Aristides Gionis

Professor

Department of Computer Science, Aalto University, Finland

Email: aristides.gionis@aalto.fi

Erkki Oja

Professor Emeritus

Department of Computer Science, Aalto University, Finland

Email: erkki.oja@aalto.fi