Ehsan Amid

Education

PhD in Computer Science, University of California, Santa Cruz, CA (2015 - 2020)

• • • Thesis Advisor: Prof. Manfred K. Warmuth

    • Thesis Title: Tempered Bregman Divergence for Continuous and Discrete Time Mirror Descent and Robust Classification (Best Thesis Award!) pdf

MSc in Machine Learning and Data Mining (with Distinction), Aalto University, Espoo, Finland (2012 - 2014)

• • • Thesis Advisor: Prof. Erkki Oja

    • Thesis Title: Application of alpha-Divergence for Stochastic Neighbor Embedding in Data Visualization pdf

BSc in Telecommunication, Tehran Polytechnic, Tehran, Iran (2007 - 2012)

• Thesis Advisor: Prof. S. M. Ahadi

• Thesis Title: Musical Instrument Classification Using Statistical Models

Publications

2024

• Andrew Hard, Antonious M Girgis, Ehsan Amid, Sean Augenstein, Lara McConnaughey, Rajiv Mathews, Rohan Anil. "Learning from straggler clients in federated learning", arXiv preprint arXiv:2403.09086, 2024. pdf

• Richard Nock, Ehsan Amid, Frank Nielsen, Alexander Soen, and Manfred K Warmuth. "Tempered Calculus for ML: Application to Hyperbolic Model Embedding",  arXiv preprint arXiv:2402.04163, 2024. pdf

• C Fifty, D Duan, RG Junkins, E Amid, J Leskovec, C Ré, S Thrun. "Context-Aware Meta-Learning", ICLR, 2024. pdf

• Ehsan Amid, Frank Nielsen, Richard Nock, and Manfred K Warmuth. "Optimal Transport with Tempered Exponential Measures", AAAI, 2024. pdf

   2023

• Gemini Team. "Gemini: a family of highly capable multimodal models", preprint arXiv:2312.11805, 2023. pdf

• Ehsan Amid, Frank Nielsen, Richard Nock, Manfred K Warmuth. "The Tempered Hilbert Simplex Distance and Its Application To Non-linear Embeddings of TEMs", arXiv preprint arXiv:2311.13459, 2023. pdf

• Jared Lichtarge, Ehsan Amid, Shankar Kumar, Tien-Ju Yang, Rohan Anil, Rajiv Mathews. "Heterogeneous Federated Learning Using Knowledge Codistillation", arXiv preprint arXiv:2310.02549, 2023. pdf

• Manfred K Warmuth and Ehsan Amid. "Open Problem: Learning sparse linear concepts by priming the features", The Thirty Sixth Annual Conference on Learning Theory (COLT), 2023. pdf

• George E Dahl, Frank Schneider, Zachary Nado, ..., Ehsan Amid et al. "Benchmarking Neural Network Training Algorithms", arXiv preprint arXiv:2306.07179, 2023. pdf

• Richard Nock, Ehsan Amid, Manfred K. Warmuth. "Boosting with Tempered Exponential Measures", NeurIPS, 2023. pdf

• Christopher Fifty, Joseph M. Paggi, Ehsan Amid, Jure Leskovec, Ron Dror. "Harnessing Simulation for Molecular Embeddings", arXiv preprint arXiv:2302.02055, 2023. pdf

• Ehsan Amid, Rohan Anil, Christopher Fifty, and Manfred K. Warmuth. "Layerwise Bregman Representation Learning of Neural Networks with Applications to Knowledge Distillation", Transactions on Machine Learning Research (TMLR), 2023. pdf

• Ehsan Amid, Richard Nock, Manfred Warmuth. "Clustering above Exponential Families with Tempered Exponential Measures", International Conference on Artificial Intelligence and Statistics (AISTATS), 2023. pdf

• Jiaheng Wei, Harikrishna Narasimhan, Ehsan Amid, Wen-Sheng Chu, Yang Liu, Abhishek Kumar. "Distributionally Robust Post-hoc Classifiers under Prior Shifts", International Conference on Learning Representations (ICLR), 2023. pdf

• Jiaheng Wei, Zhaowei Zhu, Tianyi Luo, Ehsan Amid, Abhishek Kumar, Yang Liu. "To Aggregate or Not? Learning with Separate Noisy Labels", Proceedings of the 29th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, 2023. pdf

2022

• Yatong Chen, Abhishek Kumar, Yang Liu, Ehsan Amid. "Fast Implicit Constrained Optimization of Non-decomposable Objectives for Deep Networks", Has it Trained Yet? NeurIPS 2022 Workshop, 2022. pdf

• Abel L. Peirson*, Ehsan Amid*, Yatong Chen, Vlad Feinberg, Manfred K. Warmuth, Rohan Anil. "Fishy: Layerwise Fisher Approximation for Higher-order Neural Network Optimization", Has it Trained Yet? NeurIPS 2022 Workshop, 2022 (*Equal Contribution). pdf

• Ehsan Amid*, Om Thakkar*, Arun Narayanan, Rajiv Mathews, Françoise Beaufays. "Extracting Targeted Training Data from ASR Models, and How to Mitigate It", INTERSPEECH, 2022 (*Equal Contribution). Oral - pdf

• Ehsan Amid, Rohan Anil, Wojciech Kotłowski, Manfred K Warmuth. "Learning from Randomly Initialized Neural Network Features", arXiv preprint arXiv:2202.06438, 2022. pdf

• Ehsan Amid*, Rohan Anil*, and Manfred K. Warmuth. "LocoProp: Enhancing BackProp via Local Loss Optimization",  International Conference on Artificial Intelligence and Statistics (AISTATS), 2022 (*Equal Contribution). pdf

• Ehsan Amid, Arun Ganesh*, Rajiv Mathews, Swaroop Ramaswamy, Shuang Song, Thomas Steinke, Vinith M Suriyakumar, Om Thakkar, Abhradeep Thakurta. "Public Data-Assisted Mirror Descent for Private Model Training", International Conference on Machine Learning (ICML), 2022 (*Corresponding Author). Oral - pdf

2021

• Abhishek Kumar and Ehsan Amid. "Constrained Instance and Class Reweighting for Robust Learning under Label Noise", arXiv preprint arXiv:2111.05428, 2021. pdf

• Christopher Fifty, Ehsan Amid, Zhe Zhao, Tianhe Yu, Rohan Anil, Chelsea Finn. "Efficiently Identifying Task Groupings for Multi-Task Learning", NeurIPS, 2021. Spotlight - pdf

• Negin Majidi, Ehsan Amid, Hossein Talebi, and Manfred K. Warmuth. "Exponentiated Gradient Reweighting for Robust Training Under Label Noise and Beyond",  arXiv preprint arXiv:2104.01493, 2021. pdf

• Sina Rezaei Aghdam, Ehsan Amid, Marija Furdek, and Alexandre Graell i Amat. "Privacy-Preserving Wireless Federated Learning Exploiting Inherent Hardware Impairments",  International Workshop on Computer Aided Modeling and Design of Communication Links and Networks (CAMAD), 2021. pdf

• Manfred K. Warmuth, Wojciech Kotłowski, and Ehsan Amid. "A case where a spindly two-layer linear network whips any neural network with a fully connected input layer",  ALT, 2021. pdf

   2020

• Christopher Fifty, Ehsan Amid, Zhe Zhao, Tianhe Yu, Rohan Anil, and Chelsea Finn. "Measuring and Harnessing Transference in Multi-Task Learning", arXiv preprint arXiv:2010.15413, 2020. pdf

• Ehsan Amid, Rohan Anil, Christopher Fifty, and Manfred K. Warmuth. "Step-size Adaptation Using Exponentiated Gradient Updates", Workshop on "Beyond first-order methods in ML systems" at the 37th International Conference on Machine Learning (ICML), 2020. pdf

• Ehsan Amid and Manfred K. Warmuth. "Winnowing with Gradient Descent", Conference on Learning Theory (COLT), 2020. pdf

• Ehsan Amid and Manfred K. Warmuth. "Reparameterizing Mirror Descent as Gradient Descent", NeurIPS, 2020. pdf

• Hossein Talebi, Ehsan Amid, Peyman Milanfar, and Manfred K. Warmuth. "Rank-smoothed Pairwise Learning in Perceptual Quality Assessment", IEEE International Conference on Image Processing (ICIP), 2020. pdf

• Ehsan Amid and Manfred K. Warmuth. "Divergence-based motivation for online EM and combining hidden variable models",  UAI, 2020. pdf

• Ehsan Amid and Manfred K. Warmuth. "An Implicit Form of Krasulina's k-PCA Update without the Orthonormality Constraint", AAAI, 2020. pdf

2019

• Ehsan Amid, Manfred K. Warmuth, Rohan Anil, and Tomer Koren. "Robust Bi-Tempered Logistic Loss Based on Bregman Divergences",  Neurips, 2019. pdf, code, demo

• Ehsan Amid, Manfred K. Warmuth, and Sriram Srinivasan. "Two-temperature logistic regression based on the Tsallis divergence", AISTATS, 2019. pdf

• Ehsan Amid and Manfred K. Warmuth. "TriMap: Large-scale Dimensionality Reduction Using Triplets", arXiv preprint arXiv:1910.00204, 2019. pdf, code (incorporated into the scanpy package, FlowJo implementation by Ian Taylor)

2018 and older

• Ehsan Amid and Manfred K. Warmuth. "A more globally accurate dimensionality reduction method using triplets", arXiv preprint arXiv:1803.00854, 2018. pdf

• Ehsan Amid, Nikos Vlassis, Manfred K. Warmuth. "Low-dimensional data embedding via robust ranking",  arXiv preprint arXiv:1611.09957, 2016. pdf, code

• Ehsan Amid, Aristides Gionis, and Antti Ukkonen. "Semi-supervised kernel metric learning using relative comparisons." arXiv preprint arXiv:1612.00086, 2016. pdf

• Ehsan Amid and Antti Ukkonen. "Multiview triplet embedding: Learning attributes in multiple maps." International Conference on Machine Learning (ICML), 2015. pdf, code

• Ehsan Amid, Aristides Gionis, and Antti Ukkonen. "A kernel-learning approach to semi-supervised clustering with relative distance comparisons." Joint European Conference on Machine Learning and Knowledge Discovery in Databases (ECML PKDD), 2015. pdf, code

• Ehsan Amid, Onur Dikmen, and Erkki Oja. "Optimizing the Information Retrieval Trade-off in Data Visualization Using alpha-Divergence." arXiv preprint arXiv:1505.05821, 2015. pdf, code

• Ehsan Amid, et al. "Unsupervised feature extraction for multimedia event detection and ranking using audio content." 2014 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). IEEE, 2014. pdf

• Ehsan Amid. "Bayesian Non-parametric Image Segmentation with Markov Random Field Prior", Scandinavian Conference on Image Analysis, 2013. pdf

• S. Ishikawa, M. Koskela, M. Sjöberg, J. Laaksonen, E. Oja, E. Amid, K. Palomaki, A. Mesaros, and M. Kurimo. "Picsom experiments in TRECVID 2013", in Proc. of the TRECVID 2013 Workshop, 2013. pdf

• Sina R. Aghdam and Ehsan Amid. "A Fast Method of Steel Surface Defect Detection Using Decision Trees Applied to LBP based Features", IEEE Conference on Industrial Electronics and Applications, 2012.

• Ehsan Amid, Sina R. Aghdam and Hamidreza Amindavar. "Enhanced Performance for Support Vector Machines as Multiclass Classifiers in Steel Surface Defect Detection", International Conference on Computer Vision and Image Processing, 2012.

• Ehsan Amid and Sina R. Aghdam. "Musical Instrument Classification Using Embedded Hidden Markov Models", International Conference on Computer Vision and Image Processing, 2012.

Patents

• Ehsan Amid, Om Thakkar, Rajiv Mathews, Francoise Beaufays. "Phrase Extraction for ASR Models", US Patent US20230178094A1, 2023. pdf.

• Om Thakkar, Ehsan Amid, Arun Ganesh, et al. "Leveraging Public Data in Training Neural Networks with Private Mirror Descent", US Patent US20230103911A1, 2023. pdf.

• Abhishek Kumar, Ehsan Amid. "Unified Sample Reweighting Framework for Learning with Noisy Data and for Learning Difficult Examples or Groups", US20230044078A1, 2023. pdf

• Ehsan Amid, Manfred K. Warmuth, Rohan Anil. "Training neural networks using layer-wise losses", US 20220253713A1, 2022. pdf

• Nikos Vlassis and Ehsan Amid. "Enhanced triplet embedding and triplet creation for high-dimensional data visualizations", US Patent US10127694B2, 2018. pdf

Reviewing Contributions

Journals

• JMLR

• TMLR

• Information Geometry (INGE)

• Data Mining and Knowledge Discovery Journal (DAMI)

Conferences

• NeurIPS (Area Chair)

• ICLR (Area Chair)

• ACML (Area Chair)

• COLT

• ICML

• AISTATS

• ALT

• ICLR

• AAAI

• IJCAI

Tutorial

• Instructor for UCSC Tools of the Trade Bootcamp Series (topics: editors, git, LaTeX, Make, Shell, etc.)

Website: https://sites.google.com/a/ucsc.edu/bootcamps/home

Honors & Awards

• Fall 2015 -  UCSC Regent’s Fellowship University of California, Santa Cruz

• Summer 2014 - Awarded Master’s Degree with Distinction in Machine Learning and Data Mining, Aalto University

• 2012-2014 - Honours Programme Grant in the Department of Information and Computer Science, Aalto University (ics.aalto.fi/en/studies/honours_programme)

• Summer 2007 - National Physics Olympiad: Silver Medal (www.ysc.ac.ir)