SCHEDULE

Schedule

Classes will take place Sunday 15:30-17:30

20/03 Intro

27/03 No class

03/04

Notice unusual time: 16:30-17:30

Intro to Self-Supervised Learning

10/04

BlinkEye CTO, Gilad Drozdov - behind the scenes on eye tracking

Read before class:

  • Eye Tracking for Everyone, Kyle Krafka, Aditya Khosla, Petr Kellnhofer, Harini Kannan, Suchendra Bhandarkar, Wojciech Matusik, Antonio Torralba, CVPR 2016 (paper, arxiv)


Noam Elata - Image reconstruction from human brain activity (papers)

Read before class:


Papers that will be reviewed:

  • Guohua Shen, Kshitij Dwivedi, Kei Majima, Tomoyasu Horikawa, View ORCID ProfileYukiyasu Kamitani, End-to-End Deep Image Reconstruction From Human Brain Activity, Frontiers in Computational Neuroscience 2019 (paper,github)

  • Suguru Wakita, Taiki Orima, Isamu Motoyoshi, Photorealistic reconstruction of visual texture from EEG signals, Frontiers in Computational Neuroscience 2021 (paper)

  • Madison Van Horn, Using Deep Learning for Visual Decoding and Reconstruction from Brain Activity: A Review (paper)

  • Guohua Shen, Tomoyasu Horikawa, Kei Majima, View ORCID ProfileYukiyasu Kamitani, Deep image reconstruction from human brain activity, PLOS Computational Biology 2017 (paper)

  • Roman Beliy*, Guy Gaziv*, Assaf Hoogi, Fracesca Strappini, Tal Golan, Michal Irani, From voxels to pixels and back: Self-supervision in natural-image reconstruction from fMRI, NeurIPS 2019. (paper)

17/04 Passover

24/04

Anton Agafonov & Sean Man - Synthetic data

Read before class

  • Image2StyleGAN: How to Embed Images Into the StyleGAN Latent Space?, Rameen Abdal, Yipeng Qin, Peter Wonka, ICCV 2019 (paper)


Papers that will be reviewed:

  • High Resolution Zero-Shot Domain Adaptation of Synthetically Rendered Face Images, Stephan J. Garbin, Marek Kowalski, Matthew Johnson, Jamie Shotton, ECCV 2020, (paper)


Extra material:

  • Fake It Till You Make It: Face Analysis in the Wild Using Synthetic Data Alone, Erroll Wood, Tadas Baltrušaitis, Charlie Hewitt, Sebastian Dziadzio, Thomas J. Cashman, Jamie Shotton, ICCV 2021 (arxiv, github)

  • MOTSynth: How Can Synthetic Data Help Pedestrian Detection and Tracking? Matteo Fabbri, Guillem Brasó, Gianluca Maugeri, Orcun Cetintas, Riccardo Gasparini, Aljoša Ošep, Simone Calderara, Laura Leal-Taixé, Rita Cucchiara, ICCV 2021 (arxiv)

  • Synthetic Humans for Action Recognition from Unseen Viewpoints, Gül Varol, Ivan Laptev, Cordelia Schmid, Andrew Zisserman, IJCV 2021 (arxiv)

01/05

Gabriel Gozal - Remote SpO2 measurement, Medical diagnosis

Read before class:

  • Sinhal Ruchika, K. Singh, and Raghuwanshi MM, “An Overview of Remote Photoplethysmography Methods for Vital Sign Monitoring,” Computer Vision and Machine Intelligence in Medical Image Analysis, pp. 21–31, 2020 (link)

Papers that will be reviewed?

  • Y. Mendelson, “Pulse Oximetry: Theory and Applications for Noninvasive Monitoring,” Clinical chemistry, vol. 38, no. 9, pp. 1601–1607, 1992

  • U. S. Freitas, “Remote camera-based pulse oximetry,” in eTELEMED 2014, The Sixth International Conference on eHealth, Telemedicine, and Social Medicine, 2014, pp. 59–63.

  • U. Bal, “Non-contact estimation of heart rate and oxygen saturation using ambient light,” Biomedical Optics Express, vol. 6, no. 1, p. 86, Jan. 2015, doi: 10.1364/boe.6.000086.

  • N. H. Kim, S. G. Yu, S. E. Kim, and E. C. Lee, “Non-contact oxygen saturation measurement using YCgCr color space with an RGB camera,” Sensors 2021 (link to paper)


Lior Dvir - Disease prediction (1)

Read before class:


Papers that will be reviewed

  • Deep EHR: Chronic Disease Prediction Using Medical Notes, Jingshu Liu, Zachariah Zhang, Narges Razavian, Machine Learning for Healthcare Conference PMLR 2018 (arxiv)

  • Feature selection and classification systems for chronic disease prediction: A review, Divya Jain, Vijendra Singh, Egyptian Informatics Journal 2018 (link)

  • Automatic Detection of Bowel Disease with Residual Networks, Robert Holland, Uday Patel, Phillip Lung, Elisa Chotzoglou, Bernhard Kainz, Workshop on Predictive Intelligence in Medicine, 2019 (arxiv)


08/05

RAFAEL - Nimrod Kruger - Event Cameras

  • A github resource page for most of the literature on Event-based Vision (link)

  • Event-based Vision: A Survey, Guillermo Gallego, Tobi Delbruck, Garrick Orchard, Chiara Bartolozzi, Brian Taba, Andrea Censi, Stefan Leutenegger, Andrew Davison, Joerg Conradt, Kostas Daniilidis, Davide Scaramuzza, PAMI, July 2020 (arxiv)


Hila Manor - Privacy preserving deep learning

Read before class:

A paper on attacking published models:

  • The Secret Revealer: Generative Model-Inversion Attacks Against Deep Neural Networks (link)


Papers that will be reviewed:

Anonymizing the data

  • Towards Privacy-Preserving Visual Recognition via Adversarial Training: A Pilot Study (link)

  • Learning to Anonymize Faces for Privacy Preserving Action Detection (link)

  • Differentially Private Imaging via Latent Space Manipulation (link)

  • PATE-GAN: GENERATING SYNTHETIC DATA WITH DIFFERENTIAL PRIVACY GUARANTEES (link)

  • Privacy-Preserving Generative Deep Neural Networks Support Clinical Data Sharing (link)

  • Visualizing deep convolutional neural networks using natural pre-images (link)



15/05

Yiftach Edelstein & Shai Yehezkel - Tactile Sensing

Read before class:

  1. Understanding Generative Adversarial Networks (GANs), by Joseph Rocca (link)

  2. 5 Ways Tactile Sensors Are Changing The Future Of Medical Technology (link)


Papers that will be reviewed:

  • Teaching Cameras to Feel: Estimating Tactile Physical Properties of Surfaces From Images, Matthew Purri, Kristin Dana, ECCV 2020 (arxiv)

  • ObjectFolder: A Dataset of Objects with Implicit Visual, Auditory, and Tactile Representations, Ruohan Gao, Yen-Yu Chang, Shivani Mall, Li Fei-Fei, Jiajun Wu, CoRL 2021 (arxiv)

22/05

Ari Frummer - Emotion Recognition

Read before class:

  • Learning Deep Features for Discriminative Localization, Bolei Zhou, Aditya Khosla, Agata Lapedriza, Aude Oliva, Antonio Torralba, CVPR 2016 (link)

Papers that will be reviewed:

  • Emotion Recognition in Context, Ronak Kosti, Jose M. Alvarez, Adria Recasens, Agata Lapedriza, CVPR 2017 (link)

  • Visually Interpretable Representation Learning for Depression Recognition from Facial Images, Xiuzhuang Zhou, Kai Jin, Yuanyuan Shang, G. Guo, IEEE Transactions on Affective Computing (link)


Matan Kleiner - Semi/Weakly-supervised learning

Read before class:

Papers that will be reviewed:

  • Semi-Supervised 3D Hand-Object Poses Estimation with Interactions in Time, Shaowei Liu, Hanwen Jiang, Jiarui Xu, Sifei Liu, Xiaolong Wang, CVPR 2021 (arxiv)

  • Semi-Supervised Action Recognition With Temporal Contrastive Learning, Ankit Singh, Omprakash Chakraborty, Ashutosh Varshney, Rameswar Panda, Rogerio Feris, Kate Saenko, Abir Das, CVPR 2021 (arxiv , github)

Other related papers:

  • S4L: Self-Supervised Semi-Supervised Learning, Xiaohua Zhai, Avital Oliver, Alexander Kolesnikov, Lucas Beyer, ICCV 2019 (arxiv)

  • Weakly Supervised Human-Object Interaction Detection in Video via Contrastive Spatiotemporal Regions, Shuang Li, Yilun Du, Antonio Torralba, Josef Sivic, Bryan Russell, ICCV 2021 (arxiv)

29/05

Shalev Shaer - Semi-Supervised Action Recognition

Read before class:

  • Understanding Contrastive Learning (blog post)


Papers that will be reviewed:

  • Semi-Supervised Action Recognition With Temporal Contrastive Learning, Ankit Singh, Omprakash Chakraborty, Ashutosh Varshney, Rameswar Panda, Rogerio Feris, Kate Saenko, Abir Das, CVPR 2021 (arxiv, github)

  • A comprehensive study of deep video action recognition, Yi Zhu, Xinyu Li, Chunhui Liu, Mohammadreza Zolfaghari, Yuanjun Xiong, Chongruo Wu, Zhi Zhang, Joseph Tighe, R. Manmatha, Mu Li, 2020 (arxiv)


Manor Zvi - Action recognition with transformers

Read before class:

  • The Illustrated Transformer (link)


Papers that will be reviewed:

  • TSM: Temporal Shift Module for Efficient Video Understanding, Ji Lin, Chuang Gan, Song Han, ICCV 2019, (arxiv, github)

  • Is Space-Time Attention All You Need for Video Understanding? Gedas Bertasius, Heng Wang, Lorenzo Torresani, ICML 2021 (arxiv)

  • ViViT: A Video Vision Transformer, Anurag Arnab, Mostafa Dehghani, Georg Heigold, Chen Sun, Mario Lučić, Cordelia Schmid, ICCV 2021 (arxiv, github)


05/06 Shavuot

12/06

Zebra Medical VP R&D, Ayelet Akselrod-Ballin - AI for Health Care



Eliel Aknin - Human Attention for Medical Image Analysis

Read before class:

  • Understanding Zero-Shot Learning — Making ML More Human (link)

Papers that will be reviewed:

  • Leveraging Human Selective Attention for Medical Image Analysis with Limited Training Data, Yifei Huang, Xiaoxiao Li, Lijin Yang, Lin Gu, Yingying Zhu, Hirofumi Seo, Qiuming Meng, Tatsuya Harada, Yoichi Sato, BMVC 2021, (paper)

  • Goal-Oriented Gaze Estimation for Zero-Shot Learning, Yang Liu, Lei Zhou, Xiao Bai, Yifei Huang, Lin Gu, Jun Zhou, Tatsuya Harada, CVPR 2021 (paper)

19/06

Bat-Sheva Einbinder - DeepFake detection (and generation)

  • MesoNet: a Compact Facial Video Forgery Detection Network, Darius Afchar, Vincent Nozick, Junichi Yamagishi, Isao Echizen, IEEE Workshop on Information Forensics and Security, WIFS 2018 (arxiv)

  • DeepFakes and Beyond: A Survey of Face Manipulation and Fake Detection, Ruben Tolosana, Ruben Vera-Rodriguez, Julian Fierrez, Aythami Morales, Javier Ortega-Garcia, Information Fusion, 2020 (arxiv)

  • FaceForensics++: Learning to Detect Manipulated Facial Images, Andreas Rössler, Davide Cozzolino, Luisa Verdoliva, Christian Riess, Justus Thies, Matthias Nießner, ICCV 2019 (arxiv)

  • Celeb-DF: A Large-scale Challenging Dataset for DeepFake Forensics, Yuezun Li, Xin Yang, Pu Sun, Honggang Qi, Siwei Lyu, CVPR 2020 (arxiv, github)

26/06

Eyal Hanania - Disease prediction (2)

  • D-UNet: a dimension-fusion U shape network for chronic stroke lesion segmentation, Yongjin Zhou, Weijian Huang, Pei Dong, Yong Xia, Shan-Shan Wang, ACM Transactions on Computational Biology and bioinformatics (link)

  • SynthMorph: Learning Contrast-Invariant Registration Without Acquired Images, Transactions in Medical Imaging, Malte Hoffmann, Benjamin Billot, Douglas N. Greve, Juan Eugenio Iglesias, Bruce Fischl, Adrian V. Dalca, 2021 (arxiv, project)


Ofry Livney - Video Next Frame Prediction

Papers that will be reviewed:

  • Unsupervised Learning for Physical Interaction Through Video Prediction, Chelsea Finn, Ian Goodfellow, Sergey Levine, NeurIPS 2016 (arxiv)

  • Generative Video Transformer: Can Objects be the Words? Yi-Fu Wu, Jaesik Yoon, Sungjin Ahn, ICML 2021 (arxiv)

  • Disentangling Physical Dynamics from Unknown Factors for Unsupervised Video Prediction, Vincent Le Guen, Nicolas Thome, CVPR 2020 (arxiv)

  • Stochastic Variational Video Prediction, Mohammad Babaeizadeh, Chelsea Finn, Dumitru Erhan, Roy H. Campbell, Sergey Levine, ICLR 2018 (arxiv)

  • Stochastic Latent Residual Video Prediction, Jean-Yves Franceschi, Edouard Delasalles, Mickaël Chen, Sylvain Lamprier, Patrick Gallinari, ICML 2020 (arxiv)

  • Greedy Hierarchical Variational Autoencoders for Large-Scale Video Prediction, Bohan Wu, Suraj Nair, Roberto Martin-Martin, Li Fei-Fei, Chelsea Finn, CVPR 2021 (arxiv)




Additional suggested topics and papers

2D and 3D human pose estimation

  • PoseAug: A Differentiable Pose Augmentation Framework for 3D Human Pose Estimation, Kehong Gong, Jianfeng Zhang, Jiashi Feng, CVPR21 (arxiv)

  • On Self-Contact and Human Pose, Lea Müller, Ahmed A. A. Osman, Siyu Tang, Chun-Hao P. Huang, Michael J. Black, CVPR21 (arxiv)

  • Human POSEitioning System (HPS): 3D Human Pose Estimation and Self-localization in Large Scenes from Body-Mounted Sensors, Vladimir Guzov, Aymen Mir, Torsten Sattler, Gerard Pons-Moll, CVPR21 (arxiv)

  • MHFormer: Multi-Hypothesis Transformer for 3D Human Pose Estimation, Wenhao Li, Hong Liu, Hao Tang, Pichao Wang, Luc Van Gool, CVPR21 (arxiv)

  • EgoBody: Human Body Shape, Motion and Social Interactions from Head-Mounted Devices, Siwei Zhang, Qianli Ma, Yan Zhang, Zhiyin Qian, Marc Pollefeys, Federica Bogo, Siyu Tang, (arxiv)

  • HuMoR: 3D Human Motion Model for Robust Pose Estimation, Davis Rempe, Tolga Birdal, Aaron Hertzmann, Jimei Yang, Srinath Sridhar, Leonidas J. Guibas, ICCV 2021 (arxiv)

Gaze estimation

  • Goal-Oriented Gaze Estimation for Zero-Shot Learning, Yang Liu, Lei Zhou, Xiao Bai, Yifei Huang, Lin Gu, Jun Zhou, Tatsuya Harada, CVPR 2021 (arxiv)

  • Weakly-Supervised Physically Unconstrained Gaze Estimation, Rakshit Kothari, Shalini De Mello, Umar Iqbal, Wonmin Byeon, Seonwook Park, Jan Kautz, CVPR 2021 (arxiv)

  • MTGLS: Multi-Task Gaze Estimation With Limited Supervision, Shreya Ghosh, Munawar Hayat, Abhinav Dhall, Jarrod Knibbe, WACV 2022 (arxiv)

Privacy preserving deep learning

  • Towards Privacy-Preserving Visual Recognition via Adversarial Training: A Pilot Study, Zhenyu Wu, Zhangyang Wang, Zhaowen Wang, Hailin Jin, ECCV 2018 (arxiv , github)

  • Learning to Anonymize Faces for Privacy Preserving Action Detection, Zhongzheng Ren, Yong Jae Lee, Michael S. Ryoo, ECCV 2018 (arxiv, project page)

  • Privacy Preserving Structure-from-Motion, Marcel Geppert, Viktor Larsson, Pablo Speciale, Johannes L. Sch¨onberger, and Marc Pollefeys, ECCV 2018, (gthub)

  • Mixed-Privacy Forgetting in Deep Networks, Aditya Golatkar, Alessandro Achille, Avinash Ravichandran, Marzia Polito, Stefano Soatto, CVPR 2021 (paper)

  • Privacy-Preserving Deep Learning, Reza Shokri, Vitaly Shmatikov, SIGSAC 2015, (paper)

Self-supervised learning

  • Self-Supervised Learning of Pretext-Invariant Representations, Ishan Misra, Laurens van der Maaten, CVPR 2021 (arxiv)

  • SelFlow: Self-Supervised Learning of Optical Flow, Pengpeng Liu, Michael Lyu, Irwin King, Jia Xu, CVPR 2019, (arxiv)

  • Self-Supervised 3D Skeleton Action Representation Learning With Motion Consistency and Continuity, Yukun Su, Guosheng Lin, Qingyao Wu, ICCV 2021 (CVF open access)


  • MOTSynth: How Can Synthetic Data Help Pedestrian Detection and Tracking? Matteo Fabbri, Guillem Braso, Gianluca Maugeri, Orcun Cetintas, Riccardo Gasparini, Aljosa Osep, Simone Calderara, Laura Leal-Taixe, Rita Cucchiara (link)

  • Leveraging Human Selective Attention for Medical Image Analysis with Limited Training Data, Yifei Huang, Xiaoxiao Li, Lijin Yang, Lin Gu, Yingying Zhu, Hirofumi Seo, Qiuming Meng, Tatsuya Harada, Yoichi Sato (link)