Embedded Files
University of Bretagne Sud, France
University of Bretagne Sud, France
Title: Domain adaptation by optimal transport: from basic concepts to recent advances
Abstract:
This presentation deals with the unsupervised domain adaptation problem, where one wants to estimate a prediction function in a given target domain without any labeled sample by exploiting the knowledge available from a source domain where labels are known. The aim of this talk is to discuss solutions for this problem leveraging on the theory of optimal transport. First we will present a method that estimates a mapping between the feature distributions in order to adapt the training dataset prior to learning. Next we discuss how to consider the joint feature/labels distribution in the
optimal transport problem. Since optimal transport can be hard to compute for large scale problems, potential solutions involving dual and mini-batches formulations will be presented, with recent applications to deep domain adaptation. Finally we will raise the question of dataset imbalance,
that can crucially impairs the adaptation performances. Potential new and promising research directions will conclude the talk.
Talk slides: Download
Princeton University, USA
Princeton University, USA
Title: Measuring Transferability: some recent insights
Abstract:
How much information is in the source data? How much target data should we collect if any? These are all practical questions that depend crucially on 'how far' the source domain is from the target. However, it remains generally unclear how to properly measure 'distance' between source and target.
In this talk we will argue that much of the traditional notions of 'distance' (e.g. KL-divergence, extensions of TV such as D_A discrepancy, and even density-ratios) can yield an over-pessimistic picture of transferability.
In fact, much of these measures are ill-defined or too large in common situations where, intuitively, transfer should be possible (e.g. situations with structured data of differing dimensions, or situations where the target distribution puts significant mass in regions of low source mass). Instead, we show that a notion of 'relative dimension' between source and target (which we simply term the 'transfer-exponent') captures the continuum from easy to hard transfer. The transfer-exponent uncovers a rich set of situations where transfer is possible even at fast rates, helps answer questions such as the benefit of unlabeled data, and has interesting implications for related problems such as multi-task learning.
Finally, the transfer-exponent yields sharp guidance as to when and how to sample target data and guarantee fast improvement over source data alone. We illustrate these new insights through various simulations on controlled data, and on the popular CIFAR-10 image dataset.
The talk is based on work with Guillaume Martinet, and ongoing work with Steve Hanneke.
Talk slides: Download
Tsinghua University, China
Tsinghua University, China
Title: Deep Transfer Learning
Abstract: Transfer learning is a subfield of machine learning that enables adaptation of learning machines to novel domains. While recent studies reveal that deep neural networks can learn transferable features to bridge different domains, the feature transferability drops significantly by increasing domain discrepancy. In this talk, I will introduce our recent advances in deep transfer learning, from basic approaches such as non-parametric and semi-parametric statistical distances to multiple and conditional domain adversaries, and to novel scenarios such as partial and open domain adaptation
Talk slides: Download
NVIDIA Research, USA
NVIDIA Research, USA
Title: Multimodal Image Domain Transfer
Abstract: Unsupervised image-to-image translation is an important and challenging problem in computer vision. Given an image in the source domain, the goal is to learn the conditional distribution of corresponding images in the target domain, without seeing any pairs of corresponding images. While this conditional distribution is inherently multimodal, existing approaches make an overly simplified assumption, modeling it as a deterministic one-to-one mapping. As a result, they fail to generate diverse outputs from a given source domain image. To address this limitation, we propose a Multimodal Unsupervised Image-to-image Translation (MUNIT) framework. We assume that the image representation can be decomposed into a content code that is domain-invariant, and a style code that captures domain-specific properties. To translate an image to another domain, we recombine its content code with a random style code sampled from the style space of the target domain. We analyze the proposed framework and establish several theoretical results. Extensive experiments with comparisons to the state-of-the-art approaches further demonstrates the advantage of the proposed framework. Moreover, our framework allows users to control the style of translation outputs by providing an example style image.
Talk slides: Download
Google Sites
Report abuse