tonytung.org


Dr. Tony TUNG
Research Scientist Manager
Facebook Reality Labs
San Francisco Bay Area, USA

I am a Researcher specialized in computer vision for 3D modeling and reconstruction of dynamic scenes (shape, appearance, motion). I currently work at Facebook in the San Francisco Bay Area. Our projects include the creation of the next generation of virtual humans for AR/VR applications. I received my Ph.D. degree from Telecom Paris, France, and I was an Assistant Professor at Kyoto UniversityMatsuyama Lab., JapanMy past positions also include: Lead Scientist/Vice Group Manager at Rakuten, Tokyo, Japan (CV for e-commerce), R&D Engineer at Trimble, Paris area, France (3D laser scanner), and Software Engineer at AlcatelParis area, France (mobile services).

Publication List: HERE

Contact: tony2ng _at_ gmail.com



NEWS

2020

  • (Aug) Invited Speaker at the First “3D Poses in the Wild” Challenge, held at ECCV 2020. Talk on "The Next-Gen Virtual Humans": [link]
  • (July) “SIZER” paper accepted at ECCV 2020 as Oral presentation (2% out of 5025 submissions) [ArXiv]

SIZER - ECCV 2020 Oral


  • (July) “TexMesh” paper accepted at ECCV 2020 (27%) [arXiv]

TexMesh - ECCV 2020


  • (Jun) Michael Abrash’s talk at the Computer Vision for AR/VR workshop at CVPR 2020 about some of our efforts on physically correct human avatars [video][uploadvr]

Physically correct human avatars


  • (Mar-May) Stay Home. Wear your Mask.
  • (Feb) "ARCH: Animatable Reconstruction of Clothed Humans" paper accepted at CVPR 2020 (Acceptance rate: 1470/6656 = 22.08%) [arXiv][video][venturebeat]

ARCH - CVPR 2020

Animatable full-body 3D humans from one RGB image.


2019
  • (Oct) Attending ICCV19 in Seoul, South Korea
  • (Sep-) Welcome to Yinghao Huang (Research Intern, PhD Student from MPII, Germany)!
  • (July 22) ”DenseRaC” paper accepted at ICCV 2019 as Oral presentation (4.3% acceptance rate out of 4303 submissions)! [arXiv][video]

DenseRaC - ICCV 2019 Oral

3D body estimation from monocular RGB.

  • (July-) Welcome to Dr. Yuanlu Xu (Research Scientist, from UCLA)!
  • (June-) Welcome to Dr. Nikolaos Sarafianos (Research Scientist, from Univ. of Houston)!
  • (June-) Welcome to Zerong Zheng (Research Intern, PhD student from Tsinghua Univ., China)!
  • (June) Attending CVPR19 in Long Beach, CA, USA —> we are recruiting!
  • (May-) Welcome to Zeng Huang (Research Intern, PhD student from USC, USA)!
  • (May-) Welcome to Junbang Liang (Research Intern, PhD student from UNC, USA)!
  • (May-) Welcome to Jing Li (Research Intern, PhD student from Univ. of Utah, USA)!
  • (May 1st) Check out our team effort on Virtual Human creation presented at the F8's keynote:

F8 2019: VR Full-Body Avatars

VR full-body avatars.

  • Reviewer for CVPR19, ICCV19, BMVC19, SIGGRAPH19, 3DV19, WACV19, ICCV-3DRW19,


2018
  • (Dec) Attending Siggraph Asia 18 in Tokyo, Japan
  • (Sep-) Welcome to Jinlong Yang (Research Intern, PhD student from Morpheo group, INRIA Rhône-Alpes, France)!
  • (Sep) Attending ECCV18 in Munich, Germany
  • (Aug) Attending Siggraph 2018 in Vancouver, Canada
  • (July-) Welcome to Dr. Tuur Stuyck (Postdoc Research Scientist, from Pixar)!
  • (July 6) “DeepWrinkles” paper accepted at ECCV 2018 as Oral presentation (2.4% acceptance rate out of 2439 submissions)! [paper][video]

DeepWrinkles - ECCV 2018 Oral

Generation of realistic wrinkles on 3D clothing.

  • (June-) Welcome to Yuanlu Xu (Research Intern, PhD student from UCLA)!
  • (June) Attending CVPR18 in South Lake City, Utah —> we are recruiting!
  • (May-) Welcome to Chao Li (Research Intern, PhD student from University of Texas at Dallas)!
  • (May-) Welcome to Nikolaos Sarafianos (Research Intern, PhD student from University of Houston)!
  • (May 7) Oculus Research has been renamed Facebook Reality Labs!
  • Appointed Publication Chair of IAPR MVA19 (Machine Vision and Application), main computer vision conference in Japan to be held in Tokyo in May 2019
  • Reviewer for CVPR18, ECCV18, BMVC18, 3DV18, ACCV18, ECCV-3DRW18

2017
  • (Sep-) Welcome to Nadia Robertini (Research Intern, PhD student from Graphics, Vision & Video Group, Max-Planck-Institut für Informatik, Saarbrücken, Germany)!
  • (Sep-) Welcome to Zorah Lähner (Research Intern, PhD student from Computer Vision Group, Technical University Munich, Germany)!
  • (Sep) Associate Editor of IEICE Trans. Information and Systems, special issue on Machine Vision and its Applications
  • (Aug) Attending SIGGRAPH17 in Los Angeles, CA
  • (Jul) Attending CVPR17 in Honolulu, Hawaii
  • (Jun-) Welcome to Vignesh Ganapathi-Subramanian (Research Intern, PhD student from Leo Guibas Lab, Stanford University)!
  • Reviewer for the French National Research Agency (ANR)
  • Reviewer for CVPR17, ICCV17, BMVC17, 3DV17
    2016
    • (Dec) Pano2CAD paper accepted at IEEE WACV 2017
    • (Sep) Pano2CAD paper on arXiv (Rakuten's project) [pub]
    Pano2CAD
    From single panorama image of furnished room to CAD 3D model.

    • (Jul) AR Furniture Shopping (Rakuten's project) was presented at SIGGRAPH2016 [pub
      • TV news (05/03) [video]
      • Web news (04/27) [link]
      • Rakuten news (04/26) [link]

    Siggraph16 Appyhour “AR Furniture Shopping”

    Mobile application for AR/VR furniture shopping.

    • (Jun) Attending CVPR --> We are recruiting!
    • (Apr) I joined Facebook / Oculus Research as a Research Scientist in San Francisco, USA
    • 7 patents filed with Rakuten
    • Reviewer for CVPR16, ECCV16, ACCV16, 3DV16, PG16, IJCV, CGF, IEEE SPL
    2015
    Pr. Kanade with RIT Tokyo
    Pr. Kanade with part of the Team at Rakuten, Tokyo.
    • (Nov 8-9) Gave industry talk at MathWorks Asian Faculty Research Summit 2015, Tokyo
    • (Oct) Promoted to Vice Manager of our new Visual Computing Group (Research Lead)
    • (Oct 19-22) Attend 3DV2015, Lyon, France
    • (Oct 1) Welcome to Dr. Bjorn Stenger! [scholar]
    • (Oct 1) Welcome to Dr. Tomoyuki Mukasa (Kyoto Univ.)!
    • (Oct 1) Welcome to Dr. Daiki Kimura (Tokyo Tech)!
    • (Sep 29) Attend ISMAR15, Fukuoka, Japan: Rakuten Bronze sponsorship + demo
    • (Sep 19) Attend Matsuyama Laboratory 20th anniversary symposium at Kyoto University :)
    • (Aug-) Welcome to Tommi Kerola (Intern, PhD student from Tokyo Tech)!
    • (Aug 23) Rakuten is a Gold sponsor of the Machine Learning Summer School to be held in Kyoto! Find us there! [MLSS]
    • (July 13) Rakuten acquired fits.me (UK virtual fitting room startup), now our new collaborators!
    • (July-) Welcome to Adnane Boukhayma (Intern, PhD student from Morpheo team, INRIA Grenoble)!
    • (Jun-) Welcome to Dr. Jiu Xu (Waseda Univ.)!
    • (Jun 1) SIGGRAPH 2015 Poster accepted [link]
    • (May 26) Keynote Speech at Kinovis Inaugural Workshop on dynamic 3D shape modeling, INRIA Grenoble, France [link]
    • (Mar 04) Rakuten Reality Domain Group has a paper accepted at CVPR2015!
    • (Feb 28) Summary of my research work at Matsuyama Laboratory, Kyoto University on YouTube [link] and slideshare [link
    • (Feb 20) The Visual Computer journal paper to be published in Mar. 2015 [link]
    • (Feb-) Welcome to Shintaro Murakami (Intern, MS student from Sato Laboratory, The University of Tokyo)!
    • (Jan 23) Poster&Demo at IPSJ SIG-CVIM: Computer Vision and Image Media, Nara Institute of Science and Technology
      • Collaboration with Keio University, Tokyo, Japan
    2014
    • (Dec 22) VR tourism application demo using Oculus Rift DK2 at Nagano prefecture office, Tokyo
      • Press and Media coverage by national television TV Tokyo [link]
    • (Dec 8-11) Present paper at 3DV2014, Tokyo [link]
    • (Dec 8) Session chair at 3DV WS on Dynamic Shape Measurement and Analysis
    • (Nov 21) Guest speaker at plenary session - French Chamber of Commerce in Japan (Chambre de Commerce et d'Industrie au Japon) []
      • Research seminar co-organized by the French Embassy and the CNRS (National Scientific Research Center of France)
    • (Oct 1) Joined Rakuten, Inc. as Lead Scientist of the Reality Domain Group, Rakuten Institute of Technology (Computer Vision R&D)
    Recap of my 3D Video projects at Matsuyama Lab., Kyoto University, Japan (2008-2014).
    • (Sep) Photo with Matsuyama Laboratory [here]
    • (Sep 8) Paper accepted at 3DV2014
    • (Sep 6-12) Presented paper at ECCV2014, Zurich, Switzerland
      • Collaboration with INRIA Grenoble, France
    • (Sep 1-2) Gave invited talk at IPSJ SIG-CVIM: Computer Vision and Image Media, University of Tsukuba [url]
    • (Aug 6) In Program Committee of the 3DV Workshop on Dynamic Shape Measurement and Analysis [url
    • (July 12-13) Staff at Kyoto University-Inamori Foundation Joint Kyoto Prize Symposium [url]
      • Invited Speakers: Pr. Pentland (MIT), Pr. Kanade (CMU), Pr. LeCun (Facebook/NYU)
    • (July 11) In Program Committee of ACM VRST2014 Symp. Virtual Reality Software and Technology [url]
    • (Jun 23-28) Presented paper at CVPR2014, Columbus, Ohio, USA [Publications][spotlight]
    • (Jun 16) Paper accepted at ECCV2014 [Publications]
    • (Jun 11) Released code for dynamic surface alignment using geodesic mapping (our CVPR10 and PAMI14 papers) [Downloads][Github]
    • (Jun 6) Released library libamrg to compute augmented Multiresolution Reeb Graphs and 3D shape similarity [aMRG]
    • (May 4) IEEE Trans. Human-Machine Systems journal paper accepted (multimodal human-human interaction) [Publications]
    • (Apr 2) Grant-in-aid for young scientist awarded by JSPS (2-year project funding)
    • (Apr 1) Microsoft Azure project nominated "lighthouse" project (value: sixty man-hours)
    • (Mar 24-28) Hosted Dr. Franck Hetroy-Wheeler (Assoc. Pr.) from INRIA Grenoble, France (Talk on Point Cloud Processing of Trees and Plants)
    • (Mar 13) In Program Committee of 3DV2014 Int'l Conf. 3D Vision [url]
    • (Feb 25) Paper accepted at CVPR2014 [Publications] [youtube]
    • (Feb 16) The Visual Computer journal paper accepted (inconsistent geometric data encoding) [Publications]
    • (Feb 8) Selected to participate in the Kinect for Windows v2 Developer Preview Program
    • (Jan 11) Won Microsoft Azure for Research Award (value: 40,000 USD)
    • (Jan 9) IPSJ Trans. CVA paper accepted (kinematic structure estimation) [Publications]

      2013
      • (Dec) Received Dr. Jean-Sebastien Franco (Assoc. Pr.) and Benjamin Allain (PhD stu.) from INRIA Grenoble, France
        • Dr. J.-S. Franco gave a talk on Advances in 4D Modeling
      • (Dec 2) Visited Microsoft Japan, Shinagawa HQ, Tokyo, Japan
      • (Nov 21-26) Visited Morpheo research team at INRIA Grenoble, France
        • Gave talk at seminar on 4D modeling
      • (Nov 8) Received Pr. James J. Little from University of British Columbia, Vancouver, Canada 
        • (Talks on `Tracking and Measurements in Sports Video` and `Actively Using Vision and Context for Home Robotics`)
      • (Oct 8) Online version of PAMI[ieee] and CVPR13[ieee] are available on IEEE Xplore [Publications]
      • (Oct 5) Added C++ code to compute geodesics on 3D surface mesh [Downloads]
      • (Sep 12) View online 3D video reconstruction sample here (with JAVA) [Downloads]
      • (Aug 20) IEEE Trans. PAMI paper accepted [Publications]
      • (Jul 30-Aug 1) Gave talk (invited lecture) at MIRU2013, Tokyo, Japan [link]
      • (Jul 21-26) Presented paper at HCI2013 in Las Vegas, Nevada, USA [Publications]
      • (Jul 19) In Program Committee of ICCV2013 Int'l Workshop on Dynamic Shape Capture and Analysis (4DMOD) [link]
      • (Jul 3) In Program Committee of ACPR2013 Int'l Joint Workshop on Advanced Sensing / Visual Attention and Interaction (ASVAI) [link]
      • (Jul 2) Obtained Highly Skilled Professional status from Ministry of Justice, Japan
      • (Jun 23-28) Presented paper at CVPR2013 in Portland, Oregon, USA [Publications]
      • (Jun 20) Added a mini tool to test photometric stereo using a monitor (laptop/desktop) as light source [Downloads]
      • (Jun 8) Met the President of France François Hollande at the French embassy in Tokyo, Japan [see photo]
      • (May 22) Received Dr. Tae-Kyun Kim (Lecturer) from Imperial College London, UK (Talk on Randomised Forests)
      • (May 8) Added "Downloads" section to this webpage. You will find:
        • Binaries to compute augmented Multiresolution Reeb Graphs (aMRG) from 3D models as in [Tung et al., IJSM2005]: [here]
        • 1000 meshes from the "Yoga" sequence used in [Tung et al., PAMI2012]: [here] and [here] (please don't forget to cite us)
      • (May 2) Received Dr. Alejandro Ribes (Research Eng.) from EDF R&D, France (Talk on Multispectral Imaging)
      • (Apr 18) Received Dr. Tamy Boubekeur (Assoc. Pr.) and Noura Faraj (PhD stu.) from Telecom ParisTech, France
      • (Mar 18-19) Gave talk at Kyoto University and Queen Mary University of London Symposium on Intelligent Sensing, Kyoto University [link]
      • (Mar 6) Gave talk at Hanoi University of Science and Technology, Hanoi, Vietnam
      • (Mar 5) Gave talk at Vietnam National University, Hanoi, Vietnam
      • (Mar 2) Paper accepted at CVPR2013
      • (Feb 4) In Program Committee of CVPR2013 Int'l Workshop on Human Activity Understanding from 3D Data (HAU3D) [link
      • (Feb) Hosted Benjamin Petit (PhD stu.) and Vagia Tsiminaki (PhD stu.) from INRIA Grenoble for one month, France
      • (Feb 1) In Program Committee of 3DV2013 Int'l Conf. 3D Vision [link
      • (Jan 22) Gave talk at 3D Human Behavior Analysis Workshop, Kyoto University [link]
        • Received Pr. Adrian Hilton from University of Surrey, UK (Talk at Workshop)
        • Received Pr. Edmond Boyer from INRIA Grenoble, France (Talk at Workshop)
      3D Video of traditional Japanese dance performed by Maiko-san, captured at Mastuyama Lab., Kyoto University, Japan.
      web stats