Today - Android Camera
I'm excited to now work on new ways to enhance Android to become the world's best computer vision platform. More to come as it becomes public.
2013-2017 Project Tango
Platform Software Technical Lead
First SWE on the project, working and growing the team and technology - from concept to prototype to production
2007-2013 NVIDIA GPU Computing
NVIDIA Corporation, Santa Clara, USA. Developer Technology Engineer 12/2007 -
Research and application of Graphics Processing Units (GPU) to accelerate computer vision with an aim to help bridge the gap between computer vision customers and the traditionally computer graphics based NVIDIA.
• Investigating performance peaks and bottlenecks in mapping computer vision onto GPUs to support customer developers and provide data for new GPU architecture development.
• Developed and delivered various academic and industrial talks to disseminate best practices and concepts in using GPUs for computer vision.
• Developed technology demos featuring algorithms (feature detection, face recognition etc.) and applications such as panorama stiching and robust 3D glasses head-tracking for view dependent rendering
1999 - 2007 Graduate School
Being a Cyborg and Computer Vision
Building a computer mediated reality with a 256 MHz wearable was challenging, so I needed to look to the GPU to find the compute power that could process vision in real-time.
Ph.D., Electrical and Computer Engineering,
University of Toronto, Dept. of Electrical and Computer Engineering, Toronto, Ontario, Canada, Graduated May 2007
- Thesis Topic: “Parallel General Purpose Computing Across Multiple Computer Graphics Devices”
- Supervisor: Steve Mann University of Toronto,
M.A.Sc. (Master’s of Applied Science),
Dept. of Electrical and Computer Engineering, Toronto, Ontario, Canada, Graduated January 2002
- Thesis Topic: “VideoOrbits of an Algebraic Projective Geometry and Comparametric Equations, with Applications in Computer Mediated Reality”
- Supervisor: Steve Mann
University of Toronto Applied Science and Engineering, Toronto, Ontario, Canada, B.A.Sc. (Bachelor’s of Applied Science, Engineering Science), Graduated May 1999
BOOKS & CHAPTERS
A. Munshi, B. Gaster, T. Mattson, J. Fung, D. Ginsburg, “OpenCL Programming Guide”, AddisonWesley/Pearson Education, 2011
Area editor for “Computer Vision” section of “GPU Computing Gems, Emerald Edition”, Morgan Kaufmann, 2011
J. Fung, “Computer Vision on the GPU”, in GPU Gems 2, Addison Wesley, 2005, chapter 40.
James Fung, Steve Mann, “Using Graphics Devices in Reverse: GPU-based Image Processing and Computer Vision”, Proceedings of the IEEE Int’l Conf. on Multimedia & Expo, Hannover, Germany, June 23-26, 2008
Steve Mann, James Fung, and Ariel Garten, “DECONcert: Making Waves with Water, EEG, and Music”, R. Kronland-Martinet, S. Ystad, and K. Jensen (Eds.): CMMR 2007, LNCS 4969, pp. 487-505. Springer-Verlag Berlin Heidelberg 2008
Steve Mann, James Fung and Raymond Lo, “Cyborglogging with Camera Phones: Steps Toward Equiveillance”, Proceedings of the ACM Multimedia 2006, Santa Barbara, California, Oct. 23–27, 2006
Winner of the ACM Open Source Competition Award: J. Fung, S. Mann and C. Aimone, “OpenVIDIA: Parallel GPU Computer Vision”, Proceedings of the ACM Multimedia 2005, Singapore, Nov. 6-11, 2005, pages 849-852 (151 Citations as of Apr. 2012)
J. Fung and S. Mann, “Using Multiple Graphics Cards as a General Purpose Parallel Computer : Applications to Computer Vision,” Proc. 17th Int’l Conf. on Pattern Recgonition (ICPR2004), August 2326, 2004, volume 1, pages 805808.
J. Fung and S. Mann, “Computer Vision Signal Processing on Graphics Processing Units”, Proc. IEEE Int’l Conf. on Acoustics, Speech, and Signal Processing (ICASSP 2004), May 1721, 2004, pages V-93 V96.
C. Aimone, J. Fung, S. Mann, “An Eyetap video-based featureless projective motion estimation assisted by gyroscopic tracking for wearable computer mediated reality”, Personal and Ubiquitous Computing, Springer–Verlag, vol. 7, no. 5, Oct. 2003, pp. 236–248
S. Mann, C. Manders and J. Fung, “Painting with Looks: Photographic images from video using quantimetric processing”, Proc. ACM Multimedia 2002, pp. 481–484
J. Fung, F. Tang and S. Mann, “Mediated Reality Using Computer Graphics Hardware for Computer Vision”, Proc. 6th Int’l Symposium on Wearable Computing (ISWC 02), pp. 83–89
F. Tang, C. Aimone, J. Fung, A. Marjan and S. Mann, “Seeing Eye to Eye: a shared mediated reality using EyeTap devices and the VideoOrbits Gyroscopic Head Tracker”, Proc. Int’l Symposium on Mixed and Augmented Reality (ISMAR2002), pp. 267–268
J. Fung and S. Mann, “Exploring Humanistic Intelligence Through Physiologically Mediated Reality”, Proc. Int’l Symposium on Mixed and Augmented Reality (ISMAR2002), pp. 275–276
A. Garabet, S. Mann, and J. Fung, “Exploring design through wearable computing art(ifacts)”, Proc. Conference on Human Factors and Computing Systems, Conference Extended Abstracts on Human Factors in Computer Systems 2002, pp 634–635
S. Mann and J. Fung, “Eye Tap devices for Augmented, Deliberately Diminished or otherwise al- tered visual perception of rigid planar patches of real world scenes”, Presence, MIT Press, vol. 11, no. 2, Apr. 2002, pp. 158–175
S. Mann and J. Fung, “VideoOrbits on Eye Tap devices for deliberately Diminished Reality or altering the visual perception of rigid planar patches of a real world scene”, Proc. 2nd Int’l Symposium on Mixed Reality (ISMR2001), pp. 48–55 S.
Mann, J. Fung and E. Moncrieff, “EyeTap Technology for Wireless Electronic News Gathering”, Mobile Computing and Communications Review, ACM Press, vol. 3, no. 4, October 1999, pp 19–26
Founder and contributor to the “OpenVIDIA” open-source project (http://openvidia.sourceforge.net) providing source code and examples of computer vision algorithms running on the GPU. Awarded “Best Open Source Project” by ACM Multimedia in 2005 (2005-current)
Guest journal editor “Real-Time Image Processing on Multi-Cores, Many-Cores and High-level FPGA-based Platforms”, European Association for Signal Processing, (2011)
DASIP Conference Session Chair “Signal and Image Processing on GPU” special session (2011)
Program Committee member for GPUCV (2011), CVPR GPU Workshop (2010)
Presented Technology Demos at Computer Vision & Pattern Recognition conference (CVPR2011), Face & Gesture (FG2011)
Sponsored talks at:
- ICCV 2011, Barcelona, GPU Computer Vision (GPUCV) Session sponsored talk
- GPU Technology Conference 2010: “Applying the Latest GPU Features to Computer Vision”
- GPU Summer School, Grenoble, France, 2009
- SIGGRAPH 2009 “Advances in GPU-based Image Processing and Computer Vision”
- GPU Technology Conference 2009: “GTC Computer Vision on the GPU”
Project member of “On the Identity Trail”, a multi-disciplinary (social science, law and technology) research project aimed at understanding the importance and impact of anonymity and authentication in a networked society. (2006-2007)
Honors and Awards
Canadian Business crowned the Glogger project (mobile photo & video sharing application) as having the “highest cool factor” of all the innovations featured at Research in Action, Feb. 2007
Invited Appearance on CBC (Canadian) National Radio Guest performance and interview regarding applicant’s “brainwave music” project, on CBC radio show “Go“, Nov. 2003
Television Appearance on Discovery Channel’s “Daily Planet” Segment on applicant’s “brainwave music” project on the Discovery Channel’s science series “Daily Planet”, Mar. 2003. http://bit.ly/SM49D
Ontario Graduate Scholarships/Lewfam Foundation Scholarship in Science and Technology (OGSST) (2003- 2004 & 2001-2002))
Ontario Graduate Scholarships (OGS) (2002-2003)
Centre for Information Technology Ontario (CITO) Graduate Scholarship (2000-2001)
Rutherford Scholarship for entrance to post secondary education (1995)
Governor Generals Medal, Lions Host Club Award Award for Top Academic Proficiency, received for graduating with the highest overall average of the high school graduating class (1995)
Other Work History and Projects
InteraXon, Toronto, Canada. Co-Founder 11/2007 Co-developed the initial concepts and built the first working prototypes to explore interactive brainwave software. This was an ongoing continuation of original work done for the “Deconcert” exhibit in 2003 where 48 participants’ EEG signals were collectively used to create a musical soundscape. Currently a non-employee co-founder of this active startup.
Social Dynamics, Toronto, Canada. Co-Founder 6/2006 - 11/2007 Consulting software development focused on social media projects seamlessly presented across mobile devices and web browsers. Projects included a live music based social network, a music festival guide for handheld devices (including Java based pre-smartphone support) and concept development in QR code based commerce. Glogger.mobi is still used especially in countries with pre-smartphone technology and limited desktop broadband. University of Toronto, Toronto, Canada.
Teaching Assistant 9/1999 - 2004 Duties included office hours and leading weekly computer lab exercises. Teaching included writing of Linux device drivers, operating system fundamentals, and programming fundamentals.
Savvysoft, New York, NY, USA. Software Development & Support 6/1998 – 8/1998 Financial software (derivatives pricing) software development and support.
Hyder Consulting, New York, NY, USA. I.T. Internship 5/1997 – 8/1997 Information Technology help desk support and office hardware & software support.
Parsons Brinkerhoff (Asia) Ltd. , Hong Kong Engineering Internship 5/1996 – 8/1996 Edited AutoCAD layout drawings & schematics, and assisted in engineering calculations