Work Experience
Professor, Dept. of Computer Science, National Yang Ming Chiao Tung University (formerly NCTU), Taiwan (Aug. 2006 - Pres)
Primary research focus
Neural network-based image/video coding, e.g. end-to-end learned image/video coding, reinforcement learning-assisted image/video encoder control, and learned image/video coding for machines.
ISO/IEC & ITU-T video coding standards, e.g. H.264/AVC Scalable Extension (SVC), High-Efficiency Video Coding (H.265/HEVC), Screen Content coding (SCC), Versatile Video Coding (H.266/VVC), JPEG AI, and 360 Video.
Visual signal processing, e.g. image/video super-resolution and re-scaling.
Computer vision, e.g. semantic segmentation, incremental learning, video prediction, and domain adaptation.
Multi-modality data fusion for data analytics.
Graph computing and mining.
Developed learning-based and learning-assisted image/video compression
Proposed an end-to-end learned image (i.e. I-frame) codec with augmented normalizing flows (ANF).
Proposed an end-to-end learned P-frame codec with conditional augmented normalizing flows (CANF), one of the earliest works on learned conditional video coding and won the top-performance award at ISCAS 2022.
Proposed an end-to-end learned B-frame codec with conditional augmented normalizing flows (B-CANF), which is currently the state-of-the-art learned B-frame codec and won the top-creativity award at ISCAS 2023.
Led a team from NYCU and LUH (Leibniz Universität Hannover) to respond to Call for Learning-Based Video Codecs for Study of Quality Assessment, 2024.
Led a team from NYCU and PUT (Poznan Univ. of Technology) to participate in JPEG AI Call-for-Proposals on Learning-based Image Compression, 2022.
Led a team from NCTU to participate in JPEG AI Call-for-Evidence on Learning-based Image Compression, 2020.
Led a team from NCTU to participate in the Challenge on Learned Image Compression (CLIC) at CVPR 2018, 2019, 2020, 2021, 2022.
Offered 3 CASS distinguished lecture talks on “End-to-End Learned Image and Video Compression: Design, Implementation, and Computer Vision Applications,” in Italy, US, and Singapore, ’22-'23.
Offered tutorials on “Learning-based Image and Video Coding” at ICCV 2023, WACV 2023, SBCCI 2022, ISCAS 2021, and VCIP 2020.
Organized a special session on “Towards Practical Learning-based Image and Video Coding” at PCS 2022.
Organized a special section on “Learning-Based Image and Video Compression” in 2020 July TCSVT issue.
Pioneered the research on reinforcement learning-based video encoder control.
Proposed a reinforcement learning-based parametric OBMC for video prediction.
Served as a panelist for APSIPA panel on the “Future of Video Coding”, ’21.
Offered 1 tutorial on “Application of reinforcement learning to video encoder control and video prediction” at The 4th IEEE Seasonal School on Digital Processing of Visual Signals and Applications, Brazil, Oct. 2020.
Offered 2 APSIPA distinguished lecture talks on “Reinforcement Learning for Video Encoder Control and Video Prediction” in China and Korea, ’18, ’19.
Contributed actively to the development of the ISO/IEC & ITU-T HEVC Screen Content Coding (SCC) Extensions.
Led a team from NCTU (the only school team) to participate in the ISO/IEC & ITU-T joint Call-for-Proposals competition on HEVC Screen Content Coding (SCC) Extensions held in Valencia, Spain 2014.
Main inventor of Non-square Intra Block Copying, a key feature in the HEVC SCC standard.
Coordinated several Core Experiments on Intra Line Copy.
Organized one special session on HEVC Screen Content Coding in APSIPA ASC-2014.
Organized one special issue on Screen Content Video Coding and Applications in IEEE J. Emerging Select. Topics in Circuits Syst. (JETCAS), 2016.
Organized one special issue on “Immersive Video Coding and Transmission” in IEEE J. Emerging Select. Topics in Circuits Syst. (JETCAS), 2019.
Contributed actively to the development and promotion of the ISO/IEC & ITU-T H.265/HEVC standard, a celebrated milestone in the video standards community since the completion of the award-winning standard MPEG-4 Part 10 AVC/H.264.
Led a team from NCTU (one of the two school teams) to participate in the ISO/IEC & ITU-T joint Call-for-Proposals competition on HEVC held in Dresden, Germany 2010, where NCTU received a good evaluation result ranking among top-50% in a total of 27 proposals and top-1 in 2 test cases.
Conducted tests of a low-overhead bi-prediction technique in several Core Experiments established by the ISO/IEC & ITU-T JCT-VC committee, with a modified form of the technique adopted into the H.265/HEVC standard.
Provided a theoretical justification for the widely-discussed template matching prediction technique for H.265/HEVC.
Organized (two of the first few) special sessions on HEVC in premier conferences (e.g., IEEE ICME-2010 & APSIPA ASC-2010).
Served as the Technical Program Co-Chair for 2011 IEEE Visual Communications and Image Processing (VCIP-2011) conference and included HEVC and its 3-D Video Extension as the conference’s special themes.
Contributed actively to the development and promotion of the ISO/IEC & ITU-T H.264/AVC Scalable Amendment.
Participated in ISO/IEC MPEG Call-for-Evidence on Scalable Video Coding, 2003.
Participated in ISO/IEC MPEG Call-for-Proposal on Scalable Video Coding, 2004.
Provided enabling technologies for real-time SVC multicasting.
Developed an analytical mode-dependent rate-distortion model for H.264/SVC Coarse Grain Scalability (CGS), which is one of the few works that models the CGS rate-distortion behavior from a theoretical perspective and finds applications in rate control, encoder optimization, and analysis of SVC performance bottleneck.
Invented a fast SVC mode decision scheme, which provides a 20x encoding speed-up over the reference software implementation with a negligible loss in coding efficiency.
Developed a two-pronged approach to perform rate-distortion optimal extraction of SVC bitstreams for multicasting applications.
Organized a workshop on Scalable Video Coding and Transport in IEEE ISM-2007.
Explored emerging signal processing techniques for multimedia applications.
Applied compressed sensing techniques to developing a lossy image codec.
Applied machine learning techniques to video codec optimization and visual search applications.
Built a visual-search-based second screen system that can display extended and extra information of a TV program on audiences’ companion devices in a real-time and synchronous manner.
Visiting Scholar, Dept. of Network Science, IBM T. J. Watson Research Center, New York, USA (Aug. 2015 - July 2016)
Conducted joint study with IBM System G group on cross-domain data fusion for multimedia analytics
Postdoctoral Research Fellow, Institute of Electronics, National Chiao Tung University, Taiwan (Jan. 2006 – July 2006)
Applied transaction-level modeling to virtual platform construction for electronic system-level design and simulation.
Proposed one of the earliest platform-based system-on-chip design to implement an H.264/AVC decoder using software-hardware co-design. Today, embedded processors along with hardware accelerators are widely used to implement image/video codecs through software-hardware co-design.
ISO/IEC MPEG Delegate (Aug. 2003 - Pres.)
Served, since 2011, as the principal representative of Industrial Technology Research Institute International (ITRI-USA), a subsidiary of ITRI-Taiwan, in the InterNational Committee for Information Technology Standards (INCITS, the primary U.S. focus of standardization in the field of Information and Communications Technologies).
Attended the ISO/IEC MPEG & ITU-T VCEG standard meetings on a regular basis.
Contributed actively to the standardization of the ISO/IEC & ITU-T H.264/AVC Scalable Amendments, H.265/HEVC, H.265/HEVC Screen Content Extensions, H.266/VVC, and JPEG AI standards by participating in the respective Call-for-Evidence and Call-for-Proposals competitions as well as Core Experiments and Ad-hoc Groups on various subjects.
Helped promote the development of Taiwan ICT industry and further research momentum in the video coding area by regularly organizing open seminars (each drawing approximately 100 attendees from academia and industry) to share the first-hand information of the ISO/IEC & ITU-T standards activities and by providing consulting services to and working closely with local companies.
Research Assistant, Institute of Electronics, National Chiao Tung University, Taiwan (Aug. 2002 – Dec. 2005)
Invented a context-adaptive entropy coder to improve the compression performance of MPEG-4 FGS.
Participated in the MPEG Call-for-Evidence competition on SVC, where the NCTU proposal was ranked among top-3 as one of the best algorithms.
Participated in the MPEG Call-for-Proposals competition on SVC, where the NCTU proposal was ranked among top-50%.
Coordinated a Core Experiment on the coding efficiency and subjective quality improvement for SVC fine granular SNR scalability.
Intern, Intel Microprocessor Research Lab, California, USA (Nov. 2000 – Nov. 2001)
Developed a software library for MPEG video processing.
Analyzed core instructions of video applications for next-generation processors.
Built the first real-time MPEG-4 FGS software encoder.
Developed a macroblock-adaptive FGS algorithm to improve MPEG-4 FGS.
Research Assistant, Institute of Electronics, National Chiao Tung University, Taiwan (Aug. 1997 – Aug. 1999)
Developed a proof-of-concept MPEG-4 FGS streaming system.
Designed the first hardware architectures that proposed a line-based lifting scheme to implement a 2-D discrete wavelet transform (DWT), facilitating the adoption of DWT into JPEG-2000.
Built from scratch a JPEG software codec featuring one-pass rate control.