專題演講

Prof. Hongbo Fu

Hong Kong University of Science and Technology

時間：2024/7/11 9:40am

地點：台大資訊系館103演講廳

主持人: 陽明交通大學王昱舜教授

Towards More Accessible Tools for Content Creation

Abstract:

Traditional game and filming industries heavily rely on professional artists to make 2D and 3D visual content. In contrast, future industries such as metaverse and 3D printing highly demand digital content from personal users. With modern software, ordinary users can easily produce text documents, create simple drawings, make simple 3D models consisting of primitives, take images/videos, and possibly edit them with pre-defined filters. However, creating photorealistic images from scratch, fine-grained image retouching (e.g., for body reshaping), detailed 3D models, vivid 3D animations, etc., often require extensive training with professional software and is time-consuming, even for skillful artists. Generative AI, e.g., based on ChatGPT and Midjourney, recently has taken a big step and allows the easy generation of unique and high-quality images from text prompts. However, various problems, such as controllability and generation beyond images, still need to be solved. Besides AI, the recent advances in Augmented/Virtual Reality (AR/VR) software and hardware bring unique challenges and opportunities for content creation. In this talk, I will introduce my attempts to lower the barrier of content creation, making such tools more accessible to novice users. I will mainly focus on sketch-based content generation and content creation with AR/VR.

Dr. Hsin-Ying Lee

Snap Research

時間：2024/7/11 11:00am

地點：台大資訊系館103演講廳

主持人: 陽明交通大學劉育綸教授

3D/4D Generation and Modeling with Generative Priors

Abstract:

In the ever-expanding metaverse landscape, where physical and digital realms converge seamlessly, the imperative to capture, represent, and analyze three-dimensional structures is paramount. The evolution of 3D and 4D generation technologies has revolutionized applications such as gaming, augmented reality (AR), and virtual reality (VR), offering unparalleled immersion and interactivity. Bridging the gap between physical and digital domains, 3D modeling enables realistic simulations, immersive gaming experiences, and augmented reality overlays. Introducing the temporal dimension further enriches these experiences, enabling lifelike animations, object tracking, and understanding of complex spatiotemporal relationships, thus reshaping our digital interactions across entertainment, education, and beyond. The talk will offer a holistic exploration of 3D/4D generation and modeling, ranging from fundamental techniques to cutting-edge advancements. By navigating the intricacies of object- and scene-level generation and leveraging 2D priors for enhanced realism, attendees will emerge equipped with a nuanced understanding of the evolving landscape of 3D/4D modeling in the metaverse era.

Controllable Visual Imagination

Abstract:

Generative models have empowered human creators to visualize their imaginations without artistic skills and labor. A prominent example is large-scale text-to-image generation models. However, these models often are difficult to control and do not respect 3D perspective geometry. In this talk, I will showcase several of our recent efforts to improve controllability by leveraging physics-informed priors. Specifically, I will talk about how we improve semantic control for 2D image generations, generate realistic textures from reference images for 3D objects, and synthesize novel views, lighting, and weather for 3D scenes.