Abstract
Computer Graphics and Geometry Processing are the main disciplines dealing with 3D data such as meshes and point clouds. In turn, Artificial Intelligence and Deep Learning are fundamental paradigms to manage visual data. Nevertheless, applying traditional learning paradigms on 3D data requires rethinking architectural building blocks designed for 2D images.
In this course, we will introduce different representations for 3D data, and basic geometry processing techniques that intervene in deep learning pipelines (discrete differential geometry, sampling, remeshing, conversion, …). Then, we will introduce methods able to learn tasks on 3D data. We will describe different architectures to process complex geometric domains, and the novel mechanisms introduced in the literature to preserve by design their intrinsic properties. Examples include graph learning techniques, augmented with geometric and topological information; attention modules to process unordered point sets and mesh data; transformer-like architectures for unstructured data.
Learning based, radiance oriented approaches for 3D objects representation and rendering, (like NERF, Gaussian Splatting and variants) will be also introduced and discussed.
In the final part of the course, we will present different applications where the interplay between Computer Graphics/Geometry Processing and Deep Learning is opening up to exciting results, including Computational Fabrication, Assisted Design, Architectural Geometry, and Environmental Monitoring.
Syllabus
3D Data Representation
Discrete Differential Geometry
Differentiable Rendering
Radiance based representations (NERF & Gaussian Splatting)
ML for Geometric Representations
Geometric Deep Learning
3D Design with AI
Generative Models for 3D
Lessons will start the , they will be in presence at the Department of Computer Science of Pisa University, and streamed live on Teams.
The course is 32h long, students can attend only to a portion of the course but it is mandatory that thy attend to the initial introductory set of lessons (marked with I), e.g. you can follow the first 8 hours and then the last 8 hours if you are more interested to those topics.
Exam: Paper seminar
Prerequisites: basic mathematical tools to understand concepts of differential geometry and basic machine learning concepts
Schedule:
Wed MAY 07 09:00-11:00 Course Intro & 3D Representations sala sem. est
MAY 07 14:00-16:00 3D Representations sala sem. est
Fri MAY 09 09:00-11:00 Differential Geometry (Laplacian, Geodesic, Curvature) sala sem. est
MAY 09 14:00-16:00 Geometry Processing: Chamfer Distance, up/down/re-Sampling sala sem. est
Wed MAY 28 09:00-10:00 Learning on voxels, images, point clouds; transformers sala sem. est
MAY 28 14:00-17:00 Learning on voxels, images, point clouds; transformers sala sem. est
Fri MAY 30 09:00-11:00 Learning on graphs and meshes FIB-M1
MAY 30 14:00-16:00 Case study: 3D Deep Learning in Architectural Geometry FIB-M1
Fri JUN 13 09:00-11:00 Differentiable Rendering sala sem. est
JUN 13 14:00-16:00 Differentiable Rendering sala riun. ovest
Wed JUN 18 09:00-11:00 Scene understanding sala sem. ovest
JUN 18 14:00-16:00 Software + Datasets sala sem. est
Wed JUL 02 09:00-11:00 Gaussian Splatting & Radiance Based Representations sala sem. est
JUL 02 14:00-16:00 Gaussian Splatting & Radiance Based Representations sala sem. est
Wed JUL 09 09:00-11:00 Multimodal and generative AI sala sem. est
JUL 09 14:00-16:00 Conclusions and exam planning sala sem. est
Papers & Materials:
(To be updated)
Exam Topics
(To be updated)
Lecturers
Paolo Cignoni, Massimiliano Corsini, Daniela Giorgi, Luigi Malomo