CS Colloquium: Deep 3D Representation Learning for Visual Computing

Among all digital representations we have for real physical objects, 3D is arguably the most expressive encoding. 3D representations allow storage and manipulation of high-level information (e.g. semantics, affordances, function) as well as low-level features (e.g. appearance, materials) about the object. Exploiting this 3D structure promises to improve our ability to build machines and autonomous agents that sense, understand, and act on the physical world around us. Historically, 3D visual computing has predominantly focused on single 3D models or small model collections. Now, however, with the advent of large 3D repositories of object models and inexpensive 3D scanning, the opportunity arises to re-define the field from the perspective of 3D big data.

In this talk, I will discuss a series of efforts on data-driven 3D visual computing, including constructing an information-rich large-scale 3D model repository (ShapeNet), generating synthetic data for supervising neural networks (RenderForCNN), and learning end-to-end neural networks for analyzing and synthesizing 3D geometries (PointNet and Point Set Generation Network) based on point set representations. Under the guiding principle of learning representations from 3D big data, these approaches have led to novel learning architectures resulting in top-performing algorithms for pure-3D data processing, as well as 3D-assisted semantic, geometric and physical property inference from 2D images. I will conclude my talk by describing several promising directions for future research. 

Bio:
Hao Su is currently a Ph.D. candidate in the Computer Science Department of Stanford University. He is a member of Stanford AI Lab and Geometric Computing Lab. Hao’s research interests are broad, spanning computer vision, computer graphics, and machine learning. In particular, recently he is interested in building large-scale 3D dataset (ShapeNet) and developing data-driven methods (deep learning) for 3D visual computing. He has served as the chair of multiple international conferences and workshops (Program Chair of 3DVision'17, Publication Chair of 3DVision'16, Co-chair of ECCV’16 workshop, Co-chair of CVPR’15 workshop, Co-chair of ICCV’15 workshop). He is also an invited speaker at NIPS’16 workshop and 3DV'16 workshop.