Course Description
The ability to infer, model, and utilize 3D information from perceptual input is crucial to various intelligent systems (e.g., self-driving vehicles, mobile robots) and AI tasks (e.g., 2D image/3D asset generation, robot manipulation). The course will investigate the fundamentals and the latest advances in 3D vision as well as their applications in different fields. Topics will include, but not limited to, image formation, multi-view geometry, (neural) 3D representations, learning-based 3D algorithms, neural rendering, generative models. The students will play around various algorithms and models and improve or propose a creative use of them.
Format and Prerequisites
The course will feature a mixture of lectures, paper presentations/discussions (covering both classic and modern works), and a group final project. It will help students develop research skills, such as reading, presenting, and writing papers.
As this is an advanced course, it will assume substantial familiarity with the fundamentals of computer vision and machine learning.
The course is intended for PhD students. Masters and undergraduate students are strongly encouraged to first take the CS-4670/5670/6670. If you have taken the course and would like to enroll, please register for the waitlist and come to the class in the first week.
Learning Outcomes
After taking this course, students will be able to:
- Describe the challenges and limitations in 3D
- Analyze the pros and cons of 3D techniques and properly benchmark them
- Design new solutions to address identified limitations
- Identify potential applications of different 3D algorithms and applying them to different domains to resolve respective challenges
- Provide feedback and guidance to peer researchers
- Write a technical paper that can be accepted to a workshop or conference
Course Staff
Please use Ed Discussions for all communication with course staff.
Course Instructor
Teaching Assistant
Office Hours
Wei-Chiu: Tuesday after class, 11:30am ~ 12:30pm @ Gates 316
Rundong: Monday 10am ~ 11am @ Gates G33 Seat 15
Related Courses
The development of this course heavily relies on the following outstanding courses:
Learning for 3D Vision by Shubham Tulsiani, CMU
Learning for 3D Vision by Angjoo Kanazawa, UC Berkeley
3D Vision by Derek Hoiem, UIUC
Geometry-based Methods in Vision by Shubham Tulsiani, CMU
Introduction to Machine Perception by Shenlong Wang, UIUC
Advanced Topics in Computer Vision by David Fouhey, NYU
Physics-based Rendering by Ioannis (Yannis) Gkioulekas, CMU
Machine Learning for Inverse Graphics by Vincent Sitzmann, MIT
Machine Learning meets Geometry by Hao Su, UCSD
Role-Playing Paper-Reading Seminars by Alec Jacobson and Colin Raffel, UofT