How do we get robots out of the labs and into the real world with all it's complexities?
Robots must solve two fundamental problems -- (1) Perception: Sense the world using different modalities and (2) Decision making: Act in the world by reasoning over decisions and their consequences. Machine learning promises to solve both problems in a scalable way using data. However, it has fallen short when it comes to robotics.
This course dives deep into robot learning, looks at fundamental algorithms and challenges, and case-studies of real-world applications from self-driving to manipulation. We look at:
Date | Lecture | Preread | Resources |
---|---|---|---|
Fundamentals | |||
1/23/24 | Introduction to Robot Learning [slides] | The Bitter Lesson | |
1/25/24 | Robots as Markov Decision Problems [slides] | MACRL Ch. 1 | Dan Klein slides I |
1/30/24 | Solving MDPs [slides, notes] | MACRL Ch. 1 | Dan Klein slides II |
2/1/24 | Behavior Cloning, Feedback and Covariate Shift [slides, notes] | MACRL Ch. 6, Pg. 53-57 | Three regimes of covariate shift, Watch Imitation Learning Lecture! |
2/6/24 | DAgger: Interactive Experts and No-Regret Learning [slides, notes] | MACRL Ch. 6, Pg. 53-60 | DAGGER |
2/8/24 | DAgger (pt 2): Interactive Experts and No-Regret Learning [slides, notes] | MACRL Ch. 6, Pg. 53-60 | DAGGER |
2/13/23 | DAgger (pt 3): Interactive Experts and No-Regret Learning [notes] | DAGGER | |
2/15/23 | Continuous MDPs: Linear Quadratic Regulator [slides, notes] | MACRL (Ch 2, 23-27) | |
2/20/23 | Conquering Motion Planning via Sampling and Search [slides] | LazySP | |
2/22/23 | Model Predictive Control and the Unreasonable Effectiveness of Replanning [slides, notes] | DeepMind MPC overview | |
2/29/23 | Approximate Value and Policy Iteration [slides] | MACRL (Ch 8), Pg 79-89 , PDL | |
3/05/24 | Policy Gradient [slides, notes] | MACRL (Ch 11), Pg 129-133 | |
3/07/24 | Policy Gradient II [slides, notes] | MACRL (Ch 11), Pg 133-end | |
3/12/24 | Actor Critic [slides] | MACRL (Ch 11), Pg 133-end | |
3/14/24 | Review [slides] | ||
3/19/24 | POMDP [slides] | ||
3/21/24 | In-class Prelim | ||
3/26/24 | Model-based RL [slides] | ||
3/28/24 | Generative World Models [slides] | ||
4/9/24 | Open Vocabulary Object Detection [slides] | ||
4/11/24 | 3D Perception: PointNet, Neural Radiance Fields [slides] | ||
4/16/24 | No Class | ||
4/18/24 | Reinforcement Learning from Human Feedback [slides, notes] | ||
4/23/24 | Predicting humans around robots [slides] | ||
4/25/24 | Offline RL [slides] | ||
4/30/24 | No Class | ||
5/02/24 | Sim-to-real for Reals [slides] | ||
5/07/24 | Controlling Robots via Large Language Models (LLMs) [slides] |
There will be a total of 6 assignments. The first one checks prerequisites. Three of the assignments will be written. Three of the assignments will involve programming. As the course progresses, we will release each assignment in the links below.
Every assignment has an extra credit question which is compulsory for CS 5756 and extra credit for undergraduate version CS 4756.
We will have an in-class prelim on March 21st. The prelims will test concepts taught in the lectures and written assignments.
There will also be a final project. This is your chance to get creative and apply what you have learned! For the project, you may work in groups of up to 3 people. There will be three deliverables - a project proposal, a final report and a final presentation. For more details, please refer to doc.
The course is extensively based off of the following book:
Other helpful books and notes:
Sanjiban ChoudhuryInstructorsanjibanc@cornell.eduOffice Hours: Tuesday 11:30 - 1:00 pm, Gates 413 B |
Gonzalo GonzalezTeaching Assistantgg387@cornell.eduOffice Hours: Sunday 12:00 - 2:00 pm, Rhodes 404 |
|||
Lisa AsrievTeaching Assistantlaa97@cornell.eduOffice Hours: Friday 11:00 am - 1:00 pm, Rhodes 404 |
Adam CahallTeaching Assistantabc256@cornell.eduOffice Hours: Thursday 1:15 - 3:15 pm, Rhodes 404 |
|||
Angela ChaoTeaching Assistantac2323@cornell.eduOffice Hours: Wednesday 11:00 am - 1:00 pm, Rhodes 404 |
Prithwish DanTeaching Assistantpd337@cornell.eduOffice Hours: Tuesday 2:00 - 4:00 pm, Rhodes 404 |
|||
Vaishnavi GuptaTeaching Assistantvg222@cornell.eduOffice Hours: Monday 9:00 - 11:00 am, Rhodes 404 |
Bryant HarTeaching Assistantbjh254@cornell.eduOffice Hours: Friday 9:00 - 11:00 am, Zoom |
|||
Juntao RenTeaching Assistantjlr429@cornell.eduOffice Hours: Saturday 9:00 - 11:00 am, Rhodes 404 |
Assignments, lectures, and ideas on this syllabus are partially adapted from Drew Bagnell course at Carnegie Mellon University. We thank Drew for insightful discussions and suggestions for how to structure the course.
MATH 1920/ MATH 2220, MATH 2940, CS 1110, CS 4780 or permission of instructor. This course is targeted towards senior-level undergraduate students and junior graduate students. Graduate students should enroll in the graduate version of the course. Students should have a solid background in linear algebra and probability. This course involves implementing state of the art algorithms on real world datasets and simulators. Hence, strong familiarity with Python and neural network libraries (Pytorch / TensorFlow/ JAX) is required.
Here’s a breakdown of grades:
Component | Details | %Grade |
---|---|---|
Assignments | 6 assignments (5% + 5*9%) | 50% |
In-class Prelim | 20% | |
Final Project | Final Report: 10%, Final Presentation Video: 10% |
20% |
Participation | In-class participation | 10% |
Total | 100% |
Assignments must be done individually. Each assignment will require students to turn in a writeup and code in Gradescope. It is acceptable for students to discuss problems with each other; it is not acceptable for students to share answers or code. Please indicate on each homework with whom you collaborated with and what online resources you used.
Every assignment has an extra credit question which is compulsory for CS 5756 and extra credit for undergraduate version CS 4756. Extra credit points from assignments will be added up and can contribute upto a max of 5% for your final grade.
The final project can be done in groups of up to 3. There are three deliverables - an extended abstract, a final report, and a final presentation video. We will share a template for abstract and final report in due time. The final presentation will be in form of a 5 minute video that will be reviewed by your peers!
Research has demonstrated that the best learning occurs when the learner is actively involved. We will have frequent opportunities for students to work together during lectures. We expect you to come to class prepared to focus, interact with classmates, and participate in the activities. We also expect you to participate in discussions on Ed and create an engaging environment.
Assignments must be submitted by the posted due date. You are allowed up to 3 TOTAL LATE DAYs throughout the entire semester. If you exceed the late days, assignments will incur a reduction in score by 33% for each extra day. Regrade requests, if the case is strong and a significant number of points are at stake, should be submitted via Gradescope one week of when a deliverable is returned to the student. You must provide a justification for the regrade request.
In case of a legitimate situation or medical emergency that arises during the semester that is going to hinder your ability to complete the work on time, contact Prof. Choudhury as soon as possible. Extensions (beyond the already assigned slip days) will be granted only in exceptional circumstances, such as documented illness, not for situations such as job interviews or large workloads in other courses.
This course adheres to all aspects of Cornell's Code of Academic Integrity. Any work presented as your own must be your own, with no exceptions tolerated. Submitting work created by ChatGPT, or copied from a bot or a website, as your own work violates Academic Integrity. All violations of this policy will result in a penalty depending on the severity. The penalty may be a failing grade on the relevant assignment or exam, or a failing grade in the class. The code can be found at: http://cuinfo.cornell.edu/aic.cfm
The work you do consists of writing code and natural language descriptions. To some extent, the new crop of “generative AI” (GAI) tools can do both of these things for you. However, we require that the vast majority of the intellectual work must be originated by you, not by GAI. You may use GAI to look up helper functions, or to proofread your text, but clearly document how you used it.
In this class, for every assignment and final project, you can choose between two options:
Option 1: Avoid all GAI tools. Disable GitHub Copilot in your editor, do not ask chatbots any questions related to the assignment, etc. If you choose this option, you have nothing more to do.
Option 2: Use GAI tools with caution and include a one-paragraph description of everything you used them for along with your writeup. This paragraph must:
Below we provide some guidelines for what is / is not ok when using GAI for this class:
Students in this course come from a variety of backgrounds, abilities, and identities. In order to ensure an environment conducive to learning, all members of the course must treat one another and the course staff with respect. If you feel your needs are not being adequately accommodated by the other students or instruction staff, please contact Prof. Choudhury.
Zoom recordings of lecturers are not available for absences, including absences due to illness.
If you are a close contact with someone who is diagnosed with COVID19, even if you do not experience symptoms, you should test yourself [and you may want to request masking for five days after exposure regardless of the outcome].
If you have symptoms of COVID19 and have not been tested:Your access in this course is important to me. Please request your accommodation letter early in the semester, or as soon as you become registered with Student Disability Services (SDS), so that we have adequate time to arrange your approved academic accommodations. Once SDS approves your accommodation letter, it will be emailed to both you and me. Please follow up with Prof. Choudhury to discuss the necessary logistics of your accommodations. If you are approved for exam accommodations, please consult with Prof. Choudhury at least two weeks before the scheduled exam date to confirm the testing arrangements. If you experience any access barriers in this course, such as with printed content, graphics, online materials, or any communication barriers, reach out to me or SDS right away. If you need immediate accommodation, please speak with me after class or send an email message to me and SDS at sds_cu@cornell.edu.
Your health and wellbeing are important to me. There are services and resources at Cornell designed specifically to bolster undergraduate, graduate, and professional student mental health and well-being. Remember, your mental health and emotional well-being are just as important as your physical health. If you or a friend are struggling emotionally or feeling stressed, fatigued, or burned out, there is a continuum of campus resources available to you: https://mentalhealth.cornell.edu/get-support/support-students . Help is also available any time day or night through Cornell’s 24/7 phone consultation (607-255-5155). You can also reach out to me, your college student services office, your resident advisor, or Cornel Health for support.