Instructor: Anil Damle
Contact: damle@cornell.edu
Office hours: Monday and Wednesday (starting January 29) from 10:00 AM - 11:00 AM in 423 Gates Hall, or by appointment.
TA: David Eriksson
Contact: dme65@cornell.edu
Office hours: Tuesday from 10:00 AM - 11:00 AM and Thursday from 1:00 PM - 2:00 PM in 657 Frank H.T. Rhodes Hall, Room 2. (See David's website, linked to above, for directions.)
Lectures: Monday, Wednesday, and Friday from 2:30 pm till 3:20 pm in Olin Hall 165
Course overview: Introduction to the fundamentals of numerical linear algebra: direct and iterative methods for linear systems, eigenvalue problems, singular value decomposition. In the second half of the course, the above are used to build iterative methods for nonlinear systems and for multivariate optimization. Strong emphasis is placed on understanding the advantages, disadvantages, and limits of applicability for all the covered techniques. Computer programming is required to test the theoretical concepts throughout the course.
Course websites: Homework, projects, and exams will be turned in using the course management system (CMS). The course also has a discussion website using Piazza.
Your grade in this course is comprised of three components: homework, projects, and exams. Please also read through the given references in concert with the lectures.
There will be five homework assignments throughout the course, typically made available roughly two weeks before the due date. They will include a mix of mathematical questions and implementation of algorithms. Implementation may be done in MATLAB or Julia. Homeworks should be typed and submitted via the CMS.
There will be two open ended assignments, which we will refer to as projects. These assignments will present problems (with some guidance towards a solution) that you should be able to complete using the tools and techniques covered in the course. Implementation may be done in MATLAB or Julia. Projects (both code and a typed writeup) should be submitted via the CMS.
There will be two take-home exams in this course, a midterm and a final. These exams are open book and note. However, you may not discuss the exam with anyone (besides asking myself or the TA clarifying questions), and are expected to complete them on your own. Implementation may be done in MATLAB or Julia. Your exam solutions and code should be typed and submitted via the CMS
Your final grade in the course will be computed based on the homework, projects, exams, and course participation
You are encouraged to actively participate in class. This can take the form of asking questions in class, responding to questions to the class, and actively asking/answering questions on the online discussion board. We will also be soliciting feedback mid-semester to hopefully improve the course.
You may discuss the homework and projects freely with other students, but please refrain from looking at code or writeups by others. You must ultimately implement your own code and write up your own solution. In contrast, the take home exams are to be completed yourself, and should not be discussed with anyone (besides asking myself or the TA clarifying questions).
Except for the final exam, all work is due at 11:59 pm on the due date. Homework and projects should be submitted via the CMS. For each assignment you are allowed up to two "slip days". However, over the course of the semester you may only use a total of six slip days. You may not use slip days for the take-home exams.
Grades will be posted to the CMS, and regrade requests must be submitted within one week.
Linear algebra at the level of MATH 2210 or 2940 or equivalent and a CS 1 course in any language. We will assume you remember your calculus and can pick up MATLAB or Julia. Recommended but not essential: one additional mathematics course numbered 3000 or above. This course can be taken before or after CS 4210/MATH 4250.
The Cornell Code of Academic Integrity applies to this course.
In compliance with the Cornell University policy and equal access laws, I am available to discuss appropriate academic accommodations that may be required for student with disabilities. Requests for academic accommodations are to be made during the first three weeks of the semester, except for unusual circumstances, so arrangements can be made. Students are encouraged to register with Student Disability Services to verify their eligibility for appropriate accommodations.
The text by Ascher and Greif listed below serves as the primary reference text for the material covered in the class. Importantly, it is available for free online through Cornell's SIAM subscription. I have also included some additional references that may help prove to be useful resources.
A tentative schedule follows, and includes the topics we will be covering, relevant reference material, and assignment information. It is quite possible the specific topics covered on a given day will change slightly. This is particularly true for the lectures in the latter part of the course, and this schedule will be updated as necessary.
| Date | Topic | References | Notes/assignments |
|---|---|---|---|
| 1/24 | Introduction | ||
| 1/26 | Numerical algorithms and error | AG: 1 (see 4 for the relevant linear algebra background) | |
| 1/29 | Sensitivity and conditioning | AG: 1,2 | |
| 1/31 | Floating point | AG: 2 | |
| 2/2 | Linear algebra, background | AG: 4 | |
| 2/5 | Linear Algebra, background | AG: 4 | |
| 2/7 | Gaussian elimination and Cholesky | AG: 5.1 | Homework 1 due |
| 2/9 | LU and pivoting | AG: 5.2 and 5.3 | |
| 2/12 | LU and pivoting | AG: 5.2 and 5.3 | |
| 2/14 | Error analysis of direct methods | AG: 5.4 and 5.8 | |
| 2/16 | Cholesky and Sparse matrices | AG: 5.5, 5.6, and 5.7 | |
| 2/21 | Sparse matrices and intro to least squares problems | AG: 5.6, 5.7, and 6.1 | |
| 2/23 | Least squares and QR | AG: 6.2 and 6.3 | Homework 2 due |
| 2/26 | QR factorizations | AG: 6.2 and 6.3 | |
| 2/28 | QR factorizations via Householder | AG: 6.2 and 6.3 | |
| 3/2 | Cornell closed, no class | ||
| 3/5 | Intro to eigenvalue problems | AG: 8.1 | |
| 3/7 | Power and subspace iteration | AG: 8.1 | Project 1 due |
| 3/9 | QR iteration | AG: 8.3 | Midterm posted |
| 3/12 | Upper Hessenberg and tridiagonal forms | AG: 8.3 | |
| 3/14 | Upper Hessenberg and tridiagonal forms | ||
| 3/16 | Optimization in 1D | Midterm 1 due | |
| 3/19 | Stationary iterative methods | AG: 7.1, 7.2, and 7.3 | |
| 3/21 | Krylov subspaces and CG | AG: 7.4 and 7.5 | |
| 3/23 | Preconditioning | AG: 7.4 and 7.5 | |
| 3/26 | Nonlinear equations in 1D | AG: 3.1, 3.2, and 3.4 | |
| 3/28 | Optimization in 1D | ||
| 3/30 | Nonlinear equations and optimization | AG: 9.1 and 9.2 | Homework 3 due |
| 4/9 | Newtons method | AG: 9.1 and 9.2 | |
| 4/11 | Newton iterations | AG: 9.1 and 9.2 | |
| 4/13 | Quasi-Newton and other iterations | AG: 9.1 and 9.2 | |
| 4/16 | Gradient methods | AG: 9.1 and 9.2 | |
| 4/18 | Line search | AG: 9.1 and 9.2 | Homework 4 due |
| 4/20 | BFGS | AG: 9.1 and 9.2 | |
| 4/23 | BFGS and practical tips | AG: 9.1 and 9.2 | |
| 4/25 | Modified Newton and trust region | AG: 9.1 and 9.2 | |
| 4/27 | Constraints | AG: 9.3 | Project 2 due |
| 4/30 | Constraints | AG: 9.3 | |
| 5/2 | Special topics | ||
| 5/4 | Review | Homework 5 due | |
| 5/7 | Review | Final exam posted | |
| 5/9 | Special topics | ||
| 5/14 | Final exam due at 4:30 PM | Final exam due at 4:30 PM |