Software Testing

Spring 2021

Software testing is widely used for detecting flaws in software. Systematic and organized approaches to testing will be discussed, including test adequacy criteria, manual and automatic generation of test inputs, regression testing, debugging, and at least one dynamic analysis for detecting known classes of errors. Students will learn how to design and automate the execution of high-quality software tests. Students will also learn how to generate test suites that meet coverage and other adequacy criteria.

Prerequisites. Graduate standing (Ph.D, MS, or MEng) in CS, or CS majors who have taken CS 3110 or CS 4120, or permission of instructor required. Experience with Java will be helpful for programming assignments.

This course is in Beta. CS 5154 is a brand new course. Everything might change. Nothing is certain.

 

News  

  • 5/9/2021: Project Phase 2 is assigned.

  • 5/3/2021: Homework 4 is assigned.

  • 4/13/2021: Homework 3 is assigned.

  • 4/6/2021: Project Phase 1 assigned.

  • 3/22/2021: Homework 2 is assigned on CMS.

  • 3/17/2021: Spring 0.2 is assigned on CMS.

  • 3/8/2021: Homework 1 is assigned on CMS.

  • 3/1/2021: Sprint 0.1 is assigned on CMS.

  • 2/8/2021: Homework 0 is assigned on CMS.

  • 2/4/2021: Ed Discussions for the class is set up.

  • 2/4/2021: The website is up!


Lectures:
 Monday/Wednesday
 9:40 - 10:55am
 On Zoom

Instructor:
 Owolabi Legunsen
 legunsen@cornell.edu
 Office Hours:
     After class on Zoom

TA:
  Daniel Martin
  dm839@cornell.edu
  Office Hours: TBD

Assessment  

This course involves occasional homework and a semester-long project. The homework offers a chance for students to reinforce understanding of material from class. Homework may include written components, programming exercises, or assigned readings. The project has small student groups testing a non-trivial open-source software and applying techniques and tools learned in class to that software.

Final course grades will be based on the following.

Activity Grade Details
Homeworks 50%
  • There will be 5 -- 6 homework assignments.
  • Some homework will require students to work in groups.
  • Homework must be submitted by the due date. Lateness will be penalized: each additional day of lateness will decrease the grade for that homework by 30%.
  • The lowest homework grade will be dropped for those students who make a good-faith effort on all homework by submitting something credible and on time.
Course Project 40%
  • Students will work in groups to complete the course project.
  • Students in each group will get the same scores.
  • We will discuss group sizes and possible projects in class early in the semester.
  • Each group will be required to submit a project proposal and at least three progress reports on their course project.
  • Project reports must be submitted by the due date. Lateness will be penalized: each additional day of lateness will decrease the grade for that report by 30%.
  • Project grade will be based on the work done and on the quality of the submitted reports:
    • Project proposal (1 page): 5%
    • At least two reports (up to 2 pages each) on intermediate project phases: 15%
    • Final report (up to 5 pages): 20%
Class participation and other subjective factors 10% Demonstrated engagement with the material is the defining characteristic for this element of your final grade. Lecture attendance offers one obvious opportunity, so attend and participate by asking questions. Extra-credit assignments may be offered from time to time as well.

 

Tentative Schedule  


Date Topic Notes Homework/Reading
2/8
Introduction
  slides Homework 0 is assigned
2/10
Testing Concepts
  slides
2/15
Testing Concepts (contd), Test Automation
  slides-1,  slides-2
2/17
Test Automation (contd)
  slides,  code
2/22
Criteria-Based Test Design
  slides
2/24
Hands-on Demo: Maven
  slides,  code
3/1
Input Space Partitioning
  slides (ISP),  slides (project) Sprint 0.1 is assigned,  Optional reading on ISP
3/3
In-class Exercise on ISP
  slides
3/8
In-class Exercise on ISP (contd)
  slides Homework 1 is assigned
3/15
Graph-based Coverage Criteria
  slides
3/17
Applying Graph-based Coverage Criteria
  slides Sprint 0.2 is assigned
3/22
Applying Graph-based Coverage Criteria (contd)
  slides Homework 2 is assigned
3/24
Logic Coverage
  slides
3/29
Applying Logic Coverage
  slides
3/31
Syntax-based Testing
  slides
4/5
Mutation Testing
  slides
4/7
Mutation Testing (contd)
  slides,  code Project Phase 1 assigned on 4/6
4/12
Project Clinic
4/14
Automatic Test Generation
  slides Homework 3 assigned on 4/13
4/19
Automatic Test Generation (contd)
  slides,  code Optional Reading
4/21
Flaky Tests (Guest Lecture by August Shi)
  slides Optional Reading
4/28
Introduction to Runtime Verification
  slides
5/3
Introduction to Runtime Verification (contd)
  slides
5/5
Regression Test Selection
  slides
5/10
Test Suite Reduction, Test Case Prioritization
  slides
5/12
Bug Adocacy
  slides Optional Reading
 

Course Resources  

Course Administration

  • Announcements will be posted on this web page. Check the news section regularly for updates.
  • We will use Ed Discussions as an online discussion forum.
  • We are using both CMS and Canvas as course management systems.

Textbook

There is no required textbook for this course. However, students who wish to read more about the topics that we discuss in class may consult the textbook, Introduction to Software Testing by Paul Ammann and Jeff Offutt.

Sources

  • There will be periodic homework assignments in Java. Instructions for installing Java and associated software can be found here.
  • If you need to brush up on your Java skills, invest some time studying the following resources:
  • Projects and homework will involve running tests on your code and applying cutting-edge tools on open-source projects. In both cases you will use Maven as a build system. Brief introductions to Maven can be found here and here.
  • Readings may be assigned from free, publicly available articles that will be linked in the schedule section.

Project Reports

This template may be used for preparing the various project reports.
 

General Resources  

Academic Integrity

Students are expected to know and abide by Cornell's policies on academic integrity, including: Academic integrity violations will be prosecuted aggressively. If you are not sure what constitutes an academic integrity violation, please ask.

Special Needs and Wellness

We provide accommodations for disabilities. Students with disabilities can contact Student Disability Services at 607-254-4545 or the instructor for a confidential discussion of their individual needs. If you experience personal or academic stress or need to talk to someone who can help, contact the instructor or: