5431: System Security Practicum

Project Description

CS 5431 students are expected to participate in a group project to build a software system that has non-trivial security functionality. A high-level introduction to the project is given in the project overview. More details will be supplied throughout the semester as part of each project milestone.

The course project must be programmed in Java. The course may also require the use of additional tools, such as Eclipse, Eclipse plugins, C, assembler, Unix, web servers, and other standard technologies. You either need to be familiar with these technologies or to be committed to investing extra time to learn them as you go. (Part of becoming a professional computer scientist or engineer is learning to adapt quickly to new technologies.)

Schedule

Milestone	Description	Due Date
M0	Charter	Wed February 7 at 11:59pm
M1	Requirements	Wed February 14 at 11:59pm
M2	Prototype	Wed March 7 at 11:59pm
M3	Alpha Release	Wed March 28 at 11:59pm
M4	Beta Release	Wed April 25 at 11:59pm
M5	Final Project Deadline	Wed May 16 at 11:59pm

M0: Charter

Your charter defines the members of your team and proposes the software system you intend to build. Your charter should include the following information:

• The members of your team and their contact information. List the names and NetIDs of your team members.
• A working name for your system. You are not committed to this name; it may evolve throughout the semester.
• A proposal for your system. Summarize the system you intend to build. You should regard this summary as a starting point; you will have the chance to elaborate in future milestones. Here, you should tell us what will be the most important functionality of your system. Your plan for your weekly status meeting. We strongly recommend that, as a team, you agree on a weekly meeting time and place (outside of class) that you can both attend to discuss the status of your project. Benefit from our experience: teams that actually do this seem to enjoy more success.

The proposal section of your charter should be about one page long. In this section:

• Provide a very short statement of the core vision or key idea for your system.
• Provide a short bulleted list of the key features of the system.
• Provide a bulleted list of the essential security elements, and a one sentence description of how your system will incorporate each element. Refer to the project overview to make sure your project includes all the required security elements.
• Provide a narrative description of the system you intend to build. Go into enough detail in the body that, if your proposal were given to another team, and that team were never allowed to talk to you, they could still build more or less the system that you have in mind.

Submission: Form a group on CMS. As the group, submit a PDF to the Milestone 0 assignment.

Due: February 7, 2018 at 11:59pm.

M1: Requirements

Prepare a Requirements Document that contains the six sections detailed below. This will be a living document: you will continue to update it in the remaining milestones of the project. It is also a critical document, because your final system will be evaluated, in part, against your requirements. You should think of the Requirements Document as a contract between your team and the course staff. You are responsible for fulfilling that contract.

1. Personnel. Identify your team members, including names, netids, and emails.

2. System Purpose. In a couple of paragraphs, explain the purpose of your system. The basis for this explanation should be the one-page system summary you wrote for Milestone 0. Update that summary as necessary, based on the functional requirements and security goals you develop in the current milestone, as well as any comments made by the course staff on your Milestone 0 submission.

3. Functional Requirements. Invent a list of functional requirements for the system you are building. For the purposes of this project, a functional requirement comprises the following data:

• User type. The type of user involved in this requirement.
• Assets. The critical information involved in this requirement.
• Importance. The importance of this requirement to your system's purpose. Use the following "MoSCoW" scale to rank importance:
  • M. Must have.
  • S. Should have, if at all possible.
  • C. Could have, but not necessary.
  • W. Won't have now, but Would like to add later.
• A user story. A brief description of a single kind of interaction a user can have with your system. User stories describe what your system will do, but not how. Here is a general template for a user story: "As a user, I can action, so that purpose." (See below for examples.) The purpose clause of a story might be obvious enough to omit that clause. Note: stories shouldn't be about developers or attackers.

You might find it helpful to organize your requirements in a spreadsheet, so that you can sort them in various ways.

Good user stories satisfy the following "INVEST" criteria:

• Independent. Stories shouldn't overlap, and ideally you should be able to implement them in any order—though some orders might make more sense than others.
• Negotiable. Stories might (and probably will) need to change as your project evolves.
• Valuable. Stories add value to your system for users. The general template above ensures that you think in terms of that value.
• Estimable. Stories' importance can be estimated, hence ranked. And stories' implementation time can be estimated. If you can't estimate, you probably don't understand the story. Consider adding more detail or breaking it into smaller stories.
• Small. Stories should be succinct, probably no more than one to four sentences.
• Testable. Stories represent testable goals. If you can't decide whether you've met a goal—that is, if you can't write a test that a skilled outsider could follow to determine whether your system fulfills a story—then your story isn't clear enough.

How many requirements should we invent? It's impossible to give a general answer to this question: teams write at different levels of detail, have differently scoped projects, and build systems with different tradeoffs between feature set size and implementation difficulty.

4. Threat Model. Identify threats of concern to your system. What kinds of attackers will your system defend against? What are their motivations, resources, and capabilities? Don't just list vague, generic threats. Make them specific to your system and its functionality. If there are any non-threats, you should identify them, too. For example, you may wish to assume that some system components execute on hardware that is located in a physically secured machine room reachable only by trustworthy system operators.

5. Security Goals. Identify the assets and stakeholders involved with your system. This step should be easy, because you already identified assets and users for each functional requirement. For each asset, identify what its value is to stakeholders.

Perform a harm analysis on assets. Use the template "performing action on/to/with asset could cause harm." (Although you are encouraged to rewrite statements made with that template into more natural English.) Be as thorough and creative here as possible; this is the step at which you're most likely to overlook something that's important and relevant to security.

Transform the harms you've identified into security goals, using the template "the system shall prevent action on/to/with asset." Label each goal as being exactly one of confidentiality, integrity, or availability. Examine the feasibility of each goal in light of your threat analysis. If necessary, relax goals so that it is feasible to achieve them.

You must document all of these steps, showing us your asset analysis, harm analysis, feasibility analysis, and security goals.

6. Essential Security Elements Document why each of the essential security elements (authentication, authorization, audit, confidentiality, and integrity) is indeed essential to your system. If you can't construct a persuasive argument for an element E, you need to revisit your system purpose and invent new features that will cause E to be essential.

Submission: As a group, submit a single PDF to the Milestone 1 assignment on CMS.

Due: February 14, 2018 at 11:59pm.

Evaluation: We will evaluate your Requirements Document against the following criteria:

• Whether your document is easy to understand, well organized, and clearly written. Correct English style and usage counts here, just as it does in the real world.
• Whether the system purpose gives a clear summary of the system you are building.
• Whether the functional requirements seem sensible given the purpose of the system.
• Whether the functional requirements are clear and seem complete.
• Whether the functional requirements adhere to the INVEST criteria.
• Whether the threats of concern and their characteristics are described clearly.
• Whether you have documented all the steps you went through in crafting your security goals, including your harm and feasibility analyses.
• Whether you have correctly identified each security goal as being related to confidentiality, integrity, or availability.
• Whether the security goals you identify are sensible for the system purpose, planned functionality, and threats of concern.
• Whether your have successfully argued that a majority of security elements are involved in your system.

M2: Prototype

All the remaining milestones for the project involve the same essential work. You will engage in a sprint, which is a fixed length of time during which you build a "shippable" increment of your system. Your sprints will last two weeks. At the end of each sprint, you will ship the latest increment of your system to the course staff and demo it for us. You have four sprint cycles—named prototype, alpha, beta, and final—to implement your project. Read the project sprint overview to find out more.

When to implement what. You are empowered to decide what features you will deliver as part of each sprint. However, your prototype for Milestone 2 should include some fully-functional, core functionality. Your design and implementation should be compatible your threat model and security goals, but complete implementation of "gold standard" security functionality will not be required until future milestones: Alpha (authentication), Beta (authorization), and Final (audit).

Source Control. You are strongly encouraged to keep your project in a source control system. There are many free possibilities [GitHub, Bitbucket, SourceForge, etc.] that provide code hosting, bug tracking, and wikis. Under no circumstances should you resort to emailing your entire source code between team members.

Submission. (Due: March 7, 2018)

1. Submit a PDF containing your revised Requirements Document. Name that file requirements.pdf. The Functional Requirements section of your Requirements Document should now be renamed to the "System Backlog" section. You will continue to update it throughout the project. When you complete items, mark them completed and move them to a "Completed" part of the backlog. This section of your document is especially important, because we will use it to evaluate your progress. The rest of the Requirements Document should contain your personnel, system purpose, threat analysis, security goals, and essential security elements, all of which should continue to be updated throughout the project. You do not need to include the rest of the analysis from your Milestone 1 Requirements Document, in particular the asset, stakeholder, harm, or feasibility analyses—their purpose was to get you to security goals, which you now have.
2. Create a new file, design.pdf, which describes the design of the security functionality of your system. This file should have five sections, one for each Essential Security Element. Fill in this file as appropriate for each milestone and for your system.
3. Create another new file, assurance.pdf, which describes how and what you tested in the source code being submitted. Issues to discuss include how you tested individual methods, classes, and units; how you tested the integration of units, especially those written by different programmers; how you tested the features you are delivering; how you tested to ensure that security requirements are met; whether you did any pair programming or code reviews; how much code coverage your test suite achieves; and how often you run regression tests. If you are using the FindBugs tool, present the results of a FindBugs scan. We would be especially impressed if you have managed to eliminate all bugs deemed Scary and Scariest.
4. Submit a zip file, src.zip, containing the source of your system. (That is, anything needed to build your system from scratch. Do not submit compiled files. If you submission is too large to upload to CMS, chances are you are accidentally including compiled files.) Include a plain text file named "README" in root of your source detailing how to compile and execute your system on our own machines.
5. And submit your sprint report along with your system. Name that file report.pdf.

Demo. You will demo you prototype in class on March 9, 2018. Bring laptops that have your system installed and running. The course staff and the other project team might use your system and report on how well it appears to be working and achieving its security goals. Attendance of all team members is required at your team's demo. Demos typically involve some or all of the following events, though there is necessarily some flexibility involved:

• You demo a few user stories that you've marked as completed. Any bugs seen during that demo would be especially unimpressive. The course staff might try weird inputs, strange user behavior, etc., in an attempt to find such bugs.
• You exhibit source code for features you are delivering. Code quality matters.
• You discuss the design and implementation of any security functionality you are delivering.
• The course staff asks you questions. Questions about the security of your system could be directed to any specific member of the group. All members should be able to explain and defend the security-relevant choices your group made.

Evaluation. Your demo will be evaluated on how well it demonstrates the functionality you are delivering for this milestone (i.e., the system backlog items that you have marked as completed), particularly security functionality. You should demo only completed (that is, working and tested) functionality. We will be especially unimpressed by any bugs we see during your demo, if those bugs relate to functionality that you claim to have completed.

The design and implementation of your system, including your code, will be evaluated on how well it exemplifies the various principles of security. Readability, simplicity, documentation, and testing matter. We'll also, of course, be looking for vulnerabilities.

M3: Alpha Release

Complete another sprint and ship the latest increment of your system. Your alpha release should exhibit new features and new security functionality, including complete authentication features. Remember that you might need to authenticate humans, machines, and/or programs.

Submission. (Due: March 28, 2018)

Submission requirements are essentially the same as for Milestone 2. Submit your revised requirements, design, and assurance documents, along with another sprint report. You will also submit the source code for your alpha release, including a plain text file named "README" in the root of your source detailing specifically how to compile and run your system. In addition, you will submit a FindBugs report for this milestone.

Demo. You will demo your alpha release in class on March 30, 2018.

Evaluation. This will be the same as for Milestone 2.

M4: Beta Release

Complete another sprint and ship the latest increment of your system. Your beta release should exhibit new features and new security functionality, including complete authentication and authorization features.

Submission. (Due: April 25, 2018)

Submission requirements are the same as for Milestone 2.

Demo. You will demo your beta release in class on April 27, 2018.

Evaluation. This will be the same as for Milestone 2.

M5: Final Release

Complete your last sprint and ship the final version of your system. Your final release should include full functionality and security features. This is your opportunity to impress us with the system you built!

Submission. (Due: May 16, 2018)

1. Submit a PDF containing the documentation for your system. This document should describe the functionality and features of your system, the security features of your system, the threat model those features are secured against, and how those security goals are implemented and enforced. Note: you are welcome to copy any/all content from your previous requirements and design documents.
2. Submit an assurance document.
3. Submit a findbugs report.
4. Submit a sprint report.
5. Submit a zip file containing all your source code.

Final Presentation. You will present and demo your final release on May 20, 2018 at 3pm.

Evaluation. The design and implementation of your system, including your code, will be evalutated on how well it exemplifies the principles of security. Readability, simplicity, documentation, and testing matter. We will also, of course, be looking for bugs and vulnerabilties. Extra points will be awarded for systems that successfully implement more ambitious goals.

Your final presentation will be evaluated on how well you explain the purpose, features, security goals, and security design of your project. You will also be evaluated on how well you demonstrate the functionality and security features of your system. When preparing your presentation and demo, you should assume that your audience has no prior knowledge of your system.