CS6410 Fall 2017

Tue/Thur 10:10AM-11:25AM

114 Gates Hall

CS6410: Advanced Systems

[MP 0] Getting Started on Fractus

Fractus has graciously offered to provide us with a prepaid account to support the course. This is a single account, to be shared among all the CS6410 students.

Note: Fractus is behind the firewall of the CS Department. You need to use the Cisco VPN if you are outside the Cornell network.

1. Conventions for Sharing a Single Fractus Account

Because we will all be sharing a single Fractus account, it will be possible (with some effort) for students to decrypt and read one another's Fractus machine image files, or to steal one another's static content files. This is not fundamentally different from the bad old days when CS programming projects were done on a batch computing system and printouts were delivered in sorted piles in public terminal rooms. The University Academic Integrity Policy applies to everything we put into Fractus, and you are expected to follow this policy scrupulously.

Also as a consequence of sharing a single account, we will need to modify some of the procedures from the Fractus documentation so different students' projects will not interfere with one another. We have the following issues:

• The course Fractus account is registered to Prof. Weatherspoon (hweather@cs.cornell.edu) and only he knows the password. Some of the account setup operations described in the Fractus Getting Started documentation require this password, but these operations should be done exactly once per account, and Prof. Weatherspoon has already done them.

• Authentication for Fractus operations requires either

  • a KeyID and a Secret Key, or
  • an Account ID together with Private Key and X.509 Certificate files in .pem format.

  This nominally "secret" information must be known by every CS6410 student. We have not included it in this (public) document; instead, it is available for download from CMS as part of MP 0. Effectively, we are relying on CMS authentication to restrict the Fractus account information to registered CS6410 students.

• Some Fractus resources are in either global or Fractus-account-wide name spaces. We have defined naming conventions for these resources. Here are the conventions regarding the Fractus compute service:

  • Virtual Machine Image: A machine image is a static file on disk that contains (at least) a bootable root partition. In this class, we will use the same machine image to create virtual machine instances. The image we will use is emi-F528A6E4. The EMI name is ubuntu-12-04-ebs and the root device is of type EBS.

  • Instance Name Prefixes: These should be chosen to be unique by including your netid and then appending whatever descriptive name you want according to the pattern

    vm-netid-description

    for example

    vm-ks2232-mp1
    vm-ks2232-libasync
    vm-ks2232-fuse

    As discussed below, each virtual machine name prefix occurs as a prefix of each of the (many) instances. Thus, the naming convention will make it possible for students to keep more than one VM.

  • Keypairs (discussed below): Choose keypair names using the pattern

    kp-netid-description

    for example

    kp-ks2232-0
    kp-ks2232-libasync
    kp-ks2232-mp1

    Keypair names are used in the Eucalyptus API and have to be unique across the shared account.

  • Security groups (discussed below): Choose group names using the pattern

    gp-netid-description

    for example

    gp-ks2232-libasync
    gp-ks2232-mp1

    Like keypair names, group names do not correspond directly to virtual machine instance names, but they are used in the Eucalyptus API and have to be unique across the shared account.

  To help things work smoothly with so many persons sharing the same Fractus account, we ask you to adhere scrupulously to the above conventions.

  One important thing you should observe is to mark whether you are using Fractus in a shared document. The link for the document can be found in CMS. Because we have a limited shared budget to use Fractus, we don't want idle instances to be running and using the shared budget. It is your responsibility to stop the instance after use. However, as a backup plan we ask you to mark the document, so that the TA can regularly check whether instances not being used are running. Such instances running without the usage indication in the shared document can be stopped without prior notice.

2. Downloading and Installing Eucalyptus Account Information

To get the shared Eucalyptus account information, log on to CMS go to mp0 and download the file account-info.txt. Open the file in a text editor and follow the instructions you see there. When you are done, you will have accomplished the following. You will have created a directory

xxxxxxxx-pk.pem
xxxxxxxx-cert.pem

You will also have modified your environment to contain

EC2_ACCOUNT_NUMBER=xxxxxxxx
EC2_USER_ID=xxxxxxxx
EC2_SECRET_KEY=xxxxxxxx
EC2_PRIVATE_KEY=~/.euca/fractus/xxxxxxxx-pk.pem
EC2_CERT=~/.euca/fractus/xxxxxxxx-cert.pem

source ~/.euca/fractus/eucarc

All done! In following sections we will discuss getting started with Eucalyptus in some detail.

3. General Instructions

Eucalyptus's top-level documentation page (here) has links to "Getting Started" guides for each of the many services included in Eucalyptus. The Eucalyptus guides are simple and informative, but they are not designed for a shared Eucalyptus account. As you work through them, you will sometimes be instructed to use the Eucalyptus web site to sign up for a service or to create a new X.509 certificate. In other places you will be instructed to create a new keypair, to modify the rules of the default network security group, or to do some other thing that affects the global state of the Eucalyptus account. Clearly, if several students were to try this concurrently it would be a Bad Thing. So,

You should ignore such instructions!

4. Getting Started with Eucalyptus

Follow the instructions in the Euca2ools Reference Guide to install Euca2ools. If you are installing from the source code, download a newer version from here. If you are using Ubuntu, follow the instructions for Mac OS X. If the installation complains about unmatched dependencies on some package, typically it could be solved by running:

sudo pip install <package_name> --upgrade

It is a Good Idea to read the Reference Guide for each ami/api command as you are about to use it, to make sure you understand what it is about to do.

4.1. Running an Instance

This section includes a step for "Generating a Keypair", which must be changed to conform to our shared account naming conventions. The Eucalyptus instructions tell you to name your keypair "gsg-keypair" (The "gsg" part presumably stands for "Getting Started Guide.") Instead, you should use a netid-specific name following our naming conventions; for example,

kp-netid-xxx

The Network Security Group is another important issue that is not well covered in the Eucalyptus document. Every Eucalyptus instance runs in a named security group that you specify when you start the instance. The security group has a set of firewall rules that control network connectivity between instances in the group and instances outside it.

If you start an instance without explicitly specifying a security group, the instance runs in a predefined group named "default". Clearly, it would be a Bad Idea for concurrent users of a shared account to have instances running in the (same) default security group; so we use the naming convention described above for security groups. You should create a new security group for the remainder of this exercise using a command like

euca-create-group   gp-netid-xxx   -d   "yyyyyyyy"

euca-create-group   gp-ks2232-test   -d   "test group for getting started"

You can check to make sure this worked by typing

euca-describe-groups   gp-netid-xxx

euca-describe-groups

For some of the later steps in this exercise you will need to specify the group name explicitly rather than allowing it to default to "default". The command that actually starts your instance is the first of these. For example

euca-run-instances   emi-F528A6E4   -g   gp-ks2232-test   -k   kp-ks2232-test -z fractus

In the next step, "Authorizing Network Access," you need to specify your group name in place of "default". For example,

euca-authorize   gp-ks2232-test   -P   tcp   -p   22   -s   0.0.0.0/0
euca-authorize   gp-ks2232-test   -P   tcp   -p   80   -s   0.0.0.0/0

At this point, you should be able to connect to your instance with the ssh command from the Eucalyptus documentation, using the name of the RSA private key file you saved when you created your Eucalyptus keypair, and the external network address assigned to your running instance (to find it out use euca describe-instances instance_id where instance_id is returned by the euca run-instances command), for example

ssh   -i   ~/.euca/fractus/id-rsa-kp-ks2232-test   root@128.84.xx.xx

DO NOT go away without remembering to shut down your instance -- the CS6410 course account is charged for the instance for as long as it continues to run.

4.2. Cleaning Up

Don't leave any Eucalyptus instances running!

prompt>   euca-describe-instances --filter key-name=kp-netid-xxx

prompt>   euca-stop-instances   i-nnnnn ...

5. What to Submit?

You don't have to submit anything. But after Friday August 26 2016, we will check if you have created the keypair, security group, and VM instance successfully. So make sure to leave your work correctly named with your NetID.

6. End of MP 0

At this point you are done with MP 0!