CS 513 System Security -- Security in Java

Security in Java

Lecturer: Úlfar Erlingsson

Lecture notes by Lynette I. Millett

Today we begin a discussion of language-based security. Essentially, this technique is based on making sure that "bad things cannot be said" in the language.

For example, in C, bad (invalid) pointers can cause programs (and, in some cases, even the operating system) to crash. One approach to solving this problem is to disallow pointer arithmetic. However, to completely solve this problem, we must also disallow 'free' as well, as otherwise deallocated objects might be access through pointers to them. If neither pointer arithmetic nor 'free' statements are allowed in a language, then it is not possible to access an invalid pointer. (The disadvantage is that without 'free', it is now necessary to do garbage collection.)

We see, therefore, that restricting a language can be useful for security purposes. (In fact, a language like "skip" is very secure.) Java, however, is much more expressive than "skip", so we examine it instead. Some general facts about Java:

It is object oriented.
Classes implement types (they can be thought of as templates to implement objects.)
There is single inheritance and an inheritance tree with java.lang.Object at the root.
Objects are instances of classes.
Methods are operations relevant for a particular object.
Java has threads.

Consider the following concrete example:

class Queue {
     private int[] els;
     private int pos;
     public int getFromFront() { . . . }
     public void addToBack(int i) {
          pos := pos + 1;
	  els[pos] = i; 
     }
     public boolean empty() { . . . } 
}

If we would like to run this program, the Java program is compiled into a simple assembly-like language that runs on the Java Virtual Machine (JVM).

The JVM is a stack machine that also has registers. As an example of what this language looks like, consider the expression pos + 1 in the implementation of addToBack above. This instruction translates to the following JVM code

                       Top of Stack
push this              Queue
get int Queue.pos      int
push 1                 int, int
iadd                   int

where this refers to the instance of the Queue object that is running. Consider the top of the stack after each of these statements. After the first statement, the Queue object is on top of the stack. After the second, an integer is on top. Executing push 1 means that now two integers are on top of the second. Finally, iadd pops the two integers off the top and pushes the result.

Note that there are other languages that compile to VMs as we have described. They are not as successful, primarily because Java runs more efficiently and with more safety. How is this achieved? The difference is at what stages Java does certain things. At runtime, the JVM does some checks for safety (such as divide by 0 and array bounds checking.) Many other similar languages also do type checking here (e.g., checking to be sure that an integer add operation was really operating on two integers.) Java does not do this at runtime, instead, a verifier runs at load time to ensure that, for example, iadd is called only when two integers are on top of that stack.

What does this verifier have to do? For every JVM instruction, if the instruction uses a global reference (such as Queue.pos), then it should make sure that it is correct. (For instance, only Queue can reference Queue.pos, since pos is private.) Global references are checked by name. The local state is also checked. That is, the stack and registers are checked to be sure that the types are correct. This is what we are interested in. If we can do this sort of type checking, then we have made progress towards verifying safety properties of the language. To enforce type-correctness, the verifier looks through the code, method-by-method, and makes sure that the register and stack are in proper states.

The most important property verified is the Gosling property (after one of Java's creators.) This property says that the stack and registers must always look the same whenever a JVM instruction is executed. In other words, the stack must be the same size, registers must be in the same defined/undefined state, and types must be the same. The Gosling property should hold whatever the control flow. Consider the following code segment.

                           Top of Stack
    push this              Queue              
    get int Queue.pos      int                
L1: push 1                 int, int           
    iadd                   int                
    .
    .
    .
    goto L1

At goto L1, there should be an integer on the stack. Suppose that the statement push this was just before the goto statement. Then, the verifier should reject this program. Otherwise, the JVM would add an object to an integer. The following code segment is acceptable, however.

                           Top of Stack
    push this              Queue              
    get int Queue.pos      int                
L1: push 1                 int, int           
    iadd                   int                
    goto L1

In this case, there is an integer on the stack before the goto executes, which ensures that there are two ints on the stack when we do iadd.

The Gosling property is easy to reason about and enforce, however, it is quite restrictive. There is safe code that violates the Gosling property. Consider the following pseudo-code:

 
if input == 0
then reg1 = int 7; 
else reg1 = Newspaper n;
skip                       <--- unsafe state
if input == 0
then return reg1
else return reg1.NumPages

Here, before the second if, reg1 is either an integer or a Newspaper object. The Gosling property says that the stack and registers should always look the same, regardless of control flow. In this example, they don't, and even though this code fragment makes sense and is safe, the verifier would reject it. In fact, the Gosling property is even too simple for the Java language. Consider the construct

try {p₁; p₂; p₃} finally { X }

whose semantics is: always do X, even if one of the p_i fails. X may execute after any one of the p_i, but there's nothing that says the registers and stack are in the same state after each p_i. One way to solve this is to have the verifier make sure that X does not use whatever is varying from one p_i to the next. This requires a more complicated analysis, making the verifier much more complicated than it would be if this construct were not in the language. This is rather unfortunate, as this construct is not one of the most commonly used.

What do we mean by safety? We insist that only legitimate accesses to objects are allowed. We also insist that only meaningful operations be allowed on objects. Succinctly: restrict who can access what, and how they can access it. This should sound familiar, as it is precisely what the access control matrix from models. We can formulate safety using an access control matrix. In fact, the object oriented nature of Java facilitates this. Operations and objects are explicit and we know what operations are meaningful from the class definition. If we include classes in the ACM, then we see some nice properties. For instance, file access can be excluded just be denying access to the FILE class. Further, code might still store/retrieve FILE objects as generic objects (java.lang.Object) even though it couldn't use them as FILEs. Consider the Java ACM for the Queue object discussed previously.

Here, els and pos are private, so Queue is the only object that has access to them. The addToBack and getFromFront methods are public, so every object has access to them. This can be a problem. Not all classes need access to the add and get routines. Allowing them access breaks the principle of least privilege, making this ACM too lax. On the other hand, insisting that only Queue have access to els and pos may be too strict.

We need a more powerful rights annotation system. We say that the Queue object has three types of rights: direct, add and get. In this situation, subjects that access els and pos require the direct access. Calling the method getFromFront requires the get right, calling addToBack requires the add right, and empty doesn't require any rights. In this manner, we can specify explicitly which rights are needed. Then, we could give code exactly the rights they need, and even statically check that they satisfy their rights requirements, as in the following example where one Queue is appended to another.

 
Append(Queue[add] dest, Queue[get] source)
{
   while(!src.empty()) {
       dest.addToBack(src.getFromFront());
   }
}

One problem with this type of scheme is that it can be difficult to handle dynamic rights.

It's important to note that most security issues discussed above have little to do with with the Java language itself, but rather with the verifier that works on JVM byte code. This yield two advantages: any language that can be compiled to Java byte code can be verified and even handwritten byte code (which is likely to be used for attacks) can be checked.

The verifier ensures that the code is type-correct. What exactly does type-correctness mean? In general, this means that the code does not violate interfaces. That is, integer addition should operate on integers, printf should take a string as its first argument, and so on. The original motivation for type-correctness was not security, but rather software engineering: Ensuring type-correctness reduces the chance of programmer error and allows implementations to change without changing interfaces.

Aside from the fact that reducing programmer error can help with respect to security issues, how is type-correctness relevant to security? It turns out that the properties enforced by type-correctness overlap with the properties we want from a secure program.

First, we can build upon the guarantee of interface integrity. For example, if the language is type-safe, then private members in objects are not accessible to other objects, and therefore we know the data is secure. Pointers are unforgeable, as discussed previously. This means that if a single process partitions its memory as follows: then there is no way to get a pointer from somewhere in p1 to somewhere in p2 unless the creator allows it, even though p1 and p2 are in the same process.
Second, type-correctness also allows simpler, more efficient security mechanisms. This efficiency allows for more fine-grained security. Implementations that might have been too complicated (and therefore of low trustworthiness) become feasible if type-correctness can be assumed. Consider capabilities. Recall that capabilities are associated with the rows of an access control matrix. A capability is a name and a set of rights, and needs to be unforgeable. Earlier, the mechanism we described for creating and giving out capabilities employed cryptographic sealing. In cryptographic systems, there is always a chance (however small) that the key can be guessed. Type-correctness gives us unforgeable pointers. Therefore unforgeable capabilities are easy to get; there is no longer a key to worry about it, nor any cryptography to implement.

The Sandbox: Original Java Security Policy

How does Java itself use type-correctness to achieve better security? Java's original popularity was due to applets (not to do with its security properties.) The security for applets (an example is diagrammed below)

was the sandbox. The policy is that local code (from the hard drive) is allowed to do anything, but applets (code taken from the net) can only access things like the screen, sound, etc. Applets are not allowed use of the filesystem, and can only use the network to communicate back to where they came from.

This policy is enforced by the SecurityManager (SM). The SM is hooked to code/thread at load time. If the code is an applet, then the appletSM is attached; if the code is local, then a nullSM is attached. The SM is queried by a method call when services are used. For instance, if an applet tries to read a file x, then its SM is queried: is this allowed?

Recall the Gold Standard we have been using to evaluate security policies. How does the Java sandbox measure up? Authentication consists of determining the source of the code (1 bit of information): not very expressive. Authorization is handled by the security manager using a fixed set of method calls: not very flexible; adding a new type of service would involve releasing a new version of Java. Finally, there is no audit mechanism at all. In short, the sandbox does not measure up well to the Gold Standard. The problem is that the policy is too simple and too inflexible. The local code is also too powerful, violating the principle of least privilege. This can lead to trouble. Consider the following example, where a bad applet is called by local code. The applet calls some system code that formats the disk. Since the applet had a nullSM attached (due to the initial call, recall that SecurityManagers are linked to threads of operation) this operation will succeed.

Java 1.2

This version of Java (also known as Java 2) improves on the sandbox security mechanism. Authentication uses Domains based on the origin of the code and a signature. For example: code from Cornell signed by Microsoft is domain D. Authorization is in terms of domains and Permissions, which are really just the easy capabilities got by the type-safety of the JVM. For example, domain D might receive FilePermission("/tmp/*"), which implies that domain D has FilePermission("/tmp/somefile"). Permissions are granted to Domains by a user-specified security policy. However, Java 1.2 still has no audit mechanism.

Consider an example where there are three domains: editor, encryption and filesystem. The editor makes use of the encryption domain to load and save encrypted files from the filesystem. Now suppose the editor would like to save a file. It makes a call to encryption. Encryption calls the file system which calls checkPermission(Files) to ensure that the calling domians have the Files permission. This scenario is depicted below:

In Java 1.2 the policy is that checkPermission(Files) does not succeed unless all domains crossed by the calling thread (here the three domains figured) have the Files permission. This is actually implemented in the JVM by tracing back up the call stack and examining the domains crossed.

The mechanism described above automatically attenuates the Permissions of a thread to be the intersection of the Permissions of all crossed domains. But, in addition to attenuation, it is often necessary to amplify rights. E.g., the file system may need to log activity, using the Log permission, no matter who calls it.

Consider the following scenario: A Login domain, having permissions for the screen and keyboard, makes use of a PasswdCheck domain to check the entered passwords. The PasswdCheck domain, having rights to cryptography and the password file, encrypts passwords and checks the result against the password file, using the filesystem domain. The following figure shows this scenario without amplification, and when checkPermission(PasswdFile) is called, not all crossed domains have the PasswdFile permission, and the check therefore fails.

Some kind of rights amplification is clearly needed. Java 1.2 provides two commands, beginPrivileged and endPrivileged, which amplify the Permissions of a thread to include all those in the current domain. This provides a way for a particular domain to insist that it really knows what its doing. The domain programmer needs to be careful when using this construct, e.g., endPrivileged should always be included, ideally in a finally block:

try { 
      beginPrivileged();
      security_critical_code
}
finally {
      endPrivileged();
}

The figure above shows the Login example using beginPrivileged(). In this case checkPermission(PasswdFile) succeeds, as all domains crossed after beginPrivileged() have the PasswdFile permission.

This new Java 1.2 security mechanism is much better than the previous Java Sandbox. For Example, the Permissions mechanism can be used by applications such as databases, not just by system services such as the Filesystem. On the other hand, there is still no audit facility. Moreover, amplification of beginPrivileged amplifies to all permissions for that domain, there is no way to do beginPrivileged(Files). Finally, this mechanism is too static. It is not possible to do things that rely on the history of the application, e.g., this policy does not allow enforcement of "no network send after read."