Copying Garbage Collection
The goal of a garbage collector is to automatically discover and reclaim fragments of memory that will no longer be used by the computation. It turns out that during evaluation of a high-level program, we allocate lots of little objects that are only in use for short periods of time and can be effectively recycled.
Most garbage collectors are based on the idea of reclaiming whole objects that are no longer reachable from a root set. In the case of our interpreter, we only access memory objects through the stack and through pointers. So any object that isn't reachable from the stack, following the pointers contained within other objects, can be safely reclaimed by a garbage collector.
At an abstract level, all a copying collector does is start from a set of roots (in our case, the operand stack), and traverse all of the reachable memory-allocated objects, copying them from one half of memory into the other half. The area of memory that we copy from is called old space and the area of memory that we copy to is called new space. When we copy the reachable data, we compact it so that it is in a contiguous chunk. So, in effect, we squeeze out the holes in memory that the garbage data occupied. After the copy and compaction, we end up with a compacted copy of the data in new space data and a (hopefully) large, contiguous area of memory in new space in which we can quickly and easily allocate new objects. The next time we do garbage collection, the roles of old space and new space will be reversed.
For example, suppose memory looks something like this, where the colored boxes represent different objects, and the thin black box in the middle represents the half-way point in memory.
|
Obj 1 |
Obj 2 |
Obj 3 |
Obj 4 |
Obj 5 |
|
|
|
|
|
|
At this point, we've filled up half of memory and so we initiate a collection. Old space is on the left and new space on the right. Suppose further that only the red and light-blue boxes (objects 2 and 4) are reachable from the stack. After copying and compacting, we would have a picture like this:
|
Obj 1 |
Obj 2 |
Obj 3 |
Obj 4 |
Obj 5 |
|
Obj 2' |
Obj 4' |
|
|
|
Notice that we copied the live data (the red and light-blue objects) into new space, but left the unreachable data in the first half. Now we can "throw away" the first half of memory (this doesn't really require any work):
|
|
|
|
|
|
|
Obj 2 |
Obj 4 |
|
|
|
After copying the data into new space, we restart the computation where it left off. The computation continues allocating objects, but this time allocates them in the other half of memory (i.e., new space). The fact that we compacted the data makes it easy for the interpreter to allocate objects, because it has a large, contiguous hunk of free memory. So, for instance, we might allocate a few more objects:
|
|
|
|
|
|
|
Obj 2 |
Obj 4 |
Obj 6 |
Obj 7 |
Obj 8 |
When the new space fills up and we are ready to do another collection, we flip our notions of new and old. Now old space is on the right and new space on the left. Suppose now that the light-blue (Obj 4), yellow (Obj 6), and grey (Obj 8) boxes are the reachable live objects. We copy them into the other half of memory and compact them, throwing away the old data:
|
Obj 4 |
Obj 6 |
Obj 8 |
|
|
|
|
|
|
|
|
What happens if we do a copy but there's no extra space left over? Typically, the garbage collector will ask the operating system for more memory. If the OS says that there's no more available (virtual) memory, then the collector throws up its hands and terminates the whole program.
Implementation Details
It is surprisingly simple to build a copying garbage collector.
First, we need one more register scanptr, which will be used as an index into memory. The scanptr initially contains the base address of the new space (i.e., the address of the first word where we will copy objects.) We also set the allocptr to the base address of the new space. We will use the allocptr to remember where to allocate objects as we copy them from old space to new space. The purpose of the scanptr will be made clear below.
Second, starting from an array of roots (in our case, the values pushed on the operand stack), we examine each root to see whether or not it's a pointer. If so, we copy the object that the pointer references from old space to new space. As we copy the object, we place it where the allocptr currently is, and then increment the allocptr by the appropriate amount so that it points to the next available spot in new space.
After we copy the object, we need to update the array of roots so that it points to the new copy of the object. We also need to leave a forwarding pointer in the object's previous location in old space indicating that the object has already been moved and where it was moved to, in case we run into this particular object again while traversing the graph of reachable objects in old space. (This prevents us from going into an infinite loop if there are cycles in the graph, and there typically will be cycles if we use recursive functions.)
Before trying to copy any object, we should check first to see if the object has already been forwarded. If so, then we should update the root with the address of the copied object that we find in the forwarding pointer.
After processing the roots, you've managed to copy all of the data that are immediately reachable from the roots. However, we must also process all of the data that are reachable from these objects (and then all the data reachable from those objects, and so on.) This is the purpose of the scanptr register.
After forwarding the root objects, we must then examine each of their components to see if there are any pointers. Any pointers in those objects must also be forwarded from old space to new space. The process of examining a copied object for pointers and forwarding those objects is called scanning the object.
The scanptr register is used to keep track of which objects have been forwarded but not yet scanned. In particular, the scanptr starts off as the same address as the allocptr. After forwarding all of the roots, we start scanning the object pointed to by the scanptr. This will be the first object that we copied during the root processing. After scanning this object, we increment the scan pointer so that it points to the next object to be scanned. The scan pointer thus moves only from left to right.
During scanning, we need to look at each value in the object. If the value is a pointer, it should be forwarded. This may cause the allocptr to move. But because the newly copied object comes after the object being scanned, we will eventually scan it and any objects that it references will also be copied into new space.
The whole process stops when the scanptr catches up to the allocptr, for then all reachable objects have been successfully forwarded from old space to new space. At this point, we need to reset the limitptr and the startptr appropriately. We also need to check that there is enough free space for the computation to make progress. If not, then an OutOfMemory exception should be raised. (Otherwise, the interpreter will go into an infinite loop trying to garbage collect forever.)
Note that we are effectively using a queue to keep track of those objects in the graph that have been forwarded but not yet scanned. We insert items at the end of the queue (where allocptr points) and remove objects to be scanned from the front of the queue (where scanptr points). It's possible to use a different data structure such as a stack to keep track of those objects that have been copied but not yet scanned, but doing so usually requires an extra hunk of memory.
Note that by using a queue in the graph traversal, the copying collection effectively does a breadth-first traversal of the data. If we used a stack instead of a queue, the traversal would be depth-first.
You should note that although this is a fairly simple implementation of garbage collection, it uses many of the same concepts used in more complicated methods.
Generational Garbage Collection
One major problem with a copying garbage collector is that it has to look at the entire memory every time a garbage collection must happen. Objects in memory have an important property of temporal persistance. Objects that have been in memory for a long time will likely continue to stay in memory, whereas objects that have recently entered memory will likely be discarded very soon.
To exploit this principle, we can build what is known as a generational garbage collector. Objects will initially be allocated to a chunk of memory called the first generation, or G1. When G1 becomes full, we copy the live objects into another block of memory called the second generation, or G2, and free up the entire G1.
When G2 becomes full, we continue the process to copy the live objects into G3. This continues for some number of generations n. At this nth generation, Gn, we use some other method such as copying garbage collection to reclaim memory space.
NOTE: There is one problem with the generational approach. If an object in an older generation points to an object from a newer generation, when we garbage collect the newer generation, we will corrupt the older object. This can be fixed by keeping track of pointers across generations. Fortunately for us, we will restrict our garbage collector to work only for the functional set of MiniML. This means MiniML code with refs in it will not work under this garbage collector. We will leave it as a thought exercise to see why cross generational pointers are not an issue without side effects.
Of course, you can still use refs in your SML code to garbage collect. You are not required to deal with refs in MiniML when constructing the garbage collector, even though refs implemented in the type checker and compiler.
PS: You can find lecture notes about memory management in the following link, which will give you a better idea of what goes into the heap. http://www.cs.cornell.edu/courses/cs312/2003sp/lectures/lec19.pdf