• The deadline is firm. No excuses. Plan ahead to accommodate your other assignments and Prelim #2.
• You must attend recitation and lecture regularly or you will have no hope of completing this assignment!
• As with other assignments, you may work with one partner. Turn in one copy of your assignment (see below for precise details) with both names and CUIDs.
• Read through the full assignment before you start!
• Have fun with it, eh!

What the system maintainers really need is a high-level view of the classes that can be extracted automatically from the source code. In fact, there already is such a tool that comes with Sun's standard JDK called javadoc. When you pass a Java source file to javadoc, it reads through the code and automatically creates a web page showing the various features of the class, omitting most of the details. The project web page contains a link to such a generated web page.

Your job will be to create a much simpler version of this kind of tool called JavaDUCK (Java DocUmenter of Code, oK?). Your program will read in a Java source class (i.e., a .java file), build up various data structures to represent the class and its subcomponents (i.e., its methods and variables), and then create an HTML file that formats this information in a visually pleasing manner. We will walk you through the major steps of the assignment, and we will also provide some of the parts for you. However, it is vital that you attend recitation or you will be lost.

Let’s go through this declaration step-by-step. What this says is that a file will have some number of import statements, then either zero or one package statements, and then one class definition. Each import statement (if there are any) will consist of the keyword "import" followed by an identifier followed by a semicolon. The package statement (if there is one) will consist of the keyword "package" followed by an identifier followed by a semicolon.

The class declaration part can be described like this. It can have any combination of the keywords "public", "abstract", and "final", but those that are present will be in this order. Then it has the keyword "class". Then it has an identifier (the name of the class). Then it may or may not have the keyword "extends" followed by an identifier. Then it may or may not have the keyword "implements" followed by a comma-separated list of identifiers (names of interfaces). Study the implements line carefully and see how it uses the definitions of [] and * we gave above.

We would be done except that we haven’t yet given the rules for Variable_Declaration and Method_Declaration. So let’s get it over with.

The "|" used above is new. It basically means "one or the other but not both". So a variable declaration starts with "public", "private", "protected" or none of the above (because of the brackets). Remember that if none of them appears, the visibility level is "package". Then it either has static or doesn’t. Then it either has final or doesn’t. Then it has an identifier which is the type, then a comma-separated list of identifiers which are the variables. Finally it has a semicolon. Notice that for simplicity we will not allow initialization of variables when they are declared. So the declarations:

The first step, called lexical analysis or scanning, is the process of breaking the input into tokens. Tokens are sequences of characters in the code that "mean something" in the programming language. For example, " int a,b; " would be broken up into tokens "int", "a", ",", "b", and ";". Each token would carry the underlying text (such as "int" or "a") and a constant that would specify its type (token "int" would have type INT, token "a" would have type IDENTIFIER).

Now for the good news: We have written the lexical analyzer for you! It can be downloaded from the project web page. It contains an example program and a description on how to use it (it’s very simple to use, we promise). We will give you some tips on how to use the lexer in recitation. To understand the rest of this section, you just need to know that you can create an object of type Yylex (call this object lexAnalyzer) and this will have a method yylex() which returns the next token in a file. The types of tokens are defined in TokenType.java, which we will also provide for you.

The second step is called parsing. Basically, you will keep calling lexAnalyzer.yylex() and "do the right thing" with the token you get back. How do you know what the right thing is? Use the grammar! Everywhere you call lexAnalyzer.yylex() you should know what possible things you might get back. All you have to do is figure out what you actually get back, store the relevant information, and figure out what possible things you might get back next, and so on.

Here’s an example. Let’s suppose we’ve just gotten the keyword "class" back. That is, we just got a token whose type was CLASS. Next we know we will get a token of type IDENTIFIER whose text is the name of the class. We’ll have to store that name somewhere because the class name is an important thing for our HTML file. The possibilities for the type of the next token are EXTENDS, IMPLEMENTS, and LCURLY. (Why? Look at the grammar!). We’ll decide what to do based on which of the three it is. If it’s EXTENDS, then the next token we read will be the parent class name, which we’ll need to store somewhere. After that we can expect either IMPLEMENTS or LCURLY. And so on and so forth. As a general word of advice, try to take advantage of the structure of programs as much as possible. That means, for example, while you are processing the variables in a class, you should be inside a loop whose exit condition is that the next token is the special token "//Methods". Your program will have a large set of nested loops and if statements, as well as calls to interesting auxiliary methods. Your TAs will give you some advice on how to achieve this in recitation.

What we suggest is writing storage classes, similar to the Component class you designed in assignment #3. For example, you may want to define a class called ClassInfo, an instance of which will represent one class from a given file. This class would probably have storage for a class name, a superclass name, interfaces, and all variables and methods. How do you store variables and methods? You may want to define classes VariableInfo and MethodInfo, similar to ClassInfo. This way you can organize storage of the information about your variables and methods. Since you are going to store both variables and methods in your class, it may be useful to define an abstract class MemberInfo that would contain things common to variables and methods.

• It must properly identify the class name, the superclass name (if any), the visibility level of the class, whether the class is abstract, and all variables and methods of the class.
• If the class has a superclass called "foo", then the HTML file must provide a hyperlink to a file in the same directory called "foo.html". (Do not worry if the file does not actually exist. Provide the link anyway. The file may exist later. If not, the browser will deal with it.)
• It must list all static variables, then all static methods, then all constructors, then all instance (i.e. non-static) variables, and then all instance methods. Within these five groups, members may be listed in any order. Notice that static and instance variables (and similarly static and instance methods) may be interspersed in the original Java file. Your HTML file will not list the members in the order they appear in the Java file. That’s fine — people reading the documentation don’t care in what order the implementer wrote down the members.
• For each variable, it must identify the variable’s name, type and visibility level.
• For each method, it must identify the method’s name, visibility level, return type, and all of its parameters. For each parameter, it must identify the parameter name and type. If the method is abstract, the file must say so.
• It must be nicely laid out. The creative use of graphics (e.g., pictures of ducks) is up to you.

Basically you are on your own to create the class definitions and data structures you need to implement the project in an appropriate way. However, in order for us to test your program (and for the GUI described below to work), your implementation must have a class called Documenter. This class must have a method with the following name and form:

This file must read in the file classname + ".java" and produce the file classname + ".html" according to the specifications above. Everything else is up to you.

The code we will provide you will be minimal. We will provide the lexer as described above. We will also provide a simple graphical widget for specifying what class to document (a FileDialog like in assignment #1). Basically you type in class name foo and that means Documenter.documentClass("foo") will be executed.

• A printout of your (well-written, well-documented) code in a monospaced font as with previous assignments.
• A printout of the (simplified) Java files on which you tested your program. Note: In case your program has bugs as you write it, do not test it on code files you care about.
• A printout of the resulting HTML source files. (That is, the plain text version where all the HTML tags are visible.)
• A printout from a web browser of your HTML files. (That is, the version that looks like a web page. Use the print button on your web browser.)
• A single diskette clearly labeled with your names and whether it is for a Mac or PC.

• All of the Java files you used to implement the project. Do not put them in a subdirectory. Include the files we provided you.
• Any extraneous files used by your HTML files (e.g., image files). If they don’t fit on a single diskette, then don’t use them.
• Nothing else. Specifically, do not include your test files. Do not include your .class files. Do not include your .mcp file.

We will call your Documenter.documentClass method, so this method must be called this and must do what it is supposed to (see above).

1. Just generate documentation for files that have no variables or methods, no import statements, and no package statement. In terms of our grammar, your file will look like

[public] [abstract] [final] class CLASSNAME

[extends SUPERCLASS]

[implements INTERFACE1 [, INTERFACE2]* ] {}

This stage will probably take you the most time because you will have to figure out how to use the lexer, how to create an HTML file, and how to parse. Do not forget that a class may implement more than one interface.

2. Now let your input files have import statements and package statements. This should be easy after stage one. You just have to ignore tokens until the class declaration starts. Remember that just because you’re allowing import/package statements, it doesn’t mean that input files will always have them.
3. Now add variable declarations. This will require more parsing, but it will be easier than what you did in stage 1. Remember, though, that the output needs to separate static and instance variables. Think about what data structures you want to create to hold the necessary information. Also remember to expect the "//Variables" line in the right place.
4. Now add method declarations. The parsing will probably be more difficult, but by now you should have the hang of things. Separating static methods, constructors, and instance methods shouldn’t be much harder than what you did in stage 3. You need to convey a lot of information about each method, so make sure your HTML file is easy for a human to understand. Also remember where to check for the "//Methods" line which separates variables and methods.
5. If your thoughts haven’t yet turned to barbecues, swimming holes, and warm evenings, try the extra credit. For the extra credit, first just look at the parent class (if available). Once you have that working, look at all available ancestor classes.