Beautify
The Java Code Formatter
Introduction
Beautify is a program that can be used to format Java Code depending on certain user preferences. The use of a program like this is to make it easier for one to read and edit code that they did not write, as there are almost as many ways to format code as there are people. This way the foreign code can be formatted to a user standard for easier reading. Potential uses include:
The ideal use would be something that could be called from inside of a developer, so that it can be done
Background
This project started as a project for Computer Science 501 - Software Engineering. The project was to design a developer's tool, and my group chose the code formatter. Initially, it was implemented with JavaCC for the backend, with a somewhat standard Java GUI as a frontend. The past project worked reasonably well, but beccause JavaCC generates a full recursive parse tree, the original ended up being very large, and very slow. This prompted looking for something that may be thinner and faster.
For reference, this is a description of the principal developing enviroments. The lexer and the core Java functions were developed on a handful of different Pentium PC's running Windows NT and Microsoft Developer's Studio. The remaining work was done on a Apple PowerMac 8500 running OS 8.1, as well as Codewarrior by Metrowerks for principal development and MRJ 2.0 for the JManager stuff.
Numerous questions were directed to, and answers were found at the newsgroup comp.sys.mac.programmer.codewarrior, a very good newsgroup for work with CW.
Beautify
The new and version that was made this semester was to make a plug-in that could be used with Codewarrior. This way, one did not need to worry about running a separate program or any other things like that. The ideal functionality was to have a menu itme or something as easy in Codewarrior that a user would select and it would beautify the code in the front most window. Initially, the CW plug-in API was investigated, but after looking at it for a while, it turned out that the API was mostly ofr designing linkers and compilers, and there was no way to invoke them other than telling CW to compile the file. This is really the only part of CodeWarrior that Metrowerks opens up for developers, so I had to look at other techniques as far as integrating it into CodeWarrior. Fortunately, there was enough open application communication stuff that Apple had designed that made this task easier.
First major compenent: the Lexer
The first thing that was designed was the lexer. For a lexer generator, eventually I chose JLex as recommeded to me as a good tool for the task. JLex will tkae in a grammar file and will output a lexer class that will return token objects sequentially, that will contain the token that is being returned, and the text that matched that token. The initial grammar file was done by Maksim Khavin, a fellow Master of Enginnering student, who made it for another project. It was then modified throughout the project to reflect some specific things that needed to be done for this project, as I determined that parts of it needed to be more specific or less specific.
The way the lexer works is pretty simple: after you have created the lexer class, in your main program, you create a lexer object, feeding it either a Reader object or an InputStream object. This will work for most any case, since almost any type of data can be made into one or the other. Then, you continue to call the nextToken function in the lexer class, which will return the next matching token in the data, as a Token object, which contains the token itself, and the data that matched that token (for recreation of the code)
So, simply put, the Lexer takes in the code to be Beautified, and will output a series of tokens for the relevant code-objects that it will find (ignoring white space)
Second major compenent: the Java Core
The Java core is actually where most of the work gets done, and is what I probably spent most of the time working on. This is where all of the decisions are made as to how the code gets formatted. As a test, this was assembled into an applet that could take in text that was typed (or copy and pasted) in to a window, and then output a "beautified" result into another window. This is compiled into several class files, so it could theoretically run as an applet or an application if you set up a Java VM to run the function "main" in the class "Beautify". This is also part of a project that should bele to build versions of the Beautfiy App that will either have an embedded VM or not. The first version needs to have a VM installed (so, something like MRJ), but the app is only a few hundred kbytes. The second version has the VM internally, but it is around 7 megs.
Anyway, here are all of the related files:
Sources and other related files
To build these, you need to download all the sources and the project file. The project file refers to classes.zip (which you should have, or you should not be playing with Java,) as well as "MW Java StandAlone.lib" and "MW Java StandAlone VM.lib" which build the applications that have and don't have the VM internally.
Third major compenent: the JNI/JManager functions
Probably the trickiest part to this project, and I am still not succesful with this part, has to do with crossing the line between Java and C. The core was written in Java to take advantage of JLex and the ease of programming in Java, yet the outer layer was built in C as AppleEvents have not been implemented in Java as of yet (though there are plans with Apple's future system releases, stay tuned) This meant that at some point, there needed to be a point where the C program called the Beautify function. The ideal design is to simply pass a string to the Java function, and have a string returned
The tools that are used are partially done by Sun, and partially by Apple. The main native-Java libraries were developed by Sun, and are called the Java Native Interface, or JNI for short. This allows both Java functions to call functions in C, and for C programs to instantiate their own VM's and call Java functions. The second use is what I would need it for. Apple has come up with a library called JManager, that expands upon the JNI stuff, and maps allows for a mapping of Java AWT objects to corresponding OS objects, as well as other features, generally making JNI work on the Mac.
JManager is written as a part of the Macintosh Runtime Java SDK that is produced by Apple. JNI is part of the MRJ as well, but it is also included in the CW cd.
The way that JManager works is that it starts a Java Session (basically, a Java VM). It then searches for the .zip or .jar file that contains the Java functions that are being called. It then searches for the class that you wish to invoke, and again searches for the method you wish to invoke by both name and calling specification (what is being called and returned). Then, you call a function depending on what the Java function that you are trying to run. (Example, you call functions with names like "CallStaticIntMethod()" to call a static method that returns an int.) This will then return the value that the Java function returns, just as normal. For consistency, there are associated Java types that are declared in JNI.h such as jint, jlong, jshort, to ensure that stuff is being passed and returned properly.
The JNI stuff has objects that are called "jstrings" which can be passed and returned like Java Strings, and there are assocaited functions to convert between the C and the Java Strings in C. So, a string can be converted, and passed, and then the result can be returned and converted back again.
This part of the project is not succesfully implemented yet, and here is a breif description of all of the trouble I faced, and I plan to have this resolved soon.
First of all, I tried to implement this with straight JNI. I searched around for examples, and found very little. The main documentation was a very long PDF file from Sun. After reading, I created a simple file that would call a simple Java function, following the specification for JNI very closely. It was not clear which library should be added when building this file, as there is no JNILib or anything obvious like that. I tried a number of different libraries, and the closest I got were the libraries called JavaVMLib_JAM and JavaVMLib_JIT (These were actually found in JManager example on the CW CD). The problem I had was that when I finally tried to compile the function, it found all of the functions except for JNI_GetDefaultJavaVMInitArgs(&vm_args) and JNI_CreateJavaVM(&jvm, &env, &vm_args). Not sure why, but I needed these functions according to the Sun Documentation.
The next idea was to use JManager, since that was also contained on the CW CD. The JManager example implemented JManager 1.0, and also included the two libraries: JavaVMLib_JAM and JavaVMLib_JIT. Again, I had to read through another lengthy PDF file (this time produced by Apple) that described how to use JManager. This way, I was able to get parts of the project to compile using the JManager initializers instead of the JNI Initializers. When attempting to execute the progam, I would get an error that JavaVM_JAM or JavaVM_JIT were not found (depending on which I included). This was if they were weak linked. If I tried strong linking, then the compiler would die, saying that an unexpected end of file was reached while trying to read either of the two libraries. I tried moving them aroundto see if it was an access path issue, but it doesn't seem to be, and I don't want to have to specifically install one of these things when a new user installs Beautify.
While trying to sort this out, a number of people mentioned Macintosh Runtime Java SDK ver 2.0, written for Macintosh Runtime Java ver 2.0 produced by Apple. Apparently this was a better version, and came out recently and is supposed to be improved. The installer mentions installing Examples and libraries and stuff, and the examples never appeared. The libraries appeared, as well as new .h files, and I tried building with those. Again, I read through the PDF file, since a lot changed from 1.0 to 2.0, and several of the function prototypes were changed. Fortunately, a library appeared called JManagerLib, and I figured that there was no doubt that this was the right library. I tried installing it, and running it, and it compiled fine again.This time, it was able to run. Here is what happens: It iniates a Java Session properly. It then looks for the .zip file to add it to the newly created VM. This is where the problem occurs. The prototype in the jmanager.h file says that it takes a char* and a bunch of other things as a URL and converts that to a FSSpec (Apple file specification). No matter what I do, I get a paramErr, implying that the URL is not proper. I tried relative and absolut URL's, but nothing worked. Looking through the documentation, the example for 2.0 shows the function takes a JMTextRef instead of a char* in the function. (A JMTextRef is a new text object they added in JManager 2.0, and they convert the text to a JMTextRef and pass it instead of just a char*). This was confusing as the .h file shows it taking a char*. I tried changing the .h file to use JMTextRef instead, but still no go. The Problem is that the source is not public, and there is not possible to debug and figure out what it doesn't like, or what it is looking for.
Recently, I have been getting some help from a person from Apple (Euro apple, so at noon our time, he goes home and I have to wait till the next day to email him). He said he has to do with making sure the JManager API works properly. He put some information in an email that shows that in his .h file, it is taking a JMTextRef object. So, the .h file seems to be in error, but usually the library doesn't stray too far from what the .h file's contain, so there is a chance that the library is messed up. I tried re-downloading and reinstalling, and the JManager.h file for JManager 2.0 seems to have the 1.0 prototype for this function, yet other stuff is 2.0 stuff. So, something is amiss.
Perusing Apple's Java Bug Report web site, I came across bug number 2209068 that states that "Can't get MRJ 2.0 to access C++ via JNI". Now, this is possibly related, though I am doing things the other way around, so it may not be relevant. The Apple guy game me the impression that it works, so I want to keep working with him to get it to work. Since I am not a "registered" Apple Developer (Something like a minimum 500 per year now that Apple has changed the rates) I am not eligible for the full official bug reporting and discussing, but the newsgroups are proving to be somewhat useful.
Talking recently to an Apple guy, I found out that he too has found the problem, and will let me know what he finds out, but it may take some time.
I would include code, but it is all in a very unruly state as I was adding and commenting out all sorts of stuff to try any combination of things that would get this to work. Code will be added once I manage to get this stuff to compile.
A potential problem is that it may not be possible (once these bugs are worked out) to build all of the Java stuff internally. There are ways to compile Java Applications with the VM built in (such as one of the versions on this page) but I am not sure that I will be able to build the VM in like this using JManager and JNI. It seems that they both need to assume that MRJ or something is installed on the computer. Even Netscape and Internet Explorer (which are C++ apps that have Java stuff built in) install Java running stuff when you install it. At the moment, I just want to get a version that works, and then I will try to generalize it as much as possible. I would like to find a nice little library that I can refer to that would allow me to run a Java VM without having to install a large collection on the users computer.
Fourth major compenent: the Applescript I/O functions
One of the biggest problems I had to overcome was how to get and set the text in the Codewarrior window. This part is the solution that I came up with for dealing with getting text from a Codewarrior window and then setting the text when the program is done. Apple has designed the system to allow for Interapplication Communication bya model called Apple Events, which allows applications to send events and information to other applications. So, the answer definitly lay in Apple Events to pass the original code to the Beautify application, and then to pass the new code back. The problem is that there is some learning that needs to be done with Apple Events as far as how to package them up and send them, and for one who is not familiar with Apple Events programming, it can be very time consuming. A lot of time was spent trying to develop even simple event passing functions.
One easy way to deal with Apple Events, is to use AppleScript, which is a very easy to use scrpting language that allows you to quickly write scripts to perform a large number of tasks. Most applications have the ability to receive and respond to Apple Events, and Applescript can give a nice frame work for the AppleEvents. Problem is that it is hard to mix the two without converting all of the Applescript into Apple Event calls.
Fortunately, I can across something called AS Widgets. What AS Widgets does is to allow you to embed and compile Applescripts into projects in Codewarrior, and it will in turn create an application with all of the scripts functionality built in. There are a series of steps you need to do to identify which script to run, and which function in that script. There are also setting and accessing functions for setting Applescript paramters, and receiving script results. This is still much easier than writing it all in Apple Events, and it allowed me to make the sequence a little more robust as creating dialogs in such are one-line commands.
I created two functions in main body of Beuatify that perform the "getting" and the "setting" of the text in the window. The first takes no arguments and returns a string. This function sets up the Applescript, and then calls the first of the two Applescript functions that were built into Beautify. This function takes no arguments and simply returns the text of the front-most window. The C function then goes through a couple of steps to properly retreive the text from the return of the script, and to format it into a standard string. The second function takes in the new text that was formatted by the JAVA core and returned by the JNI stuff, and sets and passes the text to the second embed Applescript. The second script simply takes text and sets the text of the front most window to the text that it was passed.
The rest of the functions and header files used come with the AS Widgets. You need to download them anyway, as there is a small compiler plug-in that you need do download and install, and I do not know what legality of me giving it away is.
This all works painlessly and fairly quickly, and I am happy with how this works out. Not only did the AS Widgets make this project easier, but now I know that anything I can do faster in Applescript, I can now carry over to the "real world" in C.
Fifth major compenent: the C++ wrapper function - the Main body, and the final result
First of all, there is one last thing that needs to be considered. The way to have this appear as a memu item is to turn on CodeWarrior's Script Menu, which can be set in the preferences for CodeWarrior. Once the menu is set, it will list several scripts that have alread been done. The one problem is that it only lists scripts in the menu, so a trivial script needs to be created that only calls the Beautify app, so a one-line script.
This is a rough overview of the sequence that is performed, to give you an idea of how everything is related. This main function needs to be C++ by the way, since JNI and JManager are written in C++, not straight C.
At the moment, I have no relevant source files for this part, as I was not able to get the JNI stuff to work. Most of this is already implemented partially in each of the subparts, so it would be fairly easy to put it together when everything works.
Progress Reports
Throughout the semester, incremental progress reports were made, and I have included those for reference, if you are interested in how the problems and solutions evolved.
| February 17, 1998 | HTML | Microsft Word |
| February 24, 1998 | HTML | Microsft Word |
| March 13, 1998 | HTML | Microsft Word |
Future Enhancements/Fixes
There are a number of things that can and should be done to continue to improve these things. They are of more and less significance, and I will hopefully change this page to reflect what happens to it as time goes on.
Last Modified: