1.5
= 1000 = 1000
May1994
T. V. Raman
Visual communication is characterized by the eye's ability to actively access parts of a two-dimensional display. The reader is active, while the display is passive. This active-passive role is reversed by the temporal nature of oral communication: information flows actively past a passive listener. This prohibits multiple views -it is impossible to first obtain a high-level view and then ``look'' at details. These shortcomings become severe when presenting complex mathematics orally.
Audio formatting, which renders information structure in a manner attuned to an auditory display, overcomes these problems. AsTeR is interactive, and the ability to browse information structure and obtain multiple views enables active listening.
T.V. Raman was born and raised in Pune, India. He was partially sighted (sufficient to be able to read and write) until he was 14. Thereafter, he learned with the help of his brother, who spent a great deal of time as his first reader/tutor.
Three years later, in 1982, he learned Braille. No Braille textbooks were available, however, so the only source of reading material was his own class notes and notes prepared from recordings made by his readers. He developed his own system for writing math in Braille, since he could not locate the standard Braille math-codes in India.
One of his first uses of Braille was to mark a Rubik's cube. He had solved the first two layers of the cube by pointing to each square and having someone tell him the color, but the last layer was too difficult to solve in this manner. So his brother coded the cube colors in Braille. It took him four days to solve the cube the first time; later, he could solve it in under 30 seconds. Working with the Braille cube yielded interesting insights. For example, he soon realized that one color should stay unmarked, since that was easiest to identify. This point can be rephrased in terms of cues: the absence of a cue is itself a very good cue!
Raman received his B.A. in Mathematics at Nowrosjee Wadia College in Pune and his Masters in Math and Computer Science at the Indian Institute of Technology, Bombay. For his final-year project, he developed CONGRATS, a program that allowed the user to visualize curves by listening to them. In his final year, he had 13 readers recording texts for three hours per week each. Raman would paraphrase the recordings to prepare his own notes before recycling the cassette tapes. This took time, but when exams came around, he would have nothing more to study. Recording, paraphrasing, and revising is an excellent way to imbibe study material, and he recommends it to all.
Many of the ideas on audio formatting mathematics come from his experiences in having math read to him, in dictating math exams and having them written by a writer, and in listening to RFB (Recordings for the Blind) books on tape.
Raman was introduced to computing in 1987 with an introductory course on programming in Fortran77. He did his computing with someone behind him to read the display. A major reason for his desire to do graduate study in the U.S. was the lack of adaptive equipment in India.
Raman joined the PhD program in Applied Math at Cornell in Fall 1989. He obtained his first talking computer and his guide dog, Aster, in early 1990, both of which have enriched his life. His own research, which is described in this thesis, has already made learning and doing PhD research easier for him, and he hopes that it will open new possibilities for the visually handicapped throughout the world.
I thank my adviser, David Gries, for his help and guidance in turning a collection of useful ideas into a practicable thesis. His insight into defining a language for audio formatting proved crucial in realizing my ultimate goal of producing a system that does for audio documents what systems like La)TeX have achieved in the world of printed documents. I also acknowledge the help and support of the other members of my committee, John Hopcroft, Dan Huttenlocher, Dexter Kozen, Keith Dennis and John Guckenheimer.
My former office-mate, M. S. Krishnamoorthy (RPI), was the first to spot the potential presented by my prototype, . He, along with John Hopcroft, Keith Dennis and Brian Kernighan (ATT), encouraged me to take up the problem of producing audio renderings from electronic markup source for my dissertation. Tim Teitelbaum and Anne Neirynck helped in the initial phase when I was defining the problem. Bruce Donald was my adviser during the first phase of the project. We had many useful discussions, and I am grateful to him for convincing me to implement my system in Lisp-CLOS. Bruce Donald and CSRVL (Computer Science Robotics and Vision Lab) supported my work with a research assistantship and equipment.
My summer experience at the Xerox Palo Alto Research Center (PARC) helped me crystalize many ideas. Dennis Arnon of Xerox PARC pointed out the importance of working with document logical structure. Xerox Corp. also supported my work with an equipment grant in spring 1992. Jim Davis (Cornell DRI) advised me on lexical choice when producing spoken mathematics, helped improve my Lisp programming skills, and also contributed some Lisp code used to communicate with the speech synthesizer.
Intel Corp. supported my work with a one-year fellowship for the academic year 1992-93 and a research grant for fall 93. I acknowledge the help and support of my Intel mentor, Murali Veeramoney, and the other members of his group. Jim Larson (Intel Architecture Labs) helped me crystalize some of my ideas on user-interface design during the many stimulating discussions we had over the summer of 1993.
I implemented AsTeR and wrote this thesis using an Intel-[tex2html_wrap5242] PC from CSRVL running IBM Screen Reader. I thank the Center for Applied Mathematics (CAM) for opening up the world of computing to me by acquiring an Accent speech synthesizer and IBM Screen Reader -Screen Reader, designed by Jim Thatcher (IBM Watson Research Center), is one of the most robust screen-reading programs available today. I acknowledge the support of our systems administrators for their untiring help in my efforts to adapt my setup to use the software available.
I also thank the USENET community for their support in helping configure the various pieces of software that I use. The Emacs editor and Screen, a public-domain window manager for ASCII terminals, have together provided a powerful computing environment that has enabled me to be fully productive. Lack of online documentation for Lisp was overcome with help from the USENET (comp.lang.lisp) community. I also thank Nelson Beebee for his invaluable help on La)TeX throughout the writing of this thesis. I thank the authors and publishers of the texts listed in t:online-books for providing me online access to the electronic sources -these proved invaluable both as online references as well as test material for AsTeR .
Taking classes at Cornell was an enjoyable experience, and I thank all the faculty for their help. Every effort was made to provide online lecture notes -AsTeR was motivated by the availability of online notes for CS681 taught by Dexter Kozen. Talking books from Recording for the Blind (RFB) proved invaluable. I was also ably assisted by a dedicated group of readers. Anindya Basu, Bill Barry (ORST), Jim Davis, Harsh Kaul and Matthai Phillipose proof-read this thesis and suggested many useful improvements. I also thank Holly Mingins, Dolores Pendell (CAM) and the rest of the administrative staff of the CS department for their help and support. I thank Bert Adams of the Cornell Physical Education program for helping me stay fit during the last four years and Mike Dillon (NYSCBVH) for orienting me around the Cornell campus.
Finally, I thank my family for their love and support throughout.