SamTokenizer

Overview

Package

Class

Tree

Deprecated

Index

Help

PREV CLASS NEXT CLASS

FRAMES NO FRAMES All Classes

SUMMARY: NESTED | FIELD | CONSTR | METHOD

DETAIL: FIELD | CONSTR | METHOD

edu.cornell.cs.sam.io
Class SamTokenizer

java.lang.Object
  edu.cornell.cs.sam.io.SamTokenizer

All Implemented Interfaces:: Tokenizer

public class SamTokenizer
extends java.lang.Object
implements Tokenizer
extends java.lang.Object
implements Tokenizer

This is a SamTokenizer implementation that has full backwards/forwards mobility and loads everything into memory as soon as it is created. By default, quote processing is disabled and comment ignoring is enabled. You may use one of the constructors with options to enabled/disable this options

Nested Class Summary
`static class`	`SamTokenizer.TokenizerOptions`

Nested classes/interfaces inherited from interface edu.cornell.cs.sam.io.Tokenizer
`Tokenizer.TokenType`

Constructor Summary
`SamTokenizer(java.io.Reader r, SamTokenizer.TokenizerOptions... opt)` Creates a new SamTokenizer with a Reader as a source
`SamTokenizer(SamTokenizer.TokenizerOptions... opt)` Creates a new SamTokenizer with System.in as a source
`SamTokenizer(java.lang.String FileName, SamTokenizer.TokenizerOptions... opt)` Creates a new SamTokenizer with a file for a source

Method Summary
`boolean`	`canPushBack()` Checks if the stream can be pushed back
`boolean`	`check(char c)` Removes the next token if it is an operator, and matches the character c.
`boolean`	`check(java.lang.String s)` Removes the next token if it is a word, and matches the string s.
`void`	`close()` Closes the input stream
`char`	`getCharacter()` Returns the next token (unless it is not a character)
`java.lang.String`	`getComment()` Returns the next token (unless it is not a comment)
`float`	`getFloat()` Returns the next token (unless it is not a float)
`int`	`getInt()` Returns the next token (unless it is not an integer)
`char`	`getOp()` Returns the next token (unless it is not an operator)
`java.lang.String`	`getString()` Returns the next token (unless it is not a string)
`java.lang.String`	`getWhitespaceBeforeToken()` Returns the whitespace in the file before the current token.
`java.lang.String`	`getWord()` Returns the next token (unless it is not a word)
`int`	`lineNo()` Returns the line number of the last token requested
`void`	`match(char c)` Removes the next token if it is an operator, and matches the given character.
`void`	`match(java.lang.String s)` Removes the next token if it is a word, and matches the given string.
`int`	`nextLineNo()` Returns the line number of the next token to be read
`Tokenizer.TokenType`	`peekAtKind()` Returns the type of the next token
`void`	`pushBack()` Pushes the last token requested back.
`void`	`skipToken()` Skips the next token
`boolean`	`test(char c)` Checks if the next token is an operator and matches the character c.
`boolean`	`test(java.lang.String s)` Checks if the next token is a word and matches the string s.

Methods inherited from class java.lang.Object
`clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait`

Constructor Detail

SamTokenizer

public SamTokenizer(java.lang.String FileName,
                    SamTokenizer.TokenizerOptions... opt)
             throws java.io.IOException,
                    java.io.FileNotFoundException,
                    TokenParseException

Creates a new SamTokenizer with a file for a source

Parameters:: FileName - The file name of the file to read; opt - Options for parsing
Throws:: java.io.IOException - If there is a file error; java.io.FileNotFoundException - If the file could not be found; TokenParseException

SamTokenizer

public SamTokenizer(java.io.Reader r,
                    SamTokenizer.TokenizerOptions... opt)
             throws java.io.IOException,
                    TokenParseException

Creates a new SamTokenizer with a Reader as a source

Parameters:: r - The source; opt - Options for parsing
Throws:: java.io.IOException - if there is a stream error; TokenParseException

SamTokenizer

public SamTokenizer(SamTokenizer.TokenizerOptions... opt)
             throws java.io.IOException,
                    TokenParseException

Creates a new SamTokenizer with System.in as a source

Parameters:: opt - Options for parsing
Throws:: java.io.IOException - if there is a stream error; TokenParseException

Method Detail