Package gnu.text

Class Lexer

java.lang.Object

java.io.Reader

gnu.text.Lexer

All Implemented Interfaces:

java.io.Closeable, java.lang.AutoCloseable, java.lang.Readable

Direct Known Subclasses:

Lexer, LispReader, XQParser, XslTranslator

public class Lexer
extends java.io.Reader

Framework for implementing lexical scanners and parsers.

Field Summary

Fields

Modifier and Type	Field	Description
protected int	nesting
protected gnu.kawa.io.InPort	port
protected boolean	tentative
char[]	tokenBuffer	For building tokens of various kinds.
int	tokenBufferLength	The number of chars of tokenBuffer that are used.

Fields inherited from class java.io.Reader

lock

Constructor Summary

Constructors

Constructor	Description
Lexer(gnu.kawa.io.InPort port)
Lexer(gnu.kawa.io.InPort port, SourceMessages messages)

Method Summary

Modifier and Type	Method	Description
boolean	checkErrors(java.io.PrintWriter out, int max)	Returns true if any error were seen.
boolean	checkNext(char ch)	Check if the next character matches a given character.
void	clearErrors()
void	close()
void	eofError(java.lang.String msg)
void	eofError(java.lang.String message, int startLine, int startColumn)
void	error(char severity, java.lang.String message)
void	error(char severity, java.lang.String filename, int line, int column, java.lang.String message)
void	error(java.lang.String message)
void	fatal(java.lang.String message)
int	getColumnNumber()	Return the current (zero-based) column number.
SourceError	getErrors()
int	getLineNumber()	Get the current line number.
SourceMessages	getMessages()
java.lang.String	getName()
gnu.kawa.io.InPort	getPort()
boolean	isInteractive()
boolean	isTentative()	True if input may be incomplete or actively edited.
void	mark()	Start tentative parsing.
int	peek()
void	popNesting(char save)	Exit a nested expression, reversing pushNesting
char	pushNesting(char promptChar)	Enter a nested expression.
int	read()
int	read(char[] buf, int offset, int length)
int	readCodePoint()	Read a Unicode character (codepoint) by checking for surrogates.
boolean	readDelimited(java.lang.String delimiter)	Scan until a given delimiter.
static long	readDigits(gnu.kawa.io.InPort port, int radix)
static long	readDigitsInBuffer(gnu.kawa.io.InPort port, long ival, int radix)	Read digits, up to the first non-digit or the buffer limit
int	readIntDigits()
int	readOptionalExponent()	Read an optional signed integer.
int	readUnicodeChar()	Deprecated. Use readCodePoint().
void	reset()	Stop tentative parsing.
boolean	seenErrors()
void	setInteractive(boolean v)
void	setMessages(SourceMessages messages)
void	setTentative(boolean v)
void	skip()
protected void	skip_quick()
void	tokenBufferAppend(int ch)	Append one character to tokenBuffer, resizing it if need be.
java.lang.String	tokenBufferString()
protected void	unread()
void	unread(int ch)
protected void	unread_quick()

Methods inherited from class java.io.Reader

mark, markSupported, nullReader, read, read, ready, skip, transferTo

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

port

protected gnu.kawa.io.InPort port

tentative

protected boolean tentative

nesting

protected int nesting

tokenBuffer

public char[] tokenBuffer

For building tokens of various kinds.

tokenBufferLength

public int tokenBufferLength

The number of chars of tokenBuffer that are used.

Constructor Detail

Lexer

public Lexer(gnu.kawa.io.InPort port)

Lexer

public Lexer(gnu.kawa.io.InPort port,
             SourceMessages messages)

Method Detail

pushNesting

public char pushNesting(char promptChar)

Enter a nested expression. This is used in interactive mode to control whether to continue past end of line, depending on whether the expression is incomplete.

Parameters:

promptChar - Used in prompt string to indicate type of nesting.

Returns:

The previous value of promptChar, to be passed to popNesting.

popNesting

public void popNesting(char save)

Exit a nested expression, reversing pushNesting

Parameters:

save - Saved values return by prior pushNesting

getPort

public final gnu.kawa.io.InPort getPort()

close

public void close()
           throws java.io.IOException

Specified by:

close in interface java.lang.AutoCloseable

Specified by:

close in interface java.io.Closeable

Specified by:

close in class java.io.Reader

Throws:

java.io.IOException

read

public int read()
         throws java.io.IOException

Overrides:

read in class java.io.Reader

Throws:

java.io.IOException

readUnicodeChar

@Deprecated
public int readUnicodeChar()
                    throws java.io.IOException

Deprecated.

Use readCodePoint().

Read a Unicode character (codepoint) by checking for surrogates.

Throws:

java.io.IOException

readCodePoint

public int readCodePoint()
                  throws java.io.IOException

Read a Unicode character (codepoint) by checking for surrogates.

Throws:

java.io.IOException

read

public int read(char[] buf,
                int offset,
                int length)
         throws java.io.IOException

Specified by:

read in class java.io.Reader

Throws:

java.io.IOException

unread

public void unread(int ch)
            throws java.io.IOException

Throws:

java.io.IOException

peek

public int peek()
         throws java.io.IOException

Throws:

java.io.IOException

skip

public void skip()
          throws java.io.IOException

Throws:

java.io.IOException

unread

protected void unread()
               throws java.io.IOException

Throws:

java.io.IOException

unread_quick

protected void unread_quick()
                     throws java.io.IOException

Throws:

java.io.IOException

checkNext

public boolean checkNext(char ch)
                  throws java.io.IOException

Check if the next character matches a given character.

Parameters:

ch - The character to match against.

Returns:

if the character read matches On a match, the position is advanced following that character.

Throws:

java.io.IOException

skip_quick

protected void skip_quick()
                   throws java.io.IOException

Throws:

java.io.IOException

getMessages

public SourceMessages getMessages()

setMessages

public void setMessages(SourceMessages messages)

checkErrors

public boolean checkErrors(java.io.PrintWriter out,
                           int max)

Returns true if any error were seen. Prints and clears the errors.

Parameters:

out - where to write the error message to

max - maximum number of messages to print (can be 0)

getErrors

public SourceError getErrors()

seenErrors

public boolean seenErrors()

clearErrors

public void clearErrors()

error

public void error(char severity,
                  java.lang.String filename,
                  int line,
                  int column,
                  java.lang.String message)

error

public void error(char severity,
                  java.lang.String message)

error

public void error(java.lang.String message)

fatal

public void fatal(java.lang.String message)
           throws SyntaxException

Throws:

SyntaxException

eofError

public void eofError(java.lang.String msg)
              throws SyntaxException

Throws:

SyntaxException

eofError

public void eofError(java.lang.String message,
                     int startLine,
                     int startColumn)
              throws SyntaxException

Throws:

SyntaxException

readOptionalExponent

public int readOptionalExponent()
                         throws java.io.IOException

Read an optional signed integer. If there is no integer in the input stream, return 1. For excessively large exponents, return Integer.MIN_VALUE or Integer.MAX_VALUE.

Throws:

java.io.IOException

readDelimited

public boolean readDelimited(java.lang.String delimiter)
                      throws java.io.IOException,
                             SyntaxException

Scan until a given delimiter. On success, text upto the delimiter is in then tokenBuffer (with tokenBufferLength marking its length); the delimiter is not included.

Throws:

java.io.IOException

SyntaxException

readDigitsInBuffer

public static long readDigitsInBuffer(gnu.kawa.io.InPort port,
                                      long ival,
                                      int radix)

Read digits, up to the first non-digit or the buffer limit

Parameters:

ival - previously-seen digits or -2 if no digits seen

Returns:

the digits seen as a non-negative long, or -1 on overflow, or -2 if no digits seen

readDigits

public static long readDigits(gnu.kawa.io.InPort port,
                              int radix)
                       throws java.io.IOException

Throws:

java.io.IOException

readIntDigits

public int readIntDigits()
                  throws java.io.IOException

Throws:

java.io.IOException

getName

public java.lang.String getName()

getLineNumber

public int getLineNumber()

Get the current line number. The "first" line is number number 0.

getColumnNumber

public int getColumnNumber()

Return the current (zero-based) column number.

isInteractive

public boolean isInteractive()

setInteractive

public void setInteractive(boolean v)

isTentative

public boolean isTentative()

True if input may be incomplete or actively edited. Used for command-completion and on-the-fly error checking.

setTentative

public void setTentative(boolean v)

tokenBufferAppend

public void tokenBufferAppend(int ch)

Append one character to tokenBuffer, resizing it if need be.

tokenBufferString

public java.lang.String tokenBufferString()

mark

public void mark()
          throws java.io.IOException

Start tentative parsing. Must be followed by a reset.

Throws:

java.io.IOException

reset

public void reset()
           throws java.io.IOException

Stop tentative parsing. Return to position where we called mark.

Overrides:

reset in class java.io.Reader

Throws:

java.io.IOException