Package org.codehaus.janino

Class TokenStreamImpl

java.lang.Object

org.codehaus.janino.TokenStreamImpl

All Implemented Interfaces:

TokenStream

public class TokenStreamImpl
extends java.lang.Object
implements TokenStream

Standard implementation for the TokenStream.

Field Summary

Fields

Modifier and Type	Field	Description
private java.lang.String	docComment	The optional JAVADOC comment preceding the nextToken.
private Token	nextButOneToken
private Token	nextToken
private Token	previousToken
private Scanner	scanner
private WarningHandler	warningHandler

Constructor Summary

Constructors

Constructor	Description
TokenStreamImpl(Scanner scanner)

Method Summary

Modifier and Type	Method	Description
protected CompileException	compileException(java.lang.String message)	Convenience method for throwing a CompileException.
java.lang.String	doc()	Gets the text of the doc comment (a.k.a.
private static int	indexOf(java.lang.String[] strings, java.lang.String subject)
private static int	indexOf(TokenType[] tta, TokenType subject)
private static java.lang.String	join(java.lang.Object[] oa, java.lang.String glue)
Location	location()
Token	peek()
boolean	peek(java.lang.String suspected)
int	peek(java.lang.String... suspected)	Checks whether the value of the next token equals any of the suspected; does not consume the next token.
boolean	peek(TokenType suspected)
int	peek(TokenType... suspected)	Checks whether the type of the next token is any of the suspected; does not consume the next token.
Token	peekNextButOne()
boolean	peekNextButOne(java.lang.String suspected)
boolean	peekRead(java.lang.String suspected)	Checks whether the value of the next token equals the suspected; if so, consumes the token.
int	peekRead(java.lang.String... suspected)	Checks whether the value of the next token is one of the suspected; if so, consumes the token.
java.lang.String	peekRead(TokenType suspected)	Checks whether the type of the next token is the suspected; if so, consumes the token.
int	peekRead(TokenType... suspected)	Checks whether the type of the next token is one of the suspected; if so, consumes the token.
private Token	produceToken()
Token	read()
void	read(java.lang.String expected)	Verifies that the value of the next token equals expected, and consumes the token.
int	read(java.lang.String... expected)	Verifies that the value of the next token equals one of the expected, and consumes the token.
java.lang.String	read(TokenType expected)	Verifies that the type of the next token is the expected, and consumes the token.
int	read(TokenType... expected)	Verifies that the type of the next token is one of the expected, and consumes the token.
void	setWarningHandler(WarningHandler warningHandler)	By default, warnings are discarded, but an application my install a WarningHandler.
java.lang.String	toString()
private void	warning(java.lang.String handle, java.lang.String message, Location location)	Issues a warning with the given message and location and returns.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Field Detail

scanner

private final Scanner scanner

docComment

@Nullable
private java.lang.String docComment

The optional JAVADOC comment preceding the nextToken.

previousToken

@Nullable
private Token previousToken

nextToken

@Nullable
private Token nextToken

nextButOneToken

@Nullable
private Token nextButOneToken

warningHandler

@Nullable
private WarningHandler warningHandler

Constructor Detail

TokenStreamImpl

public TokenStreamImpl(Scanner scanner)

Method Detail

produceToken

private Token produceToken()
                    throws CompileException,
                           java.io.IOException

Throws:

CompileException

java.io.IOException

doc

@Nullable
public java.lang.String doc()

Gets the text of the doc comment (a.k.a. "JAVADOC comment") preceding the next token.

Returns:

null if the next token is not preceded by a doc comment

peek

public Token peek()
           throws CompileException,
                  java.io.IOException

Specified by:

peek in interface TokenStream

Returns:

The next token, but does not consume it

Throws:

CompileException

java.io.IOException

peekNextButOne

public Token peekNextButOne()
                     throws CompileException,
                            java.io.IOException

Specified by:

peekNextButOne in interface TokenStream

Returns:

The next-but-one token, but consumes neither the next nor the next-but-one token

Throws:

CompileException

java.io.IOException

peek

public boolean peek(java.lang.String suspected)
             throws CompileException,
                    java.io.IOException

Specified by:

peek in interface TokenStream

Returns:

Whether the value of the next token equals suspected; does not consume the next token

Throws:

CompileException

java.io.IOException

peek

public int peek(java.lang.String... suspected)
         throws CompileException,
                java.io.IOException

Description copied from interface: TokenStream

Checks whether the value of the next token equals any of the suspected; does not consume the next token.

Specified by:

peek in interface TokenStream

Returns:

The index of the first of the suspected that equals the value of the next token, or -1 if the value of the next token equals none of the suspected

Throws:

CompileException

java.io.IOException

peek

public boolean peek(TokenType suspected)
             throws CompileException,
                    java.io.IOException

Specified by:

peek in interface TokenStream

Returns:

Whether the type of the next token is suspected

Throws:

CompileException

java.io.IOException

peek

public int peek(TokenType... suspected)
         throws CompileException,
                java.io.IOException

Description copied from interface: TokenStream

Checks whether the type of the next token is any of the suspected; does not consume the next token.

Specified by:

peek in interface TokenStream

Returns:

The index of the first of the suspected types that is the next token's type, or -1 if the type of the next token is none of the suspected types

Throws:

CompileException

java.io.IOException

peekNextButOne

public boolean peekNextButOne(java.lang.String suspected)
                       throws CompileException,
                              java.io.IOException

Specified by:

peekNextButOne in interface TokenStream

Returns:

Whether the value of the next-but-one token equals the suspected; consumes neither the next nor the next-but-one token

Throws:

CompileException

java.io.IOException

read

public Token read()
           throws CompileException,
                  java.io.IOException

Specified by:

read in interface TokenStream

Returns:

The next token, which it also consumes, or null iff the scanner is at end-of-input

Throws:

CompileException

java.io.IOException

read

public void read(java.lang.String expected)
          throws CompileException,
                 java.io.IOException

Description copied from interface: TokenStream

Verifies that the value of the next token equals expected, and consumes the token.

Specified by:

read in interface TokenStream

Throws:

CompileException - The value of the next token does not equal expected (this includes the case that the scanner is at end-of-input)

java.io.IOException

read

public int read(java.lang.String... expected)
         throws CompileException,
                java.io.IOException

Description copied from interface: TokenStream

Verifies that the value of the next token equals one of the expected, and consumes the token.

Specified by:

read in interface TokenStream

Returns:

The index of the consumed token within expected

Throws:

CompileException - The value of the next token does not equal any of the expected (this includes the case where the scanner is at end-of-input)

java.io.IOException

read

public java.lang.String read(TokenType expected)
                      throws CompileException,
                             java.io.IOException

Description copied from interface: TokenStream

Verifies that the type of the next token is the expected, and consumes the token.

Specified by:

read in interface TokenStream

Returns:

The value of the next token; an interned String iff the token represents an identifier, true, false, null, or an operator

Throws:

CompileException - The next token's type is not the expected

java.io.IOException

read

public int read(TokenType... expected)
         throws CompileException,
                java.io.IOException

Description copied from interface: TokenStream

Verifies that the type of the next token is one of the expected, and consumes the token.

Specified by:

read in interface TokenStream

Returns:

The index of the first of the expected types that is the next token's type; -1 if the type of the next token is none of the expected

Throws:

CompileException - The next token's type is none of the expected

java.io.IOException

peekRead

public boolean peekRead(java.lang.String suspected)
                 throws CompileException,
                        java.io.IOException

Description copied from interface: TokenStream

Checks whether the value of the next token equals the suspected; if so, consumes the token.

Specified by:

peekRead in interface TokenStream

Throws:

CompileException

java.io.IOException

peekRead

public int peekRead(java.lang.String... suspected)
             throws CompileException,
                    java.io.IOException

Description copied from interface: TokenStream

Checks whether the value of the next token is one of the suspected; if so, consumes the token.

Specified by:

peekRead in interface TokenStream

Returns:

The index of the first of the suspected that equals the next token's value; -1 iff the next token's value equals none of the suspected

Throws:

CompileException

java.io.IOException

peekRead

@Nullable
public java.lang.String peekRead(TokenType suspected)
                          throws CompileException,
                                 java.io.IOException

Description copied from interface: TokenStream

Checks whether the type of the next token is the suspected; if so, consumes the token.

Specified by:

peekRead in interface TokenStream

Returns:

The value of the next token; an interned String iff the token represents an identifier, true, false, null, or an operator

Throws:

CompileException

java.io.IOException

peekRead

public int peekRead(TokenType... suspected)
             throws CompileException,
                    java.io.IOException

Description copied from interface: TokenStream

Checks whether the type of the next token is one of the suspected; if so, consumes the token.

Specified by:

peekRead in interface TokenStream

Returns:

The index of the elements that is the next token's type, or -1 iff the type of next token is none of the suspected

Throws:

CompileException

java.io.IOException

location

public Location location()

Specified by:

location in interface TokenStream

Returns:

The location of the first character of the previously read (not peeked!) token

indexOf

private static int indexOf(java.lang.String[] strings,
                           java.lang.String subject)

indexOf

private static int indexOf(TokenType[] tta,
                           TokenType subject)

setWarningHandler

public void setWarningHandler(@Nullable
                              WarningHandler warningHandler)

Description copied from interface: TokenStream

By default, warnings are discarded, but an application my install a WarningHandler.

Notice that there is no Parser.setErrorHandler() method, but parse errors always throw a CompileException. The reason being is that there is no reasonable way to recover from parse errors and continue parsing, so there is no need to install a custom parse error handler.

Specified by:

setWarningHandler in interface TokenStream

Parameters:

warningHandler - null to indicate that no warnings be issued

toString

public java.lang.String toString()

Overrides:

toString in class java.lang.Object

warning

private void warning(java.lang.String handle,
                     java.lang.String message,
                     @Nullable
                     Location location)
              throws CompileException

Issues a warning with the given message and location and returns. This is done through a WarningHandler that was installed through setWarningHandler(WarningHandler).

The handle argument qualifies the warning and is typically used by the WarningHandler to suppress individual warnings.

Throws:

CompileException - The optionally installed WarningHandler decided to throw a CompileException

compileException

protected final CompileException compileException(java.lang.String message)

Convenience method for throwing a CompileException.

join

private static java.lang.String join(@Nullable
                                     java.lang.Object[] oa,
                                     java.lang.String glue)