|
||||||||||
| PREV CLASS NEXT CLASS | FRAMES NO FRAMES | |||||||||
| SUMMARY: NESTED | FIELD | CONSTR | METHOD | DETAIL: FIELD | CONSTR | METHOD | |||||||||
java.lang.Objectfr.umlv.tatoo.runtime.lexer.Tokenizer<R,B>
R - type of rulesB - type of the bufferpublic class Tokenizer<R,B extends LexerBuffer>
Provide a simple tokenizer that can be used to implement LL algorithm.
The buffer must support the read operation.
If you want an iterator, use Scanner instead.
Scanner| Method Summary | ||
|---|---|---|
static
|
createTokenizer(LexerTable<R> lexerTable,
B buffer)
Creates a tokenizer that extracts character from a buffer and matches them against rules. |
|
static
|
createTokenizer(LexerTable<R> lexerTable,
B buffer,
R... blanks)
Creates a scanner that extracts character from a buffer and matches them against rules. |
|
boolean |
eof()
Indicated wether eod-of-file is reached. |
|
B |
getBuffer()
Returns the underlying buffer. |
|
LexerTable<R> |
getLexerTable()
Returns the rule tables for this process |
|
R |
getNext()
Returns the rule successfully matched by a prior call to hasNext. |
|
boolean |
hasNext(R... rules)
Indicates if input contains one of the tokens passed as argument. |
|
R |
next(R... rules)
Returns the next matching rule. |
|
void |
reset(B buffer)
Reset the tokenizer to perform a new analysis. |
|
| Methods inherited from class java.lang.Object |
|---|
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait |
| Method Detail |
|---|
public static <R,B extends LexerBuffer> Tokenizer<R,B> createTokenizer(LexerTable<R> lexerTable,
B buffer)
read operation.
R - type of rules.B - type of bufferlexerTable - data table of the scanner.buffer - the buffer used to extract characters.
the lexing process
public static <R,B extends LexerBuffer> Tokenizer<R,B> createTokenizer(LexerTable<R> lexerTable,
B buffer,
R... blanks)
R - type of rules.B - type of bufferlexerTable - data table of the scanner.buffer - the buffer used to extract characters.
the lexing processblanks - the tokens to be skipped
public boolean hasNext(R... rules)
throws IOException
discards
the previous token from the buffer.
rules - the tokens to match
IOException
public R next(R... rules)
throws IOException
getBuffer()
and the devoted method of the buffer.
A call to this method discards
the previous token from the buffer.
rules - the tokens to match
IOExceptionTokenBuffer.view()public R getNext()
hasNext. This
method can only be called after hasNext has returned true.
hasNext.#hasNext(R...)public B getBuffer()
public LexerTable<R> getLexerTable()
public void reset(B buffer)
public boolean eof()
throws IOException
IOException - when an i/o error occurs
|
||||||||||
| PREV CLASS NEXT CLASS | FRAMES NO FRAMES | |||||||||
| SUMMARY: NESTED | FIELD | CONSTR | METHOD | DETAIL: FIELD | CONSTR | METHOD | |||||||||