java.lang.Object

org.jparsec.Terminals

public final class Terminals extends Object

Provides convenient API to build lexer and parsers for terminals. The following example is a parser snippet for Java generic type expression such as List<String>:

  Terminals terms = Terminals
      .operators("?", "<", ">", ",")
      .words(Scanners.IDENTIFIER)
      .keywords("super", "extends")
      .build();
  Parser<String> typeName = Terminals.identifier();
  Parser<?> wildcardWithUpperBound = terms.phrase("?", "extends");
  ...
  parser.from(terms.tokenizer(), Scanners.WHITESPACES.optional()).parse("List<String>");

Author:: Ben Yu

Nested Class Summary

Nested Classes

Modifier and Type

Class

Description

final class

Terminals.Builder

Builds Terminals instance by defining the words and keywords recognized.

static final class

Terminals.CharLiteral

Entry point for parser and tokenizers of character literal.

static final class

Terminals.DecimalLiteral

Entry point for parser and tokenizers of decimal number literal represented as String.

static final class

Terminals.Identifier

Entry point for parser and tokenizers of regular identifier.

static final class

Terminals.IntegerLiteral

Entry point for any arbitrary integer literal represented as a String.

static final class

Terminals.LongLiteral

Entry point for parser and tokenizers of integral number literal represented as Long.

static final class

Terminals.ScientificNumberLiteral

Entry point for parser and tokenizers of scientific notation literal.

static final class

Terminals.StringLiteral

Entry point for parser and tokenizers of string literal.
Field Summary

Fields

Modifier and Type

Field

Description

static final Parser<String>

RESERVED

Parser that recognizes reserved word tokens. i.e.
Method Summary

Modifier and Type

Method

Description

static Terminals

caseInsensitive(String[] ops, String[] keywords)

Deprecated.
Use operators(ops) .words(Scanners.IDENTIFIER) .caseInsensitiveKeywords(keywords) .build() instead.

static Terminals

caseInsensitive(Parser<String> wordScanner, String[] ops, String[] keywords)

Deprecated.
Use operators(ops) .words(wordScanner) .caseInsensitiveKeywords(keywords) .build() instead.

static Terminals

caseInsensitive(Parser<String> wordScanner, String[] ops, String[] keywords, Function<String,?> wordMap)

Deprecated.
Use operators(ops) .words(wordScanner) .tokenizeWordsWith(wordMap) .caseInsensitiveKeywords(keywords) .build() instead.

static Terminals

caseSensitive(String[] ops, String[] keywords)

Deprecated.
Use operators(ops) .words(Scanners.IDENTIFIER) .keywords(keywords) .build() instead.

static Terminals

caseSensitive(Parser<String> wordScanner, String[] ops, String[] keywords)

Deprecated.
Use operators(ops) .words(wordScanner) .keywords(keywords) .build() instead.

static Terminals

caseSensitive(Parser<String> wordScanner, String[] ops, String[] keywords, Function<String,?> wordMap)

Deprecated.
Use operators(ops) .words(wordScanner) .tokenizeWordsWith(wordMap) .keywords(keywords) .build() instead.

static Parser<String>

fragment(Object... tags)

Returns a Parser that recognizes Tokens.Fragment token values tagged with one of tags.

static Parser<String>

identifier()

Returns a Parser that recognizes identifiers (a.k.a words, variable names etc).

static Terminals

operators(String... ops)

Returns a Terminals object for lexing the operators with names specified in ops.

static Terminals

operators(Collection<String> ops)

Returns a Terminals object for lexing the operators with names specified in ops.

Parser<?>

phrase(String... tokenNames)

A Parser that recognizes a sequence of tokens identified by tokenNames, as an atomic step.

Parser<Token>

token(String tokenName)

A Parser that recognizes the token identified by tokenName.

Parser<Token>

token(String... tokenNames)

A Parser that recognizes a token identified by any of tokenNames.

Parser<?>

tokenizer()

Returns the tokenizer that tokenizes all terminals (operators, keywords, identifiers etc.) managed in this instance.

Terminals.Builder

words(Parser<String> wordScanner)

Starts to build a new Terminals instance that recognizes words not already recognized by this Terminals instance (typically operators).

Methods inherited from class Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Details
- RESERVED
  
  public static final Parser<String> RESERVED
  
  Parser that recognizes reserved word tokens. i.e. Tokens.Fragment tokens tagged as Tokens.Tag.RESERVED. Tokens.Fragment.text() is returned as parser result.
Method Details
- caseInsensitive
  
  @Deprecated public static Terminals caseInsensitive(String[] ops, String[] keywords)
  
  Deprecated.
  Use operators(ops) .words(Scanners.IDENTIFIER) .caseInsensitiveKeywords(keywords) .build() instead.
  
  Returns a Terminals object for lexing and parsing the operators with names specified in ops, and for lexing and parsing the keywords case insensitively. Parsers for operators and keywords can be obtained through token(String...); parsers for identifiers through identifier().
  In detail, keywords and operators are lexed as Tokens.Fragment with Tokens.Tag.RESERVED tag. Words that are not among keywords are lexed as Fragment with Tokens.Tag.IDENTIFIER tag.
  A word is defined as an alphanumeric string that starts with [_a - zA - Z], with 0 or more [0 - 9_a - zA - Z] following.
  
  Parameters:
  
  ops - the operator names.
  
  keywords - the keyword names.
  
  Returns:
  
  the Terminals instance.
- caseSensitive
  
  @Deprecated public static Terminals caseSensitive(String[] ops, String[] keywords)
  
  Deprecated.
  Use operators(ops) .words(Scanners.IDENTIFIER) .keywords(keywords) .build() instead.
  
  Returns a Terminals object for lexing and parsing the operators with names specified in ops, and for lexing and parsing the keywords case sensitively. Parsers for operators and keywords can be obtained through token(String...); parsers for identifiers through identifier().
  In detail, keywords and operators are lexed as Tokens.Fragment with Tokens.Tag.RESERVED tag. Words that are not among keywords are lexed as Fragment with Tokens.Tag.IDENTIFIER tag.
  A word is defined as an alphanumeric string that starts with [_a - zA - Z], with 0 or more [0 - 9_a - zA - Z] following.
  
  Parameters:
  
  ops - the operator names.
  
  keywords - the keyword names.
  
  Returns:
  
  the Terminals instance.
- caseInsensitive
  
  @Deprecated public static Terminals caseInsensitive(Parser<String> wordScanner, String[] ops, String[] keywords)
  
  Deprecated.
  Use operators(ops) .words(wordScanner) .caseInsensitiveKeywords(keywords) .build() instead.
  
  Returns a Terminals object for lexing and parsing the operators with names specified in ops, and for lexing and parsing the keywords case insensitively. Parsers for operators and keywords can be obtained through token(String...); parsers for identifiers through identifier().
  In detail, keywords and operators are lexed as Tokens.Fragment with Tokens.Tag.RESERVED tag. Words that are not among keywords are lexed as Fragment with Tokens.Tag.IDENTIFIER tag.
  
  Parameters:
  
  wordScanner - the scanner that returns a word in the language.
  
  ops - the operator names.
  
  keywords - the keyword names.
  
  Returns:
  
  the Terminals instance.
- caseSensitive
  
  @Deprecated public static Terminals caseSensitive(Parser<String> wordScanner, String[] ops, String[] keywords)
  
  Deprecated.
  Use operators(ops) .words(wordScanner) .keywords(keywords) .build() instead.
  
  Returns a Terminals object for lexing and parsing the operators with names specified in ops, and for lexing and parsing the keywords case sensitively. Parsers for operators and keywords can be obtained through token(String...); parsers for identifiers through identifier().
  In detail, keywords and operators are lexed as Tokens.Fragment with Tokens.Tag.RESERVED tag. Words that are not among keywords are lexed as Fragment with Tokens.Tag.IDENTIFIER tag.
  
  Parameters:
  
  wordScanner - the scanner that returns a word in the language.
  
  ops - the operator names.
  
  keywords - the keyword names.
  
  Returns:
  
  the Terminals instance.
- caseInsensitive
  
  @Deprecated public static Terminals caseInsensitive(Parser<String> wordScanner, String[] ops, String[] keywords, Function<String,?> wordMap)
  
  Deprecated.
  Use operators(ops) .words(wordScanner) .tokenizeWordsWith(wordMap) .caseInsensitiveKeywords(keywords) .build() instead.
  
  Returns a Terminals object for lexing and parsing the operators with names specified in ops, and for lexing and parsing the keywords case insensitively. Parsers for operators and keywords can be obtained through token(String...); parsers for identifiers through identifier().
  In detail, keywords and operators are lexed as Tokens.Fragment with Tokens.Tag.RESERVED tag. Words that are not among keywords are lexed as Fragment with Tokens.Tag.IDENTIFIER tag.
  
  Parameters:
  
  wordScanner - the scanner that returns a word in the language.
  
  ops - the operator names.
  
  keywords - the keyword names.
  
  wordMap - maps the text to a token value for non-keywords recognized by wordScanner.
  
  Returns:
  
  the Terminals instance.
- caseSensitive
  
  @Deprecated public static Terminals caseSensitive(Parser<String> wordScanner, String[] ops, String[] keywords, Function<String,?> wordMap)
  
  Deprecated.
  Use operators(ops) .words(wordScanner) .tokenizeWordsWith(wordMap) .keywords(keywords) .build() instead.
  
  Returns a Terminals object for lexing and parsing the operators with names specified in ops, and for lexing and parsing the keywords case sensitively. Parsers for operators and keywords can be obtained through token(String...); parsers for identifiers through identifier().
  In detail, keywords and operators are lexed as Tokens.Fragment with Tokens.Tag.RESERVED tag. Words that are not among keywords are lexed as Fragment with Tokens.Tag.IDENTIFIER tag.
  
  Parameters:
  
  wordScanner - the scanner that returns a word in the language.
  
  ops - the operator names.
  
  keywords - the keyword names.
  
  wordMap - maps the text to a token value for non-keywords recognized by wordScanner.
  
  Returns:
  
  the Terminals instance.
- operators
  
  public static Terminals operators(String... ops)
  
  Returns a Terminals object for lexing the operators with names specified in ops. Operators are lexed as Tokens.Fragment with Tokens.Tag.RESERVED tag. For example, to get the parser for operator "?", simply call token("?").
  If words and keywords need to be parsed, they can be configured via words(Parser).
  
  Parameters:
  
  ops - the operator names.
  
  Returns:
  
  the Terminals instance.
- operators
  
  public static Terminals operators(Collection<String> ops)
  
  Returns a Terminals object for lexing the operators with names specified in ops. Operators are lexed as Tokens.Fragment with Tokens.Tag.RESERVED tag. For example, to get the parser for operator "?", simply call token("?").
  If words and keywords need to be parsed, they can be configured via words(Parser).
  
  Parameters:
  
  ops - the operator names.
  
  Returns:
  
  the Terminals instance.
  
  Since:
  
  2.2
- words
  
  public Terminals.Builder words(Parser<String> wordScanner)
  
  Starts to build a new Terminals instance that recognizes words not already recognized by this Terminals instance (typically operators).
  By default identifiers are recognized through identifier() during token-level parsing phase. Use Terminals.Builder.tokenizeWordsWith(Function) to tokenize differently, and choose an alternative token-level parser accordingly.
  
  Parameters:
  
  wordScanner - defines words recognized by the new instance
  
  Since:
  
  2.2
- identifier
  
  public static Parser<String> identifier()
  
  Returns a Parser that recognizes identifiers (a.k.a words, variable names etc). Equivalent to Terminals.Identifier.PARSER.
  
  Since:
  
  2.2
- fragment
  
  public static Parser<String> fragment(Object... tags)
  
  Returns a Parser that recognizes Tokens.Fragment token values tagged with one of tags.
- tokenizer
  
  public Parser<?> tokenizer()
  
  Returns the tokenizer that tokenizes all terminals (operators, keywords, identifiers etc.) managed in this instance.
- phrase
  
  public Parser<?> phrase(String... tokenNames)
  
  A Parser that recognizes a sequence of tokens identified by tokenNames, as an atomic step.
- token
  
  public Parser<Token> token(String... tokenNames)
  
  A Parser that recognizes a token identified by any of tokenNames.
- token
  
  public Parser<Token> token(String tokenName)
  
  A Parser that recognizes the token identified by tokenName.

Class Terminals

Nested Class Summary

Field Summary

Method Summary

Methods inherited from class Object

Field Details

RESERVED

Method Details

caseInsensitive

caseSensitive

caseInsensitive

caseSensitive

caseInsensitive

caseSensitive

operators

operators

words

identifier

fragment

tokenizer

phrase

token

token