Documentation

InlineLexer extends AbstractLexer
in package

FinalYes

Base class for writing simple lexers, i.e. for creating small DSLs.

Tags
extends

Table of Contents

Constants

ANNOTATION_END  = 17
ANNOTATION_START  = 16
ANONYMOUS_END  = 3
ANONYMOUSE_REFERENCE  = 12
BACKTICK  = 6
COLON  = 13
DOUBLE_BACKTICK  = 18
EMAIL  = 20
EMPHASIS_DELIMITER  = 21
ESCAPED_SIGN  = 25
HYPERLINK  = 19
LITERAL  = 5
NAMED_REFERENCE  = 11
NBSP  = 23
OCTOTHORPE  = 14
STRONG_DELIMITER  = 22
UNDERSCORE  = 2
VARIABLE_DELIMITER  = 24
WHITESPACE  = 15
WORD  = 1

Properties

$lookahead  : Token<T, V>|null
The next token in the input.
$token  : Token<T, V>|null
The last matched/seen token.
$tokenPositions  : array<int, int>
Map between string position and position in token list.

Methods

getInputUntilPosition()  : string
Retrieve the original lexer's input until a given position.
getLiteral()  : int|string
Gets the literal for a given token.
glimpse()  : Token<T, V>|null
Peeks at the next token, returns it and immediately resets the peek.
isA()  : bool
Checks if given value is identical to the given token.
isNextToken()  : bool
Checks whether a given token matches the current lookahead.
isNextTokenAny()  : bool
Checks whether any of the given tokens matches the current lookahead.
moveNext()  : bool
Moves to the next token in the input string.
peek()  : Token<T, V>|null
Moves the lookahead token forward.
reset()  : void
Resets the lexer.
resetPeek()  : void
Resets the peek pointer to 0.
resetPosition()  : void
Resets the lexer position on the input to the given position.
setInput()  : void
Sets the input data to be tokenized.
skipUntil()  : void
Tells the lexer to skip input tokens until it sees a token with the given value.
getCatchablePatterns()  : array<string|int, string>
Lexical catchable patterns.
getModifiers()  : string
Regex modifiers
getNonCatchablePatterns()  : array<string|int, string>
Lexical non-catchable patterns.
getType()  : T|null
Retrieve token type. Also processes the token value if necessary.
scan()  : void
Scans the input string for tokens.

Constants

ANONYMOUSE_REFERENCE

public mixed ANONYMOUSE_REFERENCE = 12

EMPHASIS_DELIMITER

public mixed EMPHASIS_DELIMITER = 21

VARIABLE_DELIMITER

public mixed VARIABLE_DELIMITER = 24

Properties

Methods

getInputUntilPosition()

Retrieve the original lexer's input until a given position.

public getInputUntilPosition(int $position) : string
Parameters
$position : int
Return values
string

getLiteral()

Gets the literal for a given token.

public getLiteral(T $token) : int|string
Parameters
$token : T
Return values
int|string

glimpse()

Peeks at the next token, returns it and immediately resets the peek.

public glimpse() : Token<T, V>|null
Return values
Token<T, V>|null —

The next token or NULL if there are no more tokens ahead.

isA()

Checks if given value is identical to the given token.

public isA(string $value, int|string|UnitEnum $token) : bool
Parameters
$value : string
$token : int|string|UnitEnum
Return values
bool

isNextToken()

Checks whether a given token matches the current lookahead.

public isNextToken(T $type) : bool
Parameters
$type : T
Tags
psalm-assert-if-true

!=null $this->lookahead

Return values
bool

isNextTokenAny()

Checks whether any of the given tokens matches the current lookahead.

public isNextTokenAny(array<int, T> $types) : bool
Parameters
$types : array<int, T>
Tags
psalm-assert-if-true

!=null $this->lookahead

Return values
bool

moveNext()

Moves to the next token in the input string.

public moveNext() : bool
Tags
psalm-assert-if-true

!null $this->lookahead

Return values
bool

peek()

Moves the lookahead token forward.

public peek() : Token<T, V>|null
Return values
Token<T, V>|null —

The next token or NULL if there are no more tokens ahead.

resetPeek()

Resets the peek pointer to 0.

public resetPeek() : void

resetPosition()

Resets the lexer position on the input to the given position.

public resetPosition([int $position = 0 ]) : void
Parameters
$position : int = 0

setInput()

Sets the input data to be tokenized.

public setInput(string $input) : void

The Lexer is immediately reset and the new input tokenized. Any unprocessed tokens from any previous input are lost.

Parameters
$input : string

The input to be tokenized.

skipUntil()

Tells the lexer to skip input tokens until it sees a token with the given value.

public skipUntil(T $type) : void
Parameters
$type : T

The token type to skip until.

getCatchablePatterns()

Lexical catchable patterns.

protected getCatchablePatterns() : array<string|int, string>
Return values
array<string|int, string>

getModifiers()

Regex modifiers

protected getModifiers() : string
Return values
string

getNonCatchablePatterns()

Lexical non-catchable patterns.

protected getNonCatchablePatterns() : array<string|int, string>
Return values
array<string|int, string>

getType()

Retrieve token type. Also processes the token value if necessary.

protected getType(string &$value) : T|null
Parameters
$value : string
Tags
inheritDoc
Return values
T|null

scan()

Scans the input string for tokens.

protected scan(string $input) : void
Parameters
$input : string 

        

        
    




    

    

                                

                

                                
    

        On this page

        
    


                

                            

            

    

        

            
Search results