Commits

Commits on Jun 12, 2023

add initial support for escape-characters
ricwein
committedJun 12, 2023

Commits on Jun 9, 2023

php 8.1 cleanup
ricwein
committedJun 9, 2023

Commits on Apr 1, 2020

adds setable start-lineno
ricwein
committedApr 1, 2020

Commits on Mar 29, 2020

updates composer dependencies (symfony)
ricwein
committedMar 29, 2020

Commits on Mar 27, 2020

updates composer dependencies
ricwein
committedMar 27, 2020
BREAKING: tokenizer-config is now an array: Tokenizer(,,array $config)
ricwein
committedMar 27, 2020
fixes TokenStream Typehints in docblocks
ricwein
committedMar 27, 2020

Commits on Mar 26, 2020

adds BaseToken::content(): string to fetch general token content as string
ricwein
committedMar 26, 2020

Commits on Mar 25, 2020

introduces optional affix-separation
ricwein
committedMar 25, 2020

Commits on Mar 24, 2020

adds stream reseting to custom offsets
ricwein
committedMar 24, 2020

Commits on Mar 23, 2020

fixes line-counting for new-line delimited symbols
ricwein
committedMar 23, 2020
adds sources line number to result tokens
ricwein
committedMar 23, 2020

Commits on Mar 22, 2020

introduces BlockToken::content() for simple block-content access
ricwein
committedMar 22, 2020
fixes bug for not-tokenized blocks containing other blocks
ricwein
committedMar 22, 2020

Commits on Mar 20, 2020

replaces remaining symbols() keywords with tokens
ricwein
committedMar 20, 2020
replaces Result/-Symbols with Tokens and TokenStream
ricwein
committedMar 20, 2020

Commits on Mar 13, 2020

adds another, more complex example to restructuring unit test
ricwein
committedMar 13, 2020

Commits on Mar 12, 2020

adds method to change resultsymbols delimiter postinit
ricwein
committedMar 12, 2020
fixes ResultSymbol::isContextSwitching()
ricwein
committedMar 12, 2020
adds isContextSwitching flag to delimiters and resultsymbols
ricwein
committedMar 12, 2020
fixes multichar delimiter and block re-matching issues
ricwein
committedMar 12, 2020

Commits on Mar 11, 2020

reduces content-trimming for more exact result
ricwein
committedMar 11, 2020

Commits on Mar 9, 2020

fixes block prefix/suffix handling for empty strings
ricwein
committedMar 9, 2020

Commits on Mar 6, 2020

adds Delimiter::is():bool method
ricwein
committedMar 6, 2020
adds Result::symbols() methods for convenient processing access
ricwein
committedMar 6, 2020

Commits on Mar 5, 2020

adds ResultSymbol::asGuessedType()
ricwein
committedMar 5, 2020
changes behavior of blocks "shouldTokenizeContent" flag
ricwein
committedMar 5, 2020
adds missing suffix/prefix exports to resultblocks
ricwein
committedMar 5, 2020
adds Block::is() for easy block-type comparison
ricwein
committedMar 5, 2020
makes tokenizer result countable
ricwein
committedMar 5, 2020
adds rebuilding of strings from tokenized Result object
ricwein
committedMar 5, 2020
refactors result types
ricwein
committedMar 5, 2020

Commits on Mar 4, 2020

init
ricwein
committedMar 4, 2020