Module tokenizers::tokenizer::pattern

Structs

• Invert
  Invert the is_match flags for the wrapped Pattern. This is usefull for example when we use a regex that matches words instead of a delimiter, and we want to match the delimiter.

Traits

• Pattern
  Pattern used to split a NormalizedString