Module tokenizers::normalizers::utils

source ·

Structs§

  • Lowercase
    Lowercases the input
  • Sequence
    Allows concatenating multiple other Normalizer as a Sequence. All the normalizers run in sequence in the given order against the same NormalizedString.