Module tokenizers::decoders::wordpiece

source ·

Structs§

  • WordPiece
    The WordPiece decoder takes care of decoding a list of wordpiece tokens back into a readable string.

Functions§