Module tokenizers::decoders::fuse

source ·

Structs§

  • Fuse
    Fuse simply fuses all tokens into one big string. It’s usually the last decoding step anyway, but this decoder exists incase some decoders need to happen after that step