Tool
0.5.1
Softstar-Tokenizer Toolkit
Fast BPE tokenizer library with pre-trained vocabularies. Supports sentencepiece conversion and custom training pipelines.
↓ 6,200 downloads
License: MIT
Updated: Feb 2026