SentencePiece tokenizer introduced in paper: SentencePiece: A simple and language independent subword tokenizer and detokenizer for Neural Text Processing (Kudo & Richardson, 2018).
Subject
SentencePiece tokenizer
Predicate
introduced_in_paper
Object
SentencePiece: A simple and language independent subword tokenizer and detokenizer for Neural Text Processing (Kudo & Richardson, 2018)
Primary source · preprint · 2018-08-19
SentencePiece: A simple and language independent subword tokenizer and detokenizer for Neural Text Processing — arXiv (Kudo, Richardson)