- Bug fix: do not apply normalization to the user-defined-symbols.
- Bug fix: stop adding extra whitespaces before user-defined symbols
- Feature: added --minloglevel flag to suppress LOG(INFO) message
- Feature: added --split_by_number flag to allow numbers to attach other symbols.
- Feature: added --max_sentence_length flag to control the maximum byte length of input sentence for training.
- used tf-versioned so file for _sentencepiece_processor_ops to minimize ABI incompatibility for tf wapper.
For more detail: v0.1.5...master