Skip to content

v2.1.3

Choose a tag to compare

View all tags
@tsproisl tsproisl released this 05 Mar 15:04
· 172 commits to master since this release
v2.1.3
c5833f4
  • Add a few abbreviations.
  • Improve tokenization of protocol-less URLs.
  • Improve tokenization of a few emoticons and symbols/dingbats.
  • Improve tokenization of gendered nouns (gender star, gender colon).
  • Improve tokenization of simple arithmetic operations.
Assets 2
Loading