You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Potentially breaking change: The somajo-tokenizer script is automatically created upon installation and bin/somajo-tokenizer is removed. For most users, this does not make a difference. If you used to run your own modified version of SoMaJo directly via bin/somajo-tokenizer, consider installing the project in editable mode (see Development section in README.md).
Switch from setup.py to pyconfig.toml and restructure the project (source in src, tests in tests).
When creating a Token object, only known token classes can be passed.