Update Python version support and dependencies; refactor tokenizer and tests#560
Update Python version support and dependencies; refactor tokenizer and tests#560LucyMaber wants to merge 2 commits intoallenai:mainfrom
Conversation
| ) | ||
|
|
||
|
|
||
| class TestWhitespace: |
There was a problem hiding this comment.
it's more idiomatic to use importlib.util.find_spec to check for the existence of dependencies.
see https://github.com/cthoyt/ssslm/blob/50dda51b40c885082dffef340fb11d932461ae80/tests/test_ner/test_scispacy.py#L10-L27, in which a more explicit check is made for the installation of en_core_sci_sm
cthoyt
left a comment
cthoyt
left a comment
There was a problem hiding this comment.
This PR has lots of awesome stuff! But I would suggest splitting it into many smaller independent PRs, since it's very difficult to understand all at once / assess how the different changes interact.
Note: I'm not a maintainer, just an interested bystander
| ] | ||
| ) | ||
|
|
||
| infix_re = compile_infix_regex(infixes) |
There was a problem hiding this comment.
just replaced with a new function? if this is sufficient, why not re-implement the old function by using a call to the new one?
…s utility functions; enhance whitespace test for model loading
|
Any news on when this will be released? I am upgrading to Python 3.13 and spaCy 3.8, and this change is needed for that. Thanks in advance. |
|
Honestly I have stopped working on that yep I believe it works but if the maintainers want to change it or merge it in or anyone else then it's up to them |
|
@LucyMaber The maintainers might respond soon since the holiday break is about over. But you could always roll your wheel and upload to PyPi of course. Then we could use your changes to get us by and then cut back over to the original when they fold it in. |
4 participants
No description provided.