You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
### my_application.py# Two packages are installed via `pip install app tokenizer`fromappimport*fromtokenizerimport*toks=Tokenizer().tokenize(text)
words=transform_tokens(toks)
TypeError: transform_tokens(): incompatible function arguments. The following argument types are supported:
1. (arg0: List[app.Token]) -> List[app.Word]
Invoked with: [<tokenizer.Token object at 0x2b61a0719770>, <tokenizer.Token object at 0x2b61a071f7b0>, <tokenizer.Token object at 0x2b61a071fe70>]
When I ran my_application.py, I got the error message above.
Originally app and tokenizer are C++ projects, and app has a dependency on tokenizer as you can see.
Both of them are developed in our team, but tokenizer is used in many other projects so we packaged it individually. tokenizer provides only C and python interfaces (its package does not have C++ headers), so I cannot access its C++ headers that I try to imitate the declaration of class Token and wish to reverse-bind the python class tokenizer.Token to imitated class Token in C++. transformTokens function should be implemented in C++ to meet the speed requirements. tokenizer has dependency on torch, but to achieve high performance, the C++ library of tokenizer use custom build of torch, while the python package of tokenizer just uses pytorch.
Therefore, if the python package of app include the C++ library of tokenizer, conflicts occur between pytorch and tokenizer in my_application.py.
reacted with thumbs up emoji reacted with thumbs down emoji reacted with laugh emoji reacted with hooray emoji reacted with confused emoji reacted with heart emoji reacted with rocket emoji reacted with eyes emoji
-
When I ran my_application.py, I got the error message above.
Originally
app
andtokenizer
are C++ projects, andapp
has a dependency ontokenizer
as you can see.Both of them are developed in our team, but
tokenizer
is used in many other projects so we packaged it individually.tokenizer
provides only C and python interfaces (its package does not have C++ headers), so I cannot access its C++ headers that I try to imitate the declaration ofclass Token
and wish to reverse-bind the python classtokenizer.Token
to imitatedclass Token
in C++.transformTokens
function should be implemented in C++ to meet the speed requirements.tokenizer
has dependency ontorch
, but to achieve high performance, the C++ library oftokenizer
use custom build oftorch
, while the python package oftokenizer
just usespytorch
.Therefore, if the python package of
app
include the C++ library oftokenizer
, conflicts occur betweenpytorch
andtokenizer
in my_application.py.Is there any idea on it? Thank you in advance...!
Beta Was this translation helpful? Give feedback.
All reactions