You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
error: symbol `cxxbridge05$box$huggingface$tokenizers$ffi$NormalizedString$drop` is already defined
--> src\pre_tokenizers.rs:17:49
|
17 | fn pre_tokenized_string(normalized: Box<NormalizedString>) -> Box<PreTokenizedString>;
| ^^^^^^^^^^^^^^^^
This can be avoided by using references, of course, but then I need to clone the NormalizedString (which is probably safer anyway, and what I am planning to do). Or again, by defining everything in a single module, which I'd prefer to avoid.
More generally, do I understand correctly that if I want to create an object on the Rust side and own it on the C++ side, I should use functions returning Box?
The text was updated successfully, but these errors were encountered:
This isn't exactly the same issue but it is very closely related to #496, so I am going to close this issue in favor of tracking it together over there. #496 (comment) outlines how I intend to resolve both limitations.
More generally, do I understand correctly that if I want to create an object on the Rust side and own it on the C++ side, I should use functions returning Box?
I have two
extern "Rust"
blocks here https://github.com/alexeyr/tokenizers/blob/f88d84b06f2503a312a051d710bd2596eba24ed4/bindings/cpp/src/normalizers.rs and here https://github.com/alexeyr/tokenizers/blob/f88d84b06f2503a312a051d710bd2596eba24ed4/bindings/cpp/src/pre_tokenizers.rsBecause both use
Box<NormalizedString>
, I getThis can be avoided by using references, of course, but then I need to clone the
NormalizedString
(which is probably safer anyway, and what I am planning to do). Or again, by defining everything in a single module, which I'd prefer to avoid.More generally, do I understand correctly that if I want to create an object on the Rust side and own it on the C++ side, I should use functions returning
Box
?The text was updated successfully, but these errors were encountered: