Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix pointer aliasing an other issues in ItemSliceSend #1116

Merged
merged 4 commits into from
Nov 16, 2023

Conversation

martinvonz
Copy link
Contributor

No description provided.

As suggested by @cramertj in our review of unsafe code at Google.
Same reasoning as for the fix in in_parallel.rs.

Thanks to @cramertj for spotting this.
This prevents mutating the items concurrently with the
`ItemSliceSend`. For example, before this patch, one could create two
`ItemSliceSend` from the same mutable slice.
`ItemSliceSend` still has the unsafe `Clone` implementation, so let's
not make it public. That also matches the visibility of most types in
`resolve.rs` (which is the only place these types are used).
Copy link
Member

@Byron Byron left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks a lot!

I think the main achievement here is to keep track of the lifetime of the input slice, which I will remember should there ever be another requirement fro unsafe in the future.

The switch away from a pointer to a slice seems more arbitrary unless one wants to remove bound checks. By now, since the logic seems to work, I think it's fine to do that even though in future, I'd still go with pointer slices initially as they'd provide bound-checks by default. But please let me know if there is anything I am missing. After all, unsafe is a bit like a language on its own and I am not very well versed in it.

Reading the above, and with my current probably incomplete knowledge, it can boil down to a style question: Do you want data: *mut [T] possibly paired with &mut (*data).get_unchecked(index) or rather data: *mut T which hides that we are looking at a slice.

T: Send,
{
items: *mut T,
phantom: PhantomData<&'a T>,
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice! The phantom has the sole purpose of maintaining the lifetime of the input slice, even though internally it's reduced to a pointer.

This also means the bounds-check isn't happening anymore upon access, but then again, it's also not needed anymore as it's known that all indices fit within the slice.

@Byron Byron merged commit 613f018 into GitoxideLabs:main Nov 16, 2023
18 checks passed
@martinvonz
Copy link
Contributor Author

The switch away from a pointer to a slice seems more arbitrary unless one wants to remove bound checks.

Do you mean in the constructor in the first commit? I just felt like slices are preferred over pointers in general. I would personally only use a pointer if I really had to. I don't know if I've ever used pointers outside of FFI in Rust.

Do you want data: *mut [T] possibly paired with &mut (*data).get_unchecked(index) or rather data: *mut T which hides that we are looking at a slice.

That's the same reasoning as in #1115: Dereferencing that *mut [T] concurrently (e.g. in different threads) is undefined behavior. Using *mut T and pointer::add() is not. That's my understanding of the issue anyway.

@Byron
Copy link
Member

Byron commented Nov 16, 2023

Do you mean in the constructor in the first commit? I just felt like slices are preferred over pointers in general. I would personally only use a pointer if I really had to. I don't know if I've ever used pointers outside of FFI in Rust.

I meant the internal representation, having a constructor is definitely nicer as well. If using *mut [T] from multiple threads generally is undefined behaviour, it makes sense to not do that of course. However, from my limited knowledge, all reasons for that that I have in mind (and do actively remember) are… actually it's only one reason being the usage of uninitialized memory to the point where one can't even look at it. But dereferencing a mutable slice should be fine as long as memory isn't aliased.
But I write that here just to share it, not to make any claims of correctness. After all, unsafe is mostly forbidden territory for me exactly for that reason - I never feel like I know enough and with certainty 😅.

@martinvonz
Copy link
Contributor Author

martinvonz commented Nov 16, 2023

I also don't know much about these things (that's why we have review of unsafe code by people who know better) :) I suspect the old version was perfectly safe with the current rustc, but the UB could mean that some future rustc could break it somehow. @cramertj provided me this nice illustration in our internal review: https://play.rust-lang.org/?version=stable&mode=debug&edition=2021&gist=3e20c65f7c47979cf57e7effd2e1195a. Click Tools->Miri to see the error.

@Byron
Copy link
Member

Byron commented Nov 16, 2023

Thanks so much for sharing!
I'd love to say I understand why the first version doesn't work and it's a bit of a hunch that it might be related to the new borrow checker not understanding something that it should understand (even though it's too trivial, too). Removing one assignment for example also works with version 1 (that fails with two assignments), which should mean that it's generally nothing that Miri would complain about.

The borrow-checker not being able to validate borrowing rules seems separate.

Oh well, if you ever stumble over an answer I can understand, please do share it with me :).

@martinvonz
Copy link
Contributor Author

I think it's that *args_slice_ptr results in a &mut [u8], mutably borrowing the entire slice. So I think it's like writing this:

let x = &mut args[0];
let y = &mut args[1];

That's of course not allowed (I mean, it could be if Rust understood that you only borrowed a single element, but that's not how Rust currently works).

By switching to a pointer, I think we're basically telling the compiler to just assume that the mutable pointers to individual elements cannot be aliased (which is a valid assumption, barring any bugs).

@Byron
Copy link
Member

Byron commented Nov 16, 2023

Ah, I understand now, thanks for bearing with me 😅. Using a slice like was done here before is then also fighting the borrow-checker more than necessary, and who knows what that could cover up, I suppose.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants