Full world cloning #17316
Full world cloning #17316
Conversation
…ion on `Component` and `Resource`
|
@chengts95, would this implementation be good enough for your use case? |
|
I thought of a use case for this yesterday, lol. When a player finishes a race in a game like Mariokart, I wonder if it performs a fast simulation to determine the final place of the NPCs that haven't crossed the finish line yet or if it just uses their current position. |
BenjaminBrienen
added
C-Feature
I prefer my approach that can avoid modifying internal things. Also, I may want partial cloning to sync only specific types of components/ specific entity sets. That is why I used a similar reflection registry which only requires a common |
I can see how your proposed approach would be more general, however my main concern is that it doesn't support dynamic components (since they don't have The reason I though that full world cloning would make more sense for your use case is that when some resource or component didn't get cloned for some reason, it might lead to unexpected results. You also mentioned native performance, which is why I tried to make it as fast as possible (and that required trying to clone as much of internal data as possible instead of recomputing stuff). Then there's also the issue of when to run various hooks (or even if they should be run at all and for which entities) and observers, how to update archetypes if a component for an entity wasn't cloned. Maybe the functionality introduced in this PR can be extended to support partial cloning, but I think it will get a lot more complicated if we want to preserve native performance.
If there is some component that can't be cloned, it is possible to provide a new value instead by setting a custom component clone handler. The main problem is that you would still need to provide some value - it can't just ignore cloning a component (or we would lose performance to archetype moves, although there was also a suggestion to add an But in general I do agree that current approach is somewhat inflexible, I'm just not sure how to improve it yet. |
My approach already solved my problems, the only problem for Bevy is that they want to remove the API to spawn with a specific entity id. In addition, my approach also handles the types without the Clone trait such as parent and child without modifications to the original components. I believe a dynamic component already has a dynamic Clone trait or clone function to call. Generally, I avoid virtual functions in ECS design since ECS stores typed components. Since we have to register the clone function, I prefer to register it out of the world and components. Also, many data types are not cloneable and need additional information to regenerate. I prefer to leave the freedom to the user instead of fusing the logic inside the datatype, it is against the rule of data-oriented design. In some cases, full cloning is used. In other more common cases, partial cloning is used. It actually depends on the systems that will be called. |
alice-i-cecile
added
the
X-Contentious
The proposed approach does not rely on
Just to make sure we're talking about the same thing, by dynamic components I mean components registered using
That would mean we would lose automatic clone handler registration for components that implement
As long as the data needed to regenerate a type is stored within source world, it is accessible from custom clone handler:
Custom clone handlers can be set for any registered component by using I'm just clearing up some misunderstandings since this functionality is based on #16132 and I don't explain everything that was implemented there in this PR. |
|
This might be useful for saving worlds? In my game I've been manually extracting all entities that have saveable components into a separate world along with their children, de-spawning any children that don't need to be saved, then serialising. |
BenjaminBrienen
added
the
S-Waiting-on-Author
|
scripts will be reflected. Usually they have a type of scriptable to erase Their own type i think. Again every thing in rust has a type.
获取Outlook for Android<https://aka.ms/AAb9ysg>
…________________________________
From: eugineerd ***@***.***>
Sent: Sunday, January 12, 2025 6:41:43 AM
To: bevyengine/bevy ***@***.***>
Cc: CTS ***@***.***>; Mention ***@***.***>
Subject: Re: [bevyengine/bevy] Full world cloning (PR #17316)
In addition, my approach also handles the types without the Clone trait such as parent and child without modifications to the original components
The proposed approach does not rely on Clone, but it does provide default handler implementations for components that implement Clone or Reflect. It uses autoderef specialization in #[derive(Component)] to select appropriate handler based on implemented traits.
I believe a dynamic component already has a dynamic Clone trait or clone function to call. Generally, I avoid virtual functions in ECS design since ECS stores typed components.
Just to make sure we're talking about the same thing, by dynamic components I mean components registered using
world.register_component_with_descriptor, which can be used to register new components for scripting implementations. These components don't have TypeId's, but they should still be cloneable (scripting implementation would be responsible for setting the appropriate clone handler).
Since we have to register the clone function, I prefer to register it out of the world and components
That would mean we would lose automatic clone handler registration for components that implement Clone or Reflect and they would have to be registered manually in the registry.
Also, many data types are not cloneable and need additional information to regenerate.
As long as the data needed to regenerate a type is stored within source world, it is accessible from custom clone handler: pub type ComponentCloneFn = fn(&World, &mut ComponentCloneCtx).
I prefer to leave the freedom to the user instead of fusing the logic inside the datatype, it is against the rule of data-oriented design.
Custom clone handlers can be set for any registered component by using world.set_component_clone_handler(component_id, handler), you don't need to implement fn get_component_clone_handler.
________________________________
I'm just clearing up some misunderstandings since this functionality is based on #16132<#16132> and I don't explain everything that was implemented there in this PR.
―
Reply to this email directly, view it on GitHub<#17316 (comment)>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/ABSPMZ52YRABSQ5LFIC2IP32KJPIPAVCNFSM6AAAAABVALVOJKVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDKOBVG4YTQOJXGM>.
You are receiving this because you were mentioned.Message ID: ***@***.***>
|
| } | ||
|
|
||
| /// HACK: Determine if T is [`Copy`] by (ab)using array copy specialization. | ||
| /// This utilizes a maybe bug in std which is maybe unsound when used with lifetimes (see: <https://github.com/rust-lang/rust/issues/132442>). |
There was a problem hiding this comment.
Since Component must be 'static, and at least in this PR we're only calling is_copy on Components, that means the "using this trick on structs with lifetimes is unsound" concern wouldn't have any practical impact here, right?
Obviously it is still a hack, but it's also private to this module and I'd say the doc comment adequately describes the hazards involved with using it.
There was a problem hiding this comment.
I'd recommend to replace this hack with autoderef specialization since its about to be fixed: rust-lang/rust#135634. There are still some other ways to abuse std's specialization, but it's not a good idea to do so.
|
I read this PR to judge whether I'd be able to rebase it and take it over the finish line (as @eugineerd suggested), and here's what looks like needs to be done code-wise still to make this useful & robust in practice:
But that all seems secondary to the one big question:
If the answer is "yes, we want to support that", then we also need to figure out (from most to least important):
|
This is a solid plan of action as far as I can see.
As for the api changes in clone handlers - some work in this PR went into making clone handlers take Can't really say much about any other changes ECS went through since the time this PR was opened, but I don't think anything changed significantly enough to break this code too much.
I don't have a good answer for this, which is a large part of why this PR stalled in the first place. I also don't really have any projects where I would've liked to do world cloning, so I don't feel qualified enough to answer this questions in the first place. |
|
In my opinion, the clone must be implemented in a isolated runtime or user-coded context registry. Injecting trait to component is not necessary and not helpful. It depends on users' decision to define how to replicate the states. It is very obvious that many objects and structures are semantically bound to move-only or controlled-copy behavior, which is the default behavior for modern languages and even modern C++. You cannot distinguish deepcopy and shallow copy from a trait which erased such constriant. In addition, it is not possible for a type to fully implement a meaningful Consider this: a Therefore, the correct approach is to copy the data-driven description and re-register the behavior in the new context. Injecting OOP-style implicit behavior — such as cloning boxed function pointers without structure awareness — goes directly against the principles of a data-driven and data-oriented ECS architecture. |
@chengts95 in 0.16 Bevy already gained an official way to clone entities: see 0.16's release notes, Entity Cloning section for a summary. How to clone each component can be customized on a per-component (and per-clone-attempt) basis too; see EntityCloner docs. So none of that is new to this PR; this PR just extends that a little to allow "component clone behavior when cloning world" to be different from "component clone behavior when cloning a single entity". As for the full vs partial cloning of the World (and the related infectiousness of Cloning, as in "so much of Bevy's types will end up needing to be Clone in order for It seems like that approach would work well for Resources (no hooks/observers on Resources.. right? so that sidesteps all the partial cloning concerns for them), and I suspect that in practice it will mostly be Resources that can't be cloned properly. That gives an easy answer for dealing with the (almost certainly present) Schedule resource: it just gets skipped, so the clone doesn't fail. But for actual Components on entities, the concerns about partial cloning in the PR description would still be valid though, so the approach would be "Uncloneable Resources are skipped and their ComponentIds are returned to the user, but uncloneable Entity Components cause the world clone to fail". Partial cloning of entity components could be figured out later when someone asks for it - at which point we could ask them how they would expect hooks/observers to behave. Thoughts? (of course this is all presupposing that Bevy wants world cloning in the first place) |
This is certainty a solution, however the reason I decided to go with short-circuiting is because allowing only some components to be cloned complicates the code a lot and makes this approach less performant overall. For example, for each archetype we would need to keep track of all entities that had at least 1 component that failed cloning and move them to a different archetype. That would mean that we can't just clone
There is work ongoing to make resources more like components, which would allow them to have hooks as well (see #17485). That would also mean there's no need to special-case resources and the same approach that works for components would work for resources as well. |
Objective
Fix #16559 by introducing full world cloning. This would allow to "fork" a
Worldby callingtry_clonefor use cases like running a simulation of a world without affecting the main world.Solution
This PR introduces ability to clone a
&Worldby performing "full cloning". "Full cloning", in this context, means that either world will be cloned fully (all entities, resources, etc.) or it will not be cloned at all.There are many ways cloning a world can fail, here's a list of errors that can happen:
Resource clone handler
Since full world cloning implies cloning resources,
Resourcetrait has been changed to allow setting component clone handler as well.This should be fine since we're moving towards unifying
ComponentandResourceanyway.Copy-based component clone handlerTo speed up component cloning,
ComponentCloneHandlercan now be set tocopy_handlermode andComponentDescriptornow has a newis_copyflag that is set totrueonly for components/resources that implementCopy.Setting
is_copyflag uses a somewhat unsupported hack utilizing arrayCopyspecialization. It is not required for the approach to work (we can always fallback to Autoderef-based specialization if this hack ever gets fixed), but it does make code a bit cleaner and allows to set this flag forNonSendresources.World component clone handler
Another change implemented to facilitate world cloning is a new
CompnentCloneFnsignature:Commandsis now available fromComponentCloneCtxinstead of fromDeferredWorld.World cloning reuses the same component clone handlers as entity cloning, however
Commandsand source and target entity are not available when cloning component in world context. This is a bit unfortunate, but most component clone handlers that rely on commands to perform component cloning would not work well in that context. For example,ParentandChildrencomponents should not spawn a new entity - instead they should just clone inner data to new world.Sometimes it makes sense to set different handlers depending on context, which can be set like so:
Technically this is not required - branching can be performed inside the clone handler instead. I'm not entirely sure which way will be better.
Why full world cloning?
The main reason this PR implements "full cloning" and not "partial cloning" (only cloneable components and resources are cloned) is to enable fast cloning and avoid having to deal with many problems for which I don't have a good answer for:
OnRemovehooks?OnAdd/OnInserthooks run?I decided that even though this implementation is fairly limited, it should still be useful for the use case presented in the linked issue.
Testing
try_clone.There should definitely be more tests. Best-case scenario would be to somehow reuse all existing world tests by making a clone right before asserts, but I'm not sure if this is feasible.
Considerations
This PR introduces a lot of changes to various low-level components for what is arguable quite a niche feature. Going forward, all new additions to
Worldwould have to beClone-compatible, which might increase complexity too much.My question is: is this something we want to support? It might be too much of a burden, but I'm not entirely sure at this point. See "Future work" section for a list of components that would need to be supported before this functionality can be useful for a normal
bevyapp.Future work
While this PR allows to clone a
Worldas long as all components and resources are cloneable, this is not yet compatible with most defaultbevycomponent and resources. As an example, here is a list of all components and resources that fail to be cloned in a simplehello_worldapp with justDefaultPluginsadded:For some of these components it is trivial to implement
Clone, for others it is a lot harder. At this point I'm not sure if component clone handlers will ever be implemented for all ofbevy's components.