Simple scene graph using ModelNodes?

[paulmelis]

I'm in need of a very basic scene graph for an SK app and I was looking into re-using what the built-in ModelNode class and friends could provide. It doesn't seem to be a perfect match, as drawing is currently defined at the Model (and Mesh level), yet the actual graph structure is encoded using ModelNode's, which don't know anything about drawing.

I mostly need a transform per node, a way to have a node be used by multiple parents and perhaps a visibility flag per node (for now ;-)). One issue is that I also would want to include a UI.Handle as part of the scene graph and have it transform multiple objects (by making them children of the Handle). The latter doesn't easily seem possible with the existing classes, as a Handle is not know to them. Nor do I want to have something like a flag per node to indicate it's "moveable" and then use a Handle transparently, as having the Handle transform the same objects in the same way is essential, while still having control over visibility of each of those objects transformed is also needed.

My question, probably mostly for @maluoi, is would it make more sense to add my own custom small number of scene graph classes that merely use the existing Model and Mesh classes as-is, or is there a nice way to reuse the existing classes to base a scene graph on? Any experience or example code for something like this?

Edit: Also, just noticed that changing the LocalTransform of a ModelNode can potentially be an expensive operation, as it updates transforms of all the child nodes recursively as well. And since I'll be moving scene nodes around quite a bit that might not be the best basis. Why does LocalTransform have an effect on the subtree, is it merely to update all the ModelTransform's (while leaving LocalTransform of the children alone)?

[maluoi]

Philosophy of Model as Scene Graph

Model is by necessity, a small and limited scene graph! Complex model formats like GLTF basically describe an entire scene, so StereoKit does need to represent that, especially when it gets to features like animation that rely on that graph data! However, this is somewhat in conflict with StereoKit's ambition to minimize the amount of state that it manages.

Now that I've added animation anyway, it may be a good idea to just double down a bit on the concept of "Model as lightweight SceneGraph", and add some of the missing functionality. This may help with tasks like exposing some of the less predictable GLTF extension data, or getting developers started a little faster on scene description.

But I'm not certain if I want to really dig in and make it a recommended path forward. We get into some interesting questions like "Should a ModelNode be able to contain a Model?", and "How do I make a good Scene Graph API that works well in C and C# simultaneously?". I think I'd rather provide a high-level scene graph sample, and encourage people to use and customize that instead, or write up one of their own that works well for their needs.

One important idea note is that a Scene Graph can often be a very cumbersome way to manage a scene compared to some of the alternatives! So I don't want the API to necessarily push people in that direction by default.

Using Model as a Scene Graph

Okay, so for the practical concerns about this as it exists today! :D

ModelNode.ModelTransform and ModelNode.LocalTransform - This is the ModelNode's transform in Model origin relative space, or Local space relative to the parent ModelNode. Modifying either of these transform properties will cause all child ModelNodes to update their transform matrices to preserve their location relative to the parent that was just modified. This is how Scene Graphs are supposed to behave, and is a normal performance problem that any Scene Graph implementation has to solve! SK does well with performance here in the context of animations, but there is a bit of room for improvement with respect to general modifications. I wouldn't worry about it too much though, unless you have complex hierarchies and frequently move items near the root.

ModelNode.Visible - This was actually a feature added in v0.3.6-preview.6! So, fresh off the griddle, and not in the docs yet :)

In the docs, I did try and establish a pattern that I think is helpful for scraping a Model's hierarchy, and defining custom behavior with. Behavior still has to happen externally from the Model, but tagging a ModelNode's name can be used to extract the correct nodes to execute behavior on. So over here, you can see the Tagged Nodes sample:

var nodes = model.Visuals
    .Where(n => n.Name.StartsWith("[Wire]"));
foreach (var node in nodes)
{
    node.Material = node.Material.Copy();
    node.Material.Wireframe = true;
}

ModelNode as a Handle

If you want to use a particular node on a Model as a free-floating UI.Handle, you can do something kinda like this! Not sure if this is exactly what you're looking for, but hopefully this provides a bit of insight into how some of the hierarchy works in this regard.

using StereoKit;

class DocHandleNode : ITest
{
    ModelNode handleNode;
    Model     model;
    Pose      modelPose = new Pose(0,0,-1, Quat.Identity);

    public void Initialize()
    {
        model = new Model();
        model.AddNode("Cube",
            Matrix.Identity,
            Mesh.GenerateCube(Vec3.One),
            Default.Material);
        model.AddNode("HandleNode",
            Matrix.T(0,0.5f,0),
            Mesh.GenerateCube(V.XYZ(0.5f,0.2f,0.5f)),
            Default.Material);

        handleNode = model.FindNode("HandleNode");
    }

    public void Update()
    {
        UI.PushSurface(modelPose);

        handleNode.ModelTransform.Decompose(out Vec3 pos, out Quat rot, out Vec3 scale);
        Pose handlePose = new Pose(pos, rot);
        if (UI.Handle("modelHandle", ref handlePose, handleNode.Mesh.Bounds))
            handleNode.ModelTransform = handlePose.ToMatrix(scale);
        
        UI.PopSurface();
        model.Draw(modelPose.ToMatrix());
    }

    public void Shutdown() { }
}

---

Anyhow, I just wrote quite a bit! 😬 Hopefully something in here is useful, feel free to ask for more clarification! :)