Is there a way to reconstruct 468 facial feature points into a frontal view？

[FromNature]

Hello,

The non-rigid variation intensity between facial feature points is crucial for expression analysis. However, head movement can affect the distribution of these points. Is there a method to decouple head motion from facial expression motion using Mediapipe's feature points and map them to a 2D frontal view?

Thank you, and I look forward to your response.

[ArdaSenyurek]

One way that comes to my mind is the use of rigs in a 3D program like Blender. I think you can calculate the bone relative orientations without any consideration about the spatial location of the whole armature. Then the problem becomes finding the right location & orientation of the armature to match with the image frames.

[FromNature]

Thank you. That's sounds a wonderful idea. By the way, is there a tool or formula to transfer the landmarks to a Blender mode?

…

---- Replied Message ---- | From | ***@***.***> | | Date | 01/01/2025 22:10 | | To | ***@***.***> | | Cc | ***@***.***>***@***.***> | | Subject | Re: [google-ai-edge/mediapipe] Is there a way to reconstruct 468 facial feature points into a frontal view？ (Issue #5794) | One way that comes to my mind is the use of rigs in a 3D program like Blender. I think you can calculate the bone relative orientations without any consideration about the spatial location of the whole armature. Then the problem becomes finding the right location & orientation of the armature to match with the image frames. — Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you authored the thread.Message ID: ***@***.***>

[ArdaSenyurek]

You create a standard rig for the face beforehand. Then use Blender Python API to set the orientations of bones programmatically in each frame. Aside, I'd go along with the line of making use of Shape Keys then you can set drivers for a shape key value to be driven by the location of a bone.