-
Notifications
You must be signed in to change notification settings - Fork 4
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
## [5.1.5] - 2024-05-31 ### Changed - Rebuilt static libraries with Xcode version 15.4 (15F31d). You are now required to build iOS apps with Xcode 15.4 or newer.
- Loading branch information
Unity Technologies
committed
May 31, 2024
1 parent
353e3c8
commit a9e1db4
Showing
9 changed files
with
121 additions
and
11 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -3,17 +3,60 @@ uid: arkit-face-tracking | |
| --- | ||
| # Face tracking | ||
|
|
||
| This page is a supplement to the AR Foundation [Face tracking](xref:arfoundation-face-tracking) manual. The following sections only contain information about APIs where ARKit exhibits unique platform-specific behavior. | ||
|
|
||
| [!include[](snippets/arf-docs-tip.md)] | ||
|
|
||
| > [!IMPORTANT] | ||
| > To use face tracking with ARKit, you must first enable face tracking in the XR Plug-in Management settings. Refer to [Enable the Face tracking subsystem](xref:arkit-project-config#enable-face-tracking) to understand how to enable face tracking for ARKit. | ||
| ## Optional feature support | ||
|
|
||
| ARKit implements the following optional features of AR Foundation's [XRFaceSubsystem](xref:UnityEngine.XR.ARSubsystems.XRFaceSubsystem). The availability of features on specific devices depends on device hardware and software. Refer to [Requirements](#requirements) for more information. | ||
|
|
||
| | Feature | Descriptor Property | Supported | | ||
| | :------ | :------------------ | :-------: | | ||
| | **Face pose** | [supportsFacePose](xref:UnityEngine.XR.ARSubsystems.XRFaceSubsystemDescriptor.supportsFacePose) | Yes | | ||
| | **Face mesh vertices and indices** | [supportsFaceMeshVerticesAndIndices](xref:UnityEngine.XR.ARSubsystems.XRFaceSubsystemDescriptor.supportsFaceMeshVerticesAndIndices) | Yes | | ||
| | **Face mesh UVs** | [supportsFaceMeshUVs](xref:UnityEngine.XR.ARSubsystems.XRFaceSubsystemDescriptor.supportsFaceMeshUVs) | Yes | | ||
| | **Face mesh normals** | [supportsFaceMeshNormals](xref:UnityEngine.XR.ARSubsystems.XRFaceSubsystemDescriptor.supportsFaceMeshNormals) | | | ||
| | **Eye tracking** | [supportsEyeTracking](xref:UnityEngine.XR.ARSubsystems.XRFaceSubsystemDescriptor.supportsEyeTracking) | Yes | | ||
|
|
||
| > [!NOTE] | ||
| > Refer to AR Foundation [Face tracking platform support](xref:arfoundation-face-tracking-platform-support) for more information | ||
| > on the optional features of the face subsystem. | ||
| ## Session configuration | ||
|
|
||
| Face tracking requires the use of the user-facing or "selfie" camera. It is the responsibility of your session's [XRSessionSubsystem.configurationChooser](xref:UnityEngine.XR.ARSubsystems.XRSessionSubsystem.configurationChooser) to choose the camera facing direction. You can override the configuration chooser to meet your app's needs. For more information on the [ConfigurationChooser](xref:UnityEngine.XR.ARSubsystems.ConfigurationChooser), refer to the [What’s new in Unity’s AR Foundation | Unite Now 2020](https://www.youtube.com/watch?v=jBRxY2KnrUs&t=677s) video (YouTube). You can access a sample that shows how to use the `ConfigurationChooser` to choose between the user-facing and world-facing camera on the [AR Foundation samples](https://github.com/Unity-Technologies/arfoundation-samples/tree/5.1/Assets/Scenes/Configurations) GitHub repository. | ||
|
|
||
| ### Configuration chooser | ||
|
|
||
| iOS devices support different combinations of features in different camera facing directions. If your scene contains several manager components that require the world-facing camera, AR Foundation's default configuration chooser might decide to use the world-facing camera, even if the AR Face Manager component is also enabled in your scene. You can create your own [ConfigurationChooser](xref:UnityEngine.XR.ARSubsystems.ConfigurationChooser) to prioritize face tracking functionality over other features if you desire greater control over the camera's facing direction. | ||
|
|
||
| You can access an example of using a custom `ConfigurationChooser` in the `Rear Camera (ARKit)` sample on the [AR Foundations samples](https://github.com/Unity-Technologies/arfoundation-samples/blob/5.1/Assets/Scenes/FaceTracking/WorldCameraWithUserFacingFaceTracking.unity) GitHub. This example demonstrates how you can use the user-facing camera for face tracking, and the world-facing (rear) camera for passthrough video (iOS 13+). | ||
|
|
||
| ## Blend shapes | ||
|
|
||
| ARKit provides a series of [blend shapes](https://developer.apple.com/documentation/arkit/arfaceanchor/2928251-blendshapes?language=objc) to describe different features of a face. Each blend shape is modulated from 0..1. For example, one blend shape defines how open the mouth is. | ||
|
|
||
| ## Front facing camera | ||
| A blend shape represents action at a location on a face. Each blend shape is defined by an [ARKitBlendShapeLocation](xref:UnityEngine.XR.ARKit.ARKitBlendShapeLocation) to identify the location of the face action and a [ARKitBlendShapeCoefficient](xref:UnityEngine.XR.ARKit.ARKitBlendShapeCoefficient) to describe the amount of action at the location. The `ARKitBlendShapeCoefficient` is a value between `0.0` and `1.0`. | ||
|
|
||
| You can learn more about blend shapes with the `Blend shapes` sample on the [AR Foundation Samples](https://github.com/Unity-Technologies/arfoundation-samples/blob/5.1/Assets/Scripts/Runtime/ARKitBlendShapeVisualizer.cs) GitHub. This sample uses blend shapes to puppet a cartoon face which is displayed over the detected face. | ||
|
|
||
| ## Face visualizer samples | ||
|
|
||
| Face tracking requires the use of the front-facing or "selfie" camera. When the front-facing camera is active, other tracking subsystems like plane tracking or image tracking may not be available. If the rear-facing camera is active, face tracking might not be available. | ||
| The [AR Foundation Samples](https://github.com/Unity-Technologies/arfoundation-samples/tree/5.1) GitHub repository contains ARKit-specific prefabs that you can use to visualize faces in your scene, as outlined in the following table. Refer to the AR Foundation [AR Face](xref:arfoundation-face-tracking-arface) manual for more information on how to use these prefabs. | ||
|
|
||
| Different iOS devices support different combinations of features. If you `Start` a subsystem that requires the rear-facing camera, the Apple ARKit package might decide to use the rear-facing camera instead. For more information, see [Camera and Tracking Mode Selection](https://docs.unity3d.com/Packages/[email protected]/manual/migration-guide-3.html#camera-and-tracking-mode-selection). | ||
| | Prefab | Description | | ||
| | :----- | :---------- | | ||
| | [AR Eye Pose Visualizer](https://github.com/Unity-Technologies/arfoundation-samples/blob/5.1/Assets/Prefabs/AR%20Eye%20Pose%20Visualizer.prefab) | Visualize the location and direction of the eyes of a detected face. | | ||
| | [Eye Laser Visualizer](https://github.com/Unity-Technologies/arfoundation-samples/blob/5.1/Assets/Prefabs/Eye%20Laser%20Prefab.prefab) | Use the eye pose to draw laser beams emitted from the detected face. | | ||
| | [Sloth Head](https://github.com/Unity-Technologies/arfoundation-samples/blob/5.1/Assets/Prefabs/SlothHead.prefab) | Use the face blend shapes provided by ARKit to animate a 3D character. | | ||
|
|
||
| ## Technical details | ||
| <a id="requirements"></a> | ||
|
|
||
| ### Requirements | ||
| ## Requirements | ||
|
|
||
| Face tracking supports devices with Apple Neural Engine in iOS 14 and iPadOS 14 and requires a device with a TrueDepth camera on iOS 13 and iPadOS 13 and earlier. See Apple's [Tracking and Visualizing Faces](https://developer.apple.com/documentation/arkit/content_anchors/tracking_and_visualizing_faces?language=objc) documentation for more information. | ||
|
|
||
|
|
||
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -1,3 +1,3 @@ | ||
|
|
||
| <hr> | ||
| \* *Apple and ARKit are trademarks of Apple Inc., registered in the U.S. and other countries and regions.* | ||
| <p><i>Apple and ARKit are trademarks of Apple Inc., registered in the U.S. and other countries and regions.</i></p> |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,2 @@ | ||
| > [!TIP] | ||
| > When developing an AR app, refer to both the AR Foundation documentation as well as the [required packages](xref:arfoundation-manual#required-packages) for each platform you support. |
Binary file modified
BIN
+21.3 KB
(100%)
Runtime/FaceTracking/iOS/Xcode1500/libUnityARKitFaceTracking.a
Binary file not shown.
Binary file not shown.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -1,7 +1,7 @@ | ||
| { | ||
| "name": "com.unity.xr.arkit", | ||
| "displayName": "Apple ARKit XR Plugin", | ||
| "version": "5.1.4", | ||
| "version": "5.1.5", | ||
| "unity": "2021.2", | ||
| "description": "Provides native Apple ARKit integration for use with Unity's multi-platform XR API.\nSupports the following features:\n-Efficient Background Rendering\n-Horizontal Planes\n-Depth Data\n-Anchors\n-Hit Testing\n-Face Tracking\n-Environment Probes\n-Meshing\n-Occlusion\n\n* Apple and ARKit are trademarks of Apple Inc., registered in the U.S. and other countries and regions.", | ||
| "keywords": [ | ||
|
|
@@ -13,20 +13,20 @@ | |
| ], | ||
| "dependencies": { | ||
| "com.unity.editorcoroutines": "1.0.0", | ||
| "com.unity.xr.arfoundation": "5.1.4", | ||
| "com.unity.xr.arfoundation": "5.1.5", | ||
| "com.unity.xr.core-utils": "2.1.0", | ||
| "com.unity.xr.management": "4.0.1" | ||
| }, | ||
| "_upm": { | ||
| "changelog": "### Changed\n\n- Changed the AR Foundation dependency version from 5.1.3 to 5.1.4\n\n### Fixed\n\n- Fixed an issue where the tvOS build target would have compile errors in `ARKit.InputLayoutLoader`." | ||
| "changelog": "### Changed\n\n- Rebuilt static libraries with Xcode version 15.4 (15F31d). You are now required to build iOS apps with Xcode 15.4 or newer.\n- Changed the AR Foundation dependency version from 5.1.4 to 5.1.5.\n\n### Fixed\n\n- Fixed issue [ARKB-58](https://issuetracker.unity3d.com/issues/arfoundation-application-crashes-when-arocclusionmanager-is-disabled-if-multithreaded-rendering-is-enabled-on-ios-devices-that-use-lidar) where iOS apps could intermittently crash when destroying the `AROcclusionManager` with multithreaded rendering enabled." | ||
| }, | ||
| "upmCi": { | ||
| "footprint": "02444c21f06ea70f35a183ddf4dcf5d8e01e5f45" | ||
| "footprint": "dbc564c15f671b1311199382bc23b58640cf9ed3" | ||
| }, | ||
| "documentationUrl": "https://docs.unity3d.com/Packages/[email protected]/manual/index.html", | ||
| "repository": { | ||
| "url": "https://github.cds.internal.unity3d.com/unity/arfoundation.git", | ||
| "type": "git", | ||
| "revision": "17e0eb6efa2c4360e6ea85191696374b4405f5f8" | ||
| "revision": "52ff43b68913047c2a4e0ddb5111a68a428287c0" | ||
| } | ||
| } | ||