From 94bd729e1db1f5c39eadbe6911f11139979fc339 Mon Sep 17 00:00:00 2001 From: Joshua Bell Date: Tue, 28 Jan 2025 20:35:23 -0800 Subject: [PATCH 1/4] Ensure object creation specifies the realm (#810) * Ensure object creation specifies the realm "Realm" is an ECMAScript concept best explained in https://html.spec.whatwg.org/multipage/webappapis.html#realms-and-their-counterparts Newly created JS objects must be associated with a Realm; while older specs didn't do this explicitly, best practice is to be explicit about this, especially for steps running "in parallel", or in algorithms separate from method steps. Do so! This also adds lint tests to try and catch future violations. Note that dictionaries (e.g. MLOperatorDescriptor) are Infra "ordered maps" it the body of spec algorithms, not JS objects, so they don't have a realm. Conversion to a JS object when returning a dictionary to script is handled by WebIDL bindings logic. Also note that DOMExceptions, either thrown or as promise rejection values, are not given a realm. This is a known issue across all web specs and is tracked in whatwg/webidl#135. Resolves #793. * Don't double-init realm; and don't need realm for dicts * Variable name improvement from @fdwr --- index.bs | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/index.bs b/index.bs index fb3c1f55..0696b09d 100644 --- a/index.bs +++ b/index.bs @@ -1044,7 +1044,7 @@ Note: `dispatch()` itself provides no signal that graph execution has completed. 'C': outputTensorC }; context.dispatch(graph, inputs, outputs); - + // 6. Read back the computed result. const result = await context.readTensor(outputTensorC); console.log('Output value:', new Float32Array(result)); // [1, 1, 1, 1] From 7ce5a4239a2900c6a9a664119fceef4a00d573c9 Mon Sep 17 00:00:00 2001 From: Zoltan Kis Date: Mon, 27 Jan 2025 18:27:42 +0200 Subject: [PATCH 2/4] Remove MLDeviceType, fixing #749 Signed-off-by: Zoltan Kis Improve text on device selection Co-authored-by: Joshua Bell Improve text on fingerprinting Co-authored-by: Joshua Bell Improve text on underlying device selection Co-authored-by: Joshua Bell Address review comments from Ningxin Fix RFC2119 issue for Note Signed-off-by: Zoltan Kis --- index.bs | 43 +++++++++++-------------------------------- 1 file changed, 11 insertions(+), 32 deletions(-) diff --git a/index.bs b/index.bs index 0696b09d..1683cbdc 100644 --- a/index.bs +++ b/index.bs @@ -669,11 +669,11 @@ Unlike WebGPU, this API does not intrinsically support custom shader authoring; The WebGPU API identifies machine-specific artifacts as a privacy consideration. Similarly, the WebNN API's compute unit scheduling may under certain circumstances introduce a fingerprint. However, similarly to WebGPU, such fingerprints are identical across most or all of the devices of each vendor, mitigating the concern. Furthermore, software implementations can be used to further eliminate such artifacts. -The WebNN API defines two developer-settable preferences to help inform [[#programming-model-device-selection]] and allow the implementation to better select the most appropriate underlying execution device for the workload. An {{MLDeviceType}} normatively indicates the kind of device and is one of: {{MLDeviceType/"cpu"}}, {{MLDeviceType/"gpu"}}, {{MLDeviceType/"npu"}}. If this type cannot be satisfied, an "{{OperationError}}" {{DOMException}} is thrown, thus this type can in some cases add two bits of entropy to the fingerprint. An {{MLPowerPreference}} indicates preference as related to the power consumption and is considered a hint only and as such does not increase entropy of the fingerprint. +The WebNN API defines developer-settable preferences to help inform [[#programming-model-device-selection]] and allow the implementation to better select the underlying execution device for the workload. An {{MLPowerPreference}} indicates preference as related to the desired low power consumption or high performance, is considered a hint only and as such does not increase entropy of the fingerprint. Issue(623): {{MLContextOptions}} is under active development, and the design is expected to change, informed by further implementation experience and new use cases from the wider web community. -If a future version of this specification introduces support for a new {{MLDeviceType}} that can only support a subset of {{MLOperandDataType}}s, that could introduce a new fingerprint. +If a future version of this specification introduces support for a new {{MLContextOptions}} member for supporting only a subset of {{MLOperandDataType}}s, that could introduce a new fingerprint. In general, implementers of this API are expected to apply WebGPU Privacy Considerations to their implementations where applicable. @@ -729,7 +729,13 @@ An {{MLContext}} interface represents a global state of neural network execution In a situation when a GPU context executes a graph with a constant or an input in the system memory as an {{ArrayBufferView}}, the input content is automatically uploaded from the system memory to the GPU memory, and downloaded back to the system memory of an {{ArrayBufferView}} output buffer at the end of the graph execution. This data upload and download cycles will only occur whenever the execution device requires the data to be copied out of and back into the system memory, such as in the case of the GPU. It doesn't occur when the device is a CPU device. Additionally, the result of the graph execution is in a known layout format. While the execution may be optimized for a native memory access pattern in an intermediate result within the graph, the output of the last operation of the graph must convert the content back to a known layout format at the end of the graph in order to maintain the expected behavior from the caller's perspective. -When an {{MLContext}} is created with {{MLContextOptions}}, the user agent selects and creates the underlying execution device by taking into account the application's {{MLPowerPreference}} and {{MLDeviceType}} options. +When an {{MLContext}} is created with {{MLContextOptions}}, the user agent selects and creates the underlying execution device by taking into account these options, currently only the {{MLPowerPreference}} option. + +
+Depending on the underlying platform, the user agent may select different combinations of CPU, NPU and GPU devices. +
+ +For a history and rationale of this design, please see the device selection explainer. ## Task Source ## {#programming-model-task-source} @@ -764,12 +770,6 @@ WorkerNavigator includes NavigatorML; ## {{ML}} interface ## {#api-ml}