- **Called on:** {{MLGraph}} |this|.
+
+ **Called on:** {{MLExecution}} |this|.
**Arguments:**
-
- |inputs|: an {{MLNamedInputs}}. The resources and optional dimensions of inputs for the compute.
- |outputs|: an {{MLNamedOutputs}}. The pre-allocated resources of required outputs for the compute.
+
+ |graph|: an {{MLGraph}}. The compiled graph to be executed.
+ |inputs|: an {{MLNamedArrayInputs}}. The resources and optional dimensions of inputs.
+ |outputs|: an {{MLNamedArrayOutputs}}. The pre-allocated resources of required outputs.
**Returns:** {{undefined}}.
@@ -2185,8 +2227,120 @@ interface MLGraph {
1. If any of the following requirements are unmet, then throw a {{DataError}} {{DOMException}} and stop.
1. For each |key| -> |value| of |inputs|:
- 1. |this|.{{MLGraph/[[inputDescriptors]]}}[|key|] must exist.
- 1. Let |inputDesc| be |this|.{{MLGraph/[[inputDescriptors]]}}[|key|].
+ 1. |graph|.{{MLGraph/[[inputDescriptors]]}}[|key|] must exist.
+ 1. Let |inputDesc| be |graph|.{{MLGraph/[[inputDescriptors]]}}[|key|].
+ 1. Let |inputSize| be 1.
+ 1. If |value| is an {{MLArrayInput}}, then:
+ 1. The length of |value|.{{MLArrayInput/dimensions}} must be the same as the length of |inputDesc|.{{MLOperandDescriptor/dimensions}}.
+ 1. Let |i| be 0.
+ 1. While true:
+ 1. Let |dimension| be |value|.{{MLArrayInput/dimensions}}[|i|].
+ 1. |dimension| must be greater than 0.
+ 1. If |inputDesc|.{{MLOperandDescriptor/dimensions}}[|i|] is greater than 0, then |dimension| must be equal to |inputDesc|.{{MLOperandDescriptor/dimensions}}[|i|].
+ 1. Set |inputSize| to the product of |inputSize| and |dimension|.
+ 1. Increment |i| by 1.
+ 1. If |i| if equal to the length of |value|.{{MLArrayInput/dimensions}}, then break.
+ 1. Else:
+ 1. For each |dimension| of |inputDesc|.{{MLOperandDescriptor/dimensions}}:
+ 1. The value of |dimension| must be greater than 0.
+ 1. Set |inputSize| to the product of |inputSize| and |dimension|.
+ 1. If |value| is an {{MLArrayInput}}, then let |resource| be |value|.{{MLArrayInput/resource}}.
+ 1. If |value| is an {{ArrayBufferView}}, then let |resource| be |value|.
+ 1. If |resource| is an {{ArrayBufferView}}, then:
+ 1. The kind of |resource| must be compatible with |inputDesc|.{{MLOperandDescriptor/type}} according to [this table](#appendices-mloperandtype-arraybufferview-compatibility).
+ 1. The length of |resource| must be the same as |inputSize|.
+
+ 1. For each |key| -> |value| of |outputs|:
+ 1. |graph|.{{MLGraph/[[outputNames]]}}[|key|] must exist.
+
+
+ 1. For each |key| -> |value| of |inputs|:
+ 1. Let |inputDesc| be |graph|.{{MLGraph/[[inputDescriptors]]}}[|key|].
+ 1. Let |inputTensor| be a new tensor for |graph|.{{MLGraph/[[implementation]]}} of data type that is compatible with |inputDesc|.{{MLOperandDescriptor/type}}.
+ 1. If |value| is an {{MLArrayInput}}, then:
+ 1. Set the dimensions of |inputTensor| to |value|.{{MLArrayInput/dimensions}}.
+ 1. Else:
+ 1. Set the dimensions of |inputTensor| to |inputDesc|.{{MLOperandDescriptor/dimensions}}.
+ 1. If |value| is an {{MLArrayInput}}, then:
+ 1. Set the values of |inputTensor| to the values of |value|.{{MLArrayInput/resource}}.
+ 1. If |value| is an {{ArrayBufferView}}, then:
+ 1. Set the values of |inputTensor| to the values of |value|.
+ 1. Set the input of |graph|.{{MLGraph/[[implementation]]}} that is associated with |key| to |inputTensor|.
+ 1. For each |key| -> |value| of |outputs|:
+ 1. Issue a compute request for output of |graph|.{{MLGraph/[[implementation]]}} that is associated with |key|.
+ 1. Wait for the compute request to be completed.
+ 1. If there is an error returned by |graph|.{{MLGraph/[[implementation]]}}, then:
+ 1. Throw an {{OperationError}} {{DOMException}} and stop.
+ 1. Else:
+ 1. Let |outputTensor| be the output tensor returned by |graph|.{{MLGraph/[[implementation]]}}.
+ 1. If the kind of |value| is not compatible with the value type of |outputTensor|, then throw a {{DataError}} {{DOMException}} and stop.
+ 1. Let |outputSize| be 1.
+ 1. For each |dimension| of dimensions of |outputTensor|:
+ 1. Set |outputSize| to the product of |outputSize| and |dimension|.
+ 1. If |outputSize| is greater than the length of |value|, then:
+ 1. Throw a {{DataError}} {{DOMException}} and stop.
+ 1. Else:
+ 1. Set the values of |value| to the values of |outputTensor|.
+ 1. Return {{undefined}}.
+
+
+
+## MLAwaitedExecution ## {#api-mlawaitedexecution}
+The {{MLAwaitedExecution}} interface represents a method of execution that asynchronously carries out the computational workload of a compiled graph {{MLGraph}} on a worker thread to avoid blocking the calling thread while producing results as defined by the operations in the graph. This method of execution requires an {{MLContext}} created with {{MLContextOptions}} or {{WebGLRenderingContext}}.
+
+
+
+{{MLAwaitedExecution}} has the following internal slots:
+
+
+ : \[[context]] of type {{MLContext}}
+ ::
+ The context of type {{MLContext}} associated with this {{MLAwaitedExecution}}.
+
+ : \[[implementation]]
+ ::
+ The underlying implementation provided by the User Agent.
+
+
+
+ : compute(graph, inputs, outputs)
+ ::
+ Compute the {{MLGraph}} given {{MLNamedInputs}} and {{MLNamedOutputs}}. Return immediately without blocking the calling thread. The Promise<{{MLNamedOutputs}}> return value will be signaled when the computation is completed on the worker thread and that the results in {{MLNamedOutputs}} are ready to be consumed.
+
+
+ **Called on:** {{MLAwaitedExecution}} |this|.
+
+ **Arguments:**
+
+ |graph|: an {{MLGraph}}. The compiled graph to be executed.
+ |inputs|: an {{MLNamedInputs}}. The resources and optional dimensions of inputs.
+ |outputs|: an {{MLNamedOutputs}}. The pre-allocated resources of required outputs.
+
+
+ **Returns:** Promise<{{MLNamedOutputs}}>.
+
+ 1. If any of the following requirements are unmet, then throw a {{DataError}} {{DOMException}} and stop.
+
+ 1. For each |key| -> |value| of |inputs|:
+ 1. |graph|.{{MLGraph/[[inputDescriptors]]}}[|key|] must exist.
+ 1. Let |inputDesc| be |graph|.{{MLGraph/[[inputDescriptors]]}}[|key|].
1. Let |inputSize| be 1.
1. If |value| is an {{MLInput}}, then:
1. The length of |value|.{{MLInput/dimensions}} must be the same as the length of |inputDesc|.{{MLOperandDescriptor/dimensions}}.
@@ -2209,12 +2363,12 @@ interface MLGraph {
1. The length of |resource| must be the same as |inputSize|.
1. For each |key| -> |value| of |outputs|:
- 1. |this|.{{MLGraph/[[outputNames]]}}[|key|] must exist.
+ 1. |graph|.{{MLGraph/[[outputNames]]}}[|key|] must exist.
1. For each |key| -> |value| of |inputs|:
- 1. Let |inputDesc| be |this|.{{MLGraph/[[inputDescriptors]]}}[|key|].
- 1. Let |inputTensor| be a new tensor for |this|.{{MLGraph/[[implementation]]}} of data type that is compatible with |inputDesc|.{{MLOperandDescriptor/type}}.
+ 1. Let |inputDesc| be |graph|.{{MLGraph/[[inputDescriptors]]}}[|key|].
+ 1. Let |inputTensor| be a new tensor for |graph|.{{MLGraph/[[implementation]]}} of data type that is compatible with |inputDesc|.{{MLOperandDescriptor/type}}.
1. If |value| is an {{MLInput}}, then:
1. Set the dimensions of |inputTensor| to |value|.{{MLInput/dimensions}}.
1. Else:
@@ -2223,14 +2377,143 @@ interface MLGraph {
1. Set the values of |inputTensor| to the values of |value|.{{MLInput/resource}}.
1. If |value| is an {{MLResource}}, then:
1. Set the values of |inputTensor| to the values of |value|.
- 1. Set the input of |this|.{{MLGraph/[[implementation]]}} that is associated with |key| to |inputTensor|.
+ 1. Set the input of |graph|.{{MLGraph/[[implementation]]}} that is associated with |key| to |inputTensor|.
+ 1. For each |key| -> |value| of |outputs|:
+ 1. Issue a compute request for output of |graph|.{{MLGraph/[[implementation]]}} that is associated with |key|.
+ 1. Wait for the compute request to be completed.
+ 1. If there is an error returned by |graph|.{{MLGraph/[[implementation]]}}, then:
+ 1. Throw an {{OperationError}} {{DOMException}} and stop.
+ 1. Else:
+ 1. Let |outputTensor| be the output tensor returned by |graph|.{{MLGraph/[[implementation]]}}.
+ 1. If the kind of |value| is not compatible with the value type of |outputTensor|, then throw a {{DataError}} {{DOMException}} and stop.
+ 1. Let |outputSize| be 1.
+ 1. For each |dimension| of dimensions of |outputTensor|:
+ 1. Set |outputSize| to the product of |outputSize| and |dimension|.
+ 1. If |outputSize| is greater than the length of |value|, then:
+ 1. Throw a {{DataError}} {{DOMException}} and stop.
+ 1. Else:
+ 1. Set the values of |value| to the values of |outputTensor|.
+ 1. Return Promise<{{MLNamedOutputs}}>.
+
+
+## MLCommandEncoder ## {#api-mlcommandencoder}
+The {{MLCommandEncoder}} interface represents a method of execution that synchronously records the computational workload of a compiled graph {{MLGraph}} to a GPU command buffer {{GPUCommandBuffer}} on the calling thread. Since the workload is not immediately executed, just recorded, this method allows more flexibility for the caller to determine how and when the recorded commands will be submitted for execution on the GPU relative to other GPU workload on the same queue. This method of execution requires an {{MLContext}} created with {{GPUDevice}}.
+
+
+
+{{MLCommandEncoder}} has the following internal slots:
+
+
+ : \[[context]] of type {{MLContext}}
+ ::
+ The context of type {{MLContext}} associated with this {{MLCommandEncoder}}.
+
+ : \[[implementation]]
+ ::
+ The underlying implementation provided by the User Agent.
+
+
+
+ : initializeGraph(graph, inputs)
+ ::
+ Record the initialization of the graph {{MLGraph}} on the GPU command buffer {{GPUCommandBuffer}} with constant inputs {{MLNamedGPUInputs}} such as weight inputs. This is a necessary step for optimal performance as it allows the underlying platform an opportunity to prepare and optimize constant input data for the following execution of computational wordloads on the queue. It should only be done once per graph.
+
+
+ **Called on:** {{MLCommandEncoder}} |this|.
+
+ **Arguments:**
+
+ |graph|: an {{MLGraph}}. The compiled graph to be executed.
+ |inputs|: an {{MLNamedGPUInputs}}. The resources and optional dimensions of constant inputs.
+
+
+ **Returns:** {{undefined}}.
+
+ **Called on:** {{MLCommandEncoder}} |this|.
+
+ **Arguments:**
+
+ |graph|: an {{MLGraph}}. The compiled graph to be executed.
+ |inputs|: an {{MLNamedGPUInputs}}. The resources and optional dimensions of inputs.
+ |outputs|: an {{MLNamedGPUOutputs}}. The pre-allocated resources of required outputs.
+
+
+ **Returns:** {{undefined}}.
+
+ 1. If any of the following requirements are unmet, then throw a {{DataError}} {{DOMException}} and stop.
+
+ 1. For each |key| -> |value| of |inputs|:
+ 1. |graph|.{{MLGraph/[[inputDescriptors]]}}[|key|] must exist.
+ 1. Let |inputDesc| be |graph|.{{MLGraph/[[inputDescriptors]]}}[|key|].
+ 1. Let |inputSize| be 1.
+ 1. If |value| is an {{MLGPUInput}}, then:
+ 1. The length of |value|.{{MLGPUInput/dimensions}} must be the same as the length of |inputDesc|.{{MLOperandDescriptor/dimensions}}.
+ 1. Let |i| be 0.
+ 1. While true:
+ 1. Let |dimension| be |value|.{{MLGPUInput/dimensions}}[|i|].
+ 1. |dimension| must be greater than 0.
+ 1. If |inputDesc|.{{MLOperandDescriptor/dimensions}}[|i|] is greater than 0, then |dimension| must be equal to |inputDesc|.{{MLOperandDescriptor/dimensions}}[|i|].
+ 1. Set |inputSize| to the product of |inputSize| and |dimension|.
+ 1. Increment |i| by 1.
+ 1. If |i| if equal to the length of |value|.{{MLGPUInput/dimensions}}, then break.
+ 1. Else:
+ 1. For each |dimension| of |inputDesc|.{{MLOperandDescriptor/dimensions}}:
+ 1. The value of |dimension| must be greater than 0.
+ 1. Set |inputSize| to the product of |inputSize| and |dimension|.
+ 1. If |value| is an {{MLGPUInput}}, then let |resource| be |value|.{{MLGPUInput/resource}}.
+ 1. If |value| is an {{MLGPUResource}}, then let |resource| be |value|.
+
+ 1. For each |key| -> |value| of |outputs|:
+ 1. |graph|.{{MLGraph/[[outputNames]]}}[|key|] must exist.
+
+
+ 1. For each |key| -> |value| of |inputs|:
+ 1. Let |inputDesc| be |graph|.{{MLGraph/[[inputDescriptors]]}}[|key|].
+ 1. Let |inputTensor| be a new tensor for |graph|.{{MLGraph/[[implementation]]}} of data type that is compatible with |inputDesc|.{{MLOperandDescriptor/type}}.
+ 1. If |value| is an {{MLGPUInput}}, then:
+ 1. Set the dimensions of |inputTensor| to |value|.{{MLGPUInput/dimensions}}.
+ 1. Else:
+ 1. Set the dimensions of |inputTensor| to |inputDesc|.{{MLOperandDescriptor/dimensions}}.
+ 1. If |value| is an {{MLGPUInput}}, then:
+ 1. Set the values of |inputTensor| to the values of |value|.{{MLGPUInput/resource}}.
+ 1. If |value| is an {{MLGPUResource}}, then:
+ 1. Set the values of |inputTensor| to the values of |value|.
+ 1. Set the input of |graph|.{{MLGraph/[[implementation]]}} that is associated with |key| to |inputTensor|.
1. For each |key| -> |value| of |outputs|:
- 1. Issue a compute request for output of |this|.{{MLGraph/[[implementation]]}} that is associated with |key|.
+ 1. Issue a compute request for output of |graph|.{{MLGraph/[[implementation]]}} that is associated with |key|.
1. Wait for the compute request to be completed.
- 1. If there is an error returned by |this|.{{MLGraph/[[implementation]]}}, then:
+ 1. If there is an error returned by |graph|.{{MLGraph/[[implementation]]}}, then:
1. Throw an {{OperationError}} {{DOMException}} and stop.
1. Else:
- 1. Let |outputTensor| be the output tensor returned by |this|.{{MLGraph/[[implementation]]}}.
+ 1. Let |outputTensor| be the output tensor returned by |graph|.{{MLGraph/[[implementation]]}}.
1. If the kind of |value| is not compatible with the value type of |outputTensor|, then throw a {{DataError}} {{DOMException}} and stop.
1. Let |outputSize| be 1.
1. For each |dimension| of dimensions of |outputTensor|:
@@ -2240,8 +2523,21 @@ interface MLGraph {
1. Else:
1. Set the values of |value| to the values of |outputTensor|.
1. Return {{undefined}}.
+
+ **Called on:** {{MLCommandEncoder}} |this|.
+
+ **Arguments:**
+
+ |descriptor|: an {{GPUCommandBufferDescriptor}}. Descriptor of the command buffer.
+
- Issue: Describe the algorithm steps for |this|.{{MLGraph/[[context]]}} created from {{WebGLRenderingContext}} and {{GPUDevice}}.
+ **Returns:** {{GPUCommandBuffer}}.
@@ -2266,6 +2562,9 @@ const b = builder.input('b', descB);
const c = builder.matmul(a, b);
const graph = builder.build({'c': c});
+// Create a graph execution method
+const execution = new MLExecution(context);
+
function allocateAndCompute(shapeA, shapeB, shapeC) {
const bufferA = new Float32Array(sizeOfShape(shapeA)).fill(0.5);
const bufferB = new Float32Array(sizeOfShape(shapeB)).fill(0.5);
@@ -2277,7 +2576,7 @@ function allocateAndCompute(shapeA, shapeB, shapeC) {
'b': {resource: bufferB, dimensions: shapeB},
};
const outputs = {'c': bufferC};
- graph.compute(inputs, outputs);
+ execution.compute(graph, inputs, outputs);
console.log(`values: ${bufferC}`);
}
@@ -2306,17 +2605,20 @@ const d = builder.matmul(a, b);
const e = builder.add(d, c);
const graph = builder.build({'d': d, 'e': e});
+// Create a graph execution method
+const execution = new MLExecution(context);
+
const bufferA = new Float32Array(sizeOfShape(descA.dimensions)).fill(0.5);
const inputs = {'a': bufferA};
// Compute d.
const bufferD = new Float32Array(sizeOfShape([3, 3]));
-graph.compute(inputs, {'d': bufferD});
+execution.compute(graph, inputs, {'d': bufferD});
console.log(`values: ${bufferD}`);
// Compute e.
const bufferE = new Float32Array(sizeOfShape([3, 3]));
-graph.compute(inputs, {'e': bufferE});
+execution.compute(graph, inputs, {'e': bufferE});
console.log(`values: ${bufferE}`);
The following code executes the compiled graph.
@@ -2399,7 +2709,7 @@ const inputs = {
'input2': inputBuffer2,
};
const outputs = {'output': outputBuffer};
-graph.compute(inputs, outputs);
+execution.compute(graph, inputs, outputs);
console.log('Output value: ' + outputBuffer);
// Output value: 2.25,2.25,2.25,2.25,2.25,2.25,2.25,2.25