From 42bc5db85dd94fbaf4d3724120c02013c00172ba Mon Sep 17 00:00:00 2001 From: Anssi Kostiainen Date: Tue, 11 Oct 2022 15:39:33 +0300 Subject: [PATCH 01/11] Syncronize content with the initial charter [1] [1] https://www.w3.org/2021/04/web-machine-learning-charter.html --- charter.html | 30 +++++++++++------------------- 1 file changed, 11 insertions(+), 19 deletions(-) diff --git a/charter.html b/charter.html index a159a11..7556757 100644 --- a/charter.html +++ b/charter.html @@ -96,7 +96,12 @@

[DRAFT] Web Machine Learning Working Group Charter

-

This is draft charter for a new W3C Web Machine Learning Working Group for discussion. It has no formal standing.

+

+ This draft charter is available on GitHub. Feel free + to raise issues. +

The mission of the Web Machine Learning Working Group is to develop APIs for enabling efficient machine @@ -108,12 +113,6 @@

Machine Learning Working Group.

-

- This proposed charter is available on GitHub. Feel free - to raise issues. -

@@ -253,14 +252,7 @@

More detailed milestones and updated publication schedules are available on the group publication - status page. -

-

- Draft state indicates the state of the deliverable at the time - of the charter approval. Expected completion indicates when - the deliverable is projected to become a Recommendation, or otherwise - reach a stable state. + "https://www.w3.org/groups/wg/webmachinelearning">group home page.

@@ -672,16 +664,16 @@

From 6864d30682c3a59116e74ace00b72345bc706893 Mon Sep 17 00:00:00 2001 From: Anssi Kostiainen Date: Tue, 11 Oct 2022 15:43:47 +0300 Subject: [PATCH 02/11] Remove Khronos from External Organizations This reflects WG decision to drop WebGL interoperability requirements --- charter.html | 12 ------------ 1 file changed, 12 deletions(-) diff --git a/charter.html b/charter.html index 7556757..2318b5a 100644 --- a/charter.html +++ b/charter.html @@ -471,18 +471,6 @@

External Organizations

-
- Khronos -
-
- The Khronos WebGL Working Group defines a WebGL API that supports - the OpenGL ES Shading Language (GLSL). The GLSL can be used to - implement traditional machine learning algorithms efficiently. - Furthermore, this group incubates a proposal for a WebGL 2.0 - Compute specification that aims to provide an efficient way to - run general-purpose GPU (GPGPU) computing workloads such as - machine learning algorithms in the WebGL context. -
ECMA TC39
TC39 defines the JavaScript language whose primitives are key in how WebNN access data (e.g. ArrayBuffer). Possible work on operator overloading would also impact possible evolutions of the WebNN API.
From cc83e97298dd30dc5d3af30701ab646ee69284c8 Mon Sep 17 00:00:00 2001 From: Anssi Kostiainen Date: Thu, 8 Dec 2022 11:03:30 +0200 Subject: [PATCH 03/11] Clarify Out of Scope "base linear algebra operations are out of scope" refers to the Basic Linear Algebra Subprograms (BLAS) interface. Related #20 --- charter.html | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-) diff --git a/charter.html b/charter.html index 2318b5a..e76c159 100644 --- a/charter.html +++ b/charter.html @@ -236,11 +236,11 @@

algorithms.

- To avoid overlap with existing work, generic primitives used by - traditional machine learning algorithms such as base linear algebra - operations are out of scope. The WebGL and WebGPU shaders and - WebAssembly SIMD are expected to address these requirements, see - the Coordination section for details. + To avoid overlap with existing work, alignment with the Basic + Linear Algebra Subprograms (BLAS) interface is out of scope. The + WebGL and WebGPU shaders and WebAssembly SIMD are expected to + address the BLAS compatibility requirement, see the Coordination + section for details.

    From 4e781e32f86f2a12080591331e7c0bf078278925 Mon Sep 17 00:00:00 2001 From: Anssi Kostiainen Date: Thu, 8 Dec 2022 11:09:03 +0200 Subject: [PATCH 04/11] Note WebGPU interoperability expectation in Coordination Fix #20 --- charter.html | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/charter.html b/charter.html index e76c159..2e3341a 100644 --- a/charter.html +++ b/charter.html @@ -445,7 +445,8 @@

    The GPU for the Web Working Group defines a WebGPU Shading Language that may be used to implement traditional machine learning algorithms efficiently. The Web Machine Learning Working - Group should coordinate with this group to avoid overlap. + Group should coordinate with this group to avoid overlap and + to enable interoperability between the WebNN and WebGPU APIs.
    From 592643d20cf0733b159785e06f76c233ee4733e5 Mon Sep 17 00:00:00 2001 From: Anssi Kostiainen Date: Thu, 8 Dec 2022 11:13:57 +0200 Subject: [PATCH 05/11] Clarify WebRTC coordination Confidence level increased. Fix #21 --- charter.html | 6 +++++- 1 file changed, 5 insertions(+), 1 deletion(-) diff --git a/charter.html b/charter.html index 2e3341a..3703c72 100644 --- a/charter.html +++ b/charter.html @@ -462,7 +462,11 @@

    to avoid overlap.
    WebRTC Working Group
    -
    The WebRTC Working Group defines the MediaStream interface and related media processing APIs that likely make use of Machine Learning capabilities afforded by the WebNN API.
    +
    + The WebRTC Working Group defines the MediaStream + interface and related media processing APIs that enable integration + with Machine Learning capabilities afforded by the WebNN API. +
    Technical Architecture Group
    Given the well-known ethical risks of bias in the use of Machine Learning, the Web Machine Learning Working Group will work with the Technical Architecture Group to ensure its work align with the W3C TAG Ethical Web Principles.

    From 1bb533369280902ea1776c3dbb6880d65e82cf1a Mon Sep 17 00:00:00 2001 From: Anssi Kostiainen Date: Thu, 8 Dec 2022 11:24:59 +0200 Subject: [PATCH 06/11] Add high-level to low-level op decompose path These informative definitions will facilitate progressive lowering by implementations. Fix #22 Related #28 --- charter.html | 5 +++++ 1 file changed, 5 insertions(+) diff --git a/charter.html b/charter.html index 3703c72..4b855a8 100644 --- a/charter.html +++ b/charter.html @@ -220,6 +220,11 @@

    macOS/iOS Metal Performance Shaders and Basic Neural Network Subroutines.

    +

    + For each high-level building block that decomposes into well-known + lower-level operations, the APIs will informatively define a generic + emulation path to allow for future extensibility. +

    It may also work on a higher-level API to load a custom pre-trained Machine Learning model for inference in the browser.

    From dad4b0537191009cb88366d9764082c5f2f02561 Mon Sep 17 00:00:00 2001 From: Anssi Kostiainen Date: Thu, 8 Dec 2022 11:43:54 +0200 Subject: [PATCH 07/11] Expand Motivation and Background and clarify priorities in Scope Fix #23 --- charter.html | 14 ++++++++++++++ 1 file changed, 14 insertions(+) diff --git a/charter.html b/charter.html index 4b855a8..93ce058 100644 --- a/charter.html +++ b/charter.html @@ -174,6 +174,15 @@

    Motivation and Background

    +

    + Computer Vision enables computers to gain understanding from images + or videos, Natural Language Processing enables interaction between + computers and human languages, and Speech Recognition enables + computers to recognize and translate spoken language into text. + Bringing these experiences to the web in a privacy-preserving manner + requires efficient machine learning inference capabilities built into + the browser. +

    Enabling Machine Learning inference in the browser (as opposed e.g. to in the cloud) enhances privacy, since input data such as locally sourced images or video streams stay within the @@ -213,6 +222,11 @@

    and retrieve the output when hardware execution completes. +

    + This Working Group puts priority on building blocks required by + well-known model architectures in the fields of Computer Vision, + Natural Language Processing and Speech Recognition. +

    The APIs in scope of this group will not be tied to any particular platform and will be implementable on top of existing major platform From c06abcac911a105bd1eb7c0ec7eb3ef96fdd3607 Mon Sep 17 00:00:00 2001 From: Anssi Kostiainen Date: Thu, 8 Dec 2022 11:57:54 +0200 Subject: [PATCH 08/11] Add OpenXLA Project to Coordination Fix #28 --- charter.html | 12 ++++++++++++ 1 file changed, 12 insertions(+) diff --git a/charter.html b/charter.html index 93ce058..ea1000f 100644 --- a/charter.html +++ b/charter.html @@ -498,6 +498,18 @@

    ECMA TC39
    TC39 defines the JavaScript language whose primitives are key in how WebNN access data (e.g. ArrayBuffer). Possible work on operator overloading would also impact possible evolutions of the WebNN API.
    +
    +
    OpenXLA Project
    +
    + OpenXLA Project develops StableHLO, a portable ML compute + operation set that makes frameworks easier to deploy across + different hardware. WebNN API targets diverse hardware platforms + and defines an operation set whose high-level operations can be + decomposed to low level primitives that can map to StableHLO + operations. Coordination and alignment between these two + operation sets is beneficial to the open ML ecosystem. +
    +

    From 965c9defa4d62d2c4734e74f247ee0de18da9c15 Mon Sep 17 00:00:00 2001 From: Anssi Kostiainen Date: Fri, 9 Dec 2022 15:12:12 +0200 Subject: [PATCH 09/11] Address review comments - Update Expected completion date of WebNN - Remove now redundant Timeline section - Link to Ethical Principles for Web Machine Learning WG Note - Add Adopted Draft and Exclusion Draft placeholders - Update WebNN and Model Loader Draft state --- charter.html | 47 ++++++++++++++++++++++++++--------------------- 1 file changed, 26 insertions(+), 21 deletions(-) diff --git a/charter.html b/charter.html index ea1000f..3159ee9 100644 --- a/charter.html +++ b/charter.html @@ -283,8 +283,7 @@

    - Web Neural - Network API + Web Neural Network API

    @@ -292,12 +291,23 @@

    inference that can take advantage of hardware acceleration.

    - Draft state: Adopted from Web - Machine Learning Community Group + Draft state: Working Draft +

    +

    + Adopted Draft: The title, stable URL, + and publication date of the + Adopted Draft which will serve as the basis for + work on the deliverable. +

    +

    + Exclusion Draft: The title, stable + URL, and publication date of the most recent + Exclusion Draft.

    - Expected completion: [CR Q1 2022] + Expected completion: Q1 2025

    @@ -326,7 +336,8 @@

    Draft state: Explainer + "https://webmachinelearning.github.io/model-loader/">Adopted from Web + Machine Learning Community Group

    Expected completion: [N/A] @@ -341,7 +352,14 @@

    Other Deliverables

    -

    The Working Group will develop a Working Group Note documenting ethical issues associated with using Machine Learning on the Web, to help identify what mitigations its normative specifications should take into account.

    +

    + The Working Group develops Ethical + Principles for Web Machine Learning Working Group Note + documenting ethical issues associated with using Machine Learning + on the Web, to help identify what mitigations its normative + specifications should take into account. +

    Other non-normative documents may be created such as:

    @@ -355,19 +373,6 @@

    -
    -

    - Timeline -

    -
      -
    • Q2 2021: First teleconference -
      • -
    • Q2 2021: FPWD for Web Neural Network API -
      • -
    • Q2 2022: CR for Web Neural Network API -
      • -
    -

    From 33e2eb59d47a8fcc6d722dbd9c1475c663a2900c Mon Sep 17 00:00:00 2001 From: Anssi Kostiainen Date: Fri, 16 Dec 2022 11:06:21 +0200 Subject: [PATCH 10/11] Address review comments - Remove reference to WebGL from the BLAS compatibility context - Declare interop between the WebNN and WebGL APIs out of scope --- charter.html | 9 ++++++--- 1 file changed, 6 insertions(+), 3 deletions(-) diff --git a/charter.html b/charter.html index 3159ee9..11c0958 100644 --- a/charter.html +++ b/charter.html @@ -257,9 +257,12 @@

    To avoid overlap with existing work, alignment with the Basic Linear Algebra Subprograms (BLAS) interface is out of scope. The - WebGL and WebGPU shaders and WebAssembly SIMD are expected to - address the BLAS compatibility requirement, see the Coordination - section for details. + WebGPU shaders and WebAssembly SIMD are expected to address the + BLAS compatibility requirement, see the Coordination section for + details. +

    +

    + Interoperability between the WebNN and WebGL APIs is out of scope.

      From 183f792b2c97ce1f0d80ff69ed1d5c46f50e4d68 Mon Sep 17 00:00:00 2001 From: Anssi Kostiainen Date: Mon, 9 Jan 2023 11:45:01 +0200 Subject: [PATCH 11/11] Clarify Scope Move examples of well-known model architectures from the bullet list into the text section that talks about priority use cases. Add transformers as another example. Grammar fix: s/Allow/Allows Fix #23 --- charter.html | 19 ++++++++++--------- 1 file changed, 10 insertions(+), 9 deletions(-) diff --git a/charter.html b/charter.html index 11c0958..51de72c 100644 --- a/charter.html +++ b/charter.html @@ -208,24 +208,25 @@

      This Web API for neural network inference hardware acceleration:

        -
      • Allow to construct a neural network computational graph by common - building blocks, including constant values and base operations such - as convolution, pooling, softmax, normalization, fully connected, - activation, recurrent neural network (RNN) and long short-term memory - (LSTM); +
      • Allows to construct a neural network computational graph by common + building blocks required by well-known model architectures: constant + values and base operations such as convolution, pooling, softmax, + normalization, fully connected, and activation;
        • -
      • Allow to compile the neural network to native optimized format +
      • Allows to compile the neural network to native optimized format for hardware execution;
        • -
      • Allow to setup input from various sources on the Web, e.g. array +
      • Allows to setup input from various sources on the Web, e.g. array buffers, media streams, schedule the asynchronous hardware execution, and retrieve the output when hardware execution completes.

      This Working Group puts priority on building blocks required by - well-known model architectures in the fields of Computer Vision, - Natural Language Processing and Speech Recognition. + well-known model architectures such as recurrent neural network + (RNN), long short-term memory (LSTM) and transformers in the fields + of Computer Vision, Natural Language Processing and Speech + Recognition.

      The APIs in scope of this group will not be tied to any particular

      - Initial Charter + Initial Charter - [dd monthname yyyy] + 2021-04-20 - [dd monthname yyyy] + 2023-04-30 - none + Initial charter