Skip to content

Commit 81225ad

Browse files
author
Nikhil Thorat
committed
Merge tensorflow/tfjs-core into the monorepo.
2 parents 55445b4 + 8c2d9e0 commit 81225ad

File tree

471 files changed

+121977
-0
lines changed

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

471 files changed

+121977
-0
lines changed

‎.gitignore

+21
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,5 @@
11
node_modules/
2+
<<<<<<< HEAD
23
package-lock.json
34
npm-debug.log
45
.DS_Store
@@ -10,3 +11,23 @@ yalc.lock
1011
.rpt2_cache/
1112
package
1213
release-notes.md
14+
=======
15+
coverage/
16+
npm-debug.log
17+
yarn-error.log
18+
local.log
19+
.DS_Store
20+
dist/
21+
bazel-out/
22+
.idea/
23+
*.tgz
24+
**/*.pyc
25+
.yalc/
26+
yalc.lock
27+
.rpt2_cache/
28+
package/
29+
*/diff
30+
31+
tfjs-backend-wasm/dist
32+
tfjs-backend-wasm/wasm-out
33+
>>>>>>> tfjs-core/master

‎CONTRIBUTING.md

+34
Original file line numberDiff line numberDiff line change
@@ -15,6 +15,7 @@ You generally only need to submit a CLA once, so if you've already submitted one
1515
(even if it was for a different project), you probably don't need to do it
1616
again.
1717

18+
<<<<<<< HEAD
1819
## Code reviews
1920

2021
All submissions, including submissions by project members, require review. We
@@ -51,3 +52,36 @@ substantial it's better to discuss your design before submitting a PR.
5152
If you're looking for tasks to work on, we also mark issues as ["help wanted"](https://github.com/tensorflow/tfjs/issues?q=is%3Aissue+is%3Aopen+label%3A%22help+wanted%22).
5253

5354

55+
=======
56+
## Adding functionality
57+
58+
One way to ensure that your PR will be accepted is to add functionality that
59+
has been requested in Github issues. If there is something you think is
60+
important and we're missing it but does not show up in Github issues, it would
61+
be good to file an issue there first so we can have the discussion before
62+
sending us a PR.
63+
64+
In general, we're trying to add functionality when driven by use-cases instead of
65+
adding functionality for the sake of parity with TensorFlow python. This will
66+
help us keep the bundle size smaller and have less to maintain especially as we
67+
add new backends.
68+
69+
### Adding an op
70+
71+
When adding ops to the library and deciding whether to write a kernel
72+
implementation in [backend.ts](https://github.com/tensorflow/tfjs-core/blob/master/src/backends/backend.ts),
73+
be sure to check out the TensorFlow ops list [here](https://github.com/tensorflow/tensorflow/blob/master/tensorflow/core/ops/ops.pbtxt).
74+
This list shows the kernels available for the TensorFlow C API. To ensure that
75+
we can bind to this with node.js, we should ensure that our backend.ts
76+
interface matches ops in the TensorFlow C API.
77+
78+
## Code reviews
79+
80+
All submissions, including submissions by project members, require review. We
81+
use GitHub pull requests for this purpose. Consult
82+
[GitHub Help](https://help.github.com/articles/about-pull-requests/) for more
83+
information on using pull requests.
84+
85+
We require unit tests for most code, instructions for running our unit test
86+
suites are in the documentation.
87+
>>>>>>> tfjs-core/master

‎DEVELOPMENT.md

+69
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,6 @@
11
## Development
22

3+
<<<<<<< HEAD
34
This repository contains only the logic and scripts that combine
45
two packages:
56
- [TensorFlow.js Core](https://github.com/tensorflow/tfjs-core),
@@ -73,3 +74,71 @@ This will show up under "Features" as:
7374

7475
This will also show up under "Performance" as:
7576
- Improve matMul CPU speed by 100%. (Improvements to matMul.) (#900). Thanks, @externalcontributor.
77+
=======
78+
To build **TensorFlow.js Core API** from source, we need to clone the project and prepare
79+
the dev environment:
80+
81+
```bash
82+
$ git clone https://github.com/tensorflow/tfjs-core.git
83+
$ cd tfjs-core
84+
$ yarn # Installs dependencies.
85+
```
86+
87+
#### Yarn
88+
We use yarn, and if you are adding or removing dependencies you should use yarn
89+
to keep the `yarn.lock` file up to date.
90+
91+
#### Code editor
92+
We recommend using [Visual Studio Code](https://code.visualstudio.com/) for
93+
development. Make sure to install
94+
[TSLint VSCode extension](https://marketplace.visualstudio.com/items?itemName=ms-vscode.vscode-typescript-tslint-plugin)
95+
and the npm [clang-format](https://github.com/angular/clang-format) `1.2.2` or later
96+
with the
97+
[Clang-Format VSCode extension](https://marketplace.visualstudio.com/items?itemName=xaver.clang-format)
98+
for auto-formatting.
99+
100+
#### Testing
101+
Before submitting a pull request, make sure the code passes all the tests and is clean of lint errors:
102+
103+
```bash
104+
$ yarn test
105+
$ yarn lint
106+
```
107+
108+
To run a subset of tests and/or on a specific browser:
109+
110+
```bash
111+
$ yarn test --browsers=Chrome --grep='multinomial'
112+
 
113+
> ...
114+
> Chrome 62.0.3202 (Mac OS X 10.12.6): Executed 28 of 1891 (skipped 1863) SUCCESS (6.914 secs / 0.634 secs)
115+
```
116+
117+
To run the tests once and exit the karma process (helpful on Windows):
118+
119+
```bash
120+
$ yarn test --single-run
121+
```
122+
123+
To run the tests in an environment that does not have GPU support (such as Chrome Remote Desktop):
124+
125+
```bash
126+
$ yarn test --testEnv cpu
127+
```
128+
129+
Available test environments: cpu, webgl1, webgl2.
130+
131+
#### Packaging (browser and npm)
132+
133+
```bash
134+
$ yarn build-npm
135+
> Stored standalone library at dist/tf-core(.min).js
136+
> Stored also tensorflow-tf-core-VERSION.tgz
137+
```
138+
139+
To install it locally, run `yarn add ./tensorflow-tf-core-VERSION.tgz`.
140+
141+
> On Windows, use bash (available through git) to use the scripts above.
142+
143+
Looking to contribute, and don't know where to start? Check out our "stat:contributions welcome" [issues](https://github.com/tensorflow/tfjs/labels/stat%3Acontributions%20welcome).
144+
>>>>>>> tfjs-core/master

‎ISSUE_TEMPLATE.md

+5
Original file line numberDiff line numberDiff line change
@@ -7,3 +7,8 @@ To get help from the community, we encourage using Stack Overflow and the [`tens
77
#### Describe the problem or feature request
88

99
#### Code to reproduce the bug / link to feature request
10+
If you would like to get help from the community, we encourage using Stack Overflow and the [`tensorflow.js`](https://stackoverflow.com/questions/tagged/tensorflow.js) tag.
11+
12+
GitHub issues for this repository are tracked in the [tfjs union repository](https://github.com/tensorflow/tfjs/issues).
13+
14+
Please file your issue there, following the guidance in [that issue template](https://github.com/tensorflow/tfjs/blob/master/ISSUE_TEMPLATE.md).

‎WORKSPACE

+20
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,20 @@
1+
load("@bazel_tools//tools/build_defs/repo:http.bzl", "http_archive")
2+
http_archive(
3+
name = "build_bazel_rules_nodejs",
4+
sha256 = "88e5e579fb9edfbd19791b8a3c6bfbe16ae3444dba4b428e5efd36856db7cf16",
5+
urls = ["https://github.com/bazelbuild/rules_nodejs/releases/download/0.27.8/rules_nodejs-0.27.8.tar.gz"],
6+
)
7+
8+
load("@build_bazel_rules_nodejs//:defs.bzl", "yarn_install")
9+
yarn_install(
10+
name = "npm",
11+
package_json = "//:package.json",
12+
yarn_lock = "//:yarn.lock",
13+
)
14+
15+
load("@npm//:install_bazel_dependencies.bzl", "install_bazel_dependencies")
16+
install_bazel_dependencies()
17+
18+
# Setup TypeScript toolchain
19+
load("@npm_bazel_typescript//:index.bzl", "ts_setup_workspace")
20+
ts_setup_workspace()

‎cloudbuild.yml

+34
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,34 @@
1+
steps:
2+
# Install top-level deps.
3+
- name: 'node:10'
4+
entrypoint: 'yarn'
5+
id: 'yarn'
6+
args: ['install']
7+
8+
# Run diff to find modified files in each folder.
9+
- name: 'node:10'
10+
entrypoint: 'yarn'
11+
id: 'diff'
12+
args: ['diff']
13+
waitFor: ['yarn']
14+
15+
# Core.
16+
- name: 'gcr.io/cloud-builders/gcloud'
17+
entrypoint: 'bash'
18+
id: 'core'
19+
args: ['./scripts/run-build.sh', 'tfjs-core']
20+
waitFor: ['diff']
21+
22+
# WebGPU.
23+
- name: 'gcr.io/cloud-builders/gcloud'
24+
entrypoint: 'bash'
25+
id: 'webgpu'
26+
args: ['./scripts/run-build.sh', 'tfjs-webgpu']
27+
waitFor: ['diff']
28+
29+
# React Native.
30+
- name: 'gcr.io/cloud-builders/gcloud'
31+
entrypoint: 'bash'
32+
id: 'react-native'
33+
args: ['./scripts/run-build.sh', 'tfjs-react-native']
34+
waitFor: ['diff']

‎package.json

+9
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,9 @@
1+
{
2+
"devDependencies": {
3+
"clang-format": "~1.2.4",
4+
"shelljs": "~0.8.3"
5+
},
6+
"scripts": {
7+
"diff": "./scripts/diff.js"
8+
}
9+
}

‎pull_request_template.md

+3
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,3 @@
1+
To see the logs from the Cloud Build CI, please join either
2+
our [discussion](https://groups.google.com/a/tensorflow.org/forum/#!forum/tfjs)
3+
or [announcement](https://groups.google.com/a/tensorflow.org/forum/#!forum/tfjs-announce) mailing list.

‎scripts/diff.js

+42
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,42 @@
1+
#!/usr/bin/env node
2+
// Copyright 2019 Google LLC. All Rights Reserved.
3+
//
4+
// Licensed under the Apache License, Version 2.0 (the "License");
5+
// you may not use this file except in compliance with the License.
6+
// You may obtain a copy of the License at
7+
//
8+
// http://www.apache.org/licenses/LICENSE-2.0
9+
//
10+
// Unless required by applicable law or agreed to in writing, software
11+
// distributed under the License is distributed on an "AS IS" BASIS,
12+
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
13+
// See the License for the specific language governing permissions and
14+
// limitations under the License.
15+
// =============================================================================
16+
17+
const {exec} = require('./test-util');
18+
const {readdirSync, statSync, writeFileSync} = require('fs');
19+
const {join} = require('path');
20+
21+
const CLONE_PATH = 'clone';
22+
23+
const dirs = readdirSync('.').filter(f => {
24+
return f !== 'node_modules' && f !== '.git' && statSync(f).isDirectory();
25+
});
26+
27+
exec(
28+
`git clone --depth=1 --single-branch ` +
29+
`https://github.com/tensorflow/tfjs-core.git ${CLONE_PATH}`);
30+
31+
32+
dirs.forEach(dir => {
33+
const diffCmd = `diff -rq ${CLONE_PATH}/${dir}/ ./${dir}/`;
34+
const diffOutput = exec(diffCmd, {silent: true}, true).stdout.trim();
35+
36+
if (diffOutput !== '') {
37+
console.log(`${dir} has modified files.`);
38+
writeFileSync(join(dir, 'diff'), diffOutput);
39+
} else {
40+
console.log(`No modified files found in ${dir}`);
41+
}
42+
});

‎scripts/run-build.sh

+22
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,22 @@
1+
#!/usr/bin/env bash
2+
# Copyright 2019 Google LLC. All Rights Reserved.
3+
#
4+
# Licensed under the Apache License, Version 2.0 (the "License");
5+
# you may not use this file except in compliance with the License.
6+
# You may obtain a copy of the License at
7+
#
8+
# http://www.apache.org/licenses/LICENSE-2.0
9+
#
10+
# Unless required by applicable law or agreed to in writing, software
11+
# distributed under the License is distributed on an "AS IS" BASIS,
12+
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
13+
# See the License for the specific language governing permissions and
14+
# limitations under the License.
15+
# =============================================================================
16+
17+
set -e
18+
19+
DIR=$1
20+
if test -f "$DIR/diff"; then
21+
gcloud builds submit . --config=$DIR/cloudbuild.yml
22+
fi

‎scripts/test-util.js

+27
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,27 @@
1+
// Copyright 2019 Google LLC. All Rights Reserved.
2+
//
3+
// Licensed under the Apache License, Version 2.0 (the "License");
4+
// you may not use this file except in compliance with the License.
5+
// You may obtain a copy of the License at
6+
//
7+
// http://www.apache.org/licenses/LICENSE-2.0
8+
//
9+
// Unless required by applicable law or agreed to in writing, software
10+
// distributed under the License is distributed on an "AS IS" BASIS,
11+
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
12+
// See the License for the specific language governing permissions and
13+
// limitations under the License.
14+
// =============================================================================
15+
16+
const shell = require('shelljs');
17+
18+
function exec(command, opt, ignoreCode) {
19+
const res = shell.exec(command, opt);
20+
if (!ignoreCode && res.code !== 0) {
21+
shell.echo('command', command, 'returned code', res.code);
22+
process.exit(1);
23+
}
24+
return res;
25+
}
26+
27+
exports.exec = exec;

‎tfjs-backend-nodegl/.vscode/settings.json

+28
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,28 @@
1+
// Place your settings in this file to overwrite default and user settings.
2+
{
3+
"search.exclude": {
4+
"**/node_modules": true,
5+
"**/coverage/": true,
6+
"**/dist/": true,
7+
"**/yarn.lock": true,
8+
"**/.rpt2_cache/": true,
9+
"**/.yalc/**/*": true
10+
},
11+
"tslint.configFile": "tslint.json",
12+
"files.trimTrailingWhitespace": true,
13+
"editor.tabSize": 2,
14+
"editor.insertSpaces": true,
15+
"[typescript]": {
16+
"editor.formatOnSave": true
17+
},
18+
"[javascript]": {
19+
"editor.formatOnSave": true
20+
},
21+
"editor.rulers": [80],
22+
"clang-format.style": "Google",
23+
"files.insertFinalNewline": true,
24+
"editor.detectIndentation": false,
25+
"editor.wrappingIndent": "none",
26+
"typescript.tsdk": "./node_modules/typescript/lib",
27+
"clang-format.executable": "${workspaceRoot}/node_modules/.bin/clang-format"
28+
}

‎tfjs-backend-nodegl/README.md

+5
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,5 @@
1+
# Headless WebGL backend for TensorFlow.js via Node.js
2+
3+
** This project is under heavy development **
4+
5+
This new backend will provide a light-weight headless WebGL runtime for TensorFlow.js running under Node.js. This new backend is powered by the [node-gles](https://github.com/google/node-gles) module which uses [ANGLE](https://github.com/google/angle) to provide an integration layer to system GL runtime. This package aims to provide a think acceleration engine for IoT, desktop, and Node.js applications where CUDA (size/OS compatibility) is not an option.

0 commit comments

Comments
 (0)