Skip to content

Commit

Permalink
Add documentation for the case cache (#3178)
Browse files Browse the repository at this point in the history
Add `npm run node` helper to sit alongside `npm run standalone` and `npm run wpt`.

Update build.md for `out-node`.

Co-authored-by: Kai Ninomiya <[email protected]>
  • Loading branch information
ben-clayton and kainino0x authored Nov 18, 2023
1 parent 8ee974a commit 300768e
Show file tree
Hide file tree
Showing 3 changed files with 116 additions and 16 deletions.
55 changes: 39 additions & 16 deletions docs/build.md
Original file line number Diff line number Diff line change
@@ -1,16 +1,48 @@
# Building

Building the project is not usually needed for local development.
However, for exports to WPT, or deployment (https://gpuweb.github.io/cts/),
However, for exports to WPT, NodeJS, [or deployment](https://gpuweb.github.io/cts/),
files can be pre-generated.

The project builds into two directories:
## Build types

- `out/`: Built framework and test files, needed to run standalone or command line.
- `out-wpt/`: Build directory for export into WPT. Contains:
- An adapter for running WebGPU CTS tests under WPT
- A copy of the needed files from `out/`
- A copy of any `.html` test cases from `src/`
The project can be built several different ways, each with a different output directory:

### 0. on-the-fly builds (no output directory)

Use `npm run start` to launch a server that live-compiles everything as needed.
Use `tools/run_node` and other tools to run under `ts-node` which compiles at runtime.

### 1. `out` directory

**Built with**: `npm run standalone`

**Serve locally with**: `npx grunt serve`

**Used for**: Static deployment of the CTS, primarily for [gpuweb.github.io/cts](https://gpuweb.github.io/cts/).

### 2. `out-wpt` directory

**Built with**: `npm run wpt`

**Used for**: Deploying into [Web Platform Tests](https://web-platform-tests.org/). See [below](#export-to-wpt) for more information.

Contains:

- An adapter for running WebGPU CTS tests under WPT
- A copy of the needed files from `out/`
- A copy of any `.html` test cases from `src/`

### 3. `out-node` directory

**Built with**: `npm run node`

**Used for**: Running NodeJS tools, if you want to specifically avoid the live-compilation overhead of the `tools/` versions, or are running on a deployment which no longer has access to `ts-node` (which is a build-time dependency). For example:

- `node out-node/common/runtime/cmdline.js` ([source](../src/common/runtime/cmdline.ts)) - A command line interface test runner
- `node out-node/common/runtime/server.js` ([source](../src/common/runtime/server.ts)) - An HTTP server for executing CTS tests with a REST interface

## Testing

To build and run all pre-submit checks (including type and lint checks and
unittests), use:
Expand All @@ -25,15 +57,6 @@ For checks only:
npm run check
```

For a quicker iterative build:

```sh
npm run standalone
```

## Run

To serve the built files (rather than using the dev server), run `npx grunt serve`.

## Export to WPT

Expand Down
76 changes: 76 additions & 0 deletions docs/case_cache.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,76 @@
# Case Cache

The WebGPU CTS contains many tests that check that the results of an operation
fall within limits defined by the WebGPU and WGSL specifications. The
computation of these allowed limits can be very expensive to calculate, however
the values do not vary by platform or device, and can be precomputed and reused
for multiple CTS runs.

## File cache

To speed up execution of the CTS, the CTS git repo holds holds pre-computed
test cases, serialized in a set of gzip-compressed binary files under
[`src/resources/cache`](../src/resources/cache).

These files are regenerated by [`src/common/tools/gen_cache.ts`](../src/common/tools/gen_cache.ts)
which can be run with `npx grunt run:generate-cache`.
This tool is automatically run by the various Grunt build commands.

As generating the cache is expensive (hence why we build it ahead of time!) the
cache generation tool will only re-build the cache files it believes may be out
of date. To determine which files it needs to rebuild, the tool calculates a
hash of all the transitive source TypeScript files that are used to build the
output, and compares this hash to the hash stored in
[`src/resources/cache/hashes.json`](`../src/resources/cache/hashes.json`). Only
those cache files with differing hashes are rebuilt.

Since source changes will sometimes change the hash without changing the generated cache,
sometimes the cache will be regenerated unnecessarily. **This is OK, but try to avoid committing
no-op regenerations - this will happen if your version of Node produces different gzip outputs
than the original committer's Node did for the same input.**

The cache files are copied from [`src/resources/cache`](../src/resources/cache)
to the `resources/cache` subdirectory of the
[`out` and `out-node` build directories](build.md#build-types), so the runner
can load these cache files.

The GitHub presubmit checks will error if the cache files or
[`hashes.json`](`../src/resources/cache/hashes.json`) need updating.

## In memory cache

If a cache file cannot be found, then the [`CaseCache`](../src/webgpu/shader/execution/expression/case_cache.ts)
will build the cases during CTS execution and store the results in an in-memory LRU cache.

## Using the cache

To add test cases to the cache:

1. Import `makeCaseCache` from [`'case_cache.js'`](../src/webgpu/shader/execution/expression/case_cache.ts);

```ts
import { makeCaseCache } from '../case_cache.js'; // your relative path may vary
```

2. Declare an exported global variable with the name `d`, assigned with the return value of `makeCaseCache()`:

```ts
export const d = makeCaseCache('unique/path/of/your/cache/file', {
// Declare any number of fields that build the test cases
name_of_your_case: () => {
return fullI32Range().map(e => { // example case builder
return { input: i32(e), expected: i32(-e) };
});
},
});
```

3. To load the cases from the cache, use `d.get();`

```ts
const cases = await d.get('name_of_your_case');
// cases will either be loaded from the cache file, loaded from the in-memory
// LRU, or built on the fly.
```

4. Run `npx grunt run generate-cache` to generate the new cache file.
1 change: 1 addition & 0 deletions package.json
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,7 @@
"all": "grunt all",
"standalone": "grunt standalone",
"wpt": "grunt wpt",
"node": "grunt node",
"checks": "grunt checks",
"unittest": "grunt unittest",
"typecheck": "grunt typecheck",
Expand Down

0 comments on commit 300768e

Please sign in to comment.