From 300768e70537a52889f9e283465b1fbf8679ebfb Mon Sep 17 00:00:00 2001 From: Ben Clayton Date: Sat, 18 Nov 2023 00:53:05 +0000 Subject: [PATCH] Add documentation for the case cache (#3178) Add `npm run node` helper to sit alongside `npm run standalone` and `npm run wpt`. Update build.md for `out-node`. Co-authored-by: Kai Ninomiya --- docs/build.md | 55 +++++++++++++++++++++++---------- docs/case_cache.md | 76 ++++++++++++++++++++++++++++++++++++++++++++++ package.json | 1 + 3 files changed, 116 insertions(+), 16 deletions(-) create mode 100644 docs/case_cache.md diff --git a/docs/build.md b/docs/build.md index 2d7b2f968c48..d786bbc18c22 100644 --- a/docs/build.md +++ b/docs/build.md @@ -1,16 +1,48 @@ # Building Building the project is not usually needed for local development. -However, for exports to WPT, or deployment (https://gpuweb.github.io/cts/), +However, for exports to WPT, NodeJS, [or deployment](https://gpuweb.github.io/cts/), files can be pre-generated. -The project builds into two directories: +## Build types -- `out/`: Built framework and test files, needed to run standalone or command line. -- `out-wpt/`: Build directory for export into WPT. Contains: - - An adapter for running WebGPU CTS tests under WPT - - A copy of the needed files from `out/` - - A copy of any `.html` test cases from `src/` +The project can be built several different ways, each with a different output directory: + +### 0. on-the-fly builds (no output directory) + +Use `npm run start` to launch a server that live-compiles everything as needed. +Use `tools/run_node` and other tools to run under `ts-node` which compiles at runtime. + +### 1. `out` directory + +**Built with**: `npm run standalone` + +**Serve locally with**: `npx grunt serve` + +**Used for**: Static deployment of the CTS, primarily for [gpuweb.github.io/cts](https://gpuweb.github.io/cts/). + +### 2. `out-wpt` directory + +**Built with**: `npm run wpt` + +**Used for**: Deploying into [Web Platform Tests](https://web-platform-tests.org/). See [below](#export-to-wpt) for more information. + +Contains: + +- An adapter for running WebGPU CTS tests under WPT +- A copy of the needed files from `out/` +- A copy of any `.html` test cases from `src/` + +### 3. `out-node` directory + +**Built with**: `npm run node` + +**Used for**: Running NodeJS tools, if you want to specifically avoid the live-compilation overhead of the `tools/` versions, or are running on a deployment which no longer has access to `ts-node` (which is a build-time dependency). For example: + +- `node out-node/common/runtime/cmdline.js` ([source](../src/common/runtime/cmdline.ts)) - A command line interface test runner +- `node out-node/common/runtime/server.js` ([source](../src/common/runtime/server.ts)) - An HTTP server for executing CTS tests with a REST interface + +## Testing To build and run all pre-submit checks (including type and lint checks and unittests), use: @@ -25,15 +57,6 @@ For checks only: npm run check ``` -For a quicker iterative build: - -```sh -npm run standalone -``` - -## Run - -To serve the built files (rather than using the dev server), run `npx grunt serve`. ## Export to WPT diff --git a/docs/case_cache.md b/docs/case_cache.md new file mode 100644 index 000000000000..8e53834ce473 --- /dev/null +++ b/docs/case_cache.md @@ -0,0 +1,76 @@ +# Case Cache + +The WebGPU CTS contains many tests that check that the results of an operation +fall within limits defined by the WebGPU and WGSL specifications. The +computation of these allowed limits can be very expensive to calculate, however +the values do not vary by platform or device, and can be precomputed and reused +for multiple CTS runs. + +## File cache + +To speed up execution of the CTS, the CTS git repo holds holds pre-computed +test cases, serialized in a set of gzip-compressed binary files under +[`src/resources/cache`](../src/resources/cache). + +These files are regenerated by [`src/common/tools/gen_cache.ts`](../src/common/tools/gen_cache.ts) +which can be run with `npx grunt run:generate-cache`. +This tool is automatically run by the various Grunt build commands. + +As generating the cache is expensive (hence why we build it ahead of time!) the +cache generation tool will only re-build the cache files it believes may be out +of date. To determine which files it needs to rebuild, the tool calculates a +hash of all the transitive source TypeScript files that are used to build the +output, and compares this hash to the hash stored in +[`src/resources/cache/hashes.json`](`../src/resources/cache/hashes.json`). Only +those cache files with differing hashes are rebuilt. + +Since source changes will sometimes change the hash without changing the generated cache, +sometimes the cache will be regenerated unnecessarily. **This is OK, but try to avoid committing +no-op regenerations - this will happen if your version of Node produces different gzip outputs +than the original committer's Node did for the same input.** + +The cache files are copied from [`src/resources/cache`](../src/resources/cache) +to the `resources/cache` subdirectory of the +[`out` and `out-node` build directories](build.md#build-types), so the runner +can load these cache files. + +The GitHub presubmit checks will error if the cache files or +[`hashes.json`](`../src/resources/cache/hashes.json`) need updating. + +## In memory cache + +If a cache file cannot be found, then the [`CaseCache`](../src/webgpu/shader/execution/expression/case_cache.ts) +will build the cases during CTS execution and store the results in an in-memory LRU cache. + +## Using the cache + +To add test cases to the cache: + +1. Import `makeCaseCache` from [`'case_cache.js'`](../src/webgpu/shader/execution/expression/case_cache.ts); + +```ts +import { makeCaseCache } from '../case_cache.js'; // your relative path may vary +``` + +2. Declare an exported global variable with the name `d`, assigned with the return value of `makeCaseCache()`: + +```ts +export const d = makeCaseCache('unique/path/of/your/cache/file', { + // Declare any number of fields that build the test cases + name_of_your_case: () => { + return fullI32Range().map(e => { // example case builder + return { input: i32(e), expected: i32(-e) }; + }); + }, +}); +``` + +3. To load the cases from the cache, use `d.get();` + +```ts +const cases = await d.get('name_of_your_case'); +// cases will either be loaded from the cache file, loaded from the in-memory +// LRU, or built on the fly. +``` + +4. Run `npx grunt run generate-cache` to generate the new cache file. diff --git a/package.json b/package.json index 10432f343a46..608d0318f28d 100644 --- a/package.json +++ b/package.json @@ -7,6 +7,7 @@ "all": "grunt all", "standalone": "grunt standalone", "wpt": "grunt wpt", + "node": "grunt node", "checks": "grunt checks", "unittest": "grunt unittest", "typecheck": "grunt typecheck",