diff --git a/README.md b/README.md index f34468df..858c0903 100644 --- a/README.md +++ b/README.md @@ -7,6 +7,10 @@ JavaScript related workloads required by Web Developers nowadays. It's the explicit goal not to measure I/O or other non-JavaScript related tasks. +See the [in-depth +analysis](https://github.com/bmeurer/web-tooling-benchmark/blob/master/docs/in-depth.md) +for a detailed description of the tests included in this benchmark suite. + ## Philosophy I love the web! I love working on [Chromium](http://www.chromium.org) - more @@ -42,41 +46,6 @@ JavaScript heavy workloads from the most common tools used by the web platform. Specifically I/O and other non-JavaScript related aspects are excluded from the measurement. -## Test suite - -The test suite currently contains: - -- A test to stress the [Babe](https://github.com/babel/babel) - transformation logic using the `es2015` preset on a 196K - ES2015 module containing the untranspiled [Vue](https://github.com/vuejs/vue) - source code. -- A test to stress the [Babylon](https://github.com/babel/babylon) - syntax analysis library on different common inputs, i.e. on - the [jQuery](https://jquery.com) source code. Babylon is the - JavaScript parser used in [Babel](https://github.com/babel/babel) - and is built on top of [Acorn](https://github.com/ternjs/acorn). - It's representative for a whole class of popular parsing worklads. -- A test to stress the [Buble](https://github.com/Rich-Harris/buble) - ES2015 compiler, both the parser and the actual transformation - logic, on the same 196K ES2015 module containing the untranspiled - [Vue](https://github.com/vuejs/vue) source code that is also used - to stress by the `babel` and `babylon` tests. -- A test to stress the [Chai Assertion Library](http://chaijs.com), - which is commonly used to write unit and integration tests. -- A test to stress the [Lebab](https://github.com/lebab/lebab) - ES5 to ES6/ES7 transpiler, modernizing the preact bundle. -- A test to stress the [Prettier](https://github.com/prettier/prettier) - code formatter with various `.js` and `.jsx` inputs. -- A test to stress the [source-map](https://github.com/mozilla/source-map) - tool on both parsing and serializing source maps. -- A test to stress the [TypeScript](https://github.com/Microsoft/TypeScript) - compiler on the [`typescript-angular`](https://github.com/tastejs/todomvc/tree/master/examples/typescript-angular) - example from [todomvc](https://github.com/tastejs/todomvc). -- A test to stress the [UglifyJS3](https://github.com/mishoo/UglifyJS2) - minifier, running on the (concatenated) JavaScript source for - the ES2015 test in the [Speedometer](https://browserbench.org/Speedometer) - 2.0 benchmark. - ## Building ``` diff --git a/docs/in-depth.md b/docs/in-depth.md new file mode 100644 index 00000000..50b00db3 --- /dev/null +++ b/docs/in-depth.md @@ -0,0 +1,80 @@ +# In-depth Analysis + +The Web Tooling Benchmark contains interesting JavaScript workloads from a variety of +commonly used web developer tools. It executes these core workloads a couple of times +reports a single score at the end that balances them using geometric mean. + +Note that scores of different versions of the Web Tooling Benchmark are not comparable +to each other. + +## babel + +[Babel](https://github.com/babel/babel) is a transpiler that compiles modern JavaScipt +(i.e. ES2015 and later) to an older JavaScript dialect (i.e. ES3 or ES5) that is understood +by a broad set of browsers. It's probably the most popular transpiler at this point in time +and used in the vast majority of web projects nowadays. + +This benchmark runs the Babel transformation logic using the `es2015` preset on a 196KiB +ES2015 module containing the untranspiled [Vue](https://github.com/vuejs/vue) bundle. +Note that this explicitly excludes the [Babylon](https://github.com/babel/babylon) parser +and only measures the throughput on the actual transformations. The parser is tested +separately by the `babylon` benchmark below. + +## babylon + +[Babylon](https://github.com/babel/babylon) is the frontend for the Babel transpiler and +deals with transforming the JavaScript input into a so-called AST (Abstract Syntax Tree) +that can be consumed by the core transformation logic in Babel. It is built on top of +[Acorn](https://github.com/ternjs/acorn) and thus representative of a whole class of +common parsing workloads. + +This benchmark runs the Babylon parser on different popular inputs, including the +[Preact](https://github.com/developit/preact) 8.2.5 bundle, the [lodash](https://lodash.com) +4.17.4 bundle, the JSX files from the React TodoMVC example app, the +[underscore](http://underscorejs.org/) 1.8.3 bundle, and an ES2015 module containing the untranspiled [Vue](https://github.com/vuejs/vue) bundle + +## buble + +A test to stress the [Buble](https://github.com/Rich-Harris/buble) +ES2015 compiler, both the parser and the actual transformation +logic, on the same 196K ES2015 module containing the untranspiled +[Vue](https://github.com/vuejs/vue) source code that is also used +to stress by the `babel` and `babylon` tests. + +## chai + +A test to stress the [Chai Assertion Library](http://chaijs.com), +which is commonly used to write unit and integration tests. + +## lebab + +A test to stress the [Lebab](https://github.com/lebab/lebab) +ES5 to ES6/ES7 transpiler, modernizing the preact bundle. + +## prettier + +A test to stress the [Prettier](https://github.com/prettier/prettier) +code formatter with various `.js` and `.jsx` inputs. + +## source-map + +A test to stress the [source-map](https://github.com/mozilla/source-map) +tool on both parsing and serializing source maps. + +## typescript + +A test to stress the [TypeScript](https://github.com/Microsoft/TypeScript) +compiler on the [`typescript-angular`](https://github.com/tastejs/todomvc/tree/master/examples/typescript-angular) +example from [todomvc](https://github.com/tastejs/todomvc). + +## uglify-js + +A test to stress the [UglifyJS3](https://github.com/mishoo/UglifyJS2) +minifier, running on the (concatenated) JavaScript source for +the ES2015 test in the [Speedometer](https://browserbench.org/Speedometer) +2.0 benchmark. + +## uglify-es + +This benchmark stresses the new ES2015 and beyond minifier using the 196KiB +ES2015 module containing the untranspiled [Vue](https://github.com/vuejs/vue) bundle. \ No newline at end of file diff --git a/package-lock.json b/package-lock.json index 914a3b57..cbbc52fb 100644 --- a/package-lock.json +++ b/package-lock.json @@ -274,12 +274,24 @@ "readable-stream": "2.3.3" } }, + "bluebird": { + "version": "3.5.1", + "resolved": "https://registry.npmjs.org/bluebird/-/bluebird-3.5.1.tgz", + "integrity": "sha512-MKiLiV+I1AA596t9w1sQJ8jkiSr5+ZKi0WKrYGUn6d1Fx+Ij4tIj+m2WMQSGczs5jZVxV339chE8iwk6F64wjA==", + "dev": true + }, "bn.js": { "version": "4.11.8", "resolved": "https://registry.npmjs.org/bn.js/-/bn.js-4.11.8.tgz", "integrity": "sha512-ItfYfPLkWHUjckQCk8xC+LwxgK8NYcXywGigJgSwOP8Y2iyWT4f2vsZnoOXTTbo+o5yXmIUJ4gn5538SO5S3gA==", "dev": true }, + "boolbase": { + "version": "1.0.0", + "resolved": "https://registry.npmjs.org/boolbase/-/boolbase-1.0.0.tgz", + "integrity": "sha1-aN/1++YMUes3cl6p4+0xDcwed24=", + "dev": true + }, "brace-expansion": { "version": "1.1.8", "resolved": "https://registry.npmjs.org/brace-expansion/-/brace-expansion-1.1.8.tgz", @@ -427,6 +439,16 @@ "integrity": "sha1-hZgoeOIbmOHGZCXgPQF0eI9Wnug=", "dev": true }, + "camel-case": { + "version": "3.0.0", + "resolved": "https://registry.npmjs.org/camel-case/-/camel-case-3.0.0.tgz", + "integrity": "sha1-yjw2iKTpzzpM2nd9xNy8cTJJz3M=", + "dev": true, + "requires": { + "no-case": "2.3.2", + "upper-case": "1.1.3" + } + }, "camelcase": { "version": "1.2.1", "resolved": "https://registry.npmjs.org/camelcase/-/camelcase-1.2.1.tgz", @@ -518,6 +540,15 @@ "safe-buffer": "5.1.1" } }, + "clean-css": { + "version": "4.1.9", + "resolved": "https://registry.npmjs.org/clean-css/-/clean-css-4.1.9.tgz", + "integrity": "sha1-Nc7ornaHpJuYA09w3gDE7dOCYwE=", + "dev": true, + "requires": { + "source-map": "0.5.7" + } + }, "cli-cursor": { "version": "1.0.2", "resolved": "https://registry.npmjs.org/cli-cursor/-/cli-cursor-1.0.2.tgz", @@ -619,6 +650,51 @@ "integrity": "sha1-wguW2MYXdIqvHBYCF2DNJ/y4y3U=", "dev": true }, + "copy-webpack-plugin": { + "version": "4.1.1", + "resolved": "https://registry.npmjs.org/copy-webpack-plugin/-/copy-webpack-plugin-4.1.1.tgz", + "integrity": "sha512-qcjV9uj5PFuKo9GDr0xYAZ3DwFA3ugwDcfbLHfiDrvnUx66Z7C4r00/ds856GaGb2cGHqLTwrGxwfvW+lgAQew==", + "dev": true, + "requires": { + "bluebird": "3.5.1", + "fs-extra": "4.0.2", + "glob": "7.1.2", + "is-glob": "4.0.0", + "loader-utils": "0.2.17", + "lodash": "4.17.4", + "minimatch": "3.0.4", + "node-dir": "0.1.17" + }, + "dependencies": { + "is-extglob": { + "version": "2.1.1", + "resolved": "https://registry.npmjs.org/is-extglob/-/is-extglob-2.1.1.tgz", + "integrity": "sha1-qIwCU1eR8C7TfHahueqXc8gz+MI=", + "dev": true + }, + "is-glob": { + "version": "4.0.0", + "resolved": "https://registry.npmjs.org/is-glob/-/is-glob-4.0.0.tgz", + "integrity": "sha1-lSHHaEXMJhCoUgPd8ICpWML/q8A=", + "dev": true, + "requires": { + "is-extglob": "2.1.1" + } + }, + "loader-utils": { + "version": "0.2.17", + "resolved": "https://registry.npmjs.org/loader-utils/-/loader-utils-0.2.17.tgz", + "integrity": "sha1-+G5jdNQyBabmxg6RlvF8Apm/s0g=", + "dev": true, + "requires": { + "big.js": "3.2.0", + "emojis-list": "2.1.0", + "json5": "0.5.1", + "object-assign": "4.1.1" + } + } + } + }, "core-js": { "version": "2.5.1", "resolved": "https://registry.npmjs.org/core-js/-/core-js-2.5.1.tgz", @@ -710,6 +786,24 @@ "randombytes": "2.0.5" } }, + "css-select": { + "version": "1.2.0", + "resolved": "https://registry.npmjs.org/css-select/-/css-select-1.2.0.tgz", + "integrity": "sha1-KzoRBTnFNV8c2NMUYj6HCxIeyFg=", + "dev": true, + "requires": { + "boolbase": "1.0.0", + "css-what": "2.1.0", + "domutils": "1.5.1", + "nth-check": "1.0.1" + } + }, + "css-what": { + "version": "2.1.0", + "resolved": "https://registry.npmjs.org/css-what/-/css-what-2.1.0.tgz", + "integrity": "sha1-lGfQMsOM+u+58teVASUwYvh/ob0=", + "dev": true + }, "d": { "version": "1.0.0", "resolved": "https://registry.npmjs.org/d/-/d-1.0.0.tgz", @@ -775,12 +869,72 @@ "randombytes": "2.0.5" } }, + "dom-converter": { + "version": "0.1.4", + "resolved": "https://registry.npmjs.org/dom-converter/-/dom-converter-0.1.4.tgz", + "integrity": "sha1-pF71cnuJDJv/5tfIduexnLDhfzs=", + "dev": true, + "requires": { + "utila": "0.3.3" + }, + "dependencies": { + "utila": { + "version": "0.3.3", + "resolved": "https://registry.npmjs.org/utila/-/utila-0.3.3.tgz", + "integrity": "sha1-1+jn1+MJEHCSsF+NloiCTWM6QiY=", + "dev": true + } + } + }, + "dom-serializer": { + "version": "0.1.0", + "resolved": "https://registry.npmjs.org/dom-serializer/-/dom-serializer-0.1.0.tgz", + "integrity": "sha1-BzxpdUbOB4DOI75KKOKT5AvDDII=", + "dev": true, + "requires": { + "domelementtype": "1.1.3", + "entities": "1.1.1" + }, + "dependencies": { + "domelementtype": { + "version": "1.1.3", + "resolved": "https://registry.npmjs.org/domelementtype/-/domelementtype-1.1.3.tgz", + "integrity": "sha1-vSh3PiZCiBrsUVRJJCmcXNgiGFs=", + "dev": true + } + } + }, "domain-browser": { "version": "1.1.7", "resolved": "https://registry.npmjs.org/domain-browser/-/domain-browser-1.1.7.tgz", "integrity": "sha1-hnqksJP6oF8d4IwG9NeyH9+GmLw=", "dev": true }, + "domelementtype": { + "version": "1.3.0", + "resolved": "https://registry.npmjs.org/domelementtype/-/domelementtype-1.3.0.tgz", + "integrity": "sha1-sXrtguirWeUt2cGbF1bg/BhyBMI=", + "dev": true + }, + "domhandler": { + "version": "2.1.0", + "resolved": "https://registry.npmjs.org/domhandler/-/domhandler-2.1.0.tgz", + "integrity": "sha1-0mRvXlf2w7qxHPbLBdPArPdBJZQ=", + "dev": true, + "requires": { + "domelementtype": "1.3.0" + } + }, + "domutils": { + "version": "1.5.1", + "resolved": "https://registry.npmjs.org/domutils/-/domutils-1.5.1.tgz", + "integrity": "sha1-3NhIiib1Y9YQeeSMn3t+Mjc2gs8=", + "dev": true, + "requires": { + "dom-serializer": "0.1.0", + "domelementtype": "1.3.0" + } + }, "elegant-spinner": { "version": "1.0.1", "resolved": "https://registry.npmjs.org/elegant-spinner/-/elegant-spinner-1.0.1.tgz", @@ -828,6 +982,12 @@ "tapable": "0.2.8" } }, + "entities": { + "version": "1.1.1", + "resolved": "https://registry.npmjs.org/entities/-/entities-1.1.1.tgz", + "integrity": "sha1-blwtClYhtdra7O+AuQ7ftc13cvA=", + "dev": true + }, "errno": { "version": "0.1.4", "resolved": "https://registry.npmjs.org/errno/-/errno-0.1.4.tgz", @@ -1090,6 +1250,17 @@ "for-in": "1.0.2" } }, + "fs-extra": { + "version": "4.0.2", + "resolved": "https://registry.npmjs.org/fs-extra/-/fs-extra-4.0.2.tgz", + "integrity": "sha1-+RcExT0bRh+JNFKwwwfZmXZHq2s=", + "dev": true, + "requires": { + "graceful-fs": "4.1.11", + "jsonfile": "4.0.0", + "universalify": "0.1.1" + } + }, "fs.realpath": { "version": "1.0.0", "resolved": "https://registry.npmjs.org/fs.realpath/-/fs.realpath-1.0.0.tgz", @@ -2113,6 +2284,12 @@ "minimalistic-assert": "1.0.0" } }, + "he": { + "version": "1.1.1", + "resolved": "https://registry.npmjs.org/he/-/he-1.1.1.tgz", + "integrity": "sha1-k0EP0hsAlzUVH4howvJx80J+I/0=", + "dev": true + }, "hmac-drbg": { "version": "1.0.1", "resolved": "https://registry.npmjs.org/hmac-drbg/-/hmac-drbg-1.0.1.tgz", @@ -2130,6 +2307,97 @@ "integrity": "sha512-pNgbURSuab90KbTqvRPsseaTxOJCZBD0a7t+haSN33piP9cCM4l0CqdzAif2hUqm716UovKB2ROmiabGAKVXyg==", "dev": true }, + "html-minifier": { + "version": "3.5.5", + "resolved": "https://registry.npmjs.org/html-minifier/-/html-minifier-3.5.5.tgz", + "integrity": "sha512-g+1+NBycQI0fGnggd52JM8TRUweG7+9W2wrtjGitMAqc4G7maweAHvVAAjz9veHseIH3tYKE2lk2USGSoewIrQ==", + "dev": true, + "requires": { + "camel-case": "3.0.0", + "clean-css": "4.1.9", + "commander": "2.11.0", + "he": "1.1.1", + "ncname": "1.0.0", + "param-case": "2.1.1", + "relateurl": "0.2.7", + "uglify-js": "3.1.3" + } + }, + "html-webpack-plugin": { + "version": "2.30.1", + "resolved": "https://registry.npmjs.org/html-webpack-plugin/-/html-webpack-plugin-2.30.1.tgz", + "integrity": "sha1-f5xCG36pHsRg9WUn1430hO51N9U=", + "dev": true, + "requires": { + "bluebird": "3.5.1", + "html-minifier": "3.5.5", + "loader-utils": "0.2.17", + "lodash": "4.17.4", + "pretty-error": "2.1.1", + "toposort": "1.0.6" + }, + "dependencies": { + "loader-utils": { + "version": "0.2.17", + "resolved": "https://registry.npmjs.org/loader-utils/-/loader-utils-0.2.17.tgz", + "integrity": "sha1-+G5jdNQyBabmxg6RlvF8Apm/s0g=", + "dev": true, + "requires": { + "big.js": "3.2.0", + "emojis-list": "2.1.0", + "json5": "0.5.1", + "object-assign": "4.1.1" + } + } + } + }, + "htmlparser2": { + "version": "3.3.0", + "resolved": "https://registry.npmjs.org/htmlparser2/-/htmlparser2-3.3.0.tgz", + "integrity": "sha1-zHDQWln2VC5D8OaFyYLhTJJKnv4=", + "dev": true, + "requires": { + "domelementtype": "1.3.0", + "domhandler": "2.1.0", + "domutils": "1.1.6", + "readable-stream": "1.0.34" + }, + "dependencies": { + "domutils": { + "version": "1.1.6", + "resolved": "https://registry.npmjs.org/domutils/-/domutils-1.1.6.tgz", + "integrity": "sha1-vdw94Jm5ou+sxRxiPyj0FuzFdIU=", + "dev": true, + "requires": { + "domelementtype": "1.3.0" + } + }, + "isarray": { + "version": "0.0.1", + "resolved": "https://registry.npmjs.org/isarray/-/isarray-0.0.1.tgz", + "integrity": "sha1-ihis/Kmo9Bd+Cav8YDiTmwXR7t8=", + "dev": true + }, + "readable-stream": { + "version": "1.0.34", + "resolved": "https://registry.npmjs.org/readable-stream/-/readable-stream-1.0.34.tgz", + "integrity": "sha1-Elgg40vIQtLyqq+v5MKRbuMsFXw=", + "dev": true, + "requires": { + "core-util-is": "1.0.2", + "inherits": "2.0.3", + "isarray": "0.0.1", + "string_decoder": "0.10.31" + } + }, + "string_decoder": { + "version": "0.10.31", + "resolved": "https://registry.npmjs.org/string_decoder/-/string_decoder-0.10.31.tgz", + "integrity": "sha1-YuIDvEF2bGwoyfyEMB2rHFMQ+pQ=", + "dev": true + } + } + }, "https-browserify": { "version": "0.0.1", "resolved": "https://registry.npmjs.org/https-browserify/-/https-browserify-0.0.1.tgz", @@ -2440,6 +2708,15 @@ "integrity": "sha1-Hq3nrMASA0rYTiOWdn6tn6VJWCE=", "dev": true }, + "jsonfile": { + "version": "4.0.0", + "resolved": "https://registry.npmjs.org/jsonfile/-/jsonfile-4.0.0.tgz", + "integrity": "sha1-h3Gq4HmbZAdrdmQPygWPnBDjPss=", + "dev": true, + "requires": { + "graceful-fs": "4.1.11" + } + }, "jsonify": { "version": "0.0.0", "resolved": "https://registry.npmjs.org/jsonify/-/jsonify-0.0.0.tgz", @@ -2741,6 +3018,12 @@ "integrity": "sha1-MKCy2jj3N3DoKUoNIuZiXtd9AJc=", "dev": true }, + "lower-case": { + "version": "1.1.4", + "resolved": "https://registry.npmjs.org/lower-case/-/lower-case-1.1.4.tgz", + "integrity": "sha1-miyr0bno4K6ZOkv31YdcOcQujqw=", + "dev": true + }, "lru-cache": { "version": "4.1.1", "resolved": "https://registry.npmjs.org/lru-cache/-/lru-cache-4.1.1.tgz", @@ -2892,11 +3175,38 @@ "resolved": "https://registry.npmjs.org/nan/-/nan-2.6.2.tgz", "integrity": "sha1-5P805slf37WuzAjeZZb0NgWn20U=" }, + "ncname": { + "version": "1.0.0", + "resolved": "https://registry.npmjs.org/ncname/-/ncname-1.0.0.tgz", + "integrity": "sha1-W1etGLHKCShk72Kwse2BlPODtxw=", + "dev": true, + "requires": { + "xml-char-classes": "1.0.0" + } + }, + "no-case": { + "version": "2.3.2", + "resolved": "https://registry.npmjs.org/no-case/-/no-case-2.3.2.tgz", + "integrity": "sha512-rmTZ9kz+f3rCvK2TD1Ue/oZlns7OGoIWP4fc3llxxRXlOkHKoWPPWJOfFYpITabSow43QJbRIoHQXtt10VldyQ==", + "dev": true, + "requires": { + "lower-case": "1.1.4" + } + }, "node-abi": { "version": "2.1.1", "resolved": "https://registry.npmjs.org/node-abi/-/node-abi-2.1.1.tgz", "integrity": "sha512-6oxV13poCOv7TfGvhsSz6XZWpXeKkdGVh72++cs33OfMh3KAX8lN84dCvmqSETyDXAFcUHtV7eJrgFBoOqZbNQ==" }, + "node-dir": { + "version": "0.1.17", + "resolved": "https://registry.npmjs.org/node-dir/-/node-dir-0.1.17.tgz", + "integrity": "sha1-X1Zl2TNRM1yqvvjxxVRRbPXx5OU=", + "dev": true, + "requires": { + "minimatch": "3.0.4" + } + }, "node-libs-browser": { "version": "github:bmeurer/node-libs-browser#66da7bda97003b07f86efa53236a609f9cec2bf0", "dev": true, @@ -3000,6 +3310,15 @@ "set-blocking": "2.0.0" } }, + "nth-check": { + "version": "1.0.1", + "resolved": "https://registry.npmjs.org/nth-check/-/nth-check-1.0.1.tgz", + "integrity": "sha1-mSms32KPwsQQmN6rgqxYDPFJquQ=", + "dev": true, + "requires": { + "boolbase": "1.0.0" + } + }, "number-is-nan": { "version": "1.0.1", "resolved": "https://registry.npmjs.org/number-is-nan/-/number-is-nan-1.0.1.tgz", @@ -3101,6 +3420,15 @@ "integrity": "sha1-8/dSL073gjSNqBYbrZ7P1Rv4OnU=", "dev": true }, + "param-case": { + "version": "2.1.1", + "resolved": "https://registry.npmjs.org/param-case/-/param-case-2.1.1.tgz", + "integrity": "sha1-35T9jPZTHs915r75oIWPvHK+Ikc=", + "dev": true, + "requires": { + "no-case": "2.3.2" + } + }, "parse-asn1": { "version": "5.1.0", "resolved": "https://registry.npmjs.org/parse-asn1/-/parse-asn1-5.1.0.tgz", @@ -3242,6 +3570,16 @@ "resolved": "https://registry.npmjs.org/prettier/-/prettier-1.7.4.tgz", "integrity": "sha1-XoYkrpNjyA+V7GRFhOzfVddPk/o=" }, + "pretty-error": { + "version": "2.1.1", + "resolved": "https://registry.npmjs.org/pretty-error/-/pretty-error-2.1.1.tgz", + "integrity": "sha1-X0+HyPkeWuPzuoerTPXgOxoX8aM=", + "dev": true, + "requires": { + "renderkid": "2.0.1", + "utila": "0.4.0" + } + }, "pretty-format": { "version": "21.2.1", "resolved": "https://registry.npmjs.org/pretty-format/-/pretty-format-21.2.1.tgz", @@ -3481,12 +3819,39 @@ "is-equal-shallow": "0.1.3" } }, + "relateurl": { + "version": "0.2.7", + "resolved": "https://registry.npmjs.org/relateurl/-/relateurl-0.2.7.tgz", + "integrity": "sha1-VNvzd+UUQKypCkzSdGANP/LYiKk=", + "dev": true + }, "remove-trailing-separator": { "version": "1.1.0", "resolved": "https://registry.npmjs.org/remove-trailing-separator/-/remove-trailing-separator-1.1.0.tgz", "integrity": "sha1-wkvOKig62tW8P1jg1IJJuSN52O8=", "dev": true }, + "renderkid": { + "version": "2.0.1", + "resolved": "https://registry.npmjs.org/renderkid/-/renderkid-2.0.1.tgz", + "integrity": "sha1-iYyr/Ivt5Le5ETWj/9Mj5YwNsxk=", + "dev": true, + "requires": { + "css-select": "1.2.0", + "dom-converter": "0.1.4", + "htmlparser2": "3.3.0", + "strip-ansi": "3.0.1", + "utila": "0.3.3" + }, + "dependencies": { + "utila": { + "version": "0.3.3", + "resolved": "https://registry.npmjs.org/utila/-/utila-0.3.3.tgz", + "integrity": "sha1-1+jn1+MJEHCSsF+NloiCTWM6QiY=", + "dev": true + } + } + }, "repeat-element": { "version": "1.1.2", "resolved": "https://registry.npmjs.org/repeat-element/-/repeat-element-1.1.2.tgz", @@ -3852,6 +4217,12 @@ "integrity": "sha1-fSKbH8xjfkZsoIEYCDanqr/4P0M=", "dev": true }, + "toposort": { + "version": "1.0.6", + "resolved": "https://registry.npmjs.org/toposort/-/toposort-1.0.6.tgz", + "integrity": "sha1-wxdI5V0hDv/AD9zcfW5o19e7nOw=", + "dev": true + }, "tty-browserify": { "version": "0.0.0", "resolved": "https://registry.npmjs.org/tty-browserify/-/tty-browserify-0.0.0.tgz", @@ -3939,11 +4310,23 @@ } } }, + "universalify": { + "version": "0.1.1", + "resolved": "https://registry.npmjs.org/universalify/-/universalify-0.1.1.tgz", + "integrity": "sha1-+nG63UQ3r0wUiEHjs7Fl+enlkLc=", + "dev": true + }, "unzip-response": { "version": "1.0.2", "resolved": "https://registry.npmjs.org/unzip-response/-/unzip-response-1.0.2.tgz", "integrity": "sha1-uYTwh3/AqJwsdzzB73tbIytbBv4=" }, + "upper-case": { + "version": "1.1.3", + "resolved": "https://registry.npmjs.org/upper-case/-/upper-case-1.1.3.tgz", + "integrity": "sha1-9rRQHC7EzdJrp4vnIilh3ndiFZg=", + "dev": true + }, "url": { "version": "0.11.0", "resolved": "https://registry.npmjs.org/url/-/url-0.11.0.tgz", @@ -3984,6 +4367,12 @@ "resolved": "https://registry.npmjs.org/util-deprecate/-/util-deprecate-1.0.2.tgz", "integrity": "sha1-RQ1Nyfpw3nMnYvvS1KKJgUGaDM8=" }, + "utila": { + "version": "0.4.0", + "resolved": "https://registry.npmjs.org/utila/-/utila-0.4.0.tgz", + "integrity": "sha1-ihagXURWV6Oupe7MWxKk+lN5dyw=", + "dev": true + }, "validate-npm-package-license": { "version": "3.0.1", "resolved": "https://registry.npmjs.org/validate-npm-package-license/-/validate-npm-package-license-3.0.1.tgz", @@ -4143,6 +4532,12 @@ "resolved": "https://registry.npmjs.org/wrappy/-/wrappy-1.0.2.tgz", "integrity": "sha1-tSQ9jz7BqjXxNkYFvA0QNuMKtp8=" }, + "xml-char-classes": { + "version": "1.0.0", + "resolved": "https://registry.npmjs.org/xml-char-classes/-/xml-char-classes-1.0.0.tgz", + "integrity": "sha1-ZGV4SKIP/F31g6Qq2KJ3tFErvE0=", + "dev": true + }, "xtend": { "version": "4.0.1", "resolved": "https://registry.npmjs.org/xtend/-/xtend-4.0.1.tgz", diff --git a/package.json b/package.json index 23b71a6c..f224582d 100644 --- a/package.json +++ b/package.json @@ -12,6 +12,10 @@ "precommit": "lint-staged" }, "lint-staged": { + "src/**/*.css": [ + "prettier --write", + "git add" + ], "src/**/*.js": [ "prettier --write", "git add" @@ -43,6 +47,8 @@ "virtualfs": "^1.0.0" }, "devDependencies": { + "copy-webpack-plugin": "^4.1.1", + "html-webpack-plugin": "^2.30.1", "husky": "^0.14.3", "lint-staged": "^4.2.3", "node-libs-browser": "github:bmeurer/node-libs-browser#patch-1", diff --git a/src/Logo.png b/src/Logo.png new file mode 100644 index 00000000..6c0092e8 Binary files /dev/null and b/src/Logo.png differ diff --git a/src/bootstrap.js b/src/bootstrap.js new file mode 100644 index 00000000..9e2963bb --- /dev/null +++ b/src/bootstrap.js @@ -0,0 +1,144 @@ +// Copyright (c) 2017 Benedikt Meurer +// +// Permission is hereby granted, free of charge, to any person obtaining +// a copy of this software and associated documentation files (the +// "Software"), to deal in the Software without restriction, including +// without limitation the rights to use, copy, modify, merge, publish, +// distribute, sublicense, and/or sell copies of the Software, and to +// permit persons to whom the Software is furnished to do so, subject to +// the following conditions: +// +// The above copyright notice and this permission notice shall be +// included in all copies or substantial portions of the Software. +// +// THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, +// EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF +// MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND +// NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE +// LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION +// OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION +// WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. + +const gmean = require("compute-gmean"); +const suite = require("./suite"); + +function displayStatusMessage(message) { + const element = document.getElementById("status"); + element.innerHTML = `${message}`; +} + +function displayResultMessage(name, message, style) { + const element = document.getElementById("results-cell-" + name); + element.innerHTML = message || "—"; + if (element.classList) { + element.classList.remove("result"); + element.classList.remove("highlighted-result"); + element.classList.add(style); + } else { + element.className = style; + } +} + +function reset() { + const resultSummaryDiv = document.getElementById("result-summary"); + resultSummaryDiv.textContent = ""; + + const benchmarks = []; + suite.forEach(benchmark => benchmarks.push(benchmark)); + + const numColumns = 2; + const columnHeight = Math.ceil((benchmarks.length + 1) / numColumns); + const resultsTable = document.getElementById("results"); + let text = + "" + "BenchmarkRuns/Sec".repeat(numColumns) + ""; + for (let i = 0; i < columnHeight; ++i) { + text += ""; + for (let j = 0; j < numColumns; ++j) { + const index = j * columnHeight + i; + if (index > benchmarks.length) break; + if (index == benchmarks.length) { + text += `Geometric Mean`; + text += `—`; + } else { + const benchmark = benchmarks[index]; + text += ``; + text += `${benchmark.name}`; + text += `—`; + } + } + text += ""; + } + resultsTable.innerHTML = text; +} + +function initialize() { + reset(); + + const statusDiv = document.getElementById("status"); + statusDiv.innerHTML = `Start test`; + statusDiv.firstChild.onclick = start; +} + +function start() { + reset(); + + const statusDiv = document.getElementById("status"); + statusDiv.innerHTML = "Running test suite\u2026"; + suite.run({ async: true }); +} + +window.onerror = () => { + // TODO(bmeurer): Provide some sane error page here. + console.log("SOMETHING WENT WRONG!"); +}; +window.onload = initialize; + +suite.forEach(benchmark => { + benchmark.on("start", event => { + displayResultMessage( + benchmark.name, + "Running...", + "highlighted-result" + ); + displayStatusMessage(`Running iteration 1 of ${benchmark.name}...`); + }); + benchmark.on("cycle", event => { + const iteration = benchmark.stats.sample.length + 1; + displayStatusMessage( + `Running iteration ${iteration} of ${benchmark.name}...` + ); + }); + benchmark.on("complete", event => { + displayResultMessage( + benchmark.name, + `${benchmark.hz.toFixed( + 2 + )} ± ${benchmark.stats.rme.toFixed( + 2 + )}%`, + "result" + ); + const iterations = benchmark.stats.sample.length; + displayStatusMessage( + `Ran ${iterations} iterations of ${benchmark.name}...` + ); + }); +}); + +suite.on("complete", event => { + const sample = []; + suite.forEach(benchmark => { + sample.push(...benchmark.stats.sample); + }); + const hz = 1 / gmean(sample); + displayResultMessage("geomean", `${hz.toFixed(2)}`, "highlighted-result"); + + const statusDiv = document.getElementById("status"); + statusDiv.innerHTML = `Test again`; + statusDiv.firstChild.onclick = start; + + const resultSummaryDiv = document.getElementById("result-summary"); + resultSummaryDiv.innerHTML = `
${hz.toFixed( + 2 + )}`; +}); diff --git a/src/index.html b/src/index.html new file mode 100644 index 00000000..e45aa025 --- /dev/null +++ b/src/index.html @@ -0,0 +1,51 @@ + + + + + + + Web Tooling Benchmark + + <%=htmlWebpackPlugin.files.webpackManifest%> + + + +
+ + +

+ The Web Tooling Benchmark is a performance test suite focused on JavaScript related workloads found + in common web developer tools these days. For more information, read the + in-depth + analysis. Bigger scores are better. +

+ +
+
Start test
+ +
+
+ + + diff --git a/src/style.css b/src/style.css new file mode 100644 index 00000000..643d64bf --- /dev/null +++ b/src/style.css @@ -0,0 +1,194 @@ +/*- + * Copyright (c) 2017 Benedikt Meurer + * + * Permission is hereby granted, free of charge, to any person obtaining + * a copy of this software and associated documentation files (the + * "Software"), to deal in the Software without restriction, including + * without limitation the rights to use, copy, modify, merge, publish, + * distribute, sublicense, and/or sell copies of the Software, and to + * permit persons to whom the Software is furnished to do so, subject to + * the following conditions: + * + * The above copyright notice and this permission notice shall be + * included in all copies or substantial portions of the Software. + * + * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, + * EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF + * MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND + * NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE + * LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION + * OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION + * WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. + */ + +body { + font-family: Roboto, Helvetica, Verdana, sans-serif; + font-size: 16px; + line-height: 19px; + padding-bottom: 00px; +} + +body main { + display: block; + width: 850px; + margin: auto; + text-align: center; + padding: 10px; +} + +::selection { + background-color: rgb(19, 156, 7); + color: white; +} + +p.summary { + margin: 18px auto 5px; + background-image: -webkit-linear-gradient( + right, + white 70%, + rgba(255, 255, 255, 0) + ); + background-image: -moz-linear-gradient( + right, + white 70%, + rgba(255, 255, 255, 0) + ); + background-image: linear-gradient( + to right, + white 70%, + rgba(255, 255, 255, 0) + ); + padding: 15px 125px 0; + text-align: center; +} + +h1 { + color: rgb(19, 156, 7); + text-align: left; + margin-top: 40px; +} + +p { + text-align: left; + text-indent: 20px; +} + +p:first-of-type { + text-indent: 0; +} + +div#status { + margin-top: 20px; + height: 20px; +} + +div#status a:link { + font-weight: bold; +} + +div#result-summary:empty { + display: none; +} + +div#result-summary { + margin-top: 20px; +} + +div#result-summary label { + font-weight: bold; +} + +div#result-summary .score { + font-weight: bold; + font-size: 48px; + line-height: 48px; + color: rgb(19, 156, 7); +} + +div#result-summary .score .interval { + display: block; + font-size: 18px; + line-height: 18px; + font-weight: normal; +} + +.interval { + opacity: 0.75; +} + +a:link, +a:visited { + color: rgb(19, 156, 7); + text-decoration: none; +} + +a:link:hover { + text-decoration: underline; +} + +table#results { + border-spacing: 0; + border-collapse: collapse; + margin-top: 15px; + width: 100%; + table-layout: fixed; +} + +td, +th { + padding: 4px 8px; +} + +th { + font-size: 14px; +} + +tr:first-child > th:nth-child(odd):not(:first-child) { + border-left: 15px solid white; +} + +tr:first-child > th:nth-child(odd) { + text-align: left; +} + +tr:first-child > th:nth-child(even) { + width: px; +} + +tr:nth-child(even):not(:first-child) { + background-color: rgb(238, 247, 238); +} + +.result { + color: rgb(19, 156, 7); + white-space: nowrap; +} + +.highlighted-result { + background-color: rgb(19, 156, 7); + color: white; +} + +.benchmark-name { + text-align: left; + white-space: nowrap; +} + +.geometric-mean, +.benchmark-name.category, +.result.category { + font-weight: bold; +} + +.benchmark-name:not(.category):not(.geometric-mean) { + text-indent: 0.5em; +} + +.benchmark-name:not(:first-child) { + border-left: 15px solid white; +} + +.benchmark-name a:link, +.benchmark-name a:visited { + color: black; +} diff --git a/webpack.config.js b/webpack.config.js index 631bd6c5..30d65bcc 100644 --- a/webpack.config.js +++ b/webpack.config.js @@ -19,31 +19,66 @@ // OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION // WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. +const CopyWebpackPlugin = require("copy-webpack-plugin"); +const HtmlWebpackPlugin = require("html-webpack-plugin"); const path = require("path"); const webpack = require("webpack"); -module.exports = { - entry: "./src/cli.js", - output: { - filename: "run.js", - path: path.resolve(__dirname, "dist") +module.exports = [ + { + context: path.resolve("src"), + entry: "./cli.js", + output: { + filename: "run.js", + path: path.resolve("dist") + }, + bail: true, + resolve: { + alias: { + fs: require.resolve("./src/vfs"), + module: require.resolve("./src/mocks/dummy") + } + }, + plugins: [ + new webpack.BannerPlugin({ + banner: + "// Required for JavaScript engine shells.\n" + + "var global = this;\n" + + "if (typeof console === 'undefined') {\n" + + " console = {log: print};\n" + + "}", + raw: true + }) + ] }, - bail: true, - resolve: { - alias: { - fs: require.resolve("./src/vfs"), - module: require.resolve("./src/mocks/dummy") - } - }, - plugins: [ - new webpack.BannerPlugin({ - banner: - "// Required for JavaScript engine shells.\n" + - "var global = this;\n" + - "if (typeof console === 'undefined') {\n" + - " console = {log: print};\n" + - "}", - raw: true - }) - ] -}; + { + context: path.resolve("src"), + entry: "./bootstrap.js", + output: { + filename: "bundle.js", + path: path.resolve("dist") + }, + bail: true, + resolve: { + alias: { + define: require.resolve("./src/mocks/dummy"), + fs: require.resolve("./src/vfs"), + module: require.resolve("./src/mocks/dummy") + } + }, + plugins: [ + new CopyWebpackPlugin([{ from: "style.css" }, { from: "Logo.png" }]), + new webpack.BannerPlugin({ + banner: + "// Work-around for the weird JaegerMonkey\n" + + "// work-around inside benchmark.js.\n" + + "const define = { amd: {} };\n", + raw: true + }), + new HtmlWebpackPlugin({ + template: "./index.html", + inject: "head" + }) + ] + } +];