From 4749ec115523e43b395d2556679b59bf2b0c5217 Mon Sep 17 00:00:00 2001 From: Chong Gu Date: Wed, 17 Feb 2021 01:41:06 +0000 Subject: [PATCH] [Fuchsia] Create/Reorganize documentation for Fuchsia Put Fuchsia specific instructions in //docs/fuchsia Added instructions for running blink and gpu integration tests. Added instructions for running tests on remote devices. Bug: 1174296 Change-Id: I54fabb5b339fe90b3517c6341c62d7269726772f Reviewed-on: https://chromium-review.googlesource.com/c/chromium/src/+/2673628 Reviewed-by: Sharon Yang Reviewed-by: Bob Haarman Reviewed-by: Fabrice de Gans-Riberi Auto-Submit: Chong Gu Commit-Queue: Chong Gu Cr-Commit-Position: refs/heads/master@{#854575} --- docs/README.md | 4 +- .../build_instructions.md} | 56 +-------- docs/fuchsia/debug_instructions.md | 49 ++++++++ docs/fuchsia/gpu_testing.md | 45 +++++++ docs/fuchsia/gtests.md | 97 +++++++++++++++ .../sdk_updates.md} | 0 docs/fuchsia/web_tests.md | 28 +++++ docs/fuchsia_gardening.md | 40 ------- docs/get_the_code.md | 2 +- fuchsia/README.md | 110 +----------------- tools/clang/scripts/build.py | 4 +- 11 files changed, 237 insertions(+), 198 deletions(-) rename docs/{fuchsia_build_instructions.md => fuchsia/build_instructions.md} (68%) create mode 100644 docs/fuchsia/debug_instructions.md create mode 100644 docs/fuchsia/gpu_testing.md create mode 100644 docs/fuchsia/gtests.md rename docs/{fuchsia_sdk_updates.md => fuchsia/sdk_updates.md} (100%) create mode 100644 docs/fuchsia/web_tests.md delete mode 100644 docs/fuchsia_gardening.md diff --git a/docs/README.md b/docs/README.md index 0836b4350dd794..e6fb53dc6671a9 100644 --- a/docs/README.md +++ b/docs/README.md @@ -48,8 +48,8 @@ used when committed. (on a Linux host) * [Cast for Android Build Instructions](android_cast_build_instructions.md) - Cast for Android (on a Linux host) -* [Fuchsia Build Instructions](fuchsia_build_instructions.md) - Fuchsia target - (on a Linux host) +* [Fuchsia Build Instructions](fuchsia/build_instructions.md) - + Fuchsia target (on a Linux host) * [iOS Build Instructions](ios/build_instructions.md) - iOS target (on a MacOS host) * [Chrome OS Build Instructions](chromeos_build_instructions.md) - Chrome OS diff --git a/docs/fuchsia_build_instructions.md b/docs/fuchsia/build_instructions.md similarity index 68% rename from docs/fuchsia_build_instructions.md rename to docs/fuchsia/build_instructions.md index ba27e01372f5cd..0b37b748b590e7 100644 --- a/docs/fuchsia_build_instructions.md +++ b/docs/fuchsia/build_instructions.md @@ -201,54 +201,10 @@ $ exit ### Running test suites -Building test suites generate a launcher script to run them on an emulator -or a physical device. These scripts are generated at `out/fuchsia/bin`. For -instance,to run the `base_unittests` target, launch: +There are three types of tests available to run on Fuchsia: -```shell -$ out/fuchsia/bin/run_base_unittests -``` - -Common gtest arguments such as `--gtest_filter=...` are supported by the run -script. The launcher script also symbolizes backtraces. - -The test suite, by default, will run on QEMU. AEMU can be used for running -tests that interact with Fuchsia's window manager, Scenic. To change the device -that Fuchsia will run on, use `--device={aemu|qemu|device}`. - -To run a test suite on an *unprovisioned device* in a zedboot state, simply add -`-d` to the test runner script arguments. Subsequent runs of the test runner -script will be able to pick up the same device. - -To run a test suite on a device that is *already provisioned*, add the following -arguments to the test runner script: - -* `-d` to run the test suite on a device. -* `--fuchsia-out-dir=[/path/to/fuchsia/out/directory]` or - `--ssh-config=[/path/to/ssh_config]` to specify the SSH configuration used for - connecting to the target device. -* (Optional) `--host=[IP]` to specify the test device IP. Typically, this is the -value obtained from the command `fx netaddr --fuchsia`. -* (Optional) `--port=[port]` to specify the SSH port, if different from 22. - -### Troubleshooting a test - -To troubleshoot a specific test, consider a combination of the following -arguments to the test runner script: - -* `--gtest_filter="[TestSuite.TestName]"` to only run a specific test, or a - comma-separated list to run a set of tests. Wildcards can also be used to run - a set of tests or an entire test suite, e.g. `--gtest_filter="[TestSuite.*]"`. -* `--test-launcher-jobs=1` to only have one batch of tests running at a time. - This bypasses the test launcher buffering of test log output, making it - possible to access the log output from successful test runs. -* `--single-process-tests` to run all the tests in the same process. Unlike the - above option, this will run the tests directly in the test launcher process, - making it easier to attach a debugger. -* `system-log-file=[/path/to/syslog]` to specify the file to write system logs - to. Or `system-log-file=-` to write the system logs to stdout. By default, - Chromium logs are written to the system log on Fuchsia. This argument is known - to cause `IOError` python exceptions with a QEMU target. -* `--gtest_repeat=[number] --gtest_break_on_failure` to run a test or test suite - a certain number of times until it fails. This is useful to investigate flaky - tests. +1. [Gtests](gtests.md) +2. [GPU integration tests](gpu_testing.md) +3. [Blink tests](web_tests.md) + +Check the documentations to learn more about how to run these tests. diff --git a/docs/fuchsia/debug_instructions.md b/docs/fuchsia/debug_instructions.md new file mode 100644 index 00000000000000..264332bbc5daf7 --- /dev/null +++ b/docs/fuchsia/debug_instructions.md @@ -0,0 +1,49 @@ +# Debugging + +It is possible to debug Fuchsia binaries using `zxdb`. For the sake of this +example, we will be using `base_unittests` as the test suite we wish to execute: + +1. (From Chromium) Install your package(s) and its symbols onto the device. + + ```bash + $ out/fuchsia/bin/install_base_unittests + ``` + +2. (From Fuchsia source tree) Run the debugger. + + ```bash + $ fx debug + ``` + +3. Set up the debugger to attach to the process. + + ``` + [zxdb] attach base_unittests.cmx + ``` + +4. Configure breakpoint(s). + + ``` + [zxdb] break base::GetDefaultJob + ``` + +5. (In another terminal, from Fuchsia source tree) Run the test package. + + ```bash + $ fx shell run fuchsia-pkg://fuchsia.com/base_unittests#meta/base_unittests.cmx + ``` + +6. At this point, you should hit a breakpoint in `zxdb`. + + ``` + [zxdb] f + ▶ 0 base::GetDefaultJob() • default_job.cc:18 + 1 base::$anon::LaunchChildTestProcessWithOptions(…) • test_launcher.cc:335 + 2 base::$anon::DoLaunchChildTestProcess(…) • test_launcher.cc:528 + 3 base::TestLauncher::LaunchChildGTestProcess(…) • test_launcher.cc:877 + ... + ``` + +7. Enjoy debugging! Steps 2 through 6 will also work for things like services + which aren't run directly from the command line, such as WebEngine. + Run `help` inside ZXDB to see what debugger commands are available. \ No newline at end of file diff --git a/docs/fuchsia/gpu_testing.md b/docs/fuchsia/gpu_testing.md new file mode 100644 index 00000000000000..a5bcbf3fc6d1ff --- /dev/null +++ b/docs/fuchsia/gpu_testing.md @@ -0,0 +1,45 @@ +# Running GPU integration tests on Fuchsia. + +General instruction on running and debugging GPU integration tests can be +found [here](../gpu/gpu_testing.md). + +Fuchsia uses [web_engine_shell](../../fuchsia/engine/test/README.md) to run GPU +integration tests. For the sake of this example, we will be using `gpu_process` +as the test suite we wish to execute. Build the target +`fuchsia_telemetry_gpu_integration_test` and run the appropriate commands: + +## Hermetic emulation + +The test script brings up an emulator, runs the tests on it, and shuts the +emulator down when finished. + +```bash +$ content/test/gpu/run_gpu_integration_test_fuchsia.py gpu_process +--browser=web-engine-shell --out-dir=/path/to/outdir +``` + +## Run on an physical device + +```bash +$ content/test/gpu/run_gpu_integration_test_fuchsia.py gpu_process +--browser=web-engine-shell --out-dir=/path/to/outdir -d +``` + +## Run on a device paved with Fuchsia built from source + +```bash +$ content/test/gpu/run_gpu_integration_test_fuchsia.py gpu_process +--browser=web-engine-shell --out-dir=/path/to/outdir -d +--fuchsia-out-dir=/path/to/fuchsia/outdir +``` + +## Run on a device the host is connected to remotely via ssh + +Note the `--ssh-config` flag, which should point to the config file used to set +up the connection between the host and the remote device. + +```bash +$ content/test/gpu/run_gpu_integration_test_fuchsia.py gpu_process +--browser=web-engine-shell --out-dir=/path/to/outdir -d --host=localhost +--ssh-config=/path/to/ssh/config +``` \ No newline at end of file diff --git a/docs/fuchsia/gtests.md b/docs/fuchsia/gtests.md new file mode 100644 index 00000000000000..aacb9823ae8c73 --- /dev/null +++ b/docs/fuchsia/gtests.md @@ -0,0 +1,97 @@ +# Deploying and running gtests on Fuchsia. + +Fuchsia gtest binaries are deployed and executed via scripts that are +automatically generated by the `fuchsia_package_runner()` GN target. +The binaries can deploy to either emulators started by the runner script, +an existing emulator instance, or a physical device. To build a gtest binary, +check this [documentation](build_instructions.md). + +For the sake of this example, we will be using `base_unittests` as the package +we wish to install and/or execute. + +## Hermetic emulation + +The test script brings up an emulator, runs the tests on it, and +shuts the emulator down when finished. +```bash +$ out/fuchsia/bin/run_base_unittests +``` + +## Run on an physical device + +Note the `-d` flag, which is an alias for `--device`. + +```bash +$ out/fuchsia/bin/run_base_unittests -d +``` + +## Run on a device paved with Fuchsia built from source + +Make sure that the CPU architecture of your Chromium output directory matches +the architecture of the Fuchsia output directory (x64==x64, arm64==arm64, etc.). + +```bash +$ out/fuchsia/bin/run_base_unittests -d +--fuchsia-out-dir=/path/to/fuchsia/outdir +``` + +If you are frequently deploying to Fuchsia built from source, you might want to +add the following entry to your `args.gn`: + +``` +default_fuchsia_build_dir_for_installation = "/path/to/fuchsia/outdir" +``` + +With this flag in place, the `--fuchsia-out-dir` flag will automatically be +used whenever you `run_` or `install_` Fuchsia packages, making your command +lines much shorter: + +```bash +$ out/fuchsia/bin/run_base_unittests -d +``` + +## Install on a device running Fuchsia built from source + +```bash +$ out/fuchsia/bin/install_base_unittests +--fuchsia-out-dir=/path/to/fuchsia/outdir +``` + +You will need to run the package manually on your device. In this case, run the +following from your Fuchsia directory: + +``` +$ fx shell run fuchsia-pkg://fuchsia.com/base_unittests#meta/base_unittests.cmx +``` + +## Run on a device the host is connected to remotely via ssh + +Note the `--ssh-config` flag, which should point to the config file used to set +up the connection between the host and the remote device. + +```bash +$ out/fuchsia/bin/run_base_unittests -d +--host=localhost --ssh-config=/path/to/ssh/config +``` + +## Troubleshooting a test + +To troubleshoot a specific test, consider a combination of the following +arguments to the test runner script: + +* `--gtest_filter="[TestSuite.TestName]"` to only run a specific test, or a + comma-separated list to run a set of tests. Wildcards can also be used to run + a set of tests or an entire test suite, e.g. `--gtest_filter="[TestSuite.*]"`. +* `--test-launcher-jobs=1` to only have one batch of tests running at a time. + This bypasses the test launcher buffering of test log output, making it + possible to access the log output from successful test runs. +* `--single-process-tests` to run all the tests in the same process. Unlike the + above option, this will run the tests directly in the test launcher process, + making it easier to attach a debugger. +* `system-log-file=[/path/to/syslog]` to specify the file to write system logs + to. Or `system-log-file=-` to write the system logs to stdout. By default, + Chromium logs are written to the system log on Fuchsia. This argument is known + to cause `IOError` python exceptions with a QEMU target. +* `--gtest_repeat=[number] --gtest_break_on_failure` to run a test or test suite + a certain number of times until it fails. This is useful to investigate flaky + tests. \ No newline at end of file diff --git a/docs/fuchsia_sdk_updates.md b/docs/fuchsia/sdk_updates.md similarity index 100% rename from docs/fuchsia_sdk_updates.md rename to docs/fuchsia/sdk_updates.md diff --git a/docs/fuchsia/web_tests.md b/docs/fuchsia/web_tests.md new file mode 100644 index 00000000000000..6c573b21acbb47 --- /dev/null +++ b/docs/fuchsia/web_tests.md @@ -0,0 +1,28 @@ +# Deploying content_shell and running web_tests on Fuchsia. + +General instruction on running and debugging web_tests can be found +[here](../testing/web_tests.md). + +Currently, only +[a small subset of web tests](../../third_party/blink/web_tests/SmokeTests) +can be run on Fuchsia. Build the target `blink_tests` first before running any +of the commands below: + +## Hermetic emulation + +The test script brings up an emulator, runs the tests on it, and shuts the +emulator down when finished. +```bash +$ third_party/blink/tools/run_web_tests.py -t --platform=fuchsia +``` + +## Run on an physical device. + +Note the `--fuchsia-host-ip` flag, which is the ip address of the test host that +the Fuchsia device uses to establish a connection. + +```bash +$ third_party/blink/tools/run_web_tests.py -t --platform=fuchsia +--device=device --fuchsia-target-cpu= +--fuchsia-out-dir=/path/to/fuchsia/outdir --fuchsia-host-ip=test.host.ip.address +``` \ No newline at end of file diff --git a/docs/fuchsia_gardening.md b/docs/fuchsia_gardening.md deleted file mode 100644 index 63c11245f04878..00000000000000 --- a/docs/fuchsia_gardening.md +++ /dev/null @@ -1,40 +0,0 @@ -# Cr-Fuchsia Gardening - -## Gardener Responsibilities - -In priority order, the responsibilities of the Fuchsia Gardener are as follows (though see notes below!): - -1. **Chromium waterfall:** Keep [the Fuchsia bots on the Chromium waterfall](https://ci.chromium.org/p/chromium/g/chromium.linux/console) green. - 1. Join the #chromium IRC channel on Freenode. - 1. Not all waterfall bots have a corresponding try-bot configuration. - 1. E.g. the Cast Fuchsia bots are not run on the CQ by default. - 1. E.g. the CQ builds have DCHECKs enabled, whereas our Cast waterfall bots don't. -1. **Chromium try-bots:** Ensure that Fuchsia bots are not causing CQ flake. - 1. Join the #chromium IRC channel on Freenode. - 1. Watch for try-flakes emails & investigate any tests/suites reported flakey. - 1. Watch for IRC mentions of flakiness on the Fuchsia bots. -1. **Fuchsia SDK rolls:** Keep [the Fuchsia SDK auto-roller](https://autoroll.skia.org/r/fuchsia-sdk-chromium-autoroll) working. - 1. Watch for emailed status updates from auto-roller CLs. - 1. If a roll CL fails, check the failed bot to confirm SDK-related fail vs other flake. - 1. If it was an actual SDK-related failure, note the latest auto-roller patch-Id, and stop the auto-roller. - 1. Create a local branch e.g. with "`git checkout -b sdkRoll origin`". - 1. Pull-down the auto-roll CL with "`git cl patch && gclient sync`". - 1. Clear the CL metadata with "`git cl issue 0`". - 1. Make any necessary modifications for compatibility with the new SDK. - 1. Commit the changes and run "`git cl upload`" to upload a new CL. - 1. Edit the CL description, which will include the git commit description from the auto-roller CL, making it easy to provide a consistent description. - 1. Note that if the auto-roller is blocked for a long time then it may be easier to fix the most-recent failed roll, and roll from there, to avoid having to deal with several different causes of breakage in a single roll! -1. **FYI waterfall:** Keep our bots on the [FYI waterfall](https://ci.chromium.org/p/chromium/g/chromium.fyi/console) green. - 1. FYI bots don't block the CQ, but still provide early-warning of regressions. - 1. They're also our staging-ground for bringing complex test suites to the CQ/waterfall. - -While the Gardener takes primary responsibility for each of these areas during their rotation, that does not mean that they must do all the work - if another teammate happens to have started fixing the SDK roll, un-breaking the bots, or sending you CLs to fix our Debug builder (look, ma! No try-bot!), then lucky you, your Gardening job is done. :) - -## Optional Gardener Tasks - -The waterfall is green, the try-bots reliable, SDK is rolling and pigs soar majestically in the sky. Fear not, gentle Gardener, you still have a valuable role to play! - -Googlers: The current gardener can be found at [go/cr-fuchsia-cop](http://go/cr-fuchsia-cop). - -* Look for tests that have been filtered under Fuchsia, and diagnose them. - * File a new bug, or upate the filter to refer to existing bugs, as appropriate. diff --git a/docs/get_the_code.md b/docs/get_the_code.md index 0b4e1acaa921ca..06a666389be838 100644 --- a/docs/get_the_code.md +++ b/docs/get_the_code.md @@ -12,7 +12,7 @@ you might want to build: * [Android](android_build_instructions.md) * [Android Cast](android_cast_build_instructions.md) * [Chrome OS](chromeos_build_instructions.md) -* [Fuchsia](fuchsia_build_instructions.md) +* [Fuchsia](fuchsia/build_instructions.md) * [iOS](ios/build_instructions.md) * [Linux](linux/build_instructions.md) * [Linux Cast](linux/cast_build_instructions.md) diff --git a/fuchsia/README.md b/fuchsia/README.md index 31469405e92551..ead70a4eff9677 100644 --- a/fuchsia/README.md +++ b/fuchsia/README.md @@ -1,7 +1,7 @@ # Chromium-based Fuchsia services This directory contains implementation code for various Fuchsia services living in the Chromium repository. To build Chromium on Fuchsia, check this -[documentation](../docs/fuchsia_build_instructions.md). +[documentation](../docs/fuchsia/build_instructions.md). [TOC] @@ -62,112 +62,14 @@ tests. ## Deploying and running Fuchsia code. Fuchsia binaries are deployed and executed via scripts that are automatically -generated by the `fuchsia_package_runner()` GN target. -`fuchsia_package_runner()` targets exist for all tests and a number of -frequently used standalone executables like `"content_shell_fuchsia"`'. The -scripts can deploy to either emulators started by the runner script, an existing -emulator instance, or a physical device. - -For the sake of this example, we will be using `base_unittests` as the package -we wish to install and/or execute. - -**Hermetic emulation** -```bash -$ out/fuchsia/bin/run_base_unittests -``` - -**Run on an physical device running Zedboot**. Note the `-d` flag, which is an -alias for `--device`. - -```bash -$ out/fuchsia/bin/run_base_unittests -d -``` - -**Run on a device paved with Fuchsia built from source.** - -```bash -$ out/fuchsia/bin/run_base_unittests -d ---fuchsia-out-dir=/path/to/fuchsia/outdir -``` - -**Install on a device running Fuchsia built from source.** - -```bash -$ out/fuchsia/bin/install_base_unittests ---fuchsia-out-dir=/path/to/fuchsia/outdir -``` - -You will need to run the package manually on your device. In this case, run the -following from your Fuchsia directory: - -``` -$ fx shell run fuchsia-pkg://fuchsia.com/base_unittests#meta/base_unittests.cmx -``` - -If you are frequently deploying to Fuchsia built from source, you might want to -add the following entry to your `args.gn`: - -``` -default_fuchsia_build_dir_for_installation = "/path/to/fuchsia/outdir" -``` - -With this flag in place, the `--fuchsia-out-dir` flag will automatically be used -whenever you `run_` or `install_` Fuchsia packages, making your command lines -much shorter: - -```bash -$ out/fuchsia/bin/run_base_unittests -d -``` - -Make sure that the CPU architecture of your Chromium output directory matches the -architecture of the Fuchsia output directory (x64==x64, arm64==arm64, etc.). +generated by the `fuchsia_package_runner()` GN target. Details on deploying and +running Fuchsia test binaries can be found +[here](../docs/fuchsia/gtests.md) ## Debugging -1. (From Chromium) Install your package(s) and its symbols onto the device. - - ```bash - $ out/fuchsia/bin/install_base_unittests - ``` - -2. (From Fuchsia source tree) Run the debugger. - - ```bash - $ fx debug - ``` - -3. Set up the debugger to attach to the process. - - ``` - [zxdb] attach base_unittests.cmx - ``` - -4. Configure breakpoint(s). - - ``` - [zxdb] break base::GetDefaultJob - ``` - -5. (In another terminal, from Fuchsia source tree) Run the test package. - - ```bash - $ fx shell run fuchsia-pkg://fuchsia.com/base_unittests#meta/base_unittests.cmx - ``` - -6. At this point, you should hit a breakpoint in `zxdb`. - - ``` - [zxdb] f - ▶ 0 base::GetDefaultJob() • default_job.cc:18 - 1 base::$anon::LaunchChildTestProcessWithOptions(…) • test_launcher.cc:335 - 2 base::$anon::DoLaunchChildTestProcess(…) • test_launcher.cc:528 - 3 base::TestLauncher::LaunchChildGTestProcess(…) • test_launcher.cc:877 - ... - ``` - -7. Enjoy debugging! Steps 2 through 6 will also work for things like services - which aren't run directly from the command line, such as WebEngine. - Run `help` inside ZXDB to see what debugger commands are available. +Check [this documentation](../docs/fuchsia/debug_instructions.md) for +debugging tips. ## WebRunner/WebEngine diff --git a/tools/clang/scripts/build.py b/tools/clang/scripts/build.py index f99bea54b378b0..b430abf0c6b1ce 100755 --- a/tools/clang/scripts/build.py +++ b/tools/clang/scripts/build.py @@ -468,7 +468,9 @@ def main(): print('Install the Fuchsia SDK by adding fuchsia to the ') print('target_os section in your .gclient and running hooks, ') print('or pass --without-fuchsia.') - print('https://chromium.googlesource.com/chromium/src/+/master/docs/fuchsia_build_instructions.md') + print( + 'https://chromium.googlesource.com/chromium/src/+/master/docs/fuchsia/build_instructions.md' + ) print('for general Fuchsia build instructions.') return 1