Add script for triggering end-to-end tests from a local machine

This change adds a script `tests/local.sh` that can be used to trigger the end to end tests from a development machine on a remote ec2 instance. Signed-off-by: Evan Lezar <[email protected]>
NVIDIA · Jun 8, 2021 · 4951391 · 4951391
1 parent 1bd5c1c
commit 4951391
Show file tree

Hide file tree

Showing 2 changed files with 69 additions and 0 deletions.
diff --git a/tests/README.md b/tests/README.md
@@ -1,5 +1,42 @@
 # GPU operator test utilities
 
+## Testing locally
+The `local.sh` script allows for triggering basic end-to-end testing of the GPU
+operator from a local machine.
+
+Running the command:
+```bash
+./tests/local.sh
+```
+will perform the following steps:
+1. Launch an AWS instance using the `aws-kube-ci` subrepo
+1. Sync the project folder to the newly created instance.
+1. Install additional prerequsites (see `scripts/prerequisites.sh`)
+1. Execute the default test case (`cases/defaults.sh`) on the remote instance
+
+If an instance has already been launched with terraform and the launch phase is to be skipped, the following can be used:
+```
+SKIP_LAUNCH=true ./tests/local.sh
+```
+For documentation on configuring instance creation, see the associated [README](../aws-kube-ci/README.md) folder.
+
+### Cleaning up
+If the tests succeed, the operator and workloads should be uninstalled (`scripts/uninstall.sh`). If the tests fail there may be state that needs to be cleaned up. This can be confirmed by running `kubectl` commands on the remote instance:
+
+For example:
+```bash
+./tests/scripts/remote.sh kubectl get pods -A
+./tests/scripts/remote.sh kubectl get namespace
+```
+
+Which would show the pods associated with the `gpu-operator-resources` namespace and or the `test-operator` namespace.
+
+Running:
+```bash
+CLEANUP=true ./tests/local.sh
+```
+should cleanup the resources on the remote machine to allow the tests to be run again.
+
 ## Provided scripts / utilities
 
 * `./tests/scripts/remote.sh [command]`: Execute `[command]` on the remote instance via SSH. This behaves the same as the `ssh` command and if no command is specified an interactive session is started.

diff --git a/tests/local.sh b/tests/local.sh
@@ -0,0 +1,32 @@
+#! /bin/bash
+
+: ${TEST_CASE:="./tests/cases/defaults.sh"}
+if [[ $# -ge 1 ]]; then
+    TEST_CASE=${1}
+    test -n "${TEST_CASE}"
+fi
+test -f ${PROJECT_DIR}/${TEST_CASE}
+
+export PROJECT="gpu-operator"
+
+SCRIPT_DIR="$( cd "$( dirname "${BASH_SOURCE[0]}" )"/scripts && pwd )"
+source ${SCRIPT_DIR}/.definitions.sh
+
+if [[ -n "$CLEANUP" ]]; then
+    echo "Running cleanup"
+    ${SCRIPT_DIR}/sync.sh
+    remote ./tests/scripts/uninstall.sh
+    exit 0
+fi
+
+# Launch a remote instance
+${SCRIPT_DIR}/launch.sh
+
+# Sync the project folder to the remote
+${SCRIPT_DIR}/sync.sh
+
+# We trigger the installation of prerequisites on the remote instance
+remote ./tests/scripts/prerequisites.sh
+
+# We trigger the specified test case on the remote instance
+remote ${TEST_CASE}