Repository navigation
Live Site Test (idptools.com) #182
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| name: Live Site Test (idptools.com) | |
| # Runs the full Selenium suite against the DEPLOYED PROD site | |
| # (https://idptools.com) by invoking ./remote-run-tests.sh — the same script used | |
| # to run this locally, so there is no CI-specific copy of the setup to drift. | |
| # | |
| # See the test-site workflow (website-test-live.yml) for why: the previous path | |
| # used a compose file of its own that started keycloak + sts only, so the two | |
| # walt.id interoperability jobs were never added to the list and CI reported 50 | |
| # tests where a local run reports 52. | |
| # | |
| # Runs automatically after a successful prod deploy, plus nightly and on demand. | |
| # | |
| # NOTE: the workflow_run trigger only fires from this file on the DEFAULT branch | |
| # (master). | |
| on: | |
| # After the prod deploy finishes. | |
| workflow_run: | |
| workflows: ["Deploy Static Site (idptools.com)"] | |
| types: [completed] | |
| workflow_dispatch: | |
| # Nightly, staggered after the test-site live run (11:00 UTC). | |
| schedule: | |
| - cron: '0 12 * * *' | |
| permissions: | |
| contents: read | |
| jobs: | |
| live-test: | |
| runs-on: ubuntu-latest | |
| # Two Keycloaks, the mock STS and walt.id's issuer and verifier all have to | |
| # come up before ~52 browser tests run against a real site. | |
| timeout-minutes: 60 | |
| # For a post-deploy (workflow_run) trigger, only run if the deploy | |
| # succeeded. Manual dispatch and the nightly schedule always run. | |
| if: ${{ github.event_name != 'workflow_run' || github.event.workflow_run.conclusion == 'success' }} | |
| steps: | |
| - name: Checkout Repository | |
| uses: actions/checkout@v7 | |
| with: | |
| # remote-run-tests.sh builds the STS locally (keycloak-tests.yml) to run | |
| # the suite against the deployed site, and its source is the sts/ | |
| # submodule — https://github.com/rcbj/iya-sts.git. | |
| # `recursive`, not `true`, and the difference is load-bearing: there | |
| # are now submodules INSIDE submodules. api/node-ldapjs is the LDAP | |
| # library the api's client uses, and sts/node-ldapjs is the same | |
| # library inside the mock STS, which its embedded directory is built | |
| # on. `true` stops one level short of the second, and an uninitialised | |
| # submodule is an EMPTY DIRECTORY — so the sts image builds, npm | |
| # installs a package with no main, and the container dies at startup | |
| # with `Cannot find module 'ldapjs'`, naming a package rather than a | |
| # submodule. | |
| submodules: recursive | |
| # Full history, so the step below can check out whichever commit the | |
| # site was built from. | |
| fetch-depth: 0 | |
| # Test the commit idptools.com was BUILT FROM, not the one this run was | |
| # handed — the same step the test-site workflow gained in d4ab242, for | |
| # the same reason. A workflow_run or scheduled run is on master, which is | |
| # what prod deploys from, so this is usually a no-op; a MANUAL dispatch | |
| # on develop is not. Run 36376517213 was started on develop at 0cb27f1 | |
| # against a site built from master at 2d46301, and failed scim_page on a | |
| # fix the site did not have yet. The site's own version.json is the one | |
| # record of what it is. | |
| - name: Check out the deployed commit | |
| run: | | |
| set -euo pipefail | |
| url=https://idptools.com/version.json | |
| commit="$(curl -fsSL "$url" | jq -r '.commit // empty')" || commit="" | |
| if [ -z "$commit" ]; then | |
| echo "::warning::$url named no commit; testing origin/master," \ | |
| "which is the branch this site deploys from." | |
| commit=origin/master | |
| fi | |
| echo "Deployed commit: $commit" | |
| git checkout --detach "$commit" | |
| git submodule update --init --recursive | |
| git log -1 --oneline | |
| # The script runs run-report.js and the test scripts on the runner itself, | |
| # not in a container, so the runner needs the node the images use. | |
| - name: Set up Node | |
| uses: actions/setup-node@v4 | |
| with: | |
| node-version: '24' | |
| # xmllint is what the WS-Trust schema test validates with when the optional | |
| # native libxmljs2 binding is unusable. The tests image installs | |
| # libxml2-utils for that reason; here the suite runs on the host instead. | |
| - name: Install xmllint | |
| run: sudo apt-get update -qq && sudo apt-get install -y -qq libxml2-utils | |
| # Selenium drives the runner's own Chrome (preinstalled on ubuntu-latest) | |
| # and fetches a matching driver itself. Printed so that a browser-version | |
| # problem shows up in the log rather than as a mystery timeout. | |
| - name: Show tool versions | |
| run: | | |
| node --version | |
| docker compose version | |
| google-chrome --version || true | |
| # The script picks CONFIG_FILE from the URL (./env/prod.js for this one) and | |
| # derives the SAML/API env itself, so the URL is the only input. It exits | |
| # non-zero when a test fails: run-report.js exits 1 and check_return_code | |
| # propagates it. | |
| - name: Run the suite against the live PROD site | |
| run: ./remote-run-tests.sh https://idptools.com | |
| # The script deliberately leaves the stack up for fast local re-runs. A | |
| # runner is thrown away, but tearing down keeps the log honest about what | |
| # was left running. | |
| - name: Tear down the identity stack | |
| if: always() | |
| run: sudo docker compose -p idptools-kctest -f keycloak-tests.yml down -v --remove-orphans || true | |
| - name: Normalize report permissions | |
| if: always() | |
| run: sudo chown -R "$(id -u):$(id -g)" tests/report || true | |
| - name: Upload test report | |
| if: always() | |
| uses: actions/upload-artifact@v7 | |
| with: | |
| name: live-test-report-prod | |
| path: tests/report/latest | |
| if-no-files-found: warn | |
| retention-days: 30 |