[CI/BUILD] Use build cache for Spyre tests

[ckadner]

Changes in this PR:

• Update GH workflow for Spyre tests to ...
  • build and cache a test image
  • use the cached test image for both v0 and v1 tests
  • run v0 and v1 tests in parallel
• Add build stages to Dockerfile, with separate stage for tests
• Add .dockerignore
• Delete untagged cache images
• AFTER MERGE: Update branch protection rules for test-spyre as merge requirement
  • Remove test-spyre:

    

  • Add test-spyre-v0 and test-spyre-v1:

    

Optional:

• Setup registry for large build cache images (Quay.io)
  • Setup Github secrets to authenticate to image registry (requires repo admin permissions)
• Reconsider running tests sequentially if image uplodad/download time outweighs overall time savings
  • parallel wins if one of the 2 test suites runs for more than 3 mins (90s upload + 90s download/load)
• Consider building cache images for branches (release, dev, nighlty, feature-*, ...)

Resolves #34

[github-actions]

👋 Hi! Thank you for contributing to vLLM support on Spyre.
Just a reminder: Make sure that your code passes all the linting checks, otherwise your PR won't be able to be merged. To do so, first install the linting requirements, then run format.sh and commit the changes:

pip install -r requirements-lint.txt
bash format.sh

Now you are good to go 🚀

[ckadner]

The full image build here took 3+ min. A cached image build takes 30+ sec.

There won't be any actual cached images until the next PR after this PR gets merged. Here is a snapshot of the execution times for a cached build on my fork:

[ckadner]

I updated the TODO list in the PR description (initial comment)

...

• Delete untagged cache images
• AFTER MERGE: Update branch protection rules for test-spyre/test-spyre as merge requirement

Optional:

• Setup registry for large build cache images (Quay.io)
  • Setup Github secrets to authenticate to image registry (requires repo admin permissions)
• Reconsider running tests sequentially if image uplodad/download time outweighs overall time savings
  • parallel wins if one of the 2 test suites runs for more than 3 mins (90s upload + 90s download/load)
• Consider building cache images for branches (release, dev, nighlty, feature-*, ...)

[ckadner]

@joerunde -- I am done with my intended changes and removed draft status for this PR.

Would you kindly give it another review?

[joerunde]

@ckadner are we still planning on merging this, or moving to a non-docker approach instead?

[ckadner]

@ckadner are we still planning on merging this, or moving to a non-docker approach instead?

@joerunde I have a pip based version and a uv based version of the test workflow. The one using uv is in PR #70

Either of them take about a minute longer than this Docker based version (when cached) 😄 🤷🏻 🤯

I don't know what other platforms we may want to test on in the future. Sticking with Docker, we could use cross-platform compilation to build and test Z/OS images.

[joerunde]

Thanks @ckadner - just to stick this on here for posterity I think for now all of our multi-platform builds are being done internally, so we're safe to delete the docker workflow here.

Mind re-purposing this PR to delete the Dockerfile after merging #70?

[ckadner]

Close in favor of #70