[CI] improve the ci efficiency by parallelizing the build tasks

[nwangfw]

Pull Request Description

In the initial version, we do not care the efficiency that much and we use single runner to build the images one by one.

In this PR, we create four runners to build images in parallel and then use another runner to conduct tests.

Related Issues

Resolves: #[128]

[nwangfw]

Tested on local mac machine, reduced about 40% docker building time. However, based on online Integration Test, there is no performance gain.

[Jeffwan]

did you try to use make -j4 here?

https://github.com/aibrix/aibrix/blob/b0413e9ca4a1e912b783b346d1be0a04f187f54b/.github/workflows/installation-tests.yml#L37-L39

[Jeffwan]

there's another option

leverage the matrix to increase the parallelisme
https://docs.github.com/en/actions/writing-workflows/workflow-syntax-for-github-actions#jobsjob_idstrategymatrix

    strategy:
      matrix:
        image: [controller-manager, plugins, runtime, users]
# then let's do not use make docker-build-all, instead, let's build separate ones.

[Jeffwan]

Please help write the PR description and linked issue. check #205 as an example

[xieus]

Tested on local mac machine, reduced about 40% docker building time. However, based on online Integration Test, there is performance gain.

No performance gain for online integration test?

[nwangfw]

Tested on local mac machine, reduced about 40% docker building time. However, based on online Integration Test, there is performance gain.

No performance gain for online integration test?

Tested on local mac machine, reduced about 40% docker building time. However, based on online Integration Test, there is performance gain.

No performance gain for online integration test?

No, thanks for finding this typo. Will try some new ideas as Jiaxin suggested.

[nwangfw]

there's another option

leverage the matrix to increase the parallelisme https://docs.github.com/en/actions/writing-workflows/workflow-syntax-for-github-actions#jobsjob_idstrategymatrix

    strategy:
      matrix:
        image: [controller-manager, plugins, runtime, users]
# then let's do not use make docker-build-all, instead, let's build separate ones.

The matrix approach works after some dependency modification! Now, we are able to have 4 runners to build images in parallel. The overall test time reduced 1/3 (from ~9m to ~6m)

[Jeffwan]

em. are those images not visible to further loading process? is artifact upload required?

[nwangfw]

In the previous version, we only used 1 runner and we don't need upload and download steps. However, we use 5 runners in total (the first 4 runners to build images and the last runner to run the installation test) and that's why we need upload process for the first four runners and the download process for the last runner.