intel/llvm-test-suite CI can be confusing

[sarnex]

Sorry for posting this here, discussions seem disabled at https://github.com/intel/llvm-test-suite

For the case where you need an intel/llvm PR for the intel/llvm-test-suite PR to pass, the CI is pretty confusing. The
pre-ci-linux and pre-ci-windows will fail because they aren't using the intel/llvm PR, leading some developers to think there's a real issue. This just happened to someone on my team.

Is there a way we can improve this? Maybe we can skip/cancel/something the pre-ci-{linux,windows} CI if there is a comment with "/verify with" or something?

[sarnex]

FYI @bader @intel/dpcpp-devops-reviewers

[bader]

This is going to be resolved by moving SYCL tests from llvm-test-suite to intel/llvm repository. Both DPC++ and test changes will be tested together within a single PR.

[sarnex]

Ah ok nice, if we don't have to make PRs for intel/llvm-test-suite anymore that will solve this. If there is an internal issue tracker for this please send it to me.

[bader]

Okay.

[v-klochkov]

In many cases seeing a fail for LLMV-LIT test is good because such fail shows that the test really verifies new functionality.
If some test claims to test new functionality in intel/llvm, but fails without PR in intel/llvm - then such test is not good for it's purpose.

[sarnex]

that could be fine, but it should be more clear somehow i think, maybe the CI item is named like "tests without PR" or something.