Decouple E2E Tests from Cluster Setup #409
Description
Currently, the Go test suite (e2e_suite_test.go) is tightly coupled to Kind (Kubernetes in Docker), as it contains hardcoded commands for creating the cluster, loading images, and deploying prerequisite resources like CRDs and Envoy. This coupling limits flexibility and increases the complexity of the tests themselves.
Problem Statement
Embedding cluster setup logic directly within the Go e2e tests introduces several significant problems:
Tight Coupling to Kind:
The test suite contains hardcoded exec.Command calls specifically for kind (e.g., kind create cluster, kind load image-archive). This locks the entire test suite to Kind, preventing it from being run against potentially more representative environments like OpenShift Local (CRC), ROSA, OSD, or other OpenShift clusters without significant code changes.
Ex: https://github.com/llm-d/llm-d-inference-scheduler/blob/main/test/e2e/e2e_suite_test.go#L101 and https://github.com/llm-d/llm-d-inference-scheduler/blob/main/test/e2e/e2e_suite_test.go#L134
Reduced Flexibility and Portability:
Developers or CI systems cannot easily substitute a different cluster type (e.g., a pre-existing CRC or cloud-based OpenShift cluster) for testing without modifying the Go test code. This hinders debugging downstream-specific issues and limits our ability to validate against diverse environments.
Increased Test Complexity and Blurry Concerns:
The Go tests currently mix infrastructure setup concerns (creating clusters, installing CRDs, deploying Envoy) with application testing concerns (creating LLMInferenceServices, verifying behavior). This makes the tests longer, harder to read, and more difficult to debug, as failures could stem from either the setup logic or the actual application logic.
Difficult Maintenance:
Any changes required for the underlying test infrastructure (e.g., updating Kind configuration, changing prerequisite deployments like Envoy, modifying CRD installation) necessitate modifying and recompiling the Go test code. Infrastructure setup should ideally be managed through declarative configurations or scripts, separate from the application test logic.
Hidden Dependencies and Setup Logic:
Critical setup steps (like CRD installation or Envoy deployment) are buried within Go functions (BeforeSuite). This makes the test prerequisites less visible and harder to manage compared to having explicit setup steps defined in a Makefile or dedicated setup scripts.