KEP-2401: Validate fine-tuning configurations in torch plugin

[Electronic-Waste]

What you would like to be added?

In order to ensure the validity of the configurations propagated by SDK, we plan to add some validating requirements to the TrainJob Webhook. We'll implement validations in torch plugin CustomValidationPlugin:

• The ClusterTrainingRuntime referenced by runtime_ref exists in the control plane.

Ref: https://github.com/kubeflow/trainer/tree/master/docs/proposals/2401-llm-trainer-v2#validate-fine-tuning-configurations

Why is this needed?

Scheduled in #2410

Love this feature?

Give it a 👍 We prioritize the features with most 👍

[Electronic-Waste]

/remove-label lifecycle/needs-triage

[Electronic-Waste]

/area llm