add aws support to ai-inference in pkb #6201
Conversation
…ing custom branch
perfkitbenchmarker/resources/kubernetes/wg_serving_inference_server.py
Outdated
Show resolved
Hide resolved
perfkitbenchmarker/resources/kubernetes/wg_serving_inference_server.py
Outdated
Show resolved
Hide resolved
perfkitbenchmarker/data/container/kubernetes_ai_inference/serving_catalog_cli.yaml.j2
Outdated
Show resolved
Hide resolved
…repo and branch to flags, removed legacy list()
|
|
||
| flags.DEFINE_string( | ||
| 'wg_serving_repo_branch', | ||
| 'main', |
There was a problem hiding this comment.
Nice. I was wondering if you'd have to leave off values altogether with like a jinja if, but this totally makes sense as defaults.
| self.cluster.ApplyManifest( | ||
| 'container/kubernetes_ai_inference/serving_catalog_cli.yaml.j2', | ||
| image_repo=FLAG_IMAGE_REPO.value, | ||
| wg_serving_repo_url=FLAGS.wg_serving_repo_url, |
There was a problem hiding this comment.
Ok, flag location is always tricky though. Let's put the flags in this file (wg_serving_inference_server) & use flagholders with eg REPO_URL = FLAGS.DEFINE.... REPO_URL.value.
|
/gcbrun |
…o-ai-inference-pkb PiperOrigin-RevId: 836297609
|
|
||
| def _GetInferenceServerManifest(self) -> str: | ||
| """Generates and retrieves the inference server manifest content.""" | ||
| # Ensure GPU capacity exists before scheduling GPU workloads |
There was a problem hiding this comment.
On my end this looks merged in:
https://screenshot.googleplex.com/AREUuCaL76yAdQg
Ooh ok actually perhaps this has made it in but been superceded by follow up PRs? https://screenshot.googleplex.com/BNVuvNLwNKckBF5 is the current state of wg_serving_inference_server.py & it does have a cloud == 'AWS' section.
I suppose we simply close this PR?
2 participants
add aws support to ai-inference in pkb with temporary link to wg-serving custom branch