[feat]: GPU Optimizer and Simulator development app#430

Merged

Jeffwan merged 45 commits intomainfrom jingyuan/autoscaler

Nov 27, 2024

+9,700-42

Commits on Sep 26, 2024

Integrate Vidur as a LLM simulator.
Jingyuan Zhang
committed

Commits on Sep 27, 2024

Merge commit 'main' into jingyuan/autoscaler
Jingyuan Zhang
committed

Commits on Oct 3, 2024

Test deployment
Jingyuan Zhang
committed
Merge commit 'f9702c3d8da7ab4d0e6fd8c5877c1e57ce528fb9' into jingyuan/autoscaler
Jingyuan Zhang
committed

Commits on Oct 8, 2024

Fix debug log output
Jingyuan Zhang
committed

Commits on Oct 24, 2024

Merge commit '5d8d8439077b08b12fd3de62ad7c4e6e2fb6e4ed' into jingyuan/autoscaler
Jingyuan Zhang
committed

Commits on Oct 28, 2024

Integrate gpu optimizer server that exposes customized pod metrics to guide autoscaling.
Jingyuan Zhang
committed
Merge commit 'ea5dc7784fa767fcf40b61417627ab47b6dba426' into jingyuan/autoscaler
Jingyuan Zhang
committed
bug fix: autoscaler with metricsSources now works.
Jingyuan Zhang
committed

Commits on Oct 31, 2024

Decoupled workload monitoring and visualizer
Jingyuan Zhang
committed

Commits on Nov 8, 2024

Integrate ILP solver and profile to GPU optimizer. Aggregated traces are supported now.
Jingyuan Zhang
committed

Commits on Nov 11, 2024

Add support for minimum replicas.
Jingyuan Zhang
committed
Merge branch 'main' into jingyuan/autoscaler
Jingyuan Zhang
committed

Commits on Nov 15, 2024

Debugged GPU profile benchmark, generation, and loading from file.
Jingyuan Zhang
committed

Commits on Nov 18, 2024

Add redis support for profile exchange.
Jingyuan Zhang
committed

Commits on Nov 20, 2024

bug fix: Duplicate 'http://' on calling RestMetricsFetcher.FetchPodMetrics (#408 ), Add abstractMetricsFetcher to make CustomMetricsFetcher, ResourceMetricsFetcher, and KubernetesMetricsFetcher comp…
Jingyuan Zhang
committed
bug fix
Jingyuan Zhang
committed
Merge branch 'issues/408_Duplicated_http_in_RestMetricsFetcher' into jingyuan/autoscaler
Jingyuan Zhang
committed

Commits on Nov 21, 2024

Tuning the granularity of aggregated profile
Jingyuan Zhang
committed
Adjust request trace to finer granularity.
Jingyuan Zhang
committed
Make request trace self-explanatory on time interval
Jingyuan Zhang
committed
Merge branch 'jingyuan/finer_profile_granuality' into jingyuan/autoscaler
Jingyuan Zhang
committed

Commits on Nov 22, 2024

Apply new request trace schema.
Jingyuan Zhang
committed
Add Readme.md for demo walkthrough.
Jingyuan Zhang
committed
Merge branch 'main' into jingyuan/finer_profile_granuality
Jingyuan Zhang
committed
Merge branch 'jingyuan/finer_profile_granuality' into jingyuan/autoscaler
Jingyuan Zhang
committed
Remove model cache
Jingyuan Zhang
committed
Remove TargetPort changes, which is not used.
Jingyuan Zhang
committed
Fix deployment
Jingyuan Zhang
committed
Organize Imports
Jingyuan Zhang
committed

Commits on Nov 23, 2024

Lint fix
Jingyuan Zhang
committed
Lint fix
Jingyuan Zhang
committed
ruff reformat
Jingyuan Zhang
committed
Passed mypy
Jingyuan Zhang
committed
Lint fix
Jingyuan Zhang
committed
pass mypy for redis
Jingyuan Zhang
committed
ruff again
Jingyuan Zhang
committed
Pass lint
Jingyuan Zhang
committed
Python 3.9 format check
Jingyuan Zhang
committed
Python 3.8 format check
Jingyuan Zhang
committed

Commits on Nov 25, 2024

Add a40 deployment
Jingyuan Zhang
committed

Commits on Nov 26, 2024

Bug fix
Jingyuan Zhang
committed
Improve benchmark stability.
Jingyuan Zhang
committed

Commits on Nov 27, 2024

Fix python file names.
Jingyuan Zhang
committed
python CI test bump to 3.10
nwangfw
committed