Skip to content

Commit aa6b666

Browse files
add example for llama 3.1 8b
1 parent 5f191fc commit aa6b666

File tree

1 file changed

+23
-0
lines changed

1 file changed

+23
-0
lines changed
Lines changed: 23 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,23 @@
1+
# ---
2+
# apiVersion: v1
3+
# kind: Namespace
4+
# metadata:
5+
# name: llama-3-1-8b-instruct
6+
---
7+
8+
apiVersion: ome.io/v1beta1
9+
kind: InferenceService
10+
metadata:
11+
name: llama-3-1-8b-instruct
12+
namespace: llama-3-1-8b-instruct
13+
spec:
14+
model:
15+
name: llama-3-1-8b-instruct
16+
engine:
17+
minReplicas: 8
18+
maxReplicas: 8
19+
runtime:
20+
name: srt-llama-3-1-8b-instruct
21+
router:
22+
minReplicas: 1
23+
maxReplicas: 1

0 commit comments

Comments
 (0)