Skip to content

Commit

Permalink
Update config.py
Browse files Browse the repository at this point in the history
  • Loading branch information
Esmail-ibraheem authored May 7, 2024
1 parent 58845b0 commit 1fc99e4
Showing 1 changed file with 1 addition and 1 deletion.
2 changes: 1 addition & 1 deletion config.py
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@
class LlamaConfig:
dim: int = 4096
n_layers: int = 32
n_heads: int = 32
n_heads: int = 32
n_kv_heads: Optional[int] = None
vocab_size: int = -1 # Later set in the build method
multiple_of: int = 256
Expand Down

0 comments on commit 1fc99e4

Please sign in to comment.