Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

没有fastllm文件 #1

Open
ak646260187 opened this issue Aug 16, 2023 · 2 comments
Open

没有fastllm文件 #1

ak646260187 opened this issue Aug 16, 2023 · 2 comments

Comments

@ak646260187
Copy link

您好,可以分享下fastllm文件吗,想试下效果

@ArtificialZeng
Copy link
Owner

baichuan模型导出 (默认脚本导出baichuan-13b-chat模型)

# 需要先安装baichuan环境
# 如果使用自己finetune的模型需要修改baichuan2flm.py文件中创建tokenizer, model的代码
# 根据所需的精度,导出相应的模型
cd build
python3 tools/baichuan2flm.py baichuan-13b-fp16.flm float16 #导出float16模型
python3 tools/baichuan2flm.py baichuan-13b-int8.flm int8 #导出int8模型
python3 tools/baichuan2flm.py baichuan-13b-int4.flm int4 #导出int4模型

@ArtificialZeng
Copy link
Owner

是通过这个命令导出, 转化的

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants