This repository was archived by the owner on Oct 19, 2024. It is now read-only.

How to use Alpa to serve BERT models #962

Open

opened

on Oct 20, 2023

Hi,

As title, are we able to run BERT models with Alpa backends? The llm_serving package includes OPT, BLOOM, and CodeGen models, and I'm wondering if we have similar ways to run BERT or other non-autoregressive models with Alpa.

Thanks

to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Assignees

No one assigned

Labels

No labels

No labels

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests