Skip to content

memory consumption calculation method for deploying LLM using vllm #3

@Projoke

Description

@Projoke

Hi, can you provide the memory consumption calculation method for deploying LLM using vllm, as well as the computational approach for multi-machine and multi-GPU?

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions