Skip to content

Cannot create managed on-host memory with the alignment specified in buffer_requirements #5373

Description

@xuke537

Hi,

I am implementing an NPU dispatch shared object for our NPU. Our NPU has special alignment requirements (aligned with the burst size) for input/output buffers allocated in host memory. I have set the alignment to 256 in buffer_requirements, but it always allocates aligned memory according to LITERT_HOST_MEMORY_BUFFER_ALIGNMENT 64.

Call STACK:

LiteRtTensorBufferT::CreateManagedWithAlignment(LiteRtEnvironment env, LiteRtTensorBufferType buffer_type, const LiteRtRankedTensorType & tensor_type, size_t buffer_size, size_t alignment) (\workspaces\liteRT\LiteRT\litert\runtime\tensor_buffer.cc:676)
LiteRtTensorBufferT::CreateManaged(LiteRtEnvironment env, LiteRtTensorBufferType buffer_type, const LiteRtRankedTensorType & tensor_type, size_t buffer_size) (\workspaces\liteRT\LiteRT\litert\runtime\tensor_buffer.cc:666)
LiteRtCreateManagedTensorBuffer(LiteRtEnvironment env, LiteRtTensorBufferType buffer_type, const LiteRtRankedTensorType * tensor_type, size_t buffer_size, LiteRtTensorBuffer * tensor_buffer) (\workspaces\liteRT\LiteRT\litert\c\litert_tensor_buffer.cc:404)
litert::TensorBuffer::CreateManaged(const litert::Environment & env, litert::TensorBufferType buffer_type, const litert::RankedTensorType & tensor_type, size_t buffer_size) (\workspaces\liteRT\LiteRT\litert\cc\litert_tensor_buffer.cc:51)
litert::CompiledModel::CreateBufferImpl(const litert::Environment & env, const litert::TensorBufferRequirements & buffer_requirements, const litert::RankedTensorType & tensor_type) (\workspaces\liteRT\LiteRT\litert\cc\litert_compiled_model.cc:80)
litert::CompiledModel::CreateInputOutputBuffer(const litert::CompiledModel * const this, size_t signature_index, absl::lts_20250814::string_view tensor_name, bool is_input) (\workspaces\liteRT\LiteRT\litert\cc\litert_compiled_model.cc:108)
litert::CompiledModel::CreateInputOutputBuffers(const litert::CompiledModel * const this, size_t signature_index, bool is_input) (\workspaces\liteRT\LiteRT\litert\cc\litert_compiled_model.cc:136)
litert::CompiledModel::CreateInputBuffers(const litert::CompiledModel * const this, size_t signature_index) (\workspaces\liteRT\LiteRT\litert\cc\litert_compiled_model.h:366)
litert::(anonymous namespace)::RunModel() (\workspaces\liteRT\LiteRT\litert\tools\run_model.cc:319)
main(int argc, char ** argv) (\workspaces\liteRT\LiteRT\litert\tools\run_model.cc:397)

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Labels

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions