How to get the parameters introduction in config.pbtxt and the relationship between them?

[activezhao]

When I just want to know the meaning of the parameter introduction in config.pbtxt, I do not know where the introduction is.

For example, I want to open inflight_batching, but I can not find a place about the introduction of inflight_batching until I found this https://github.com/triton-inference-server/tensorrtllm_backend/blob/release/0.5.0/inflight_batcher_llm/README.md.

This is actually very inconvenient, especially for some new features. It would be nice if a special place could be provided to introduce these parameters.

Thanks.

[byshiue]

Hi, @activezhao . The document of inflight-batching is at TensorRT-LLM repo, you can find them here https://github.com/NVIDIA/TensorRT-LLM/blob/release/0.5.0/docs/source/batch_manager.md#the-batch-manager-in-tensorrt-llm. We are also going to improve the document to prevent confusion.

[activezhao]

Hi, @activezhao . The document of inflight-batching is at TensorRT-LLM repo, you can find them here https://github.com/NVIDIA/TensorRT-LLM/blob/release/0.5.0/docs/source/batch_manager.md#the-batch-manager-in-tensorrt-llm. We are also going to improve the document to prevent confusion.

@byshiue Cool, look forward to the new document.
Thanks.