Help on serving OpenPose to Triton backend

[JJLimmm]

Similar to this issue #3648 , i wanted to ask if anybody managed to use Openpose with triton backend for inference.
I am planning to use the Body_25 model, but it is in caffe and that's not one of the supported backends for Triton...

Or has anybody managed to convert the caffe model from Openpose to ONNX/Pytorch/Tensorflow?

[tanmayv25]

Does this Tensorflow model resolve your issue: https://modelzoo.co/model/lightweight-openpose ?

[tanmayv25]

@JJLimmm If it is difficult to convert the OpenPose model to one of the Triton supported frameworks, you can write the serving in a python script and use Triton's Python backend to run the caffe model in Triton.
https://github.com/triton-inference-server/python_backend

[JJLimmm]

@tanmayv25 I saw this Python backend serving on triton as well. A little confused at their documentation. Not sure if i can just containerize the whole project in a docker image and run in with triton or i need to write a python script to start the triton inference.

[tanmayv25]

You can containerize the whole project. See the simple example here: https://github.com/triton-inference-server/python_backend/tree/main/examples/pytorch

Python backend allows users to deploy a python script as a model in Triton Inference Server.

The overall workflow would look like:

Triton Client -> Triton Server Endpoint -> Triton Core -> Triton Python Backend -> model.py

In your case the solution should look like:

Triton Client -> Triton Server Endpoint -> Triton Core -> Triton Python Backend -> model.py -> caffe model

[JJLimmm]

I see, so i will have to write out the TritonPythonModel() class.
Am i right if i do this below?

1. write initialize() method in TritonPythonModel() class to load the openpose caffe model via their Openpose Python Wrapper.
2. write the execute() method for handling the inference request, but via the OpenPose Wrapper.

For the second part, am i even still able to perform the inference request without using pb_utils library API (python backend utils)?
Or do you know of how i can still use the Openpose python wrapper within the pb_utils.InferenceResponse() API call?

[tanmayv25]

This looks accurate. You'd have to obtain the output as numpy array. Then create pb_utils.Tensor object from this numpy array and feed these pb_utils.Tensor objects as output tensors to pb_utils.InferenceResponse().

I am assuming Openpose python wrapper does provide results in form of numpy array, or in a format that can be converted to numpy.