linear_random_pytorch

Linear model in PyTorch

Overview

The example presents a simple Linear model implemented in PyTorch

Example consists of following scripts:

• server.py - start the model with Triton Inference Server
• client.py - execute HTTP/gRPC requests to the deployed model

Requirements

The example requires the torch package. It can be installed in your current environment using pip:

pip install torch

Or you can use NVIDIA PyTorch container:

docker run -it --gpus 1 --shm-size 8gb -v {repository_path}:{repository_path} -w {repository_path} nvcr.io/nvidia/pytorch:24.08-py3 bash

If you select to use container we recommend to install NVIDIA Container Toolkit.

Quick Start

The step-by-step guide:

1. Install PyTriton following the installation instruction
2. In current terminal start the model on Triton using server.py

./server.py

3. Open new terminal tab (ex. Ctrl + T on Ubuntu) or window
4. Go to the example directory
5. Run the client.py to perform queries on model:

./client.py