Question: is there any docker image available?

[hdnh2006]

I think to build a docker image would not be hard.

I can do it myself, but can you please recommend some base image? maybe something like this?

# Use the huggingface/transformers-pytorch-gpu image as the base image
FROM huggingface/transformers-pytorch-gpu

# Set the working directory inside the container
WORKDIR /workspace

# Update and install git
RUN apt-get update && apt-get install -y git

# Clone the GPTQModel repository and navigate into the directory
RUN git clone https://github.com/ModelCloud/GPTQModel.git && cd GPTQModel

# Install the GPTQModel with optional modules
RUN cd GPTQModel && pip install -v . --no-build-isolation

Then:

docker build -t gptqmodel-image .
docker run -it --gpus all --name gptqmodel-container gptqmodel-image /bin/bash

Unfortunately this image builf from huggingface is giving me the following log:

==========
== CUDA ==
==========

CUDA Version 12.1.0

Container image Copyright (c) 2016-2023, NVIDIA CORPORATION & AFFILIATES. All rights reserved.

This container image and its contents are governed by the NVIDIA Deep Learning Container License.
By pulling and using the container, you accept the terms and conditions of this license:
https://developer.nvidia.com/ngc/nvidia-deep-learning-container-license

A copy of this license is made available in this container at /NGC-DL-CONTAINER-LICENSE for your convenience.

*************************
** DEPRECATION NOTICE! **
*************************
THIS IMAGE IS DEPRECATED and is scheduled for DELETION.
    https://gitlab.com/nvidia/container-images/cuda/blob/master/doc/support-policy.md

So I don't know which one to use.

Any recommendation?

[CSY-ModelCloud]

we build our CI docker from nvidia/cuda.

you can try nvidia/cuda:12.6.3-devel-ubuntu24.04

[hdnh2006]

Ok, I am using the next Dockerfile (Ubuntu22 instead) and it seems to work

cat Dockerfile 
# Use NVIDIA CUDA base image
FROM nvidia/cuda:12.6.3-devel-ubuntu22.04

# Set the working directory in the container
WORKDIR /workspace

# Install system dependencies
RUN apt-get update && apt-get install -y \
    git \
    python3 \
    python3-pip \
    && rm -rf /var/lib/apt/lists/*

# Upgrade pip
RUN python3 -m pip install --upgrade pip

# Install PyTorch and other dependencies
RUN pip3 install torch torchvision torchaudio

# Install transformers and other required packages
RUN pip3 install transformers datasets scikit-learn scipy numpy

# Clone the GPTQModel repository
RUN git clone https://github.com/ModelCloud/GPTQModel.git

# Install GPTQModel
RUN cd GPTQModel && pip3 install -v . --no-build-isolation

# Set the default command to bash
CMD ["/bin/bash"]

Any suggestion would be appreciated.

[Qubitium]

Check what version of python is installed by default in 22.04 and 24.xx . We recommend 24.xx as it would likely use python 3.12 which is faster.