Will we ever get training tools like axolotl for gpt-oss type models. Because 8 B60's RIP through distill

[cdscustoms-coder]

I ran a distill off gpt-oss-120b today, at 32 concurrent I was going over 750 t/s at like 85 watts per GPU.

Then disappointment set it, I have to dequant the model and run on CPU and system ram for training gpt-oss-20b.

I am training the smaller model for my business on my business domains.

distill makes nice screenshots though. See attached.

[cdscustoms-coder]

Or at least Lora adapter support?

I mean these cards are incredible for distillation, would be nice to be able to use them to train a model.

[Wesley-Du]

@cdscustoms-coder thanks for the suggestions, we may look into training when there is resource, you are more than welcome to contribute!

[cdscustoms-coder]

Hi Wesley — thanks for the response!
I’m new to contributing at this level, but very motivated to help.

I’d like to start working toward LoRA adapter support and (eventually) full training support on Arc Pro / Battlematrix GPUs. Before I begin, I want to confirm the environment Intel prefers so that any work I do is reproducible on your side.

Here is my understanding — can you please confirm or correct anything?

1. OS + Kernel

Intel’s docs recommend Ubuntu 25.04, and I see kernel 6.18 just landed as the new LTS, so I plan to install a clean 25.04 environment on a separate NVMe so I can test without any contamination from my main system.

2. GPU / oneAPI stack

I plan to install:

Intel GPU compute drivers

oneAPI Base Toolkit

IPEX-LLM / Intel Extension for PyTorch

Latest llm-scaler (currently using the “b6” release)

Is this the stack you want LoRA work to target?

3. Scope of contribution

My initial goals would be:

1. Get a minimal LoRA fine-tuning example running on XPU (even on a small model).

2. Extend that pipeline toward gpt-oss-20b MXFP4.

3. Upstream any working scripts / configs so others can build on them.

GPT-OSS-20B actually writes great Python, so I’ll even let it help generate clean, documented training code as I go. 😊

4. What I need from Intel

To avoid wasting time on the wrong versions, could you clarify:

Preferred kernel version for training development?

Preferred IPEX-LLM version / branch?

Any reference model you want used for the first LoRA demo (1B–8B)?

Any internal guidelines you want followed for adding CLI flags and configs inside llm-scaler?

Once I have this, I’ll set up the clean environment and start testing.
Happy to contribute however I can.

Thanks again!

[liu-shaojun]

Hi, thanks for the interest and detailed plan!

For the environment setup, please follow our documented guide here:
https://github.com/intel/llm-scaler/tree/main/vllm#11-install-bare-metal-environment

We provide an Offline Installer to complete the full bare-metal install. The included install.sh already installs the target kernel 6.14.0-1006-intel and all required dependencies (oneCCL, xpu-smi, Intel GPU drivers, oneAPI, etc.), so no manual kernel selection is needed.

After installation, please use our b6 image, which is built from:
https://github.com/intel/llm-scaler/blob/main/vllm/docker/Dockerfile

As shown here:
https://github.com/intel/llm-scaler/blob/main/vllm/docker/Dockerfile#L51-L57

we currently apply changes by patching vLLM. For development, you can modify code inside the container, uninstall vLLM, then rebuild it to test your changes.

Hope this helps, and thanks for contributing!