[Question] Resource allocation for ray tuner

[PierrePeng]

Hi @garylvov @binw666. Great work on enabling resource allocation across different tasks in #2847. However, I’m wondering if this can be integrated with ray tuner.

For example, suppose I launch a Ray Tune job with 100 times trials but only have one GPU. Is it possible to run two trials concurrently on that single GPU? If so, could you provide a YAML example? Or is this feature not designed to work with ray tuner?

[RandomOakForest]

Thank you for posting this. The team will review, and will aim to post here an update.

[garylvov]

Currently, the resource allocation is meant for manual sweeps.

As for several isaac lab tune trials per GPU, if Isaac Sim supports several instances of itself on a GPU, then this is possible (or would be possible with minimal modifications).

I apologize I don't have the bandwidth to test this (as I have to focus on my PhD research), but if someone posts a log attempting this I would be happy to help debug. That being said, if Isaac Sim can't do multiple instances per GPU, then the Ray tune functionality definitely can't do this due to fundamental limitations in what it's trying to orchestrate.

[garylvov]

Also, training/tuning speed wise, it's probably better to have a single sim+learning instance with more samples, that uses closer to your total vram, than to have more than one. There is compute overhead associated with each instance.

[PierrePeng]

Hi @garylvov thanks for your response! That completely cleared up all my doubts.

From my tests, it seems that Isaac Sim does not support running multiple instances on a single GPU. I tried running three trials on one GPU, and I found that two of the trials had identical training curves (even with different hyperparameters).

The strategy you mentioned for improving sampling efficiency is indeed a wise approach. In my current RL task, I’m not using image-based observations, so even when I increase num_envs to 4096, I still can’t fully utilize the GPU’s VRAM (RTX4090) — but I can increase the batch_size instead.

Thanks a lot for your advice!

[garylvov]

No problem! I regularly train with 8192 environments for state-based or even vision-based. I even train with 16384, but I think for some reason 8192 maximizes performance most of the time from what I remember. I'd go as high as you can to utilize the vram, but just benchmark performance with a few trial runs