Implement thread management for parallel simulations

[arnaudgabas]

Hello!

I have to share with you a feature that I think would be very beneficial in the way parallel simulations are handled with spicelib.

Context

Most SPICE simulators provide a feature that allows the user to specify the maximum number of threads allowed for the simulation. LTspice has this feature, but only in the GUI, in the SPICE "Engine" parameters:

Two problems rises from that:

1. This "Max threads" option is only available in the GUI: it's not possible to set it with a .option directive, or with a command line switch ;
2. The option is not saved between invocations.

This means that even if you run a single simulation with the library, you can't really limit the number of threads/resources used by the LTspice process.

Why is it bad?

If the user wants to use the parallel simulation feature (SimRunner.run(parallel_sims=8) for example), each LTspice process created will try to use as many threads as possible, resulting in slower speed. As a result, the parallel simulation feature cannot really be fully exploited.

My suggestion

My suggestion would be to change the way LTspice processes are created in the simulator.py run_function() function. Instead of launching the LTspice batch command with subprocess.run(), each process invocation would be allocated a certain number of threads, and release them when the simulation is complete. This can be done with psutil and some thread allocate/release functions with threading. The function can still return a "retcode" with this method. I've made a first prototype of that, which seems to work correctly:

def run_function(command, timeout=None, stdout=None, stderr=None, thread_count=8):

    def run_with_affinity():
        
        process = subprocess.Popen(command, stdout=stdout, stderr=stderr)
        
        allocated_threads = thread_manager.allocate_threads(thread_count)
        
        if allocated_threads:
            p = psutil.Process(process.pid)
            p.cpu_affinity(allocated_threads)
            _logger.debug(f"Allocated threads {allocated_threads} to process {process.pid}")
        
        return_code = process.wait(timeout=timeout)
        
        if allocated_threads:
            thread_manager.release_threads(allocated_threads)
        
        return return_code

    with concurrent.futures.ThreadPoolExecutor(max_workers=1) as executor:
        future = executor.submit(run_with_affinity)
        return_code = future.result(timeout=timeout)
        return return_code

This method applies no thread restrictions to the main script.
Also, with this prototype, if there are no more threads available to run a simulation, the LTspice process will run without any thread restriction. This implies that the thread management (and therefore the number of parallel simulations) should be more restricted so that each process (or simulation, whatever you want to call it) runs with a specific number of allocated threads.
Of course, this run_function prototype generates exceptions which are handled in the RunTask.run function.

What do you think of that?

[hb020]

looks nice. My remarks:

• threadmanager is not too recent, and doesn't seem to have been tested on MacOS, so it would need at least some more testing. On the other hand, you do not seem to use many of the advanced features of it, and just use it to manage reservations in a pool (where is the init of the pool?). And that can be done with more mainstream methods.
• to make it easier for the users, we should auto detect the CPUs on the host, to provide default values.
• Is this potentially useful under other simulators? I guess so.
• do you have some test workloads that would be good? I can set them up, but it seems to me you already have some, so just to speed up...

[arnaudgabas]

threadmanager is not too recent, and doesn't seem to have been tested on MacOS, so it would need at least some more testing.

About that, I don't really know. I've only tested this first code on two machines (12 threads and 64 threads).

On the other hand, you do not seem to use many of the advanced features of it, and just use it to manage reservations in a pool (where is the init of the pool?). And that can be done with more mainstream methods.

This prototype is a simple approach: there must be better ways. The pool is initialized each time the function is called (with concurrent.futures.ThreadPoolExecutor(max_workers=1) as executor:) which is also not well optimized. Maybe creating a global thread pool would be preferable (especially if several simulations are run in parallel).

to make it easier for the users, we should auto detect the CPUs on the host, to provide default values.

For the example I've set the number of threads to a fixed number so that I can monitor CPU activity, but something can be done with psutil.cpu_count() or multiprocessing.cpu_count().

do you have some test workloads that would be good? I can set them up, but it seems to me you already have some, so just to speed up...

The simplest test you can do is to run a simulation with a very large simulation time. I took Batch_Test.net for my tests and I put .tran 1k as a simple trick.

[nunobrum]

Nice initiative. I'm mostly in favor of improving things. If you decide to advance in this direction and propose a pull-request, I only have a few recommendation and caution points.

• As @hb020 said, spicelib is multi-os, and also multi-simulator, I would like that it won't be just fine tuned to LTspice or just windows. It took quite a while to get up to a solution that could work with all these different ecosystems. I guess it would be wiser to keep this in a separate branch/version till we make sure we have nailed it correctly on all different OSes/Simulators/Simulation Types.
• The number of threads might not be the only bottleneck on the system. Memory and I/O might be also limitation factors.
• Although LTspice has this Max Threads configuration, since you have been looking at processes, how confident are we that it really uses all those threads and in which circumstances. All these spice simulators that were derived from Berkley's 3F5 spice simulator, and solving one single matrix, I guess they are mostly one core. Of course this limitation is not applicable when stepping values or for threads that are managing UI. Understanding how/when the threads are allocated might help to optimize.
• I guess, that a full optimized solution is going to be something that transcend a bit just a code on the run_function. If you haven't done it yet, I invite you to take a look at the sources in simulators folder where we define commands for each simulator.
• If there was only thing that this library taught me, is to always start the development by writing the unittest. I wasn't doing it at the beginning and for a while, it seemed that each new version was worst than the previous.