Question: threads for async/sync version of function

[dhowe]

My use case is a library (for node/browser) with a couple of long-running cpu-intensive functions. So I'd like to offer sync and async version of these functions (similar to the 'fs' module) with web workers... how to do this with 'threads' when the code already exists in a module (using require) ? thanks!

[andywer]

Hey @dhowe. You cannot easily offer a synchronous version of your library when using workers as communication with the workers is always async.

Also: Why would you want to offer a sync version? Blocking the main thread until the workers are done would destroy most of the performance gained by using workers 😉

Feel free to re-open if that didn't answer your question.

[dhowe]

Why would you want to offer a sync version?

One of the uses of this library is for teaching, and we want users to be able to use these functions before they understand sync/async

So the sync version wouldn't use threads, but the point is that it is the same function. I guess I was looking for an example like this...

ps. dont think I can reopen the issue

[andywer]

Tbh that sounds like you are trying to solve the issue in the wrong spot. If the library provides the functionality "do CPU-heavy thing X" then it probably makes more sense to just offer the sync version in the library and let the application that uses the library manage the threading logic.

Also keep in mind that

• there may be users of your library who don't want to use multi-threading – for them it would be a burden if the library comes with threads.js as a production dependency and
• bundling libraries that use threads.js is still not fully supported – that's a pretty tricky subject, see Spawn worker from blob / inlined bundle #211

[dhowe]

Well, the lack of bundling may make it a non-starter. But I think the use-case is valid. For teaching and other cases with unsophisticated users, expecting them to manage threading logic is not viable. Basically what I'm looking for is the the functionality of the 'fs' module for node, where you can simply choose fs.loadFile (async) or fs.loadFileSync depending on what you want (and understand).

[dhowe]

Putting the bundling aside for now, is this feasible with threads.js ?

[andywer]

Apart from the bundling, sure you could. You could just export the sync function and then an async one that spawns a worker and executes the function in the worker.

But then you don't want to spawn a new thread on every call and you probably want to have a pool of workers if you call that function frequently (worse: the library probably won't know if it will be called frequently or not). In case the application needs to do other CPU-intensive tasks, too, then you should ideally coordinate thread usage with those other functionalities as well, probably have a shared worker pool.

That's why I want to stress that dispatching work to threads should in most cases not be handled by the library, but the application. Long story short… technically it is feasible, but from a software architectural point of view it probably won't make sense, since the library usually lacks the knowledge how it's gonna be used.

I guess it comes down to threaded code being stateful due to the worker(s) management. The fs module functions are all stateless. That's why you cannot necessarily apply the same mindset to fs and a threaded library 😉