Parallelize only offline rendering, keep playback serial

Author: juzzlin

CHANGELOG

@@ -70,14 +70,14 @@ Bug fixes:
 
 Other:
 
-* Process audio serially on the audio thread by default
-  - The parallel device worker pool added per-buffer synchronization and cache
-    overhead that outweighed its benefit for realistic projects and could cause
-    stutter under load; serial processing is smoother and uses less CPU
-  - Opt into the pool with NOTEAHEAD_AUDIO_WORKERS ("auto" for a hardware-based
-    count, or a specific number of worker threads)
-  - Warn in the log when worker threads are requested but real-time scheduling
-    is unavailable
+* Process real-time playback serially, parallelize only offline rendering
+  - Splitting per-buffer device processing across threads added synchronization
+    and cache overhead that caused stutter during playback, so playback now runs
+    serially on the audio thread (smoother, lower CPU)
+  - Offline rendering/export still fans out across worker threads (at normal
+    priority) for faster exports without affecting playback or the UI
+  - NOTEAHEAD_AUDIO_WORKERS tunes the render worker count ("auto", a number, or
+    0 for single-threaded rendering)
 
 * Double the maximum number of devices and effects per rack to 16
 

README.md

@@ -321,23 +321,20 @@ Transposition (also available via right-clicking on the editor):
 ##
 ## Audio performance
 
-The internal devices and effects are rendered on the audio thread. By default all device processing
-runs **serially** on that thread, which for realistic projects is both smoother and lighter on the
-CPU than splitting the work across threads.
-
-An optional parallel worker pool is available for very large projects that a single core cannot keep
-up with. It is opt-in via the `NOTEAHEAD_AUDIO_WORKERS` environment variable:
-
-    $ NOTEAHEAD_AUDIO_WORKERS=auto ./noteahead   # hardware-based number of worker threads
-    $ NOTEAHEAD_AUDIO_WORKERS=4 ./noteahead       # a specific number of worker threads
-    $ NOTEAHEAD_AUDIO_WORKERS=0 ./noteahead       # serial (the default)
-
-**Note**: The worker threads need real-time scheduling to be useful. Without it (i.e. no `rtprio`
-limit for your user, usually granted by being in the `audio` group) they get preempted while the
-audio thread waits for them, which causes stutter under load — Noteahead warns about this in the
-log. Because of the added per-buffer synchronization and CPU-cache overhead, the pool can be slower
-and glitchier than serial processing even when real-time scheduling *is* available, so only enable
-it if you measure a real improvement.
+**Real-time playback** processes all internal devices and effects **serially on the audio thread**.
+For realistic projects this is both smoother and lighter on the CPU than splitting the per-buffer
+work across threads, which adds synchronization and CPU-cache overhead that can cause stutter.
+
+**Offline rendering / export**, on the other hand, has no real-time deadline and is parallelized
+across worker threads for faster exports. The number of render worker threads can be tuned with the
+`NOTEAHEAD_AUDIO_WORKERS` environment variable:
+
+    $ NOTEAHEAD_AUDIO_WORKERS=auto ./noteahead   # hardware-based number of render threads (the default)
+    $ NOTEAHEAD_AUDIO_WORKERS=4 ./noteahead       # a specific number of render threads
+    $ NOTEAHEAD_AUDIO_WORKERS=0 ./noteahead       # render single-threaded
+
+These worker threads run at normal priority and only during export, so they do not affect playback
+smoothness or preempt the UI.
 
 ##
 ## Real-world test cases

src/infra/audio/audio_engine.cpp

@@ -352,7 +352,13 @@ void AudioEngine::process(AudioContext & context)
     }
     auto effects = m_sendEffectRack->effects();
     const size_t sendCount = effects.size();
-    const size_t laneCount = m_workerPool->laneCount();
+
+    // Parallelize across worker threads only for offline rendering/export (exclusive mode). Real-time
+    // playback runs serially on the audio thread: splitting the per-buffer work adds synchronization
+    // and cross-core cache overhead that hurts real-time performance, whereas offline rendering has no
+    // deadline and benefits from the extra throughput.
+    const bool useWorkers = m_isExclusive.load();
+    const size_t laneCount = useWorkers ? m_workerPool->laneCount() : 1;
 
     ensureWorkBuffers(laneCount, sendCount, bufferSize);
     ensureEffectWetBuffers(sendCount, bufferSize);
@@ -418,7 +424,13 @@ void AudioEngine::process(AudioContext & context)
                 bufferSize,
                 context.bpm
             };
-            m_workerPool->run(layer.size(), &deviceContext, processDeviceTask);
+            if (useWorkers) {
+                m_workerPool->run(layer.size(), &deviceContext, processDeviceTask);
+            } else {
+                for (size_t taskIndex = 0; taskIndex < layer.size(); taskIndex++) {
+                    processDeviceTask(&deviceContext, taskIndex, 0);
+                }
+            }
         }
 
         // Sum parallel results into the main output and send buses
@@ -446,7 +458,13 @@ void AudioEngine::process(AudioContext & context)
             context.sampleRate,
             context.bpm
         };
-        m_workerPool->run(sendCount, &effectContext, processEffectTask);
+        if (useWorkers) {
+            m_workerPool->run(sendCount, &effectContext, processEffectTask);
+        } else {
+            for (size_t taskIndex = 0; taskIndex < sendCount; taskIndex++) {
+                processEffectTask(&effectContext, taskIndex, 0);
+            }
+        }
 
         for (const auto & wetBuffer : m_effectWetBuffers) {
             for (uint32_t i = 0; i < bufferSize; i++) {

src/infra/audio/real_time_worker_pool.cpp

@@ -21,36 +21,13 @@
 #include <cstdlib>
 #include <pthread.h>
 #include <string_view>
-#include <sys/resource.h>
 
 namespace noteahead {
 
-namespace {
-
-// Real-time priority the worker threads request. Kept in one place so the availability probe and
-// the per-thread setup stay in sync.
-constexpr int workerRealtimePriority = 80;
-
-// Whether this process can actually give its worker threads real-time (SCHED_FIFO) priority. Without
-// it, non-real-time workers get preempted while the real-time audio thread blocks waiting for them,
-// which causes stutter under load — so the pool is worse than serial processing. RLIMIT_RTPRIO is
-// the binding constraint for a self-set priority (RTKit, used by e.g. PulseAudio for its own thread,
-// is not involved here), so it predicts whether pthread_setschedparam will succeed.
-bool canObtainRealtimePriority()
-{
-    struct rlimit rl;
-    if (getrlimit(RLIMIT_RTPRIO, &rl) != 0) {
-        return false;
-    }
-    return rl.rlim_cur == RLIM_INFINITY || rl.rlim_cur >= static_cast<rlim_t>(workerRealtimePriority);
-}
-
-} // namespace
-
 RealTimeWorkerPool::RealTimeWorkerPool(size_t workerCount)
 {
-    juzzlin::L("RealTimeWorkerPool").info() << "Starting with " << workerCount << " worker thread(s)"
-                                            << (workerCount == 0 ? " (serial processing on the audio thread)" : "");
+    juzzlin::L("RealTimeWorkerPool").info() << "Render worker pool: " << workerCount << " worker thread(s)"
+                                            << (workerCount == 0 ? " (single-threaded rendering)" : "");
 
     m_startSemaphores.reserve(workerCount);
     m_workers.reserve(workerCount);
@@ -64,17 +41,12 @@ RealTimeWorkerPool::RealTimeWorkerPool(size_t workerCount)
             enableHardwareDenormalProtection();
 
             // Set thread name for easier debugging
-            const std::string threadName = "AudioWorker-" + std::to_string(i);
+            const std::string threadName = "RenderWorker-" + std::to_string(i);
             pthread_setname_np(pthread_self(), threadName.c_str());
 
-            // Set real-time priority
-            struct sched_param param;
-            param.sched_priority = workerRealtimePriority; // High priority for audio
-            if (pthread_setschedparam(pthread_self(), SCHED_FIFO, &param) != 0) {
-                juzzlin::L("RealTimeWorkerPool").warning() << "Failed to set RT priority for " << threadName
-                                                          << " (needs rtprio limits, e.g. the 'audio' group); "
-                                                          << "audio may stutter under load. Set NOTEAHEAD_AUDIO_WORKERS=0 to disable the pool.";
-            }
+            // These workers only run during offline rendering/export, which has no real-time deadline,
+            // so they run at normal priority. Elevating them would risk preempting the UI thread while
+            // an export is in progress.
 
             workerLoop(i);
         });
@@ -159,39 +131,28 @@ size_t RealTimeWorkerPool::hardwareBasedWorkerCount()
 
 size_t RealTimeWorkerPool::defaultWorkerCount()
 {
-    // Serial processing on the audio thread is the default. Splitting the per-buffer device
-    // processing across worker threads adds fixed synchronization cost and bounces device/effect
-    // state between CPU caches every callback; for realistic projects (which fit comfortably in one
-    // core) that overhead outweighs any benefit and can cause stutter, even when the workers do get
-    // real-time priority. Opt in via NOTEAHEAD_AUDIO_WORKERS: "auto" for a hardware-based count, or a
-    // specific number of worker threads.
+    // The pool parallelizes offline rendering/export only (real-time playback runs serially on the
+    // audio thread), so a hardware-based worker count is a sensible default. Override with
+    // NOTEAHEAD_AUDIO_WORKERS: "auto" for the hardware-based count, a specific number of threads, or 0
+    // to render single-threaded.
     const char * env = std::getenv("NOTEAHEAD_AUDIO_WORKERS");
     if (!env || !*env) {
-        return 0;
+        return hardwareBasedWorkerCount();
     }
 
-    size_t workers = 0;
     if (std::string_view { env } == "auto") {
-        workers = hardwareBasedWorkerCount();
-    } else {
-        char * end = nullptr;
-        const long requested = std::strtol(env, &end, 10);
-        if (end == env || requested < 0) {
-            juzzlin::L("RealTimeWorkerPool").warning() << "Ignoring invalid NOTEAHEAD_AUDIO_WORKERS value: " << env;
-            return 0;
-        }
-        const auto hardwareThreads = std::thread::hardware_concurrency();
-        workers = static_cast<size_t>(std::min<long>(requested, hardwareThreads > 0 ? hardwareThreads : requested));
+        return hardwareBasedWorkerCount();
     }
 
-    if (workers > 0 && !canObtainRealtimePriority()) {
-        juzzlin::L("RealTimeWorkerPool").warning()
-          << "NOTEAHEAD_AUDIO_WORKERS requested worker threads but real-time scheduling is unavailable "
-          << "(RLIMIT_RTPRIO too low); expect stutter under load. Grant rtprio, e.g. join the 'audio' group.";
+    char * end = nullptr;
+    const long requested = std::strtol(env, &end, 10);
+    if (end == env || requested < 0) {
+        juzzlin::L("RealTimeWorkerPool").warning() << "Ignoring invalid NOTEAHEAD_AUDIO_WORKERS value: " << env;
+        return hardwareBasedWorkerCount();
     }
 
-    juzzlin::L("RealTimeWorkerPool").info() << "Audio worker threads requested via NOTEAHEAD_AUDIO_WORKERS: " << workers;
-    return workers;
+    const auto hardwareThreads = std::thread::hardware_concurrency();
+    return static_cast<size_t>(std::min<long>(requested, hardwareThreads > 0 ? hardwareThreads : requested));
 }
 
 void RealTimeWorkerPool::workerLoop(size_t workerIndex)

src/infra/audio/real_time_worker_pool.hpp

@@ -26,6 +26,9 @@
 
 namespace noteahead {
 
+//! Fan-out/fan-in worker pool used to parallelize offline rendering/export (see AudioEngine::process).
+//! Real-time playback runs serially on the audio thread and does not use this pool, so the workers run
+//! at normal priority. The name is historical.
 class RealTimeWorkerPool
 {
 public:
@@ -42,7 +45,7 @@ class RealTimeWorkerPool
 
     void run(size_t taskCount, void * context, TaskCallback callback);
 
-    //! Worker count used by default. Serial (0) unless opted into via NOTEAHEAD_AUDIO_WORKERS.
+    //! Worker count used by default for rendering. Hardware-based unless NOTEAHEAD_AUDIO_WORKERS overrides.
     static size_t defaultWorkerCount();
     //! Heuristic worker count based on the number of hardware threads (leaving headroom for UI).
     static size_t hardwareBasedWorkerCount();

src/unit_tests/real_time_worker_pool_test/real_time_worker_pool_test.cpp

@@ -81,10 +81,11 @@ void RealTimeWorkerPoolTest::test_singleTaskUsesCallerThread_shouldExecuteOnCurr
 
 void RealTimeWorkerPoolTest::test_defaultWorkerCount_envOverride_shouldBeRespected()
 {
-    // Default is serial (no worker threads) unless explicitly opted in.
+    // Unset uses the hardware-based render worker count.
     qunsetenv("NOTEAHEAD_AUDIO_WORKERS");
-    QCOMPARE(RealTimeWorkerPool::defaultWorkerCount(), static_cast<size_t>(0));
+    QCOMPARE(RealTimeWorkerPool::defaultWorkerCount(), RealTimeWorkerPool::hardwareBasedWorkerCount());
 
+    // 0 forces single-threaded rendering.
     qputenv("NOTEAHEAD_AUDIO_WORKERS", "0");
     QCOMPARE(RealTimeWorkerPool::defaultWorkerCount(), static_cast<size_t>(0));
 
@@ -95,9 +96,9 @@ void RealTimeWorkerPoolTest::test_defaultWorkerCount_envOverride_shouldBeRespect
     qputenv("NOTEAHEAD_AUDIO_WORKERS", "auto");
     QCOMPARE(RealTimeWorkerPool::defaultWorkerCount(), RealTimeWorkerPool::hardwareBasedWorkerCount());
 
-    // Invalid values fall back to serial.
+    // Invalid values fall back to the hardware-based default.
     qputenv("NOTEAHEAD_AUDIO_WORKERS", "not-a-number");
-    QCOMPARE(RealTimeWorkerPool::defaultWorkerCount(), static_cast<size_t>(0));
+    QCOMPARE(RealTimeWorkerPool::defaultWorkerCount(), RealTimeWorkerPool::hardwareBasedWorkerCount());
 
     qunsetenv("NOTEAHEAD_AUDIO_WORKERS");
 }

src/unit_tests/side_chain_audio_test/side_chain_audio_test.cpp

@@ -190,6 +190,37 @@ void SideChainAudioTest::test_audioEngine_process_runtimeSidechainChange_shouldR
     QCOMPARE(compressor->sidechainSourceDeviceIndex().value_or(999), 2u);
 }
 
+void SideChainAudioTest::test_audioEngine_serialAndExclusive_shouldProduceIdenticalOutput()
+{
+    // Real-time playback processes serially; offline render (exclusive mode) may fan out to worker
+    // threads. The two paths must produce identical output. Several independent devices in one layer
+    // exercise the parallel fan-out and the per-lane summing.
+    const auto render = [](bool exclusive, std::vector<double> & out) {
+        AudioEngine engine;
+        engine.setIsExclusive(exclusive);
+        for (int i = 0; i < 5; i++) {
+            const auto device = std::make_shared<MockDevice>("Device " + std::to_string(i));
+            device->setGenerateSignal(true);
+            engine.setDevice(static_cast<size_t>(i), device);
+        }
+        out.assign(128, 0.0);
+        AudioContext context { std::span(out.data(), 128), 64, 44100 };
+        engine.process(context);
+    };
+
+    std::vector<double> serialOut;
+    std::vector<double> exclusiveOut;
+    render(false, serialOut);
+    render(true, exclusiveOut);
+
+    QCOMPARE(serialOut.size(), exclusiveOut.size());
+    for (size_t i = 0; i < serialOut.size(); i++) {
+        QVERIFY(std::abs(serialOut[i] - exclusiveOut[i]) < 1e-9);
+    }
+    // Five devices each emitting 1.0 sum to 5.0 per sample.
+    QVERIFY(std::abs(serialOut[0] - 5.0) < 1e-9);
+}
+
 void SideChainAudioTest::test_audioEngine_rebuildProcessingGraph_shouldHandleCircularDependencyGracefully()
 {
     AudioEngine engine;

src/unit_tests/side_chain_audio_test/side_chain_audio_test.hpp

@@ -28,6 +28,7 @@ private slots:
     void test_audioEngine_rebuildProcessingGraph_shouldCorrectlySortIndependentDevices();
     void test_audioEngine_rebuildProcessingGraph_shouldCorrectlySortDependentDevices();
     void test_audioEngine_process_runtimeSidechainChange_shouldRebuildGraph();
+    void test_audioEngine_serialAndExclusive_shouldProduceIdenticalOutput();
     void test_audioEngine_rebuildProcessingGraph_shouldHandleCircularDependencyGracefully();
     void test_compressorEffect_process_shouldApplySidechainGainReduction();
     void test_compressorEffect_sideChainLpf_bypass_shouldPreserveGainReduction();

src/view/qml/Manual.html

@@ -388,13 +388,13 @@ <h2 style="color: orange; border-bottom: 1px solid #444; margin-top: 20px;">Tool
 </ul>
 
 <h3 style="color: orange;">Audio Engine Performance</h3>
-<p>The internal devices and effects are rendered on the audio thread. By default all device processing runs <strong>serially</strong> on that thread, which for realistic projects is both smoother and lighter on the CPU than splitting the work across threads.</p>
-<p>An optional parallel worker pool is available for very large projects that a single core cannot keep up with. It is opt-in via the <code>NOTEAHEAD_AUDIO_WORKERS</code> environment variable, read at startup:</p>
+<p><strong>Real-time playback</strong> processes all internal devices and effects <strong>serially on the audio thread</strong>. For realistic projects this is both smoother and lighter on the CPU than splitting the per-buffer work across threads, which adds synchronization and CPU-cache overhead that can cause stutter.</p>
+<p><strong>Offline rendering / export</strong> has no real-time deadline and is parallelized across worker threads for faster exports. The number of render worker threads is read at startup from the <code>NOTEAHEAD_AUDIO_WORKERS</code> environment variable:</p>
 <ul>
-    <li><code>NOTEAHEAD_AUDIO_WORKERS=auto</code>: use a worker count based on the number of hardware threads.</li>
-    <li><code>NOTEAHEAD_AUDIO_WORKERS=4</code>: use a specific number of worker threads.</li>
-    <li><code>NOTEAHEAD_AUDIO_WORKERS=0</code> (or unset): serial processing on the audio thread (the default).</li>
+    <li><code>NOTEAHEAD_AUDIO_WORKERS=auto</code> (or unset): a render thread count based on the number of hardware threads (the default).</li>
+    <li><code>NOTEAHEAD_AUDIO_WORKERS=4</code>: a specific number of render threads.</li>
+    <li><code>NOTEAHEAD_AUDIO_WORKERS=0</code>: render single-threaded.</li>
 </ul>
-<p>The worker threads need real-time scheduling to be useful. Without it (no <code>rtprio</code> limit for your user, usually granted by membership in the <code>audio</code> group) they get preempted while the audio thread waits for them, which causes stutter under load &mdash; Noteahead warns about this in the log. Because of the added per-buffer synchronization and CPU-cache overhead, the pool can be slower and glitchier than serial processing even when real-time scheduling <em>is</em> available, so only enable it if you measure a real improvement.</p>
+<p>These worker threads run at normal priority and only during export, so they do not affect playback smoothness or preempt the UI.</p>
 
 <p style="margin-top: 30px; font-size: 0.8em; color: #888;">Noteahead is licensed under the GNU GPLv3.</p>