Multi-core processing

Settings › Audio › Processing Threads (AudioConfig::processing_threads, in threads.rs) lets independent strips play at the same time, on a small pool of real-time threads with the audio callback working too. Its output is bit for bit the serial loop’s, so nothing else in the program has to know. With the setting at 1 there is no pool: process_block runs the plain serial loop.

One strip, two kinds of route. run_strip in dsp/strip.rs is one strip’s whole block, and the serial loop and the workers run that same code. They differ only in Routes, how a strip reaches other strips. The serial Direct writes straight into later strips’ sum_l/sum_r and the bus. On the pool, in dsp/parallel.rs, nothing writes across strips: Deferred leaves a strip’s output in its own l/r and renders each send into its own Outlets, and a bus adds up its feeders’ outlets when its turn comes.

Why the sum is bit-identical. Float addition isn’t associative, so the order must be the serial loop’s: feeders by processing order, and within a feeder its pre-fader sends by slot, its output, its post-fader sends by slot. Deferred::inputs reads them in exactly that order (a task’s dependencies in TaskGraph are sorted by processing position), and Stereo Out’s bus is summed the same way once the strips are done. A send’s product l[i] * g is the same bits wherever it is computed. This is why a switch between serial and parallel at any block boundary is inaudible.

The graph. TaskGraph::of_project (dsp/graph.rs) makes every track a task and every route an edge: outputs, sends, side-chain keys and multi-output sources, from Project::feeds and keyed_by. Tasks are in process_order, so edges only point forward and a cycle can’t be built (TaskGraph::new refuses one). VCAs and stacks need no edge, since the session folds them into each track’s TrackParams. Folder automation doesn’t either: a subtrack doesn’t wait for its folder, because Parallel::run reads the folder’s lanes (fold_at) before the strips start.

Keeping the graph true. The session builds the graph off the audio thread and sends it in Cmd::Graph (Session::sync_parallel, called after every Tauri command and from idle, whenever a route-changing Cmd went out). The old one comes back as Garbage. The commands of one edit can straddle blocks, so the engine doesn’t take the graph on trust: whenever routes or the graph changed (resolve_keys), parallel::covers checks at the next block that every route the engine resolved has its edge, and until that holds it plays serially. So a route never races and never arrives a block late; at worst an edit costs a few serial blocks. Deferred::earlier also checks the edge on each read, so a side-chain key whose edge is somehow missing is silence rather than undefined behaviour. soak_check checks after every random edit that the graph fits.

The pool (dsp/pool.rs) holds no lock or queue. A task is a bit in a few words of atomics; finishing a task’s last dependency sets its bit, and whoever clears a bit owns the task, preferring tasks “at home” on that thread so a strip tends to stay on one core from block to block. The guarantees:

• The callback is never stuck behind a worker for ready work. A ready task can be seen and taken by anyone, so there is no slot a preempted worker could be holding up, and a preempted or late worker costs only the one task it has taken. Rarely does the callback have to wait with nothing to take: it spins for 300 µs, then naps 50 µs at a time, which also lets a worker at normal priority run if it shares this core.
• A stuck worker can’t hang the callback. After GIVE_UP_SECS (0.1 s, or four buffers if that’s longer) process gives the block up: silence, counted as an overload. What then depends on where the stuck worker is. Out in a plug-in, it’s left there (next point), and the block ends without its strip. Anywhere else, the session stops the transport and says so, and the block stays open (Parallel::stalled): the strips and all it reads belong to the threads playing it. Until it ends, each callback plays silence and applies no commands; then the engine flushes notes and carries on, shedding as after any overload. Anything that changes the graph off the audio thread (replace_graph, render, dropping the engine) first waits for it (Engine::settle).
• A worker left in a plug-in costs its strip, not the engine. run_strip brackets every call into a plug-in (Routes::enter and leave, a pool::Call per thread, with the call’s number and start time), and a call touches only its own strip, its thread’s scratch (the side-chain key is copied there first) and what the block reads. At the give-up Pool::abandon moves each worker that has been in one such call for three quarters of the limit to LEFT (one compare-exchange on the call’s number and phase, which the worker’s own leave loses if the plug-in returns at that moment); a shorter call is more likely a hiccup than a hang. It then counts the worker’s task and every task the graph has waiting for it out of remaining: none of them ran, and none ever gets ready. The caller finishes the rest of the block (with a quarter of the limit) and plays on. Past that, or with the worker held up outside a plug-in, it falls back to retire_stuck and a stalled block, during which each callback (and Engine::settle) calls Parallel::press_on: a worker retired in that block that has since been in a plug-in for long enough is left after all, and the caller does what’s ready of the block, never waiting. Parallel::leave_out, once the block is over, swaps the strip in the engine’s list for a stand-in made ahead in Workers::new, which takes the commands meant for the track, and marks the stand-in and every strip counted out left_out: they play nothing in any later block, serial or pooled. EngineStatus::held lists the track and plug-in, and left_events tells the session, which says so, lets playback go on (stopping a recording), refuses renders while left_out counts any strip (they’d be missing), and asks no plug-in a thread is left in for its state (capture_plugin_states, the VST3 undo watch), saving the state the project last had. The strip left behind is never touched again until its thread is gone: if the plug-in ever returns, leave fails, the strip returns at once, the worker skips its bookkeeping, marks its Call GONE and ends, and Parallel::take_released sends the strip back as garbage and marks its entry in held back. The session’s idle then builds the track a new strip, plug-ins from their state now, and sends Cmd::Back, which puts it in the stand-in’s place and brings back every strip sitting out but those still fed by a plug-in that hasn’t returned (with a fade-in, as for Track On). Reset Audio Engine rebuilds every strip while any sits out. Until a thread is gone, the job it’s in (a new cell per time workers are left), its scratch and the graph (Parallel::graph_held) stay, and dropping the Workers or the pool leaks them and lets the thread go rather than join it.
• A worker given up on is retired. At the give-up Pool::retire_stuck takes every worker still in a task (Slot::busy, set before a worker claims, so one preempted between taking a task and starting it counts) out of the pool: it takes no task again and its thread ends once it gets out. A real-time one is demoted to SCHED_OTHER there and then, by its kernel thread id. RLIMIT_RTTIME only counts time under a real-time policy, so a plug-in that never returns can now spin on without the kernel killing the app at 200 ms. The callback gives up after 100 ms (four sub-blocks of at most MAX_BLOCK take no longer at 44.1 kHz and up), and RT_GUARD keeps a worker from having gone more than 50 ms without sleeping when the block started, so the demotion comes in time. That one sched_setscheduler call is the only system call the callback makes for this, and only on a give-up. The rest of the pool plays on with the other workers once the block is over, or serially if none is left. The overload message says the thread is left out; Reset Audio Engine (restart_audio sets ThreadState::renew) starts the pool afresh. A worker held up outside a plug-in call that never ends would still leave the engine silent, as a plug-in that never returns on the callback does, but the app stays up to save the project.
• A panic stays in its task. It is caught, counted in engine_errors, its dependants still run and the block is silenced. The pool isn’t poisoned.
• Workers really sleep. A real-time thread that never blocks is killed at RLIMIT_RTTIME’s hard limit (200 ms). A worker with nothing to do spins for 30 µs at most, then parks on a futex. Blocks arriving back to back could keep it from ever truly sleeping, so one that hasn’t slept for RT_GUARD (50 ms) naps before its next task; the callback does the work meanwhile, and OVERLOAD_SECS sheds load as ever when the callback itself falls behind. survive_slow_blocks covers workers too, and every worker’s load counts toward the same block, so the meter and the shedding still see a late block. A plug-in that never returns on a worker is demoted before the kernel’s limit (above); on the callback the limit still applies.

Real time. Workers call audio_thread_priority::promote_current_thread_to_real_time, as cpal does for its callback (rtkit, the same priority and limits). A worker refused it plays nothing: at normal priority beside a real-time callback it would be late whenever the system is busy. Each worker starts STARTING, takes tasks only once ACTIVE, and a refused one ends (LEFT_OUT), so the pool plays on with the workers that were granted it, and when none was Parallel::wanted plays the serial loop. The counts are in pool::Health, which the session keeps a handle to: it logs once what was left out and why, and thread_choices carries it (LeftOut) for Settings › Audio’s note. PoolOptions::refuse_for_checks refuses chosen workers for tests and parallel_check. Away from Linux promotion isn’t tried yet, so a number chosen by hand gets workers at normal priority. On a machine with performance and efficiency cores they keep to the performance cores (Topology::fast), as the scheduler places real-time threads without regard to a core’s speed: measured at 128 frames, that took 24 CHOWTapeModels from a mean of 340 µs to 270 µs and the 99th percentile of 5 TONE3000s from 1510 µs to about 900 µs, repeatably. They keep to the set, not one core each, so the kernel still balances them, and the callback, the driver’s thread, is left alone. Workers aren’t real time on the null device, as cpal’s callback isn’t there either. A worker has an 8 MB stack, as plug-ins expect of an audio thread, and touches the first part of it before its first block.

When it runs. Parallel::wanted: there are workers, the graph holds, two strips with plug-ins can overlap (heavy_width), and the load has reached 15% of the buffer (ENGAGE_LOAD), going back to one thread once the average load times the threads, the most it could come to alone, is under that. The session doesn’t even start the threads for a project with nothing to gain. Below that a project pays nothing at all for the feature. Automatic is a fast (performance) physical core count less one, at most 6 (threads::automatic); on 1 or 2 cores it is 1, and away from Linux too, where workers can’t get real-time priority yet.

Offline renders stay serial: their blocks come back to back with no real-time deadline, and real-time workers that never sleep would be killed for it. ParallelTest::renders sends them through the pool for parallel_check and soak_check, so the route is tested for identity. ParallelTest::patient is for checks that drive Engine::process flat out: no block is given up on and no load is shed, as either would land at a block that depends on how busy the machine is (a debug build with its cores shared plays slower than real time, which once made parallel_check’s two serial runs differ).

What a block touches, and who may:

State	Where it’s used	How it’s kept safe
Commands, Garbage, cmd_rx	process, before the strips	Calling thread only; the graph and workers change only here, between blocks.
garbage: RefCell	discard, from apply	Calling thread only; the strips’ context doesn’t hold the engine, so the compiler sees to it.
live_midi, live	Filled before the strips	Read-only for the strips.
midi_rec (recorded MIDI ring)	Also pushed by process (cycle marks)	Moved before the strips, for the serial loop too; it comes from one track anyway.
analyzer_tx (SPSC, several pushers)	A strip’s tap, Stereo Out’s	On the pool the watched strip fills Outlets::analysis, and the calling thread pushes it after the strips. Stereo Out’s is on the calling thread.
inputs	Copied in before the strips	Read-only for the strips.
bus_l, bus_r	Stereo Out	Summed by the calling thread after the strips, in order.
tap_*, gain_*	A strip’s scratch	Scratch, one per thread.
sum_l, sum_r	Serial loop only	Unused on the pool, where a bus sums into its own l/r.
Strip’s own state (TrackDsp)	Its task	One task, one thread at a time, handed over by the pool’s Acquire/Release.
A feeder’s l, r, outs, key_*	Its receivers	Only read once its task is done (an edge), and by any number of receivers at once.
An instrument’s output buses, instrument_ran	Output auxes	The same: the instrument’s task is done.
A folder’s params, automation	Its subtracks’ faders	Read before the strips start (fold_now).
params, tempo, Isolate, chase, midi_target, transport values	Every strip	BlockCtx: copied or shared immutably; commands are applied only between blocks.
Regions and clips	Their strip	Shared read-only behind Arc.
TransportInfo for plug-ins	Each strip	A copy per strip, moved to its own input_time.
Plug-in instance	Its strip	Only its strip’s task calls it, so one thread at a time; which thread can change from block to block, as VST3 allows.
Gate, OutGate (VST3 bus changes)	process, suspend, the controller	Atomics, unchanged. Cmd::VstGate is applied between blocks; the task holds OutGate through the call.
Meters (Meter)	Each strip’s own, Stereo Out’s	Atomics, per strip.
EngineStatus counters (bad_audio, overruns, engine_errors, midi_activity)	Any strip	Atomic adds. bad_audio_uid is a plain store, so with two bad plug-ins in a block either may be named.
Input meters, cpu_*, late_blocks	Before and after the strips	Calling thread only.
t.rec (recording ring)	Its strip	One producer per track.
Bad-audio containment (check_output)	Each plug-in, its strip	Works on the strip’s own buffers and effect.
Effect ticks (FxShared)	Each effect	Atomics, per effect.
Global state in plug-ins	Inside third-party code	As in any host that plays instances in parallel, which is their concern. No mutable globals in fx/, instrument/, synth_dsp/, synth.rs or vst3-host’s processing; an instrument’s randomness comes from its slot’s seed.

The unsafe is in parallel.rs and pool.rs, each with its argument beside it: StripJob::run and Deferred::dep (who holds which strip), the job cell (only written with no block open), and extending BlockCtx’s lifetime (what it points at is on the heap and unchanged while a block is open). The parallel tests, parallel_check and soak_check ran clean under ThreadSanitizer: see DEVELOPING.md.

Plug-ins with threads of their own. TONE3000 runs its model on its own pool of worker threads, and the Linear Phase EQ designs filters on a thread. Their output depends on timing, serial or not, which is why parallel_check doesn’t use TONE3000, and swaps the EQ out live (a render waits for it).

The CPU meter (Tick.cpu, cpu_peak, late_blocks) is still the wall-clock time of the callback over the buffer’s duration, as that is what decides a dropout. Tick.threads says how many threads are playing right now, for its tooltip.

Measured on an i7-13700KF (8 performance and 8 efficiency cores, 24 hardware threads), driving Engine::process at real-time pace with real-time workers: block time in µs, mean / worst of about 8 s. The user’s own app was running on the same machine. Real plug-ins: 5 TONE3000s with a real project’s models, 24 CHOWTapeModels, 16 Dragonfly Hall Reverbs, and “buses”: 12 CHOWTapeModels in 3 summing stacks with a shared Dragonfly Plate Reverb on an aux.

Project	Buffer	1 thread	2	4	8	Automatic (6)
5 TONE3000	512	2822 / 5447	1723 / 2714	1237 / 2165	1060 / 1393	1099 / 1833
24 CHOWTapeModel	512	4822 / 5852	2512 / 3009	1463 / 1603	754 / 859	979 / 1086
16 Dragonfly Hall	512	1686 / 2834	891 / 1234	622 / 737	387 / 495	546 / 682
buses	512	2458 / 3033	1254 / 1556	707 / 885	496 / 573	778 / 881
5 TONE3000	128	1371 / 2729	549 / 1700	337 / 927	331 / 1049	350 / 1036
24 CHOWTapeModel	128	1330 / 1988	794 / 1026	457 / 582	206 / 312	269 / 418
16 Dragonfly Hall	128	430 / 805	221 / 400	187 / 262	110 / 173	111 / 174
buses	128	1186 / 1448	350 / 547	240 / 367	142 / 236	155 / 235

A small project, where threads have nothing to gain, at 128 and 512 frames: mean / worst µs.

Project	Buffer	1 thread	Automatic	Forced onto 6 threads	Forced onto 2 threads
2 audio tracks	512	8 / 15	10 / 24	16 / 42	12 / 27
2 tracks with 2 built-in effects each	512	81 / 130	134 / 168	68 / 93	48 / 92
2 audio tracks	128	3 / 11	5 / 16	4 / 18	4 / 23
2 tracks with 2 built-in effects each	128	35 / 58	20 / 45	16 / 29	21 / 43

Reading it: wide projects scale close to the thread count. The worst block falls with the mean, except where one plug-in sets it: at 128 frames one of the TONE3000 models takes about 850 µs in some blocks, so past 4 threads its worst block stops improving. TONE3000’s own worker threads did no harm to it or to ours: no thread count made it worse than the one before beyond the noise. The one-thread column varies by up to a factor of two from run to run (5 TONE3000s at 128: mean 870 and 1370 µs), probably from whether the callback lands on an efficiency core. Going to 8 threads helps wide projects another 10 to 25% and does nothing for the 5-track project, and it would leave a performance core for nothing else, so Automatic stops at 6. Since the gate keeps a light project on the callback, these small projects pay nothing: without the gate the pool’s cost is about 10 µs a block, or none next to any real plug-in.