Skip to content

feat(audio)!: accept Noctalia spectrum snapshots in effects - #455

Open
barrulus wants to merge 2 commits into
noctalia-dev:mainfrom
barrulus:feat/noctalia-audio-spectrum
Open

barrulus wants to merge 2 commits into
noctalia-dev:mainfrom
barrulus:feat/noctalia-audio-spectrum

Conversation

@barrulus

@barrulus barrulus commented Oct 14, 2026 •

Copy link
Copy Markdown
Contributor

Summary

Expose externally supplied 64-band audio snapshots to effects through umbriel_audio_spectrum(position). Noctalia can supply its existing spectrum through the companion provider, allowing effects to react to different parts of the spectrum while capture and analysis remain outside the compositor.

The effect-audio v2 contract accepts exactly 64 finite values in [0, 1]. Sampling clamps normalized positions and interpolates between adjacent bands. A single-owner connection, acknowledgements, 4096-byte message bound and 250 ms expiry bound the feed. Documentation, bundled shaders and configuration guidance use the spectrum API.

Breaking change: replaces the v1 scalar payload and removes umbriel_audio_level(). External producers must send v2 bands; custom shaders must sample or aggregate the spectrum. The band order is low-to-high; no frequency-to-Hz mapping is promised.

Motivation

A scalar level makes every reactive effect follow the same signal. Sharing Noctalia's spectrum gives shaders independent band responses without introducing another audio capture process in Umbriel. This stays within the external-input mechanism for compositor effects.

Type of Change

  • New feature
  • Breaking change

Testing

  • Debug build and 62 compositor unit tests passed, including 13 audio parser cases. Focused just lint and just format passed; configuration example validation and git diff --check passed.
  • Software-rendered effect/audio, effect/audio_kinds and effect/audio_spectrum harness checks passed. Coverage includes ownership, ACKs, message limits, invalid input, expiry, lock/unlock, renderer recovery, frame-rate limits, output isolation, all 64 bands, interpolation and clamping. Spectrum stress passed 4/4 runs.
  • Deliberate-failure validation: an isolated binary copy forced every shader sample to band zero. The unchanged spectrum harness failed its interpolation pixel assertion; the untouched binary passed the same check with llvmpipe. Source and test assertions were unchanged.
  • Native effects-audio-spectrum and effects-output-effects GPU suites passed on both Intel and NVIDIA render nodes.
  • The Noctalia provider drove actual IPC and rendered pixels through nonzero input, healthy silence and disconnect. A native Intel eDP session at 1920x1080 was manually exercised with music; the border and smoke effects responded correctly.

Bounded performance comparison

One paired Tracy capture compared baseline 693b6899 with implementation a9a06a8f, using the same profiling-only instrumentation and Tracy 0.13.1 toolchain. The desktop was exited for the comparison: Intel ARL/i915, Mesa 26.2.4, two hardware-headless 1280x720 outputs at 60 Hz, one mapped window and one full-screen audio shader on output A, no audio consumer on B. Both revisions received the same wave; the candidate supplied it in all 64 bands. Pixel checks passed.

Metric Baseline Spectrum
CPU frame mean / p95 0.475 / 0.678 ms 0.424 / 0.685 ms
GPU broad pass mean / p95 0.476 / 0.608 ms 0.820 / 0.981 ms
GPU draw zone mean / p95 0.178 / 0.191 ms 0.552 / 0.556 ms
Disconnected-profiler active rate 57.16 Hz 56.99 Hz
Disconnected-profiler interval p95 18.51 ms 18.67 ms

The draw zone includes uniform/state/draw work; it is not a pure shader-ALU measurement. Nested zones are not added together. The feed was ACK-limited and headless presents do not establish physical scanout pacing. With profiling disconnected, neither revision produced extra effect frames on unused output B or during fresh silence/no feed. Process CPU was 7.33% → 8.17% of one core active, 1.25% → 1.75% during fresh silence and 1.75% → 1.75% without a feed; short sampling windows limit precision.

The comparison measured additional GPU cost. No compositor budget has been agreed, and native border/smoke performance was not measured, so these results do not establish performance-budget compliance.

Manual Coverage

  • Tested in a native Umbriel session
  • Tested with native Wayland applications
  • Tested with the scrolling layout

Checklist

  • This PR is ready for review, or it is marked as Draft.
  • This change fits SCOPE.md, or its scope was agreed in an issue or on Discord first.
  • I read and followed the relevant guidance in CONTRIBUTING.md.
  • I ran just format, or this PR has no C++ changes.
  • I ran the relevant build, test, lint, or verification commands, or explained why they were not run.
  • I functionally verified compositor behavior where automated checks are insufficient.
  • I self-reviewed the changes.
  • I checked for new warnings or errors.
  • I updated docs/ and examples/config.toml, or this PR does not change user-facing configuration or behavior.
  • I used canonical names for config keys, IPC actions, paths, and identifiers.

Additional Notes

Companion: noctalia-dev/noctalia#4860

@barrulus
barrulus marked this pull request as ready for review October 14, 2026 20:01
@ItsLemmy ItsLemmy added the talia-review Ask Talia for a preliminary AI review (removed once posted) label Oct 15, 2026

@talia-review talia-review Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Talia preliminary review

Check Result
Merge score 🟢 4/5 — No issues found; likely ready after CI and human review
Findings None found
Review depth 3/5 — how thoroughly the changed code could be checked by reading it
Coverage 24 of 24 changed files reviewed

What Talia checked: Inspected every changed hunk and callers: IPC v2 bands parse and 4096 limit, registry/output 65-float latching, scene audio uniform names/reflection, GLSL index math, docs (64-entry example) and updated tests; no reachable introduced defect found.

Run details
  • Commit reviewed: 7bbb1e0
  • Source lookups: 39 file reads, 32 searches
  • Prompt version: 859b7490ca3d

Preliminary automated review: Talia read the changed code and related source; nothing was built, run or tested, and AI can make mistakes. It is not a scope acceptance or a promise to merge; a human reviewer makes the final call. See our ethos.

@talia-review talia-review Bot removed the talia-review Ask Talia for a preliminary AI review (removed once posted) label Oct 15, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants