Skip to content

Commit d18cbcb

Browse files
vanceingallsclaude
andauthored
feat(studio): the carve is one module in the rack (#3213)
* fix(ci): allowlist the build-script consolidation in the no-main-deletions guard build-audio-fx-runtime.ts and build-position-edits-render.ts were merged into build-inline-artifact.ts to kill a fallow duplication finding; the deletion guard flagged that as an accidental loss since main still has both originals. * fix(core): dedupe the wet/dry mix math between delayFeedback and chorusLfo Both effect builders set wet.gain to the mix and dry.gain to its complement in identical two-line blocks; fallow kept re-flagging it as a 10-line clone on every unrelated change. Extracted setWetDryMix. * fix(core): remove the build-audio-fx-runtime.ts stray resurrected by a main merge An earlier merge with main brought this deleted file back (git's merge/delete handling on an unchanged-on-one-side file); package.json already points at build-inline-artifact.ts, so it sat unreachable and duplicating that file's config, both of which fallow flagged. * fix(studio): pull TimelineLanes under the 600-line cap TimelineLanes.tsx hit 620 lines. Extracted the three per-clip pointer gestures (resize-start, pointer-down move-arm, click/razor-split) into createClipGestureHandlers — one factory call per rendered clip instead of ~120 lines of inline handler bodies in the render loop. 529 lines now. * fix(studio): split the extracted pointerdown handler under the CRAP threshold Moving the ~120-line gesture logic into timelineClipGestureHandlers.ts concentrated it into two functions fallow flagged (onPointerDown at CRAP 63.6, onResizeStart at 31.6). Split the decision logic (which gesture a pointerdown implies) into a pure resolvePointerDownAction, then split its own intent-blocking check into isIntentBlocked. onResizeStart's guard moved into canStartResize. Every function now scores under 30. * fix(studio): drop the unused DomEditSelection import in PropertyPanelFlat CI caught it on PR #3026 (wa-12-panel-params); a later refactor in the stack removed the last use of the type here without removing the import. * fix(studio): close the typecheck and fallow gaps wa-18b-reschedule opened useAutomationLanes.ts's write() assumed gesture-scoped coalescing and a preview-only commit that useDomEditAttributeCommits.ts never grew — backported that option support from its own later commit so the two sides of the API agree. The paste path and its tests were missing the box selection's v0/v1 bounds a sibling commit added to AutomationSelection. The FX panel's carve controls still edited the six mechanism numbers (maxCutDb, bands, intelligibilityBias) after carveProfile() collapsed authoring to one Strength knob, so those fields no longer existed on HfCarveSettings; UI now edits strength, and analyseCarveBands is called with carveProfile(strength). Also closes fallow's complexity, dead-code and duplication findings on this PR's diff: extracted automationLaneDragMath.ts (pure group/point-move math) and useAutomationRangeDrag.ts (the marquee-select gesture) out of useAutomationLaneGestures.ts, pulled a couple of render-loop ternaries and a resolver into named functions, dropped an export nothing outside its file used, and shared a step-simplifier between audioCarve's two envelope builders. The edge-stretch vs. box-select priority test in TimelineAutomationLane.test was still pinning the pre-box-select rule (edge wins over a point sitting on it) that a sibling commit deliberately reversed — a point inside the box is now selected content, so grabbing it drags the group instead. Updated the test to the shipped rule instead of the old one. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix(core): cap the via conic's weight so an edge-clamped via point can't NaN A via point pulled out past the segment (viaX: 5, viaY: -3) clamps to (0.999, 0.001) — exactly on the steady region's edge, where edge - viaX is 0. viaConic divided by that zero to get an infinite weight, and shapeVia turned Infinity into NaN a few steps later (Infinity - Infinity in the quadratic coefficient). NaN reaching setValueCurveAtTime silences the automated parameter for the rest of the render. Capped the weight at 1e6 instead of leaving it unbounded — past that point the arc already reads as touching the via point, so nothing visible is lost. Also hardened shapeVia's existing denominator guard (`<= 0`) to `!(> 0)`, since NaN fails the original comparison and fell through it. Review by Miga (PR #3208). * fix(studio-server): fingerprint the proactive waveform cache key too The route already keys the waveform cache on the asset's size and mtime as well as its path, so a rebuilt-in-place file gets fresh peaks instead of stale ones. generateWaveformCache — the proactive path that runs on upload — still called buildWaveformCacheKey with the path alone, so it wrote to a different key than the route reads from (making the pre-generated cache never found) and kept the exact collision bug this fingerprint exists to fix on its own path. Review by Miga (PR #3211). * style(docs): run oxfmt on the /hyperframes-audio skill docs Table column widths had drifted out of alignment with oxfmt's own rules, failing format:check and blocking the Preflight gate every downstream branch inherits. Whitespace only, no content change. * fix(core): stop \b from missing underscore-separated names, guard clipsOverlap's negative duration \b treats `_` as a word character, so \bbed\b never matched bed_01, music_bed_loop, or theme_song, and \bvo\b/\bvox\b/\btts\b had the same gap — an underscore-separated bed classified as "unknown" and could end up offered as its own carve source. Replaced the short hints with a boundary that actually excludes letters and digits on both sides. clipsOverlap computed end = start + duration without guarding sign, so a negative duration put end before start — an interval that does not describe anything, and one specific case showed it silently dropping a real overlap (a shorter, earlier broken end rejected a clip that genuinely contained the point). Duration clamps to zero instead: a clip cannot un-play time, and a zero-length clip at its start is the sane reading of "duration nobody wrote down as positive." Review by Miga (PR #3212). * fix(studio): widen PropertyPanel's resetModules render timeout again The 20s margin (already once widened for the same reason) is timing out in CI's full-monorepo Test run — the resetModules()+fresh-import render this test needs is uncached and competes with every other package's test suite for the same worker pool, and the same test passes in well under 2s standalone. Went to 45s rather than re-tuning to whatever number happens to clear the current CI load, since that number moves every time CI gains a package. * fix(studio): stop the single-candidate auto-apply carve firing twice Two auto-apply effects both fire when sourceOptions.length === 1: the multi-candidate effect only guards length === 0, so a single candidate passes it too, and the single-candidate effect passes its own guard right after — both compute the same sources list and both call setCarve, so the common case (one narrator, one bed) triggered two decodes, two FFT runs, and two concurrent attribute writes for one decision. The multi-candidate effect now defers to its sibling for exactly one candidate, which already has its own detailed handling for that case. Review by Miga (PR #3213). * feat(core): carve against every voice over a bed, always (#3212) * feat(core): carve against every voice over a bed, always dynamically A bed usually runs under a whole sequence — a narrator, an interview answer, a second presenter — and carving against one of them left the others fighting it. `source` becomes `sources`, and `mixCarveSources` sums every voice onto the BED's clock before anything is measured. That is what keeps one analysis sufficient: the chain is fixed, so there is no per-voice filter to switch between, and bands drawn from all the speech there is with envelopes that rise wherever any of it happens answer the actual question — where and when is speech masking this bed. Summed rather than averaged: two people talking at once mask more than either alone. Audio before the bed starts is dropped rather than folded in at zero, since it plays over nothing and shifting it would put a cut where there is no voice. `dynamic` is gone. A fixed depth thins the bed through every pause, and once both have been heard there is no reason to want it, so every carve follows the speech. Two helpers the panel and the headless script now share instead of each carrying a copy — two definitions of "what does this name suggest" drift, and then the two disagree about which track is the voice: - `classifyAudioName` reads a track's kind from its id and filename together. `unknown` is deliberately common: treating an unrecognised name as "not a voice" would hide the one track somebody needs to pick. - `clipsOverlap` keeps out a voice that never plays while the bed does. An unwritten duration counts as unbounded, not zero — refusing a clip whose length the composition leaves to the media would drop the commonest case there is. Files written before this still load: a single `source` reads as a one-voice list, a stored `dynamic` is ignored, and an absent attribute means the defaults whole. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix(core): stop \b from missing underscore-separated names, guard clipsOverlap's negative duration \b treats `_` as a word character, so \bbed\b never matched bed_01, music_bed_loop, or theme_song, and \bvo\b/\bvox\b/\btts\b had the same gap — an underscore-separated bed classified as "unknown" and could end up offered as its own carve source. Replaced the short hints with a boundary that actually excludes letters and digits on both sides. clipsOverlap computed end = start + duration without guarding sign, so a negative duration put end before start — an interval that does not describe anything, and one specific case showed it silently dropping a real overlap (a shorter, earlier broken end rejected a clip that genuinely contained the point). Duration clamps to zero instead: a clip cannot un-play time, and a zero-length clip at its start is the sane reading of "duration nobody wrote down as positive." Review by Miga (PR #3212). --------- Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix(studio): port the carve UI off the removed source/dynamic fields #3212 (accidentally squash-merged into this branch instead of main) changed HfCarveSettings from a single `source` + `dynamic` toggle to a `sources` list with dynamic mode removed outright — the multi-voice UI consumer that goes with that shape lands in the very next PR, so this branch was left with a type that no longer matched its own code. Minimal port, not the multi-voice redesign that PR does properly: the "Listen to" picker and analyse() treat sources[0] as the one voice this UI still understands, and every dynamic-mode branch (the automated envelope lanes, the toggle, the checkbox) is gone along with the field — a carve is now always the static value the analysis computes, matching what the type change made permanent. Test suite trimmed the same way: the automation-lane and toggle tests covered behavior that no longer exists. --------- Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
1 parent 56d8df6 commit d18cbcb

9 files changed

Lines changed: 1366 additions & 508 deletions

File tree

packages/studio/src/components/editor/propertyPanelAudioFxGroup.test.tsx

Lines changed: 585 additions & 80 deletions
Large diffs are not rendered by default.

packages/studio/src/components/editor/propertyPanelAudioFxGroup.tsx

Lines changed: 203 additions & 40 deletions
Large diffs are not rendered by default.

packages/studio/src/components/editor/propertyPanelFxControls.tsx

Lines changed: 19 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -120,9 +120,26 @@ export function FxParamRow({
120120
const [dragging, setDragging] = useState(false);
121121
const [local, setLocal] = useState(value);
122122
const latest = useRef(value);
123+
/**
124+
* What the gesture last asked for, held until the world agrees.
125+
*
126+
* Releasing ends the drag, but the value only comes back after the attribute is
127+
* written and the selection resynced — and for a carve, after the analysis it
128+
* kicks off. In that gap the prop still holds the pre-drag number, so dropping
129+
* straight back to it made the control snap to where it started and then jump to
130+
* where it was dropped. Keeping the gesture's own number until a NEW one arrives
131+
* is honest either way: it is what the audio is already doing, since the live
132+
* write applied on the way down.
133+
*/
134+
const [pending, setPending] = useState<number | null>(null);
123135
useEffect(() => {
124136
if (!dragging) setLocal(value);
125137
}, [value, dragging]);
138+
// Any inbound value is newer information than the gesture's guess — including a
139+
// value that came back different from what was asked for, or an undo.
140+
useEffect(() => {
141+
setPending(null);
142+
}, [value]);
126143

127144
const handleNumber = useCallback(
128145
(raw: number) => {
@@ -137,6 +154,7 @@ export function FxParamRow({
137154

138155
const commit = useCallback(() => {
139156
setDragging(false);
157+
if (typeof latest.current === "number") setPending(latest.current);
140158
onCommit?.(param.key, latest.current);
141159
}, [onCommit, param.key]);
142160

@@ -170,7 +188,7 @@ export function FxParamRow({
170188
// because it is the one the audio is using — the stored number is only the seed
171189
// the lane replaced. Safe against the pointer: an automated control is locked
172190
// (see `locked` below), so there is no drag for this to fight.
173-
const shown = dragging ? local : (liveValue ?? value);
191+
const shown = dragging ? local : (liveValue ?? pending ?? value);
174192
const numeric = typeof shown === "number" ? shown : Number(shown);
175193
const current = Number.isFinite(numeric) ? numeric : param.default;
176194

packages/studio/src/components/editor/propertyPanelFxSection.test.tsx

Lines changed: 179 additions & 50 deletions
Large diffs are not rendered by default.

packages/studio/src/components/editor/propertyPanelFxSection.tsx

Lines changed: 222 additions & 167 deletions
Large diffs are not rendered by default.

skills-manifest.json

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -26,7 +26,7 @@
2626
"files": 121
2727
},
2828
"hyperframes-audio": {
29-
"hash": "08f593a7c159ce04",
29+
"hash": "533dcc01de09db53",
3030
"files": 4
3131
},
3232
"hyperframes-cli": {

skills/hyperframes-audio/SKILL.md

Lines changed: 33 additions & 26 deletions
Original file line numberDiff line numberDiff line change
@@ -150,10 +150,17 @@ takes only those, and the bed keeps its low end and its top, so it is still musi
150150
while the voice is still intelligible.
151151

152152
**It is a relationship, not an effect.** The settings live on the _bed_ — the
153-
track that gets processed — and they name the voice to listen to, exactly as a
153+
track that gets processed — and they name the voices to listen to, exactly as a
154154
sidechain compressor does: you select the track that gets quieter and pick what
155-
makes it quieter. **Never put a carve on the voice track.** A voice carved
156-
against itself is a bug, not a subtle mix choice.
155+
makes it quieter. **Never put a carve on a voice track.** A voice carved against
156+
itself is a bug, not a subtle mix choice.
157+
158+
**Every voice, not one of them.** `sources` is a list, because a bed usually runs
159+
under a whole sequence — a narrator, an interview answer, a second presenter. They
160+
are summed onto the bed's own clock before anything is measured (`mixCarveSources`),
161+
so one analysis covers all of them: the bands come from all the speech there is, and
162+
the envelopes rise wherever any of it is happening. Voices that never play while the
163+
bed does are left out; they cannot mask it.
157164

158165
**One knob.** `strength` is 0..1 and derives everything: how deep to cut, how
159166
many bands, how wide, how far to favour intelligibility over raw voice energy,
@@ -172,13 +179,11 @@ polish step to get to if there is time. Place both tracks, run the command below
172179
listen. Skip it only when there is no narration for the music to sit under — a
173180
music video, a title card, a montage cut to the track.
174181

175-
**Static or dynamic — dynamic unless you know otherwise.** A static carve holds
176-
its cuts for the whole clip, including every pause, so the bed is thinned where
177-
there is nothing to make room for. Dynamic turns every value into an envelope of
178-
the voice's own level: silence leaves the bed alone, a loud passage pushes the
179-
carve to full depth. That is what almost every voiceover wants, so it is the
180-
default. Reach for `--static` only for wall-to-wall narration with no real gaps,
181-
where an envelope is hundreds of breakpoints describing a constant.
182+
**It always follows the voice.** There is no static mode: a fixed depth thins the
183+
bed through every pause, and once you have heard both there is no reason to want it.
184+
Every value becomes an envelope of the speech's own level — silence leaves the bed
185+
alone, a loud passage pushes the carve to full depth — written as ordinary automation,
186+
which is why the lanes show up in the timeline and can be edited afterwards.
182187

183188
**Level matching is part of it.** Spectral carving cannot fix a bed that is
184189
simply louder than the voice. So the carve also measures how far over the voice
@@ -187,8 +192,13 @@ driven by an envelope for a dynamic one. That envelope releases slowly on
187192
purpose — music that snaps back to full the instant a word ends sounds like a
188193
machine doing it.
189194

190-
**Running it.** In Studio: pick the voice in the bed's Voiceover carve control;
191-
turning it on adds the modules and strength adjusts what is there. Headless —
195+
**Running it.** In Studio the carve is one module at the top of a track's effect
196+
rack — voice, strength, dynamic, and the analysis it produced, in one card. It is
197+
there whenever another track could be the voice, and a bed with exactly **one**
198+
candidate above it is carved by default, dynamically, at the default strength:
199+
that is what a bed under narration wants, and the module is where you change or
200+
switch it off. Several candidates leaves the picker waiting rather than guessing.
201+
Headless —
192202
which is the path when you are authoring a composition rather than editing one:
193203

194204
```bash
@@ -206,20 +216,17 @@ bands 400Hz -6dB q1.4, 1000Hz -3dB q1.4, 1600Hz -3.17dB q1.4
206216
level 216-point envelope, floor -6 dB
207217
```
208218

209-
Name the pair with `--bed` / `--voice` when the composition has several plausible
210-
tracks, `--strength` to push it, `--static` to hold one depth, `--dry-run` to see
211-
that report and write nothing.
212-
213-
**How it picks the pair.** Names first, because that is what you already told it
214-
and the answer is explainable: a track whose id or filename looks like music
215-
(`music`, `bgm`, `bed`, `score`…) is the bed, one that looks like a voice
216-
(`voice`, `vo`, `narration`, `speech`…) is the voice, and SFX-shaped names are not
217-
candidates for either. If one role is filled and a single track is left, that
218-
track takes the other role. Only when names decide nothing does it listen: it
219-
measures how much of each track is quiet, and the one that stops between phrases
220-
is the voice. **When two tracks are too close to call it refuses and asks you to
221-
name them** rather than carving the wrong one — a bed carved against a bed is
222-
silent and confusing, and typing two ids is cheap.
219+
Name the tracks with `--bed` / `--voice` (repeatable) when the automatic choice is
220+
wrong, `--strength` to push it, `--dry-run` to see that report and write nothing.
221+
222+
**How it picks the tracks.** Names first, because that is what you already told it
223+
and the answer is explainable — `classifyAudioName` in core, the same classifier
224+
Studio's own picker uses, so the two cannot disagree. A track whose id or filename
225+
looks like music (`music`, `bgm`, `bed`, `score`…) is the bed; everything else that
226+
plays over it and is not SFX-shaped is a voice. Audio elements are preferred: video
227+
counts only when no audio track is left to be the voice, or every B-roll clip in the
228+
composition would read as somebody talking. **It refuses when it cannot tell which
229+
track is the bed** rather than carving the wrong one — typing one id is cheap.
223230

224231
Same analysis functions as the panel, so the result is identical. Needs `ffmpeg`
225232
on PATH and `@hyperframes/core` installed in the project (`npm i -D

skills/hyperframes-audio/references/attributes.md

Lines changed: 12 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -92,13 +92,18 @@ worklet effects (`compressor`, `limiter`, `gate`, `bitcrush`) have none at all.
9292
## `data-fx-carve` — the carve's settings
9393

9494
```json
95-
{ "source": "narration", "strength": 0.35, "dynamic": true }
95+
{ "enabled": true, "sources": ["narration", "interview-guest"], "strength": 0.35 }
9696
```
9797

98-
- `source` is the **element id of the voice track to listen to**. It lives on the
99-
bed being processed, not on the voice.
98+
- `sources` are the **element ids of every voice this bed makes room for**. They live
99+
on the bed being processed, not on the voices. Summed onto the bed's clock before
100+
the analysis, so one set of filters and envelopes covers all of them.
100101
- `strength` 0..1 derives the whole mechanism (see `carveProfile`).
101-
- `dynamic` follows the voice moment to moment instead of holding one depth.
102+
- There is no `dynamic`: a carve always follows the speech.
103+
- `enabled` is whether the carve applies. It exists because a bed with exactly one
104+
candidate voice is carved by default: with "off" represented by an absent
105+
attribute, switching it off would read as never-configured and the default would
106+
put it back. `enabled: false` keeps the settings and stops the carve.
102107

103108
This attribute is not read at playback — the chain and lanes it produced are what
104109
play. It exists so the settings can be read back and re-derived rather than
@@ -107,4 +112,6 @@ carve possible.
107112

108113
Older projects may carry the six mechanism numbers (`maxCutDb`, `bands`, `q`,
109114
`intelligibilityBias`, `duckDb`, `headroomDb`) instead of `strength`. They still
110-
load: the depth maps back onto a strength and everything else is re-derived.
115+
load: the depth maps back onto a strength and everything else is re-derived. A
116+
stored carve with no `enabled` reads as on, and a single `source` reads as a one-voice
117+
`sources` list. A stored `dynamic` is ignored — every carve follows the speech now.

0 commit comments

Comments
 (0)