feat: 8 video-use-inspired clip-quality improvements

Port video-use Hard Rules + superpowers verification into the pipeline:

1. Audio-pop fix — 30ms afade at every smartcut concat boundary
   (pipeline/cut_ops.audio_fade_filter)
2. Word-boundary snap — snap Gemini clip [start,end] to nearest transcript
   word + pad, before metadata write (pipeline/cut_ops.snap_clip_to_words)
3. Self-eval QA — ffprobe the composed clip, log duration/audio/empty issues
   (domain/clip_qa + compose._self_eval); subsumes #6 z-order sanity
4. Colour grade — domain/grade.py compose layer (runs first), Create+Edit UI
5. Animated hooks — fade + ease-out-cubic slide entrance
   (hooks.build_hook_overlay_filter, style.animate)
6. Overlay z-order — folded into #3 QA
7. Conversational trim — domain/clip_edit_ai + POST /api/edit-ai → drop_ranges,
   AI-trim input in EditClipModal
8. Cross-job taste memory — lib/taste.js records kept/discarded, injects a hint
   into the next job's Gemini instructions

Pure cores host-tested; ffmpeg paths integration-tested
(tests/domain/test_ffmpeg_render_integration.py). New endpoint /api/edit-ai
guarded by tests/api/test_edit_ai_api.py. CLAUDE.md documents all of it.

Verified: 511 host + 15 JS + 28 Docker integration tests pass; frontend builds.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Kggb11c98UYEA648Q2Y629

Author: fralapo

CLAUDE.md

@@ -12,7 +12,16 @@ All Python backend code lives under `src/clippyme/` (src-layout, installed via `
 
 - `src/clippyme/api/` — FastAPI app (`app.py`), `schemas.py`, `security.py`
 - `src/clippyme/pipeline/` — `main.py` orchestrator (~800 LOC, was 2216), `deepgram_transcribe.py`, `gemini_parser.py`, `gemini_service.py`, plus extracted stages: `transcribe_cache.py`, `download.py`, `scene_detection.py`, `postprocess.py`, `diarization.py`, `hardware.py` (DEVICE/Whisper-model selection), `reframe.py` (cv2/YOLO/MediaPipe core: `process_video_to_vertical` + `SmoothedCameraman`/`SpeakerTracker`/`DetectionSmoother` + face/person detection; `ASPECT_RATIO` is a module global `main` sets per-job via `reframe.ASPECT_RATIO = ...`), pure-math `reframe_ops.py`, and `media_probe.py` (cv2-free ffprobe wrappers + pure A/V-sync helpers: VFR detection, stream `start_time` compensation, frame-rate parsing — host-unit-tested; wired into `reframe.py`/`main.py` to fix VFR drift + YouTube audio desync. Ported from `kamilstanuch/Autocrop-vertical`, see `docs/autocrop-vertical-analysis.md`), and `texttiling_ops.py` (cv2-free, dependency-light **lexical TextTiling** topic segmentation — gap/smooth/depth/boundary math ported from `ClipsAI/clipsai` but with a classic Hearst bag-of-words front-end instead of neural sentence embeddings, so **zero new deps**; host-unit-tested. Powers `main.build_texttiling_fallback`, the smarter **no-AI whole-video fallback**: when Gemini viral detection is unavailable/fails, the transcript is topic-segmented into several clips — shaped exactly like `get_viral_clips` output (`viral_score=0` + heuristic `viral_reason`) so they flow through the identical clip loop — instead of dumping the whole source as one vertical clip. Whole-video render remains the final safety net. Ported from `ClipsAI/clipsai`, see `docs/clipsai-analysis.md`). `main` re-exports the reframe classes for back-compat. Verify pipeline changes with `docker compose run --rm -u root backend sh -lc "pip install -q pytest && pytest -m integration"`.
-- `src/clippyme/domain/` — `compose.py`, `clip_endpoints.py`, `job_results.py`, `job_artifacts.py`, `job_worker.py`, `history_service.py`, `subtitles.py`, `smartcut.py`, `hooks.py`
+- `src/clippyme/domain/` — `compose.py`, `clip_endpoints.py`, `job_results.py`, `job_artifacts.py`, `job_worker.py`, `history_service.py`, `subtitles.py`, `smartcut.py`, `hooks.py`, `grade.py` (cv2-free colour-grade filter builder + ffmpeg wrapper, host-tested), `clip_qa.py` (pure post-render QA evaluator, host-tested), `clip_edit_ai.py` (conversational clip-trim: pure prompt/parse + thin Gemini call, host-tested)
+
+  **video-use-inspired improvements** (8 changes ported from the `/video-use` skill's Hard Rules + superpowers verification; pure cores host-tested, ffmpeg paths integration-tested in `tests/domain/test_ffmpeg_render_integration.py`):
+  - **#1 Audio-pop fix** — `pipeline/cut_ops.audio_fade_filter` adds a 30ms `afade` in/out at every segment boundary in `smartcut._render_with_ffmpeg` (the ffmpeg-concat fallback). The auto-editor v3 primary path cuts at silence keep-segments so it doesn't pop.
+  - **#2 Word-boundary snap** — `pipeline/cut_ops.snap_clip_to_words`/`flatten_words` snap each Gemini clip `[start,end]` to the nearest transcript word boundary + 50/80ms pad, applied in `main.py` BEFORE the metadata write so subtitles/Smart Cut stay aligned. No-op when the transcript lacks word timing.
+  - **#3 Self-eval QA** — `domain/clip_qa.evaluate_clip_qa` + `compose._self_eval` ffprobe the final composed clip and log duration/audio/empty-file issues (soft check, never raises). This also subsumes the #6 overlay/subtitle z-order sanity concern.
+  - **#4 Colour grade** — `domain/grade.py` compose layer (`GRADE_PRESETS`: warm_cinematic/cool_crisp/neutral_punch/vivid_pop), runs FIRST in compose so overlays keep authored colour. Toggle in both Create (`opts.gradePreset`) and EditClipModal (Grade tab).
+  - **#5 Animated hooks** — `hooks.build_hook_overlay_filter` animates the hook PNG with a fade + ease-out-cubic (`pow(1-p,3)`) slide-up entrance; `style.animate` toggle (synced in `data.js` + `hooks.py` defaults). animate=False is byte-identical to the legacy static overlay.
+  - **#7 Conversational trim** — `domain/clip_edit_ai` + `POST /api/edit-ai/{job}/{clip}`: a plain-English instruction → Gemini → spans to cut, fed through the existing `drop_ranges` machinery. Frontend "AI trim" input in the EditClipModal Trim tab.
+  - **#8 Cross-job taste memory** — `dashboard/src/lib/taste.js` records kept(publish)/discarded(remove) clip signals and distils a one-line hint appended to the Gemini `instructions` on the next job (rides the existing channel, no backend change). 120-event rolling window.
 - `src/clippyme/integrations/` — `social_publisher.py`, `auto_editor_updater.py`
 - `src/clippyme/storage/` — `config_store.py`
 
@@ -171,7 +180,7 @@ The redesign uses hand-rolled primitives in `dashboard/src/redesign/primitives.j
 - **Per-job LLM model override**: the Gemini model for viral detection is global (Settings → `GEMINI_MODEL`) but can be overridden **per job** via `ProcessRequest.model`/`BatchRequest.model` → `build_main_cmd(model=...)` → `--model` CLI arg → `main.py` sets `os.environ["GEMINI_MODEL"]` before `get_viral_clips` (mirrors the `--language` override). Validated at the boundary by `job_results.GEMINI_MODEL_RE` (`^gemini-[A-Za-z0-9.\-]{1,64}$` — blocks argv injection; allows future `gemini-3*`). Frontend: a quick-picker in MediaInput's Clip Options (`preselections.model`) + the live-discovery dropdown in Settings (`/api/config/models`, allow-list prefixes `gemini-2.5-`/`gemini-3` in `gemini_service.py`). Unknown models fall through to a `$0.00` "Pricing not available" cost note (`main.py:MODEL_PRICING`).
 - **Batch processing**: `POST /api/batch` accepts up to 20 URLs, creates one job per URL, and returns the list of `job_id`s. The frontend polls each job individually via `GET /api/status/{job_id}` and aggregates progress client-side. Supports `reframe_mode` parameter.
 - **Mixed batch (URLs + files)**: The frontend `useJobSubmission.handleBatchProcess` supports both. URLs are submitted in one shot to `/api/batch`; each file is submitted individually to `/api/process`. The hook then unifies polling across all returned `job_id`s using `/api/status/{job_id}`, aggregating progress until every job reaches a terminal state. No backend change is needed for mixed batches.
-- **Compose endpoint**: `POST /api/compose/{job_id}/{clip_index}` accepts `toggles` (smartcut/hook/subtitles/**logo** booleans), `hook_params`, `subtitle_params`, **`logo_params`**. Composes layers in order: **Subtitles → Smart Cut → Hook → Logo** (subtitle-drift-safe; logo absolutely last so the brand mark sits on top of every other layer). Returns `composed_url`. Cleans up intermediate files.
+- **Compose endpoint**: `POST /api/compose/{job_id}/{clip_index}` accepts `toggles` (**grade**/smartcut/hook/subtitles/**logo** booleans), `hook_params`, `subtitle_params`, **`logo_params`**, **`grade_params`** (`{preset}`). Composes layers in order: **Grade → Subtitles → Smart Cut → Hook → Logo** (grade first so overlays keep authored colour; subtitle-drift-safe; logo absolutely last so the brand mark sits on top of every other layer). After the final copy, `compose._self_eval` ffprobes the output and logs any QA issue (soft). Returns `composed_url`. Cleans up intermediate files.
 - **Brand assets** (client deliverables — e.g. ASCENSORE): a persistent **logo overlay** + **custom subtitle fonts**, both managed in Settings → *Brand assets*.
   - **Logo** (`domain/logo.py:add_logo_to_video`): ffmpeg `overlay` of an uploaded transparent PNG (`data/logo.png`, set via `POST /api/config/logo`). Placement is a position preset (7 anchors: corners + edge-centers + center) × size preset (`S/M/L` → 0.12/0.18/0.26 of frame width) × opacity. Geometry helper `logo_overlay_xy` is pure (host-tested, no ffmpeg). The compose **logo** layer reads `LOGO_PATH` and skips silently if no logo is uploaded.
   - **Custom fonts** (`subtitles.py:list_available_fonts` / `effective_fonts_dir`): user TTF/OTF uploads (`POST /api/config/fonts`) land in the writable `data/fonts/` volume; `effective_fonts_dir()` seeds it with copies of the bundled `fonts/` faces so a single `fontsdir` serves both (libass's `ass`/`subtitles` filter takes only one dir). Both burn branches pass `fontsdir`, so an uploaded face (e.g. a licensed **Stratos**) resolves at burn time. Upload validates the sfnt magic + the strict `_FONT_NAME_RE` (the stem becomes the libass font name injected into the ASS style). Frontend font dropdowns live-merge bundled + uploaded via `hooks/useFontList.js`.
@@ -204,6 +213,7 @@ python -m clippyme.pipeline.main <url_or_path> [options]
 | POST | `/api/compose/{job_id}/{clip_index}` | Compose final video from active toggles |
 | POST | `/api/smartcut/{job_id}/{clip_index}` | Generate smart-cut version of a clip (optional `drop_ranges` body for manual trim) |
 | GET | `/api/transcript/{job_id}/{clip_index}` | Per-clip transcript segments (clip-relative) for the manual-trim UI |
+| POST | `/api/edit-ai/{job_id}/{clip_index}` | Conversational trim: NL instruction → Gemini → `drop_ranges` to cut |
 | POST | `/api/reframe/{job_id}/{clip_index}` | Switch a clip between `auto` / `object` / `disabled` reframe mode (requires preserved source slice) |
 | POST | `/api/config/cookies` | Upload persistent cookies file |
 | GET | `/api/config/cookies/status` | Check if cookies are configured |

dashboard/src/hooks/useJobSubmission.js

@@ -1,6 +1,16 @@
 import { useEffect, useRef } from 'react';
 import { submitProcessJob, submitBatchJob } from '../lib/api';
 import { getApiUrl } from '../config';
+import { tasteInstructionSuffix } from '../lib/taste';
+
+// Append the cross-job taste hint (#8) to a job's AI instructions so Gemini
+// biases viral detection toward what the user actually kept in past jobs.
+function withTaste(data) {
+  const hint = tasteInstructionSuffix();
+  if (!hint) return data;
+  const base = (data.instructions || '').trim();
+  return { ...data, instructions: base ? `${base} ${hint}` : hint };
+}
 
 /**
  * Custom hook factory that returns process/batch submission handlers.
@@ -43,7 +53,7 @@ export function useJobSubmission({
     if (data.preselections) setPreselections(data.preselections);
 
     try {
-      const resData = await submitProcessJob(data, apiKey);
+      const resData = await submitProcessJob(withTaste(data), apiKey);
       setJobId(resData.job_id);
     } catch (e) {
       setStatus('error');
@@ -69,7 +79,7 @@ export function useJobSubmission({
 
       // 1. Submit URLs as a single backend batch (if any)
       if (urls.length > 0) {
-        const batchRes = await submitBatchJob({ ...data, urls }, apiKey);
+        const batchRes = await submitBatchJob(withTaste({ ...data, urls }), apiKey);
         allJobIds.push(...batchRes.jobs.map((j) => j.job_id));
         setLogs((l) => [...l, `Submitted ${batchRes.total} URL job(s)`]);
       }
@@ -78,12 +88,12 @@ export function useJobSubmission({
       for (const f of files) {
         try {
           const fileRes = await submitProcessJob(
-            {
+            withTaste({
               type: 'file',
               payload: f,
               instructions: data.instructions,
               preselections: data.preselections,
-            },
+            }),
             apiKey,
           );
           allJobIds.push(fileRes.job_id);

dashboard/src/lib/bulkApply.js

@@ -31,6 +31,7 @@ export function clipStateToParams(state, preselections, clip) {
     subtitleParams: state?.subtitleParams || seedSubtitleParams(preselections),
     hookParams: state?.hookParams || seedHookParams(clip, preselections),
     logoParams: state?.logoParams || seedLogoParams(preselections),
+    gradeParams: state?.gradeParams || { preset: preselections?.grade?.preset || 'none' },
   };
 }
 
@@ -67,6 +68,7 @@ export function buildClipParams(srcParams, targetClip, targetState) {
     // Copy the hook STYLE but keep this clip's own text.
     hookParams: { ...srcParams.hookParams, text: targetHookText(srcParams, targetClip, targetState) },
     logoParams: { ...srcParams.logoParams },
+    gradeParams: { ...(srcParams.gradeParams || { preset: 'none' }) },
     // Manual trim is never propagated.
     dropRanges: [],
   };

dashboard/src/lib/seedClipParams.js

@@ -14,9 +14,14 @@ export function seedToggles(preselections) {
         hook: !!preselections?.hook,
         subtitles: !!preselections?.subtitles,
         logo: !!preselections?.logo,
+        grade: !!(preselections?.grade && preselections.grade.preset && preselections.grade.preset !== 'none'),
     };
 }
 
+export function seedGradeParams(preselections) {
+    return { preset: preselections?.grade?.preset || 'none' };
+}
+
 export function seedLogoParams(preselections) {
     const logo = preselections?.logo;
     return {

dashboard/src/lib/taste.js

@@ -0,0 +1,82 @@
+// Cross-job taste memory (#8). ClippyMe never learned from which clips a user
+// actually kept vs threw away — every job started cold. This records a small
+// rolling signal (clip viral_score + duration + whether kept/published or
+// discarded) and distils it into a one-line natural-language hint that is
+// appended to the Gemini `instructions` on the NEXT job. The hint rides the
+// EXISTING instructions channel, so no backend change is needed: it reaches
+// `get_viral_clips` exactly like a user-typed directive.
+//
+// Pure functions (summarizeTaste) are unit-tested via `node --test`; the
+// localStorage I/O is thin and guarded.
+
+const KEY = 'clippyme_taste_v1';
+const MAX_EVENTS = 120;        // rolling window — old taste decays out
+const MIN_EVENTS = 6;          // below this we don't have enough signal to hint
+
+export function loadTasteEvents() {
+  try {
+    const raw = localStorage.getItem(KEY);
+    const arr = raw ? JSON.parse(raw) : [];
+    return Array.isArray(arr) ? arr : [];
+  } catch {
+    return [];
+  }
+}
+
+// action: 'kept' (published) | 'discarded' (removed/disabled).
+export function recordTasteEvent({ viralScore, duration, action }) {
+  if (action !== 'kept' && action !== 'discarded') return;
+  const ev = {
+    s: Number.isFinite(viralScore) ? Math.round(viralScore) : null,
+    d: Number.isFinite(duration) ? Math.round(duration) : null,
+    a: action,
+  };
+  try {
+    const events = loadTasteEvents();
+    events.push(ev);
+    const trimmed = events.slice(-MAX_EVENTS);
+    localStorage.setItem(KEY, JSON.stringify(trimmed));
+  } catch {
+    /* localStorage unavailable — taste memory is best-effort */
+  }
+}
+
+function median(nums) {
+  const xs = nums.filter((n) => Number.isFinite(n)).sort((a, b) => a - b);
+  if (!xs.length) return null;
+  const m = Math.floor(xs.length / 2);
+  return xs.length % 2 ? xs[m] : Math.round((xs[m - 1] + xs[m]) / 2);
+}
+
+// Pure: events → short hint string (or '' when there isn't enough signal).
+export function summarizeTaste(events) {
+  const evs = (events || []).filter((e) => e && (e.a === 'kept' || e.a === 'discarded'));
+  if (evs.length < MIN_EVENTS) return '';
+
+  const kept = evs.filter((e) => e.a === 'kept');
+  const discarded = evs.filter((e) => e.a === 'discarded');
+  const parts = [];
+
+  // Preferred length from clips the user kept.
+  const keptDur = median(kept.map((e) => e.d));
+  if (keptDur) {
+    const lo = Math.max(5, keptDur - 6);
+    const hi = keptDur + 6;
+    parts.push(`prefer clips roughly ${lo}-${hi}s long`);
+  }
+
+  // Score band the user tends to discard: if discarded clips skew low-score.
+  const discMedScore = median(discarded.map((e) => e.s));
+  const keptMedScore = median(kept.map((e) => e.s));
+  if (discMedScore != null && keptMedScore != null && keptMedScore - discMedScore >= 8) {
+    parts.push(`avoid weak moments — the user discards clips scoring below about ${keptMedScore}`);
+  }
+
+  if (!parts.length) return '';
+  return `Based on the user's past edits, ${parts.join(' and ')}.`;
+}
+
+// Convenience: the suffix to append to a job's AI instructions.
+export function tasteInstructionSuffix() {
+  return summarizeTaste(loadTasteEvents());
+}

dashboard/src/lib/taste.test.js

@@ -0,0 +1,27 @@
+import { test } from 'node:test';
+import assert from 'node:assert/strict';
+import { summarizeTaste } from './taste.js';
+
+test('summarizeTaste returns empty below the signal threshold', () => {
+  assert.equal(summarizeTaste([]), '');
+  assert.equal(summarizeTaste([{ a: 'kept', d: 20, s: 80 }]), '');
+});
+
+test('summarizeTaste suggests a preferred length from kept clips', () => {
+  const evs = Array.from({ length: 8 }, () => ({ a: 'kept', d: 20, s: 80 }));
+  const out = summarizeTaste(evs);
+  assert.match(out, /14-26s/);
+  assert.match(out, /past edits/);
+});
+
+test('summarizeTaste flags a discard score band when kept >> discarded', () => {
+  const kept = Array.from({ length: 5 }, () => ({ a: 'kept', d: 25, s: 85 }));
+  const disc = Array.from({ length: 5 }, () => ({ a: 'discarded', d: 25, s: 40 }));
+  const out = summarizeTaste([...kept, ...disc]);
+  assert.match(out, /scoring below about 85/);
+});
+
+test('summarizeTaste ignores invalid actions', () => {
+  const evs = Array.from({ length: 10 }, () => ({ a: 'bogus', d: 20, s: 80 }));
+  assert.equal(summarizeTaste(evs), '');
+});