Skip to content

Audio caching policy: cache music + plain speech, NOT segment-aware speech #220

Description

@SecurityQQ

Policy

Audio is cached at the render layer (render/src/api/render/cache-utils.ts), with one exception: segment-aware speech is not media-cached.

Audio kind Media-cached? Why
Music (generateMusic) ✅ yes no timing metadata to lose
Plain speech (no segments) ✅ yes no timing metadata to lose
Segment-aware speech (await Speech({ children: [...] })) ❌ no carries segments/words/duration a media blob can't hold

Segment-aware speech is still cached losslessly via the SDK's own resolveSpeech cache key (sdk/src/react/resolve.ts), which preserves the full CachedSpeechResult.

How it's enforced

  • extractMediaData returns null for anything shaped like CachedSpeechResult ({ file, duration, words?, segments? }) → not written to the media cache.
  • extractMediaData extracts { audio: {...} } payloads (music + plain generations) → written to the media cache.
  • reconstructCachedValue returns audio in the { audio: {...} } shape the SDK destructures (const { audio } = ...).

History (why this exists)

  1. PR chore: remove fal-timing debug prints #91 media-cached all audio → dropped segments → crashed segments[0].duration.
  2. Revert feat(fal): persist queue request IDs to recover from timeouts #93 (f2e7ee9) removed the { audio } wrapper but left stale entries → crashed audio.mediaType.
  3. Current fix: cache audio except segment-aware speech, and restore the { audio } wrapper.

Related: vargHQ/render#97, #219.

If you touch this

Never media-cache a value that has segments/words/duration — it must round-trip through the SDK resolveSpeech cache instead.

Metadata

Metadata

Assignees

No one assigned

    Labels

    documentationImprovements or additions to documentation

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions