YouTube transcript fails on iOS Safari (m.youtube.com): User-Agent header on caption fetch breaks CORS preflight

[OrdenDelfai]

Symptom: In Obsidian Web Clipper on iOS Safari, {{transcript}} is always empty for YouTube videos, while the same template works on macOS (www.youtube.com). Previously reported as symptoms in obsidianmd/obsidian-clipper#728 and obsidianmd/obsidian-clipper#758, but the underlying fetch bug was never identified. Verified against current main (9db7260).

Root cause

On iPhone, Safari is redirected to m.youtube.com, so the caption XML fetch to https://www.youtube.com/api/timedtext?... is cross-origin. fetchCaptionXml attaches a custom header:

https://github.com/kepano/defuddle/blob/9db72600/src/extractors/youtube.ts#L585

const captionHeaders: Record<string, string> = { 'User-Agent': 'Mozilla/5.0' };

User-Agent is no longer a forbidden header in WebKit, so the browser does not strip it. On a cross-origin request it is a non-safelisted header, which forces a CORS preflight. The timedtext endpoint's preflight response contains no Access-Control-Allow-Headers, so the request is blocked and the fetch rejects with TypeError: Load failed. The transcript then falls through to the mobile DOM-clicking fallback, which frequently fails (see "Secondary issues" below), so the user gets an empty transcript.

On desktop (www.youtube.com) the same fetch is same-origin, CORS never applies, and everything works. That is the entire macOS/iOS asymmetry.

Reproduction (Playwright WebKit 26.4, which is the iOS Safari engine)

On a loaded https://m.youtube.com/watch?v=dQw4w9WgXcQ page, in page context:

Step (mirrors youtube.ts logic)	Result
InnerTube IOS client POST to /youtubei/v1/player	PASS, status 200, 6 captionTracks
fetch(track.baseUrl, { headers: { 'User-Agent': 'Mozilla/5.0' } })	FAIL, TypeError: Load failed
Identical fetch(track.baseUrl) with no headers	PASS, status 200, 4555 bytes of transcript XML

Preflight evidence via curl (note: no Access-Control-Allow-Headers in the response, so the requested user-agent header is not approved):

$ curl -X OPTIONS "https://www.youtube.com/api/timedtext?..." \
    -H "Origin: https://m.youtube.com" \
    -H "Access-Control-Request-Method: GET" \
    -H "Access-Control-Request-Headers: user-agent" -D -
HTTP/2 200
access-control-allow-origin: https://m.youtube.com
access-control-allow-credentials: true

Suggested fix

Only set the User-Agent header outside browser contexts (it exists for the server-side CLI; in a browser the real UA is sent anyway and the custom value is at best ignored):

const captionHeaders: Record<string, string> = {};
if (typeof window === 'undefined') {
    captionHeaders['User-Agent'] = 'Mozilla/5.0';
}

The same consideration applies to the Android InnerTube client path in fetchPlayerData, which also sends a custom User-Agent and cannot pass preflight from m.youtube.com (the player endpoint's preflight only allows content-type). The IOS client path is unaffected since Content-Type: application/json is approved.

Secondary issues found while diagnosing (can split into separate issues if preferred)

1. Relative inline baseUrl on m.youtube.com. The mobile page's ytInitialPlayerResponse caption baseUrl is relative (/api/timedtext?...), so the SSRF guard new URL(track.baseUrl) throws and kills the inline path even though that fetch would be same-origin on mobile. Fix: new URL(track.baseUrl, 'https://www.youtube.com') (https://github.com/kepano/defuddle/blob/9db72600/src/extractors/youtube.ts#L582).
2. Mobile DOM fallback rarely fires. openMobileTranscriptPanel requires button[aria-label="View all"], which only exists when the video has chapters, and the full chain (3 sequential 4s API attempts plus up to 5s polling per step) can exceed Obsidian Web Clipper's 8s parseAsync race, after which the sync fallback drops async variables like {{transcript}} while all sync variables fill in normally. That matches the "everything but the transcript" reports exactly.

Happy to submit a PR for the User-Agent and relative-URL fixes if useful.

[OrdenDelfai]

Additional field data after testing the proposed workaround on a physical iPhone (iOS Safari, Web Clipper 1.6.3):

Using Request Desktop Website so the page stays on www.youtube.com (same-origin, so the User-Agent/CORS issue above does not apply) was not sufficient by itself. {{transcript}} was still empty until the transcript panel was opened manually (expand description, tap "Show transcript"), after which clipping captured it correctly via the existing-DOM path (extractTranscriptFromExistingDom).

This suggests a second, iOS-specific issue independent of the CORS one: the fetch-based path (fetchTranscript) does not appear to complete on iOS even when same-origin. A plausible mechanism is that when the extension popup sheet opens over the page on iOS, the page context's async work is throttled or suspended, so the InnerTube/timedtext fetches never resolve and Web Clipper's 8s parseAsync race falls back to the sync parse, which drops async variables. I have not been able to instrument this on-device, so treat the mechanism as a hypothesis; the observable fact is fetch-path failure on www.youtube.com in iOS Safari while the same build works on macOS Safari.

Practical implication: on iOS the existing-DOM path is currently the only reliable one, so making it as robust as possible (and/or having the clipper kick off the transcript fetch before the popup opens) may matter as much as the CORS fix for m.youtube.com.

Workaround for anyone landing here from search, until a fix ships — on iPhone:

1. Open the video in Safari and enable Request Desktop Website for youtube.com.
2. Expand the description and tap Show transcript so the panel is visible.
3. Then clip. The transcript is captured from the open panel.