Google Drive uploads + unified Storage layer

[richiemcilroy]

This change introduces optional Google Drive as a first-class recording/upload destination alongside existing S3-style storage, wired through a new Storage Effect service in @cap/web-backend that abstracts presigned PUT/POST, resumable Drive uploads, and per-video storageIntegrationId resolution.

Greptile Summary

This PR adds Google Drive as a first-class storage destination alongside S3, wired through a new Storage Effect service in @cap/web-backend. The unified layer abstracts presigned PUT/POST, resumable Drive uploads, folder management, and per-video storageIntegrationId resolution through four new database migrations.

• Drive storage layer (Storage/GoogleDrive.ts, Storage/index.ts, Storage/StorageRepo.ts): implements Drive resumable upload sessions, folder reservation with DB-level optimistic locking, token caching with lease-based refresh coordination, and a proxy route (/api/storage/object) for serving Drive-hosted files.
• Upload path migration: multipart.ts, signed.ts, and the desktop Rust client now resolve storage access per-video (Storage.getAccessForVideo) rather than per-user-bucket, propagating the provider type back to clients so Drive-specific Content-Range headers are set correctly.
• DB schema (schema.ts, migrations 0017–0020): adds storage_integrations and storage_objects tables; FK constraints land in migration 0018 (not 0017).

Confidence Score: 4/5

Broadly safe to merge; all previously flagged blocking issues are addressed and no new blocking defects were found.

The Drive storage layer is substantially more complete than the previous revision — token-refresh-on-every-call, thundering-herd DB polling, quota-cache credential-overwrite, and JSON.parse-throws paths are all addressed. No new blocking defects were found. The net risk is the sheer volume of new Drive-specific code and the interactions between DB-level reservation locking, in-process token cache, and resumable upload session lifecycle, all exercised in production for the first time.

The Drive folder-reservation path in packages/web-backend/src/Storage/GoogleDrive.ts and the sequential chunking logic in instant-mp4-uploader.ts are the most complex new paths and deserve close monitoring in production.

Important Files Changed

Filename	Overview
packages/web-backend/src/Storage/GoogleDrive.ts	New Drive storage implementation with token caching (lease-based), resumable upload session creation, folder reservation, and in-process token deduplication. Token refresh is now cached. Folder reservation uses exponential backoff (8 retries) replacing the old 150x100ms polling loop.
packages/web-backend/src/Storage/index.ts	Unified Storage Effect service abstracting S3 and Drive. Drive getPresignedUploadPartUrl returns the same session URL for all parts. Drive multipart.complete marks the DB record complete trusting the client committed the Drive file before calling the endpoint.
packages/web-backend/src/Storage/StorageRepo.ts	New Effect service for DB access. Token refresh lease uses a conditional UPDATE (CAS). Access token stored encrypted in a dedicated column, resolving the previously flagged quota-cache race.
packages/web-backend/src/Storage/SignedObject.ts	Token signing/verification. JSON.parse now wrapped in try/catch; malformed tokens return null rather than throwing.
apps/web/app/api/storage/object/route.ts	Drive object proxy. StorageError mapped to 502. Token verification no longer throws. Key-path ownership check gates access correctly.
apps/web/app/api/desktop/[...route]/storage.ts	OAuth flow for Google Drive. htmlResponse uses escapeHtml. set-active returns 404 instead of 500 when no Drive integration exists.
apps/web/app/api/upload/[...route]/multipart.ts	Multipart upload endpoints resolve storage access per-video. Drive complete passes MpuObjectSize. Media-server processing and metadata copy guarded to S3 only.
apps/web/app/(org)/dashboard/caps/components/web-recorder-dialog/instant-mp4-uploader.ts	Drive upload: sequential slots (1), 16 MB part size, Content-Range tracking via partOffsets, resolveFinalTotalBytes guarantees the last chunk carries a concrete total.
apps/web/lib/google-drive-storage-quota.ts	Quota cached in a dedicated JSON column. The previously flagged read-modify-write race that could overwrite the refresh token is resolved.
packages/database/migrations/0017_productive_betty_brant.sql	Creates storage_integrations and storage_objects tables without FK constraints. FK constraints are added in migration 0018.
apps/desktop/src-tauri/src/upload.rs	Rust client propagates provider from initiate response, detects Drive session URLs, adds Content-Range headers, limits concurrent Drive uploads to 1.
packages/database/schema.ts	Added storageIntegrations and storageObjects tables with FK references and onDelete: restrict.

Comments Outside Diff (2)

1. apps/web/app/(org)/dashboard/caps/components/web-recorder-dialog/instant-mp4-uploader.ts, line 661-678 (link)

   Drive upload may never finalize if finalTotalBytes is null when the last chunk is sent

   uploadBlobWithProgress sets Content-Range: bytes N-M/* whenever this.finalTotalBytes is null. Google Drive's resumable upload protocol requires the final request to supply a concrete total; a request with total "*" always returns 308 Resume Incomplete and the file is never closed. If any finalisation path that doesn't set finalTotalBytes flushes the remaining Drive buffer, the upload session stays open indefinitely and bucket.multipart.complete then calls getGoogleDriveFileMetadata against a file Drive has not yet committed.

   Prompt To Fix With AI

   This is a comment left during a code review.
   Path: apps/web/app/(org)/dashboard/caps/components/web-recorder-dialog/instant-mp4-uploader.ts
   Line: 661-678
   
   Comment:
   **Drive upload may never finalize if `finalTotalBytes` is null when the last chunk is sent**
   
   `uploadBlobWithProgress` sets `Content-Range: bytes N-M/*` whenever `this.finalTotalBytes` is null. Google Drive's resumable upload protocol requires the final request to supply a concrete total; a request with total `"*"` always returns 308 Resume Incomplete and the file is never closed. If any finalisation path that doesn't set `finalTotalBytes` flushes the remaining Drive buffer, the upload session stays open indefinitely and `bucket.multipart.complete` then calls `getGoogleDriveFileMetadata` against a file Drive has not yet committed.
   
   How can I resolve this? If you propose a fix, please make it concise.

2. apps/web/app/api/upload/[...route]/signed.ts, line 87-97 (link)

   Batch signed-URL endpoint discards upload-target type for Drive

   The batch endpoint returns only upload.url, dropping type and headers. For Google Drive, upload.url is a resumable-upload session URL. Any caller that PUTs to that URL without Content-Range will get an error — the header is mandatory for resumable uploads. The single-URL endpoint below correctly preserves type and headers; the batch path should do the same so callers can detect the Drive case and set the required header.

   Prompt To Fix With AI

   This is a comment left during a code review.
   Path: apps/web/app/api/upload/[...route]/signed.ts
   Line: 87-97
   
   Comment:
   **Batch signed-URL endpoint discards upload-target type for Drive**
   
   The batch endpoint returns only `upload.url`, dropping `type` and `headers`. For Google Drive, `upload.url` is a resumable-upload session URL. Any caller that PUTs to that URL without `Content-Range` will get an error — the header is mandatory for resumable uploads. The single-URL endpoint below correctly preserves `type` and `headers`; the batch path should do the same so callers can detect the Drive case and set the required header.
   
   How can I resolve this? If you propose a fix, please make it concise.

_{Reviews (6): Last reviewed commit: "fix(web): map storage proxy errors and h..." | Re-trigger Greptile}

[richiemcilroy]

hey @greptileai please rereview the PR

[richiemcilroy]

hey @greptileai please re-review the PR big fella

[richiemcilroy]

hey @greptileai please re-review the PR.

[greptile-apps]

Want your agent to iterate on Greptile's feedback? Try greploops.