A spatial audio platform for creating and experiencing immersive 3D music mixes. Position individual stems in 3D space and let listeners experience your music with headphones-based HRTF spatial audio.
Frontend: SvelteKit + TypeScript + PixiJS + Resonance Audio Backend: Express + TypeScript + MongoDB + GridFS Auth: Auth.js with Google OAuth & Email/Password Deployment: Vercel (frontend) + Railway (backend)
- 🎵 Spatial Audio Editor - Position stems in 3D space using visual map interface
- 🎧 HRTF Playback - Headphones-only binaural spatial audio using Google Resonance Audio
- 📊 Dual Coordinate Systems - Cartesian (meters) and Spherical (azimuth/elevation/radius)
- 🎚️ Stem Management - Upload, position, adjust gain, and delete individual stems
▶️ Real-time Preview - Play/pause/restart with synchronized spatial audio- 🔄 Audio Deduplication - Efficient GridFS storage with MD5 hash deduplication
- 🔐 Multi-method Auth - Google OAuth or Email/Password via Auth.js
- 📱 Phone Recovery - SMS-based password reset using Twilio
- 👤 User Ownership - Songs tied to creators, edit permissions enforced
- 👁️ Public Listening - Anyone can listen, only owners can edit
- 🛡️ Protected Routes - JWT-like session validation on all write operations
- 🎨 Visual Feedback - Real-time waveform visualization during playback
- 🎯 Interactive Map - Drag-and-drop stem positioning with visual guides
- 📏 Scale Control - Zoom in/out on the spatial map
- ⌨️ Keyboard Shortcuts - Play/pause (Space), coordinate mode toggle, etc.
- 🔊 Per-stem Gain - Individual volume control for each stem
- Node.js 18+
- MongoDB URI (MongoDB Atlas or local instance)
- Google OAuth credentials (for sign-in)
- Twilio account (for SMS password recovery)
# Auth Configuration
AUTH_SECRET=your-random-secret-string-here
GOOGLE_CLIENT_ID=your-google-client-id
GOOGLE_CLIENT_SECRET=your-google-client-secret
# MongoDB
MONGODB_URI=mongodb://localhost:27017 # or your Atlas URI
# API
VITE_API_URL=http://localhost:3001# MongoDB
MONGODB_URI=mongodb://localhost:27017 # or your Atlas URI
# Port
PORT=3001# Install all dependencies
npm install
# Build shared package
npm run build:shared
# Run development servers (client + server)
npm run devThe client will run on http://localhost:5173 and the server on http://localhost:3001.
Frontend (Vercel):
- Set
VITE_API_URLto your Railway backend URL - Set Auth.js environment variables
Backend (Railway):
- Set
MONGODB_URIto your MongoDB Atlas connection string - Auto-deploys from GitHub pushes
audio-xr/
├── client/ # SvelteKit frontend
│ ├── src/
│ │ ├── routes/ # Pages and API routes
│ │ │ ├── +page.svelte # Home page
│ │ │ ├── songs/+page.svelte # Songs list
│ │ │ ├── songs/[id]/edit/ # Spatial audio editor
│ │ │ └── auth/signin/ # Authentication
│ │ ├── lib/
│ │ │ ├── components/ # Reusable UI components
│ │ │ │ ├── MapEditor.svelte # 3D positioning editor
│ │ │ │ └── UploadSongForm.svelte
│ │ │ └── api.ts # API configuration
│ │ └── hooks.server.ts # Auth.js configuration
│ └── package.json
├── server/ # Express backend API
│ ├── src/
│ │ ├── routes/
│ │ │ ├── songs.ts # Song CRUD
│ │ │ ├── stems.ts # Stem streaming
│ │ │ ├── stems-upload.ts # Stem upload/update
│ │ │ └── users.ts # User registration/recovery
│ │ ├── middleware/
│ │ │ └── auth.ts # Authentication middleware
│ │ └── index.ts # Express server
│ └── package.json
├── shared/ # Shared TypeScript types
│ └── src/types/song.ts
└── package.json # Root workspace config
- SvelteKit - Full-stack framework with SSR
- TypeScript - Type safety
- PixiJS - 2D map canvas rendering
- Resonance Audio - Google's spatial audio library (HRTF)
- Auth.js - Authentication framework
- Express - REST API server
- MongoDB - Document database
- GridFS - Audio file storage with deduplication
- bcryptjs - Password hashing
- Multer - File upload handling
- Auth.js (SvelteKit) - Session management
- Google OAuth - One-click sign-in
- Credentials Provider - Email/password auth
- Twilio (optional) - SMS password recovery
- Upload: Creator uploads a song and individual stems (vocals, drums, bass, etc.)
- Position: Creator positions each stem in 3D space using the map editor
- Store: Stems stored in MongoDB GridFS with position metadata
- Listen: Listener loads the song, downloads stems, and hears spatial audio
- HRTF: Resonance Audio processes each stem based on its 3D position
- Output: Binaural audio played through headphones
- User clicks "Sign In to Create" or "Upload"
- Redirected to
/auth/signin - Choose Google OAuth or Email/Password
- If Email/Password signup, optionally add phone number for recovery
- Session created via Auth.js
- User ID attached to all created songs
- Edit actions check ownership before allowing changes
- User clicks "Forgot Password"
- Enters phone number
- Receives 6-digit SMS code via Twilio
- Enters code + new password
- Password reset, can log in again
GET /api/songs- List all songs (public)GET /api/songs/:id- Get song details (public)POST /api/songs- Create song (requires auth)
POST /api/songs/:id/stems- Upload stem (requires ownership)PATCH /api/songs/:id/stems/:stemId- Update stem position (requires ownership)DELETE /api/songs/:id/stems/:stemId- Delete stem (requires ownership)GET /api/stems/:fileId- Stream stem audio (public)
POST /api/users/register- Register new userPOST /api/users/reset-password/request- Request SMS reset codePOST /api/users/reset-password/verify- Verify code and reset password
For Listeners:
- Browse songs at
/songs - Click any song to listen
- Put on headphones for full spatial effect
- Press play and experience 3D audio
For Creators:
- Sign in with Google or create account
- Click "Create New" to upload a song
- Add individual stems (vocals, drums, bass, etc.)
- Position stems on the 3D map
- Preview with play button
- Share the song URL with listeners
- Passwords hashed with bcryptjs (never stored plain)
- Google OAuth users have no password stored
- Sessions validated via Auth.js
- API routes protected with ownership checks
- CORS configured for production domains
- Admin has MongoDB access for manual recovery
See LICENSE.txt for details.