Stash

Stash is a personal AI-powered content organization system. Forward URLs, images, videos, and text to a Telegram bot; Stash stores the artifact, classifies it with Gemini Flash, and exposes a searchable dashboard.

Prerequisites

• Python 3.11
• Node.js 20
• Docker and Docker Compose
• Telegram bot token from BotFather
• Google Cloud project with Vertex AI enabled
• Cloudflare R2 bucket and API token
• PostgreSQL and Redis, locally via Docker Compose or managed by Railway

Local Development

Copy the backend environment file and fill in secrets:

cp backend/.env.example backend/.env
cp frontend/.env.example frontend/.env

Start local infrastructure and services:

docker compose up --build

Run database migrations from the backend directory:

cd backend
alembic upgrade head

Start the frontend during development:

cd frontend
npm install
npm run dev

The API runs on http://localhost:8000 and the Vite frontend runs on http://localhost:5173.

Environment Variables

Backend variables live in backend/.env.

Required production variables:

• TELEGRAM_BOT_TOKEN: Bot token from BotFather.
• TELEGRAM_WEBHOOK_URL: Public webhook URL ending in /webhook.
• YOUR_CHAT_ID: Telegram chat ID for subcategory proposals and weekly digests.
• GOOGLE_CLOUD_PROJECT: Vertex AI project ID.
• VERTEX_REGION: Vertex region, usually us-central1.
• DATABASE_URL: PostgreSQL URL.
• REDIS_URL: Redis URL.
• R2_ACCOUNT_ID, R2_ACCESS_KEY, R2_SECRET_KEY: Cloudflare R2 credentials.
• R2_BUCKET_NAME, R2_BUCKET_ID: R2 storage bucket and public bucket ID.
• SECRET_KEY: JWT signing key.
• DASHBOARD_URL: Public frontend URL used in digests and Telegram dashboard magic links.
• CORS_ORIGINS: Comma-separated dashboard origins allowed to call the API.
• GEMINI_MODEL: Vertex AI Gemini model used for classification.
• GEMINI_VIDEO_MODEL: Vertex AI Gemini model used when real video bytes are available.
• GEMINI_INLINE_VIDEO_MAX_BYTES: Maximum Telegram-uploaded video size sent inline to Gemini.
• GEMINI_TRANSCRIPTION_INLINE_MAX_BYTES: Maximum stored video size sent inline to Gemini for delayed transcript-style analysis.

Local-only helpers:

• SKIP_AUTH=true disables dashboard auth locally.
• POSTGRES_DB, POSTGRES_USER, POSTGRES_PASSWORD configure the local Compose database.
• STASH_TMP_DIR controls where temporary media downloads are written.
• VIDEO_URL_ANALYSIS_ENABLED, VIDEO_URL_MAX_BYTES, VIDEO_URL_MAX_DURATION_SECONDS, and VIDEO_URL_DOWNLOAD_FORMAT control social/reel URL video downloads for Gemini analysis.
• YTDLP_COOKIES_BROWSER or YTDLP_COOKIES_FILE can be set when public video extraction needs cookies.
• DASHBOARD_ALLOWED_CHAT_IDS: Optional comma-separated chat ID allowlist for dashboard magic links. Defaults to YOUR_CHAT_ID.
• DASHBOARD_MAGIC_LINK_TTL_SECONDS: Optional one-time dashboard link TTL, default 600.
• DASHBOARD_SESSION_TTL_SECONDS: Optional browser dashboard session TTL, default 2592000.
• TASK_EXECUTION_MODE: inline for a single free-tier web service prototype, or celery for normal API + worker deployments.

Frontend variables live in frontend/.env.

• VITE_API_URL: API base URL.
• VITE_SKIP_AUTH: Set to true only for local dashboard auth bypass.
• VITE_TELEGRAM_BOT_NAME: Telegram bot name used for the dashboard link request button.

Never commit real .env files. Use .env.example files for safe placeholders only.

Deploying to Railway

Create Railway services for:

• web: FastAPI API using backend/Dockerfile.
• worker: Celery worker using the same image.
• beat: Celery Beat scheduler using the same image.
• Railway Redis.
• Railway PostgreSQL.

The service commands are defined in railway.toml.

Before deploying, run:

python scripts/check_deploy.py

Then apply migrations against the production database:

cd backend
alembic upgrade head

After deployment, verify:

curl https://your-api.railway.app/health

Expected response:

{"status":"ok","db":"ok","redis":"ok","r2":"ok"}

Deploying Backend to Render

Deploy the backend on Render as a normal web service, not as a Blueprint. Keep the frontend on Vercel and create database/Redis resources separately.

Follow the Render runbook in docs/render-deployment.md. After Render assigns the backend URL, set VITE_API_URL in Vercel to that API base URL and redeploy the frontend.

How To Use

Forward content to your Telegram bot:

• Instagram, LinkedIn, or regular URLs.
• Screenshots and images.
• Video files.
• Plain text snippets.

The bot replies immediately with Got it, processing..., then sends a saved confirmation after Celery finishes classification and storage.

Open the dashboard to browse categories, search artifacts, view details, re-categorize items, or delete stale saves.

For production dashboard access, send /dashboard to the Telegram bot. The bot replies with a private one-time link that opens the web dashboard and stores a signed session token in the browser.

Dynamic Categories

Stash starts with AI-generated top-level categories. As a category grows:

• At 10+ uncategorized items, the nightly evolution task proposes Tier 2 subcategories in Telegram.
• You can apply, skip for 30 days, or edit proposed names.
• At 50+ items in a subcategory, Tier 3 micro-clusters can be created automatically.

Manual dashboard corrections are stored as learning signals. When the same correction pattern appears at least three times, Stash creates a few-shot prompt example so future Gemini classifications are steered toward your preferences.

Troubleshooting

/health returns 503: Check the failed component names in the response. DB usually means DATABASE_URL; Redis means REDIS_URL; R2 means bucket credentials or bucket name.

Telegram webhook does not fire: Confirm TELEGRAM_WEBHOOK_URL is public HTTPS and ends with /webhook. Check Railway logs for webhook registration failures.

Artifacts stay in processing: Check Celery worker logs and Redis connectivity. If you are running only one free Render web service with no worker, use TASK_EXECUTION_MODE=inline so the API process runs the job after sending the Telegram acknowledgement.

Video transcription fails: Delayed video analysis uses Gemini. If a stored video is larger than GEMINI_TRANSCRIPTION_INLINE_MAX_BYTES, analysis is skipped and logged. The runtime image still installs ffmpeg because social/reel extraction uses yt-dlp.

Gemini errors: Verify GOOGLE_CLOUD_PROJECT, VERTEX_REGION, and Vertex AI permissions. Quota errors are wrapped as classification errors in worker logs.

R2 images do not load: Check R2_BUCKET_ID and that the bucket public URL is enabled. Stored object keys are private unless the public bucket URL is configured.