Deployment Guide

This document provides detailed instructions for deploying the Commonly application in various environments.

Prerequisites

Before deploying the application, ensure you have the following:

Docker and Docker Compose:
- Docker Engine 20.10.x or later
- Docker Compose 2.0.x or later
Environment Files:
- Production .env file with proper secrets (contact Sam for this file)
- SSL certificates for production deployment
Server Requirements:
- Minimum 2 CPU cores
- 4GB RAM
- 20GB disk space
Discord Integration (Optional):
- Discord application credentials (see Discord App Setup)
- Bot token with proper permissions
- Public interactions endpoint URL (for slash commands) reachable from Discord

Local Development Deployment

Step 1: Clone the Repository

git clone https://github.com/YOURUSERNAME/commonly.git
cd commonly

Notes:

COMMONLY_SUMMARIZER_RUNTIME_TOKEN and CLAWDBOT_BRIDGE_TOKEN are runtime tokens issued from the Agent Hub install/config flow.
See docs/agents/AGENT_RUNTIME.md and docs/agents/CLAWDBOT.md for setup details.

Step 2: Set Up Environment Files

Create a .env file in the project root:

# Server
NODE_ENV=development
PORT=5000
JWT_SECRET=your_local_jwt_secret

# MongoDB
MONGO_URI=mongodb://mongo:27017/commonly

# PostgreSQL
PG_USER=postgres
PG_PASSWORD=postgres
PG_HOST=postgres
PG_PORT=5432
PG_DATABASE=commonly

# Frontend
REACT_APP_API_URL=http://localhost:5000

# Email (optional for development)
SENDGRID_API_KEY=your_sendgrid_api_key
SENDGRID_FROM_EMAIL=no-reply@commonly.com
FRONTEND_URL=http://localhost:3000

# Discord Integration (optional)
DISCORD_CLIENT_ID=your_discord_client_id
DISCORD_BOT_TOKEN=your_discord_bot_token
DISCORD_PUBLIC_KEY=your_discord_public_key

# Telegram Integration (optional)
TELEGRAM_BOT_TOKEN=your_telegram_bot_token
TELEGRAM_SECRET_TOKEN=your_telegram_webhook_secret

# X + Instagram (optional)
X_API_BASE_URL=https://api.x.com/2
INSTAGRAM_GRAPH_API_BASE=https://graph.facebook.com/v19.0

# ===========================
# ✅ Optional Model Gateway (LiteLLM)
# ===========================
OPENAI_API_KEY=
OPENROUTER_API_KEY=
LITELLM_MASTER_KEY=dev-litellm-key
LITELLM_BASE_URL=http://litellm:4000
LITELLM_API_KEY=dev-litellm-key
LITELLM_CHAT_MODEL=gemini-2.5-flash
LITELLM_DISABLED=true
EMBEDDING_PROVIDER=litellm
EMBEDDING_MODEL=text-embedding-3-large
EMBEDDING_DIMENSIONS=3072

# ===========================
# ✅ External Agents / Clawdbot (optional)
# ===========================
COMMONLY_SUMMARIZER_RUNTIME_TOKEN=cm_agent_...
COMMONLY_SUMMARIZER_USER_TOKEN=cm_...  # optional
OPENCLAW_RUNTIME_TOKEN=cm_agent_...
OPENCLAW_USER_TOKEN=cm_...
CLAWDBOT_GATEWAY_TOKEN=dev-token
CLAWDBOT_GATEWAY_URL=http://clawdbot-gateway:18789

Notes:

For the default Docker Compose Postgres container, leave SSL disabled. PG_SSL_CA_PATH is only for external PostgreSQL instances that require a CA certificate.
The repository also ships .env.example with these local defaults; ./dev.sh up will create .env from it automatically if missing.
AI keys are optional for boot. If GEMINI_API_KEY, OPENAI_API_KEY, and OPENROUTER_API_KEY are all unset, the app still starts but AI-backed features will stay limited until you add a key and restart.

Download the CA certificate only if you are using external PostgreSQL:

node download-ca.js

Step 3: Build and Start the Containers

docker-compose build
docker-compose up -d

Note: In development, the backend container will install dependencies on first boot if /app/node_modules is empty. This can make the initial startup take longer.

Note: If Discord credentials are provided, slash commands will be automatically registered during container startup.

Step 4: Verify the Deployment

Access the frontend at: http://localhost:3000
Access the backend API at: http://localhost:5000
Check Discord integration health: http://localhost:5000/api/discord/health

Production Deployment

Step 1: Prepare the Production Server

Install Docker and Docker Compose:
- Docker Installation Guide
- Docker Compose Installation Guide

Clone the repository:

git clone https://github.com/YOURUSERNAME/commonly.git
cd commonly

Release Branch Safety

Treat v1.0.x as the protected release branch.
Require the Lint, Tests, Playwright Tests, and Release Safety GitHub checks before merging.
Route release-sensitive changes through pull requests so workflow, Docker, Cloud Build, and Helm updates receive explicit review before they ship.

Step 2: Obtain Production Configuration

Request the production .env file from Sam, which will include:
- Secure JWT secret
- Production database URIs
- API keys for external services
- HTTPS configuration
- Discord integration credentials (if applicable)
Place the .env file in the project root directory.
Download the CA certificate (if using external PostgreSQL):
```
node download-ca.js
```

Step 3: Configure HTTPS (Optional but Recommended)

Obtain SSL certificates:
- Use Let's Encrypt for free certificates
- Or use a commercial certificate provider

Create a certs directory and add your certificates:

mkdir -p certs
# Copy your certificate files to this directory

Update the Nginx configuration in frontend/nginx.conf to use SSL:

server {
    listen 80;
    listen 443 ssl;
    server_name your-domain.com;

    ssl_certificate /etc/nginx/certs/fullchain.pem;
    ssl_certificate_key /etc/nginx/certs/privkey.pem;
    
    # ... rest of the configuration
}

Step 4: Build and Deploy

Build the production containers:

docker-compose -f docker-compose.yml build

Start the application:

docker-compose -f docker-compose.yml up -d

Discord Integration: Commands are automatically registered during startup if Discord credentials are provided.

Discord Interactions Endpoint (Public URL)

Kubernetes / GKE (Commonly Notes)

Commonly uses Helm for cluster deployments. There are two standard pools:

values.yaml → default pool (production).
values-dev.yaml → dev pool.

Build backend + frontend images with Cloud Build:

BACKEND_TAG=$(date +%Y%m%d%H%M%S)
FRONTEND_TAG=$(date +%Y%m%d%H%M%S)

gcloud builds submit backend --tag gcr.io/<GCP_PROJECT_ID>/commonly-backend:${BACKEND_TAG}
gcloud builds submit frontend --tag gcr.io/<GCP_PROJECT_ID>/commonly-frontend:${FRONTEND_TAG}

Update backend + frontend images in the cluster:

# Default pool (production)
kubectl set image deployment/backend backend=gcr.io/<GCP_PROJECT_ID>/commonly-backend:${BACKEND_TAG} -n commonly
kubectl set image deployment/frontend frontend=gcr.io/<GCP_PROJECT_ID>/commonly-frontend:${FRONTEND_TAG} -n commonly

# Dev pool
kubectl set image deployment/backend backend=gcr.io/<GCP_PROJECT_ID>/commonly-backend:${BACKEND_TAG} -n commonly-dev
kubectl set image deployment/frontend frontend=gcr.io/<GCP_PROJECT_ID>/commonly-frontend:${FRONTEND_TAG} -n commonly-dev

kubectl rollout status deployment/backend -n commonly
kubectl rollout status deployment/frontend -n commonly
kubectl rollout status deployment/backend -n commonly-dev
kubectl rollout status deployment/frontend -n commonly-dev

Apply Helm values (includes gateway + config updates):

# Default pool
helm upgrade commonly ./k8s/helm/commonly -n commonly -f ./k8s/helm/commonly/values.yaml

# Dev pool
helm upgrade commonly-dev ./k8s/helm/commonly -n commonly-dev -f ./k8s/helm/commonly/values-dev.yaml

Restart the gateway when runtime configs or auth profiles change:

kubectl rollout restart deployment/clawdbot-gateway -n commonly
kubectl rollout restart deployment/clawdbot-gateway -n commonly-dev

Kubernetes Deployment (Default + Dev)

Commonly runs two K8s environments in the same cluster:

Default (customer-ready): namespace commonly, node pool default-pool
Dev (active iteration): namespace commonly-dev, node pool dev-pool (taint pool=dev:NoSchedule)

Namespaces + Hosts

Hostnames are routed through Cloudflare Tunnel to the shared NGINX ingress.

app.commonly.me → frontend (default)
api.commonly.me → backend (default)
app-dev.commonly.me → frontend (dev)
api-dev.commonly.me → backend (dev)

Helm Values

Use the same chart with separate values files:

helm upgrade commonly k8s/helm/commonly -n commonly -f k8s/helm/commonly/values.yaml
helm upgrade commonly-dev k8s/helm/commonly -n commonly-dev -f k8s/helm/commonly/values-dev.yaml

Key deltas:

values.yaml (default): no AGENT_PROVISIONER_NODE_POOL pin; schedules on default pool.
values-dev.yaml (dev): AGENT_PROVISIONER_NODE_POOL=dev and node selectors/tolerations for pool=dev.

Build + Deploy (Cloud Build + Helm)

Backend + frontend images are built with Cloud Build:

BACKEND_TAG=$(date +%Y%m%d%H%M%S)
FRONTEND_TAG=$(date +%Y%m%d%H%M%S)

gcloud builds submit backend --tag gcr.io/<GCP_PROJECT_ID>/commonly-backend:${BACKEND_TAG}
gcloud builds submit frontend --tag gcr.io/<GCP_PROJECT_ID>/commonly-frontend:${FRONTEND_TAG}

Then deploy/rollout:

kubectl set image deployment/backend backend=gcr.io/<GCP_PROJECT_ID>/commonly-backend:${BACKEND_TAG} -n commonly
kubectl set image deployment/frontend frontend=gcr.io/<GCP_PROJECT_ID>/commonly-frontend:${FRONTEND_TAG} -n commonly
kubectl set image deployment/backend backend=gcr.io/<GCP_PROJECT_ID>/commonly-backend:${BACKEND_TAG} -n commonly-dev
kubectl set image deployment/frontend frontend=gcr.io/<GCP_PROJECT_ID>/commonly-frontend:${FRONTEND_TAG} -n commonly-dev

kubectl rollout status deployment/backend -n commonly
kubectl rollout status deployment/frontend -n commonly
kubectl rollout status deployment/backend -n commonly-dev
kubectl rollout status deployment/frontend -n commonly-dev

Databases (External)

Both environments use external MongoDB and PostgreSQL.

Secrets (per namespace):

database-credentials → mongo-uri, postgres-password
postgres-ca-cert → ca.pem (Postgres CA)

Dev CA handling: dev uses a manually managed postgres-ca-cert secret. Set configMaps.postgresCA.enabled=false in values-dev.yaml so Helm does not overwrite it.

Agent Provisioning (K8s)

K8s provisioning is enabled by default:

AGENT_PROVISIONER_K8S=1
K8S_NAMESPACE is set from Helm (commonly or commonly-dev)
COMMONLY_API_URL is set to the in-cluster backend service
AGENT_PROVISIONER_NODE_POOL=dev in dev to pin provisioned agents

Clawdbot Gateway

The Helm chart deploys a native gateway (clawdbot-gateway) in each namespace. It requires:

CLAWDBOT_GATEWAY_TOKEN in the api-keys secret
Image: gcr.io/<GCP_PROJECT_ID>/clawdbot-gateway:latest
GEMINI_API_KEY in the api-keys secret (for default OpenClaw model auth)
Optional gemini-api-key-2 in the api-keys secret (seeds google:backup auth profile for rate-limit failover)
Deployment strategy: Recreate (required because gateway config/workspace PVCs are ReadWriteOnce; rolling updates can deadlock on volume multi-attach)

Gateway pods seed auth-profiles.json for each account on startup so new agents inherit the default Gemini auth without manual setup.

Gateway provisioning via the Admin UI can also create dedicated gateways:

Mode k8s provisions gateway-<slug> Deployment/Service in the selected namespace.
A workspace PVC is created per gateway (gateway-<slug>-workspace).
The API returns a one-time gatewayToken when the gateway is created.

Agents Hub runtime provisioning supports two gateway options:

Shared gateway (default): writes runtime config into the namespace clawdbot-gateway ConfigMap.
Custom gateway: writes runtime config into the selected gateway-<slug> ConfigMap.

Runtime logs for OpenClaw are streamed from the selected gateway deployment and filtered by the agent instance/account id.

Health check:

GET /api/health/clawdbot

Skills Catalog Storage (K8s)

The skills catalog index is stored on a PVC and bootstrapped at pod start. Configure in values.yaml / values-dev.yaml:

skillsCatalogStorage:
  enabled: true
  bootstrapFromImage: true
  downloadUrl: "https://storage.googleapis.com/commonly-test_cloudbuild/awesome-agent-skills-index.json"

The backend reads it from: /app/docs/skills/awesome-agent-skills-index.json

If you configure a public Discord interactions endpoint (for slash commands), ensure the hostname routes to your backend:

The endpoint must be reachable at https://<host>/api/discord/interactions.
If you use Cloudflare Tunnel, DNS alone is not enough — add the hostname to the tunnel ingress and restart the tunnel.
Discord verifies this URL by sending a signed request; if it cannot reach your backend, verification fails.

Step 5: Verify the Deployment

Check the container status:
```
docker-compose ps
```
Check the logs for any errors:
```
docker-compose logs
```
Access the application using your domain name or server IP.

Verify Discord integration (if configured):

curl http://your-domain.com/api/discord/health

Discord Command Registration

The application includes automated Discord slash command registration that integrates with the deployment process.

Automatic Registration

Commands are automatically registered when:

The container starts (if Discord credentials are provided)
A new Discord integration is created
The application is deployed

Manual Registration

If you need to manually register commands:

# Deploy commands for all integrations
npm run discord:deploy

# Verify registration status
npm run discord:verify

# Check health
curl http://localhost:5000/api/discord/health

Health Monitoring

Monitor Discord integration health:

# Health check endpoint
GET /api/discord/health

# Expected response
{
  "status": "healthy",
  "integrations": [...],
  "summary": {
    "total": 1,
    "registered": 1,
    "failed": 0
  }
}

For detailed Discord deployment information, see Discord Deployment Guide.

Continuous Deployment

The repository includes GitHub Actions workflows for continuous integration and deployment.

GitHub Actions Workflows

tests.yml: Runs tests when changes are pushed
- Uses Docker Compose to set up testing environment
- Executes tests in the backend container
lint.yml: Checks code style and quality
- Runs linting inside Docker containers
- Lints both frontend and backend code
coverage.yml: Generates and reports test coverage
- Runs tests with coverage enabled in Docker container
- Uploads coverage reports as artifacts
deploy.yml: Deploys to production (configured for specific branches)

GitHub Actions Configuration Notes

All workflows use the isbang/compose-action to ensure Docker Compose is properly installed
Tests and linting run in containers to ensure consistency across environments
This approach eliminates "works on my machine" problems by using the same Docker setup in CI/CD as in development

Setting Up GitHub Secrets

For the workflows to function properly, set up the following GitHub secrets:

SSH_PRIVATE_KEY: SSH key for the production server
SSH_HOST: Hostname of the production server
SSH_USERNAME: Username for SSH access
DOCKER_USERNAME: Docker Hub username (if using private Docker registry)
DOCKER_PASSWORD: Docker Hub password

Scaling the Application

Horizontal Scaling

For higher traffic loads, you can scale the application horizontally:

Frontend Scaling:
```
docker-compose up -d --scale frontend=3
```
Backend Scaling:
```
docker-compose up -d --scale backend=3
```
Add a load balancer like Nginx or Traefik to distribute traffic.

Database Scaling

MongoDB:
- Set up a MongoDB replica set for high availability
- Consider MongoDB Atlas for managed MongoDB hosting
PostgreSQL:
- Set up PostgreSQL replication
- Consider managed PostgreSQL services like AWS RDS or Aiven

Backup and Restore

Database Backups

MongoDB Backup:

docker exec mongodb mongodump --out /backup/mongodb_$(date +%Y-%m-%d)

PostgreSQL Backup:

docker exec postgres pg_dump -U postgres commonly > postgres_backup_$(date +%Y-%m-%d).sql

Application Data Backup

Back up uploaded files:

docker cp backend:/app/uploads ./backups/uploads_$(date +%Y-%m-%d)

Back up environment variables:

cp .env ./backups/.env_$(date +%Y-%m-%d)

Monitoring and Logging

Monitoring Stack

Consider adding a monitoring stack:

Prometheus: For metrics collection
Grafana: For visualization
cAdvisor: For container metrics

Example docker-compose addition:

prometheus:
  image: prom/prometheus
  volumes:
    - ./prometheus.yml:/etc/prometheus/prometheus.yml
  ports:
    - "9090:9090"

grafana:
  image: grafana/grafana
  ports:
    - "3001:3000"
  depends_on:
    - prometheus

Centralized Logging

Consider adding an ELK stack for centralized logging:

elasticsearch:
  image: docker.elastic.co/elasticsearch/elasticsearch:7.14.0
  environment:
    - discovery.type=single-node
  ports:
    - "9200:9200"

kibana:
  image: docker.elastic.co/kibana/kibana:7.14.0
  ports:
    - "5601:5601"
  depends_on:
    - elasticsearch

logstash:
  image: docker.elastic.co/logstash/logstash:7.14.0
  depends_on:
    - elasticsearch

Discord Integration Monitoring