Skip to content

lasect/back-the-f-up

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

15 Commits
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

back-the-f-up

back-the-f-up is a self-hostable PostgreSQL recovery service built for environments where you only have a connection string.

It provides logical-tier near point-in-time recovery (PITR) without requiring filesystem access on the source database host.

Why this exists

I wanted to mess around with backups for Postgres for a while now. I used PgBackRest a lot for Lasect Orca's backup module. PITR was still something left on the list and the final nail in the coffin was coming across Sam Lambert's tweet about Supabase charging $100/mo for PITR. That's insane.

That kicked off my "research" on how PITR has been handled in Postgres so far and if I can frankenstein something for Supabase users, and coincidentally anyone running a managed Postgres instance. This is the result of that.

What it does today

  • Registers a source database and runs capability probes/scans.
  • Creates publication + logical replication slot.
  • Takes base backup via EXPORT_SNAPSHOT + pg_dump -Fc.
  • Streams logical changes and stores compressed NDJSON WAL segments.
  • Tracks metadata in SQLite or Postgres: connections, base backups, WAL segments, schema & sequence snapshots
  • Restores by applying base dump (pg_restore) and replaying WAL up to:
    • target LSN
    • target timestamp
  • Runs scheduled online backups with cron expressions.
  • Supports pause/resume per connection.

Architecture

  1. Source registration creates or validates replication objects.
  2. Engine opens replication connection and exports consistent snapshot.
  3. pg_dump reads that snapshot and writes base.dump artifact.
  4. Replication stream is decoded to WAL events and flushed as segments.
  5. Restore loads base.dump, then replays WAL events until cutoff.

Storage backends:

  • local filesystem (STORAGE_TYPE=local)
  • S3-compatible object storage (STORAGE_TYPE=s3)

Requirements

On the service host:

  • Go 1.25+
  • pg_dump and pg_restore on PATH
  • network access to source and restore targets

On source Postgres:

  • logical replication support
  • ability to create/use publication and replication slot
  • privileges needed by probe/scan/backup flow

Notes:

  • matching pg_dump/pg_restore major version with source is strongly recommended
  • connection-pooler endpoints that do not support logical replication are not suitable sources

Quick start

git clone https://github.com/lasect/back-the-f-up
cd back-the-f-up
go run ./cmd/server

Server defaults:

  • API: http://localhost:8080
  • metadata DB: local SQLite at ./pitr.db
  • artifact storage: local directory ./data/storage

Health check:

curl http://localhost:8080/api/v1/health

Current limitations

  • Current implementation is logical-tier PITR only (no physical/WAL archiving tier).
  • Pause keeps behavior stable but replication slot cleanup on pause can be improved.

Roadmap

  • Registration + probe + scans + DDL capture fallback logic
  • EXPORT_SNAPSHOT + pg_dump base backup
  • WAL streaming and segment persistence
  • Restore pipeline (base dump + replay)
  • Coverage and schema snapshot APIs
  • Scheduled online backups
  • Pause/resume connection control
  • API authentication (GitHub OAuth + session cookie; allowlist optional)
  • Physical backup tier (pg_receivewal / pg_basebackup)
  • Streamed base backup upload path for large databases
  • More granular observability (metrics and structured failure categories)

Project status

Work in progress. I post updates on X @ hiwinit.

About

hacky self-hostable near PITR service for postgres with just sql level access

Resources

Stars

Watchers

Forks

Releases

No releases published

Contributors