|
1 | 1 | # legacy-proxy |
2 | 2 |
|
3 | | -> IMAP / SMTP / ManageSieve → JMAP gateway. Use any JMAP client with Gmail, |
4 | | -> your hosting provider's mailbox, or any RFC 3501 IMAP server. |
| 3 | +IMAP / SMTP / ManageSieve to JMAP gateway. Lets any JMAP client talk to |
| 4 | +Gmail, your hosting provider's mailbox, or any RFC 3501 IMAP server. |
5 | 5 |
|
6 | | -`legacy-proxy` speaks **JMAP for Mail** (RFC 8620 + RFC 8621) on the front |
7 | | -side and **IMAP4rev1 + Submission (SMTP) + ManageSieve** on the back side. |
8 | | -Drop it in front of a legacy mailserver and modern JMAP apps (Bulwark |
9 | | -Webmail, the official `jmap-test-suite`, etc.) treat it as a native JMAP |
10 | | -endpoint. |
| 6 | +Front: JMAP for Mail (RFC 8620 + RFC 8621). Back: IMAP4rev1, SMTP |
| 7 | +Submission, ManageSieve. |
11 | 8 |
|
12 | | -The full design lives in [`PLAN.md`](./PLAN.md). This README is a quickstart. |
13 | | - |
14 | | ---- |
| 9 | +Design notes in [PLAN.md](./PLAN.md). |
15 | 10 |
|
16 | 11 | ## Status |
17 | 12 |
|
18 | | -This is the v0.1 scaffold described by the plan. It includes: |
19 | | - |
20 | | -- ✅ JMAP `Session` resource with full capability table |
21 | | -- ✅ JMAP request envelope, batched `methodCalls`, back-references (`#ref`) |
22 | | -- ✅ `Mailbox/get`, `Email/get`, `Email/query`, `Identity/get`, |
23 | | - `VacationResponse/get|set` |
24 | | -- ✅ IMAP backend via `imapflow` with per-account connection pool |
25 | | -- ✅ ManageSieve client (RFC 5804) used by `VacationResponse` |
26 | | -- ✅ SMTP submission via `nodemailer` |
27 | | -- ✅ Auth (PLAIN / LOGIN / XOAUTH2), HMAC session tokens, AES-256-GCM |
28 | | - credential vault |
29 | | -- ✅ State store on `better-sqlite3` |
30 | | -- ✅ EventSource push transport |
31 | | -- ✅ Docker image, `docker-compose.yml`, integration `compose.test.yml` |
32 | | - with Stalwart |
33 | | -- ✅ Unit tests (vitest, 28 green) and a `jmap-test-suite` runner |
34 | | -- 🚧 `Email/set`, `Email/copy`, `Email/import`, `EmailSubmission/set`, |
35 | | - `Thread/get`, IMAP IDLE → SSE wiring, WebSocket transport - see |
36 | | - milestones M3–M6 in `PLAN.md` |
37 | | - |
38 | | ---- |
| 13 | +v0.1. Working: |
| 14 | + |
| 15 | +- JMAP `Session` with the full capability table |
| 16 | +- Batched `methodCalls`, back-references (`#ref`) |
| 17 | +- `Mailbox/get`, `Email/get`, `Email/query`, `Identity/get`, `VacationResponse/get|set` |
| 18 | +- IMAP via imapflow, per-account connection pool |
| 19 | +- ManageSieve (RFC 5804), used by `VacationResponse` |
| 20 | +- SMTP submission (nodemailer) |
| 21 | +- Auth: PLAIN, LOGIN, XOAUTH2. HMAC session tokens. AES-256-GCM vault |
| 22 | +- better-sqlite3 state store |
| 23 | +- EventSource push |
| 24 | +- Docker image + compose files (prod, integration) |
| 25 | +- 28 unit tests, jmap-test-suite runner |
| 26 | + |
| 27 | +Not done yet (M3-M6 in PLAN.md): `Email/set`, `Email/copy`, `Email/import`, |
| 28 | +`EmailSubmission/set`, `Thread/get`, IMAP IDLE -> SSE, WebSocket transport. |
39 | 29 |
|
40 | 30 | ## Quickstart |
41 | 31 |
|
42 | | -### Run the published image (no clone needed) |
| 32 | +You need Docker. Two ways to run it: |
43 | 33 |
|
44 | | -Requires Docker. Three files land in your working directory; the image is |
45 | | -pulled from `ghcr.io/bulwarkmail/legacy-proxy`. |
| 34 | +### Pull the published image |
46 | 35 |
|
47 | 36 | ```bash |
48 | 37 | mkdir legacy-proxy && cd legacy-proxy |
49 | 38 |
|
50 | | -# .env with two random 32-byte keys |
51 | 39 | cat > .env <<EOF |
52 | 40 | VAULT_KEY=$(openssl rand -base64 32) |
53 | 41 | SESSION_HMAC_KEY=$(openssl rand -base64 32) |
54 | 42 | EOF |
55 | 43 | chmod 600 .env |
56 | 44 |
|
57 | | -# config + compose |
58 | 45 | curl -fsSLo providers.json https://raw.githubusercontent.com/bulwarkmail/legacy-proxy/main/providers.example.json |
59 | 46 | curl -fsSLo compose.prod.yml https://raw.githubusercontent.com/bulwarkmail/legacy-proxy/main/compose.prod.yml |
60 | 47 |
|
61 | | -# edit providers.json so the `generic` entry points at your IMAP/SMTP host |
| 48 | +$EDITOR providers.json # point the `generic` entry at your IMAP/SMTP host |
| 49 | + |
62 | 50 | docker compose -f compose.prod.yml up -d |
63 | 51 | ``` |
64 | 52 |
|
65 | | -Verify with `curl http://localhost:8080/healthz` (should return `ok`). |
66 | | -Then point a JMAP client at `http://localhost:8080/.well-known/jmap`. |
| 53 | +`curl http://localhost:8080/healthz` returns `ok` if it's up. JMAP clients |
| 54 | +connect to `http://localhost:8080/.well-known/jmap`. |
67 | 55 |
|
68 | | -### Run from a clone (for development, or if you want to build locally) |
| 56 | +### Build from source |
69 | 57 |
|
70 | 58 | ```bash |
71 | 59 | git clone https://github.com/bulwarkmail/legacy-proxy.git |
72 | 60 | cd legacy-proxy |
73 | | -npm run setup # generates .env + providers.json |
74 | | -docker compose up -d # builds locally; for the published image use compose.prod.yml |
| 61 | +npm run setup |
| 62 | +docker compose up -d |
75 | 63 | ``` |
76 | 64 |
|
77 | | -`npm run setup` is idempotent and won't overwrite an existing `.env` unless |
78 | | -you pass `-- --force`. |
| 65 | +`npm run setup` writes `.env` and `providers.json`. It won't clobber |
| 66 | +existing files. Pass `-- --force` to overwrite. |
79 | 67 |
|
80 | | -### Logging in |
| 68 | +## Logging in |
81 | 69 |
|
82 | | -Once the proxy is up, exchange IMAP credentials for a JMAP session token: |
| 70 | +Trade IMAP credentials for a JMAP session token: |
83 | 71 |
|
84 | 72 | ```bash |
85 | 73 | curl -s http://localhost:8080/api/login \ |
86 | 74 | -H 'content-type: application/json' \ |
87 | | - -d '{"username":"you@example.com","password":"…","provider":"generic"}' |
| 75 | + -d '{"username":"you@example.com","password":"...","provider":"generic"}' |
88 | 76 | ``` |
89 | 77 |
|
90 | | -Use the returned `token` as `Authorization: Bearer …` on all JMAP requests. |
| 78 | +Use the returned `token` as `Authorization: Bearer ...` on JMAP requests. |
91 | 79 |
|
92 | | -### Connecting Gmail |
| 80 | +### Gmail |
93 | 81 |
|
94 | | -Gmail no longer accepts plain account passwords over IMAP. Use an |
95 | | -[App Password](https://support.google.com/accounts/answer/185833) (requires |
96 | | -2-Step Verification) and pass `"provider": "gmail"`: |
| 82 | +Gmail wants an [App Password](https://support.google.com/accounts/answer/185833) |
| 83 | +(2FA must be on). Use `"provider": "gmail"`. XOAUTH2 works too if you bring |
| 84 | +your own tokens. |
97 | 85 |
|
98 | | -```bash |
99 | | -curl -s http://localhost:8080/api/login \ |
100 | | - -H 'content-type: application/json' \ |
101 | | - -d '{"username":"you@gmail.com","password":"<16-char app password>","provider":"gmail"}' |
102 | | -``` |
| 86 | +### TLS and public hosts |
103 | 87 |
|
104 | | -XOAUTH2 is also supported if you bring your own OAuth tokens. |
105 | | - |
106 | | -### Exposing on a real host |
107 | | - |
108 | | -The proxy speaks plain HTTP - terminate TLS in front of it (Caddy, Traefik, |
109 | | -nginx) and set `PUBLIC_URL` in `.env` to the URL clients will see, e.g. |
110 | | -`PUBLIC_URL=https://jmap.example.com`. The Session resource hands out URLs |
111 | | -based on `PUBLIC_URL`, so JMAP clients break if it's wrong. |
112 | | - |
113 | | ---- |
| 88 | +The proxy serves plain HTTP. Put Caddy, Traefik, or nginx in front of it |
| 89 | +and set `PUBLIC_URL` to the URL clients see. The Session resource bakes |
| 90 | +URLs from `PUBLIC_URL`, so a wrong value breaks every JMAP client silently. |
114 | 91 |
|
115 | 92 | ## Configuration |
116 | 93 |
|
117 | | -| env var | default | meaning | |
118 | | -| ------------------ | ---------------------------------- | -------------------------------------- | |
119 | | -| `PORT` | `8080` | HTTP listen port | |
120 | | -| `PUBLIC_URL` | `http://localhost:$PORT` | URL clients see (used in Session URLs) | |
121 | | -| `DATA_DIR` | `/data` | SQLite + blob cache | |
122 | | -| `VAULT_KEY` | (required) | base64 32-byte AES-GCM key | |
123 | | -| `SESSION_HMAC_KEY` | (required) | base64 32-byte HMAC-SHA-256 key | |
124 | | -| `DEFAULT_PROVIDER` | `generic` | provider key when `/api/login` omits | |
125 | | -| `PROVIDERS_FILE` | `/etc/legacy-proxy/providers.json` | provider catalogue | |
126 | | -| `LOG_LEVEL` | `info` | pino level | |
127 | | - |
128 | | -`providers.example.json` ships ready-to-use templates for Gmail, |
129 | | -and a generic `$IMAP_HOST` / `$SMTP_HOST` / `$SIEVE_HOST` form. |
| 94 | +| env var | default | notes | |
| 95 | +| ------------------ | ---------------------------------- | ------------------------------------ | |
| 96 | +| `PORT` | `8080` | HTTP listen port | |
| 97 | +| `PUBLIC_URL` | `http://localhost:$PORT` | URL clients see; baked into Session | |
| 98 | +| `DATA_DIR` | `/data` | SQLite and blob cache | |
| 99 | +| `VAULT_KEY` | required | base64 32-byte AES-GCM key | |
| 100 | +| `SESSION_HMAC_KEY` | required | base64 32-byte HMAC-SHA-256 key | |
| 101 | +| `DEFAULT_PROVIDER` | `generic` | provider key when `/api/login` omits | |
| 102 | +| `PROVIDERS_FILE` | `/etc/legacy-proxy/providers.json` | provider catalogue | |
| 103 | +| `LOG_LEVEL` | `info` | pino level | |
130 | 104 |
|
131 | | ---- |
| 105 | +`providers.example.json` ships Gmail and a generic |
| 106 | +`$IMAP_HOST`/`$SMTP_HOST`/`$SIEVE_HOST` template. |
132 | 107 |
|
133 | 108 | ## Tests |
134 | 109 |
|
135 | 110 | ```bash |
136 | | -npm test # unit tests (vitest) |
137 | | -npm run test:integration # requires compose.test.yml up; gated by RUN_INTEGRATION=1 |
138 | | -npm run test:compliance # runs jmap-test-suite against a live proxy |
139 | | -npm run test:all # all three |
| 111 | +npm test # unit tests |
| 112 | +npm run test:integration # needs compose.test.yml; gated by RUN_INTEGRATION=1 |
| 113 | +npm run test:compliance # jmap-test-suite against a live proxy |
| 114 | +npm run test:all |
140 | 115 | ``` |
141 | 116 |
|
142 | | -The compliance script clones [`jmapio/jmap-test-suite`][1] into |
143 | | -`vendor/jmap-test-suite/` on first run and executes it against the URL in |
144 | | -`PROXY_URL`. Documented allow-listed failures live in |
145 | | -`test/compliance/known-failures.txt` - these are features the upstream |
146 | | -IMAP server cannot offer (e.g. `queryChanges` without `CONDSTORE`). |
147 | | - |
148 | | -[1]: https://github.com/jmapio/jmap-test-suite |
149 | | - |
150 | | ---- |
| 117 | +`test:compliance` clones [jmap-test-suite](https://github.com/jmapio/jmap-test-suite) |
| 118 | +into `vendor/jmap-test-suite/` and runs it against `PROXY_URL`. Allow-listed |
| 119 | +upstream failures live in `test/compliance/known-failures.txt`, mostly |
| 120 | +things the IMAP server can't offer (e.g. `queryChanges` without CONDSTORE). |
151 | 121 |
|
152 | 122 | ## Architecture |
153 | 123 |
|
154 | | -See [`PLAN.md`](./PLAN.md) §2 for the full picture. Module map: |
| 124 | +PLAN.md §2 has the long version. Layout: |
155 | 125 |
|
156 | 126 | ``` |
157 | 127 | src/ |
158 | | - server.ts fastify bootstrap |
159 | | - jmap/ session, router, capabilities, methods |
160 | | - imap/ imapflow pool, fetcher, search compiler |
161 | | - smtp/ nodemailer submission |
162 | | - sieve/ ManageSieve client + vacation |
163 | | - auth/ session tokens, credential vault, providers |
164 | | - mapping/ IMAP↔JMAP id codecs, body structure, flags |
165 | | - state/ SQLite store + opaque state strings |
166 | | - push/ eventsource + websocket |
| 128 | + server.ts fastify bootstrap |
| 129 | + jmap/ session, router, capabilities, methods |
| 130 | + imap/ imapflow pool, fetcher, search compiler |
| 131 | + smtp/ nodemailer submission |
| 132 | + sieve/ ManageSieve client, vacation |
| 133 | + auth/ session tokens, credential vault, providers |
| 134 | + mapping/ IMAP <-> JMAP id codecs, body structure, flags |
| 135 | + state/ SQLite store, opaque state strings |
| 136 | + push/ eventsource, websocket |
167 | 137 | ``` |
168 | 138 |
|
169 | | ---- |
170 | | - |
171 | 139 | ## License |
172 | 140 |
|
173 | 141 | AGPL-3.0 |
0 commit comments