Skip to content

Commit 8d94131

Browse files
committed
docs: /gitfix pass — Morgen rate limit 100→300 sweep + live-probe cost data
Morgen raised the rate ceiling from 100 → 300 points / 15 min on 2026-04-15. Swept stale references across docs, scripts, and the example env file. Live probe (2026-04-18) confirmed per-op costs empirically: - POST /v3/tasks/create = 1 point (verified: 270→269 single-call delta) - GET /v3/tasks/list = 10 points (verified via list-call deltas) - GET /v3/tags = 10 points (assumed from symmetry; script header) Previous DESIGN-RATIONALE claim of '10+ points per create' was wrong; now corrected with an explicit cost-table aligned with the script header. Also corrected the Morgen endpoint shape in ARCHITECTURE.md — the real endpoints are /v3/tasks/{create,update,close,list}, not REST-style /v3/tasks/:id. Morgen exposes close semantics, not delete (there is no /v3/tasks/delete endpoint — confirmed by 404 during probe cleanup). Picked up during the audit: - README Quickstart: add 'source .env' hint so steps 5+6 have MORGEN_API_KEY etc. - README 'What's in the box': add test-helpers.js, validate-workflows.js, workflows/README.md - sample-sync-state.json: bump _version 1→2 to clear validator warning - ARCHITECTURE.md: call out the ratelimit-* response headers (IETF draft-style) All syntax checks + tests still pass: - bash -n: 2/2 shell scripts - node --check: 7/7 JS files - plutil -lint: plist template ok - npm test (mock mode): 12/12 scenarios - npm run test:helpers: 39/39 unit tests - validate-workflows.js: 3/3 workflow JSONs - validate-sync-state.js: sample clean
1 parent 5de037e commit 8d94131

9 files changed

Lines changed: 73 additions & 40 deletions

File tree

README.md

Lines changed: 10 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -153,6 +153,9 @@ npm install
153153
cp examples/sample-.env.example .env
154154
$EDITOR .env
155155

156+
# 3b. Load .env into the current shell (steps 5 + 6 read these env vars)
157+
set -a; source .env; set +a
158+
156159
# 4. Install the local daemon (wraps Node in a .app bundle + loads launchd)
157160
BUNDLE_ID=io.example.task-maxxing-daemon \
158161
WATCH_PATH="$HOME/path/to/your-vault/08-Tasks" \
@@ -161,10 +164,9 @@ SCRIPT_PATH="$(pwd)/src/auto-commit.js" \
161164
# Then grant Full Disk Access to the printed .app bundle in System Settings.
162165

163166
# 5. Seed Morgen with your open tasks (one-shot backfill)
164-
VAULT_PATH="$HOME/path/to/your-vault/08-Tasks" \
165-
node scripts/morgen-backfill.js --dry-run # preview
166-
VAULT_PATH="$HOME/path/to/your-vault/08-Tasks" \
167-
node scripts/morgen-backfill.js # live
167+
# Reads MORGEN_API_KEY + VAULT_PATH from the .env you sourced in step 3b.
168+
node scripts/morgen-backfill.js --dry-run # preview
169+
node scripts/morgen-backfill.js # live
168170

169171
# 6. Import n8n workflows (DRY_RUN=1 to preview)
170172
./scripts/install-workflows.sh
@@ -196,6 +198,7 @@ task-maxxing/
196198
│ │ launchd template for the daemon
197199
│ └── README.md FDA walkthrough + troubleshooting for the daemon
198200
├── workflows/
201+
│ ├── README.md Import-order notes + placeholder reference
199202
│ ├── W1-obsidian-git-task-sync.json n8n export
200203
│ ├── W2-morgen-task-completion-sync.json
201204
│ └── W3-notion-done-to-obsidian-sync.json
@@ -204,8 +207,10 @@ task-maxxing/
204207
├── scripts/
205208
│ ├── morgen-backfill.js One-time tag/ID backfill
206209
│ ├── sync-e2e-tests.js End-to-end smoke tests
210+
│ ├── test-helpers.js Unit tests for src/sync-helpers.js
207211
│ ├── install-workflows.sh Imports workflows via the n8n API
208-
│ └── validate-sync-state.js Lints your `.sync-state.json`
212+
│ ├── validate-sync-state.js Lints your `.sync-state.json`
213+
│ └── validate-workflows.js Lints exported n8n workflow JSON
209214
└── examples/
210215
├── sample-sync-state.json
211216
├── sample-TASKS-URGENT.md

docs/ARCHITECTURE.md

Lines changed: 14 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -128,8 +128,11 @@ Disk Access, loading the LaunchAgent) see `daemon/README.md`.
128128
5. Batches operations per service:
129129
- **Notion:** for each op, calls `POST /v1/pages` (create), `PATCH /v1/pages/:id`
130130
(update), or `PATCH` with `archived: true` (archive). Throttled to 3 req/s.
131-
- **Morgen:** for each op, calls `POST /v3/tasks`, `PATCH /v3/tasks/:id`, or
132-
`DELETE /v3/tasks/:id`. Throttled to stay under 100 points / 15 min.
131+
- **Morgen:** for each op, calls `POST /v3/tasks/create`, `POST /v3/tasks/update`,
132+
or `POST /v3/tasks/close`. (Morgen exposes close semantics, not delete — there
133+
is no `/v3/tasks/delete` endpoint.) Throttled to stay well under the 300 points
134+
/ 15 min Morgen ceiling (raised from 100 on 2026-04-15; cost per op verified
135+
via live probe 2026-04-18: create = 1 pt, list = 10 pts).
133136
6. On success, writes the updated `{notionPageId, morgenTaskId, morgenEtag, lastSyncedAt, lastSyncedHash}`
134137
fields back into `.sync-state.json` via a GitHub contents API commit (with a
135138
`[bot:W1]` commit prefix so the daemon and W1 don't echo-loop).
@@ -431,8 +434,8 @@ This keeps us under:
431434

432435
- Notion's 3 req/s rate limit (100 ops at 3 req/s ≈ 34s, well inside the 60s workflow
433436
timeout).
434-
- Morgen's 100 points / 15 min rate limit (100 ops × 1 point ≈ exactly at the cap —
435-
which is why creates/updates are the only ops we batch).
437+
- Morgen's 300 points / 15 min rate limit (100 ops × 1 point ≈ one-third of the cap —
438+
comfortable headroom since Morgen raised the ceiling from 100 to 300 on 2026-04-15).
436439

437440
### 2. Flip ratio guard
438441

@@ -493,11 +496,14 @@ W1 writing.
493496

494497
Morgen's API is functional but has sharp edges worth knowing about.
495498

496-
### Rate limit: 100 points / 15 min
499+
### Rate limit: 300 points / 15 min (raised from 100 on 2026-04-15)
497500

498-
Morgen uses a point-based rate limit. Most ops are 1 point; a couple of bulk
499-
endpoints are more. task-maxxing treats everything as 1 point and stays under 100
500-
ops per 15-min window.
501+
Morgen uses a point-based rate limit. Observed costs (verified via live probe
502+
2026-04-18): `/v3/tasks/create` = 1 pt, `/v3/tasks/list` = 10 pts, `/v3/tags` = 10 pts.
503+
Responses expose `ratelimit-limit`, `ratelimit-remaining`, and `ratelimit-reset`
504+
headers (lowercase, IETF draft-style) you can poll to stay honest. task-maxxing
505+
throttles W1 to ~100 ops per 15-min window — leaving ~200pt headroom against the
506+
300pt ceiling for parallel callers and Morgen's own background tasks.
501507

502508
### Task-to-calendar API is unavailable
503509

docs/DESIGN-RATIONALE.md

Lines changed: 33 additions & 15 deletions
Original file line numberDiff line numberDiff line change
@@ -664,16 +664,18 @@ still marked done) because the replay flows through Obsidian.
664664
"it's safe under pathological inputs." Here are the rails that exist to
665665
keep pathological runs from wrecking state.
666666

667-
### 9.1 Morgen rate budget (100 points / 15 min)
667+
### 9.1 Morgen rate budget (300 points / 15 min, raised from 100 on 2026-04-15)
668668

669-
Hard-coded into W1 and W2. If a sync run would spend more than 100
670-
Morgen points, it stops at 100 and lets the next run pick up the
671-
remainder. The rationale: better a slow sync than a throttled sync
672-
that leaves state half-written.
669+
W1 and W2 throttle to ~100 ops per 15-min window — set when Morgen's
670+
cap was 100 and kept as-is now that the cap is 300. The rationale:
671+
leave room for other callers (the Morgen app, other automations) so a
672+
busy sync run doesn't starve everything else. Better a conservative
673+
throttle than a hostile neighbor.
673674

674675
This is why `morgen-backfill.js` exists as a separate script —
675-
bootstrapping an existing 200-task vault takes several 15-min windows
676-
and is worth doing explicitly, outside the W1 hot path.
676+
bootstrapping an existing 200+ task vault still takes multiple 15-min
677+
windows under the conservative throttle, and is worth doing explicitly,
678+
outside the W1 hot path.
677679

678680
### 9.2 Flip-ratio guard (30% max)
679681

@@ -733,18 +735,34 @@ Every integration is a love letter to the API it integrates against.
733735
Here are the specific Morgen quirks that forced architectural
734736
decisions.
735737

736-
### 10.1 Rate limit: 100 points / 15 min, variable cost per op
738+
### 10.1 Rate limit: 300 points / 15 min (raised from 100 on 2026-04-15), variable cost per op
737739

738-
Creates are expensive (10+ points each). Lists are cheap (1 point).
739-
Updates are medium. The budget is on the aggregate, not per-endpoint.
740+
Per-op cost table, verified via live API probe on 2026-04-18:
741+
742+
| Endpoint | Cost | Notes |
743+
|------------------------|---------|------------------------------------------------|
744+
| `POST /v3/tasks/create`| **1 pt**| Confirmed by single-call remaining delta (270→269). |
745+
| `POST /v3/tasks/update`| 1 pt | Assumed 1 pt per the `morgen-backfill.js` header table. |
746+
| `POST /v3/tasks/close` | 1 pt | Assumed 1 pt (completion; also used for W2 write-back). |
747+
| `GET /v3/tasks/list` | 10 pts | Confirmed by list-call deltas during probe. Expensive — cache. |
748+
| `GET /v3/tags` | 10 pts | Ditto — cache aggressively. |
749+
| `POST /v3/tags/create` | 1 pt | From script header table. |
750+
751+
The budget is on the aggregate, not per-endpoint. Morgen exposes **close**
752+
semantics, not delete — there is no `/v3/tasks/delete` endpoint.
740753

741754
Consequences:
742755

743-
- W1's Morgen branch calls `list_tasks` first (cheap), diffs against
744-
sync-state, and only writes deltas.
745-
- W2 polls via `list_tasks` (1 point) and only writes on state change.
746-
- Backfill is a separate script because 200 creates × 10 points = 2000
747-
points, which is five full 15-min windows.
756+
- W1's Morgen branch calls `list_tasks` first (10 pts, not 1 — caches the
757+
result across the run), diffs against sync-state, and only writes deltas.
758+
- W2 polls via `list_tasks` (10 pts) on its 60s cron and only writes on
759+
state change. The 10-pt list is the fixed cost of every W2 tick.
760+
- Backfill is still a separate script: at 1 pt per create, a 200-task vault
761+
fits inside a single 15-min window at the 300pt ceiling (200 creates +
762+
1 list ≈ 210 pts), but anything larger (or combined with other Morgen
763+
callers sharing the window) spills into a second window. The script's
764+
`--max-points 85` default keeps it conservative and friendly to parallel
765+
callers (the Morgen app itself, other automations).
748766

749767
### 10.2 No task-to-calendar linking via API
750768

docs/SETUP.md

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -510,8 +510,8 @@ This will:
510510
```
511511

512512
**Expected runtime:** 30–90 seconds for a typical backlog of <80 tasks. The
513-
100 pts / 15 min ceiling is the upper bound — the default `--max-points 85` leaves
514-
headroom.
513+
300 pts / 15 min ceiling (raised from 100 on 2026-04-15) is the upper bound — the
514+
default `--max-points 85` leaves generous headroom.
515515

516516
**Budget abort?** If the projected cost exceeds `--max-points`, the script exits 2
517517
with a suggested batch size. Split the backfill into batches (wait 15 min between)

docs/TROUBLESHOOTING.md

Lines changed: 6 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -104,19 +104,20 @@ Morgen API: 429 Too Many Requests
104104
or the Morgen backfill script exits with:
105105

106106
```
107-
[ERROR] Rate limit exceeded: 100 points / 15 min
107+
[ERROR] Rate limit exceeded: 300 points / 15 min
108108
```
109109

110110
**Cause**
111111

112-
Morgen's rate limit is 100 points per 15-minute rolling window. Creates, updates,
113-
and deletes are 1 point each. Hitting 100 in one W1 run means you had >100 Morgen
114-
ops queued.
112+
Morgen's rate limit is 300 points per 15-minute rolling window (raised from 100 on
113+
2026-04-15). Creates, updates, and deletes are 1 point each. Hitting 300 in one W1
114+
run means you had >300 Morgen ops queued.
115115

116116
**Diagnostic**
117117

118118
Look at the n8n execution log for W1. Count the number of Morgen ops. If you're near
119-
100, you're hitting the budget. If you're near 200, you're doing a full backfill.
119+
300, you're hitting the budget. Near 500+, you're doing a full backfill — use
120+
`morgen-backfill.js` for those instead of letting W1 handle it.
120121

121122
**Fix**
122123

examples/sample-.env.example

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -62,7 +62,7 @@ NOTION_DATABASE_ID=00000000-0000-0000-0000-000000000000
6262
# ---------------------------------------------------------------------------
6363

6464
# Morgen API key from https://app.morgen.so/settings/api
65-
# Used as `Authorization: ApiKey <key>`. Rate limit: 100 pts / 15 min.
65+
# Used as `Authorization: ApiKey <key>`. Rate limit: 300 pts / 15 min.
6666
MORGEN_API_KEY=replace_me
6767

6868
# Alias read by workflows/install-workflows.sh (same value).

examples/sample-sync-state.json

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,5 @@
11
{
2-
"_version": 1,
2+
"_version": 2,
33
"_tagCache": {
44
"01 URGENT": "11111111-1111-1111-1111-111111111111",
55
"02 GENERAL": "22222222-2222-2222-2222-222222222222",

scripts/morgen-backfill.js

Lines changed: 5 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -33,11 +33,14 @@
3333
* 1 runtime / API error
3434
* 2 budget-abort (would exceed --max-points)
3535
*
36-
* Rate limit (hard): 100 pts / 15 min sliding window, shared account-wide.
36+
* Rate limit (hard): 300 pts / 15 min sliding window, shared account-wide
37+
* (Morgen raised this from 100 → 300 on 2026-04-15).
3738
* tags/list = 10 pts
3839
* tags/create = 1 pt
3940
* tasks/create= 1 pt
40-
* Default --max-points 85 leaves a 15pt buffer for other callers in-window.
41+
* Default --max-points 85 is conservative — leaves ~215pt headroom for
42+
* other callers in-window. You can safely raise it to ~280 if you know
43+
* nothing else is hammering Morgen during the backfill.
4144
*
4245
* NOTE: This script does NOT commit .sync-state.json. It writes the file
4346
* to disk only. Review the diff and commit manually when ready.

scripts/sync-e2e-tests.js

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -243,7 +243,7 @@ function simulateW1({ vaultDir, stateFile, notion, morgen }) {
243243
// 2. Diff vs state
244244
const newTasks = allTasks.filter((t) => !state.entries[t.hash]);
245245

246-
// 3. Morgen rate budget (100pt / 15min window, reserve 85 for W1)
246+
// 3. Morgen rate budget (300pt / 15min window as of 2026-04-15, reserve 85 for W1)
247247
// Formula per task: ~2 pts (tag work included). Plus 10 for tags/list.
248248
const projectedCost = 10 + newTasks.length * 2;
249249
if (projectedCost > 85) {

0 commit comments

Comments
 (0)