Skip to content

Commit 75c5ec9

Browse files
fidgetcodingfidgetcoding
committed
[bot:fix] installer: ship W0 sync orchestrator + fix macOS mktemp + NOTION_DB_ID alias
The three sub-workflows W1/W2/W3 are complete but they race each other on the shared .sync-state.json file when each has its own schedule trigger. This adds the missing W0-Sync-Orchestrator (scheduleTrigger every 15 min → executeWorkflow W2 → W3 → W1, wait=true on each) so the state file mutates serially. The orchestrator is the only workflow the user activates. Changes: - workflows/W0-orchestrator-sync-sequencer.json (NEW) — scheduleTrigger → 3× executeWorkflow with placeholders {{W1_WORKFLOW_ID}}, {{W2_WORKFLOW_ID}}, {{W3_WORKFLOW_ID}}. availableInMCP + callerPolicy=workflowsFromSameOwner so the orchestrator can invoke the three sub-workflows. - scripts/install-workflows.sh — after W1/W2/W3 are created, capture their real n8n-assigned IDs, template them into W0, POST the orchestrator last. SKIP_ORCHESTRATOR=1 escape hatch for users running their own scheduler. Also: * Fixed macOS BSD mktemp incompatibility — the old XXXXXX.json template silently fails on darwin ("File exists"). Switched to `mktemp -t` + .json suffix concat, which works on both BSD and GNU mktemp. * Added {{NOTION_DB_ID}} alias (W1 template uses this shorter form while W2/W3 use {{NOTION_DATABASE_ID}}). Prevents unreplaced-placeholder abort partway through install. - README.md — workflow glossary now covers W0 + explains the race-condition reasoning, architecture diagram adds the orchestrator box, quickstart points at W0-only activation, added step 8 for wiring the n8n MCP to Claude Code (reuses N8N_API_KEY + N8N_BASE_URL from .env), "outside the sync" section picks up n8n MCP as a third MCP option. Verified end-to-end with DRY_RUN=1 against fake env vars — all four workflows render with zero unreplaced placeholders.
1 parent 7011079 commit 75c5ec9

3 files changed

Lines changed: 273 additions & 28 deletions

File tree

‎README.md‎

Lines changed: 44 additions & 16 deletions
Original file line numberDiff line numberDiff line change
@@ -79,7 +79,7 @@ The missing piece is **three-way sync with one canonical source**. `task-maxxing
7979

8080
## Architecture
8181

82-
Six directed edges, three workflows, one local daemon.
82+
Six directed edges, three sub-workflows, one orchestrator, one local daemon.
8383

8484
```
8585
┌───────────────────────┐
@@ -90,10 +90,10 @@ Six directed edges, three workflows, one local daemon.
9090
│
9191
┌────────────────────┼─────────────────────┐
9292
│ local daemon │ W1 (n8n) │
93-
│ watches files, │ git push → │
94-
│ git commit, │ Notion create/ │
95-
│ git push │ update + Morgen │
96-
│ │ create/update │
93+
│ watches files, │ git push webhook │
94+
│ git commit, │ ─or─ called by W0 │
95+
│ git push │ → Notion + Morgen │
96+
│ │ │
9797
▼ ▼ │
9898
┌──────────────┐ ┌──────────────┐ │
9999
│ GitHub │ │ Notion │◄──────────────┤
@@ -107,15 +107,24 @@ Six directed edges, three workflows, one local daemon.
107107
│ Tasks (inbox│ W2 (n8n)
108108
│ list only) │ closed → Obsidian
109109
└──────────────┘ (commit back)
110+
111+
┌──────────────────────────────────────────┐
112+
│ W0 — Sync Orchestrator (every 15 min) │
113+
│ executeWorkflow W2 → W3 → W1, wait=true │
114+
│ The ONLY workflow you activate. │
115+
└──────────────────────────────────────────┘
110116
```
111117

112118
### Workflow glossary
113119

114-
| Label | Direction | Trigger | What it does |
115-
|-------|------------------------------------|---------------------|----------------------------------------------------------------------------------|
116-
| **W1**| Obsidian → Notion + Morgen | GitHub push webhook | Parses changed `TASKS-*.md` files, creates / updates / archives rows in Notion, creates / updates / closes tasks in Morgen. |
117-
| **W2**| Morgen → Obsidian | 60s cron | Polls Morgen tasks. On a `closed` task, commits `- [x]` back to the source markdown file. |
118-
| **W3**| Notion → Obsidian | 60s cron | Polls Notion for rows where Status changed to Done or Due/Scheduled changed. Commits the change back to the source markdown file. |
120+
| Label | Direction | Trigger | What it does |
121+
|-------|------------------------------------|-----------------------------|----------------------------------------------------------------------------------|
122+
| **W0**| *meta* | Schedule (every 15 min) | **The orchestrator.** Runs W2 → W3 → W1 in sequence via `executeWorkflow` (wait=true). This is the *only* workflow you activate — it serializes the other three so they never race on `.sync-state.json`. |
123+
| **W1**| Obsidian → Notion + Morgen | GitHub push (+ called by W0) | Parses changed `TASKS-*.md` files, creates / updates / archives rows in Notion, creates / updates / closes tasks in Morgen. |
124+
| **W2**| Morgen → Obsidian | Called by W0 | Polls Morgen tasks. On a `closed` task, commits `- [x]` back to the source markdown file. |
125+
| **W3**| Notion → Obsidian | Called by W0 | Polls Notion for rows where Status changed to Done or Due/Scheduled changed. Commits the change back to the source markdown file. |
126+
127+
> **Why an orchestrator?** Three independent 15-min cron triggers race each other and can interleave commits — W1 mid-run clobbers a `.sync-state.json` update from W2, or vice versa. The orchestrator sequences `pull from Morgen → pull from Notion → push merged state to both` so the state file mutates serially. If you really want the un-sequenced version (e.g. you're self-hosting and have your own scheduling), pass `SKIP_ORCHESTRATOR=1` to the installer and leave W1/W2/W3's own triggers active.
119128
120129
### Daemon (local, macOS)
121130

@@ -170,11 +179,22 @@ SCRIPT_PATH="$(pwd)/src/auto-commit.js" \
170179
node scripts/morgen-backfill.js --dry-run # preview
171180
node scripts/morgen-backfill.js # live
172181

173-
# 6. Import n8n workflows (DRY_RUN=1 to preview)
182+
# 6. Import n8n workflows — W1/W2/W3 + the W0 orchestrator, with IDs
183+
# auto-templated into W0 after W1/W2/W3 are created.
184+
# (DRY_RUN=1 to preview, SKIP_ORCHESTRATOR=1 to import W1/W2/W3 only.)
174185
./scripts/install-workflows.sh
175186

176-
# 7. Activate W1, W3, then W2 in the n8n UI
177-
# 8. Smoke test — see docs/SETUP.md section 14
187+
# 7. Activate ONLY the W0-Sync-Orchestrator in the n8n UI.
188+
# Leave W1/W2/W3 inactive — W0 calls them directly, in sequence, every 15 min.
189+
190+
# 8. (Optional) Wire the n8n MCP to Claude Code so you can manage workflows
191+
# from the terminal. Your N8N_API_KEY + N8N_BASE_URL are already in .env.
192+
claude mcp add n8n-mcp \
193+
--env N8N_API_URL="${N8N_BASE_URL}/api/v1" \
194+
--env N8N_API_KEY="${N8N_API_KEY}" \
195+
-- npx -y n8n-mcp
196+
197+
# 9. Smoke test — see docs/SETUP.md section 14
178198
```
179199

180200
Full walkthrough with every click: **[docs/SETUP.md](docs/SETUP.md)**.
@@ -201,9 +221,10 @@ task-maxxing/
201221
│ └── README.md FDA walkthrough + troubleshooting for the daemon
202222
├── workflows/
203223
│ ├── README.md Import-order notes + placeholder reference
204-
│ ├── W1-obsidian-git-task-sync.json n8n export
205-
│ ├── W2-morgen-task-completion-sync.json
206-
│ └── W3-notion-done-to-obsidian-sync.json
224+
│ ├── W0-orchestrator-sync-sequencer.json Sequences W2 → W3 → W1 every 15 min
225+
│ ├── W1-obsidian-git-task-sync.json n8n export (called by W0)
226+
│ ├── W2-morgen-task-completion-sync.json n8n export (called by W0)
227+
│ └── W3-notion-done-to-obsidian-sync.json n8n export (called by W0)
207228
├── notion/
208229
│ └── tasks-db-schema.md Copy-pasteable database schema for Notion
209230
├── scripts/
@@ -255,6 +276,13 @@ None of these are required for the three-way sync — W1/W2/W3 talk to Notion an
255276
claude mcp add --transport http notion https://mcp.notion.com/mcp
256277
```
257278
Or see [developers.notion.com/docs/mcp](https://developers.notion.com/docs/mcp) for the local-stdio + OAuth variants.
279+
- **n8n MCP** — manage the sync workflows from Claude Code after they're installed. Step 8 of the [Quickstart](#quickstart) wires this up with the same `N8N_API_KEY` / `N8N_BASE_URL` the installer already uses:
280+
```bash
281+
claude mcp add n8n-mcp \
282+
--env N8N_API_URL="${N8N_BASE_URL}/api/v1" \
283+
--env N8N_API_KEY="${N8N_API_KEY}" \
284+
-- npx -y n8n-mcp
285+
```
258286
- **Obsidian** — the app itself. Download at [obsidian.md](https://obsidian.md). Pair it with the [Obsidian Tasks plugin](https://publish.obsidian.md/tasks/) (Clare Macrae) — that's the plugin whose syntax `task-maxxing` parses.
259287

260288
If you want all of this pre-wired alongside Claude Code, shell aliases, and a dozen other productivity MCPs, [`cli-maxxing`](https://github.com/lorecraft-io/cli-maxxing) is the one-shot installer.

‎scripts/install-workflows.sh‎

Lines changed: 98 additions & 12 deletions
Original file line numberDiff line numberDiff line change
@@ -1,7 +1,12 @@
11
#!/usr/bin/env bash
22
#
3-
# install-workflows.sh — import W1/W2/W3 into an n8n instance after
4-
# substituting placeholders with real tokens from env vars.
3+
# install-workflows.sh — import W1/W2/W3 + the W0 sync-orchestrator into an
4+
# n8n instance after substituting placeholders with real tokens from env vars.
5+
#
6+
# The orchestrator sequences W2 → W3 → W1 every 15 min via executeWorkflow
7+
# (wait=true) so the three sub-workflows never race each other on the shared
8+
# .sync-state.json file. After the script captures real W1/W2/W3 workflow IDs
9+
# from the n8n API, it templates them into W0 before posting it last.
510
#
611
# Required env vars:
712
# GITHUB_TOKEN OAuth/PAT used by W1/W3 (fine-grained PAT ok)
@@ -19,9 +24,11 @@
1924
# Optional env vars:
2025
# N8N_BASE_URL Your n8n instance base URL (no default — must be set)
2126
# DRY_RUN=1 Render substituted JSON to /tmp, do not POST
27+
# SKIP_ORCHESTRATOR=1 Only import W1/W2/W3 (advanced — you're wiring your
28+
# own scheduling and accept that W1/W2/W3 can race)
2229
#
23-
# After import, activate the workflows in the n8n UI in this order:
24-
# W1 → W3 → W2
30+
# After import, activate ONLY the W0-Sync-Orchestrator in the n8n UI.
31+
# Leave W1/W2/W3 inactive — the orchestrator calls them directly.
2532

2633
set -euo pipefail
2734

@@ -96,13 +103,15 @@ for pair in "${WORKFLOWS[@]}"; do
96103
exit 1
97104
fi
98105

99-
rendered="$(mktemp "/tmp/${label}.rendered.XXXXXX.json")"
106+
rendered="$(mktemp -t "${label}.rendered.XXXXXX").json"
107+
: > "${rendered}"
100108

101109
sed \
102110
-e "s|{{GITHUB_TOKEN}}|${GH_E}|g" \
103111
-e "s|{{NOTION_TOKEN}}|${NOTION_E}|g" \
104112
-e "s|{{MORGEN_KEY}}|${MORGEN_E}|g" \
105113
-e "s|{{NOTION_DATABASE_ID}}|${NOTION_DB_E}|g" \
114+
-e "s|{{NOTION_DB_ID}}|${NOTION_DB_E}|g" \
106115
-e "s|{{GITHUB_REPO}}|${GH_REPO_E}|g" \
107116
-e "s|{{GITHUB_OWNER}}|${GH_OWNER_E}|g" \
108117
-e "s|{{GITHUB_REPO_NAME}}|${GH_REPO_NAME_E}|g" \
@@ -114,7 +123,7 @@ for pair in "${WORKFLOWS[@]}"; do
114123
exit 1
115124
fi
116125

117-
clean="$(mktemp "/tmp/${label}.clean.XXXXXX.json")"
126+
clean="$(mktemp -t "${label}.clean.XXXXXX").json"
118127
jq '{name, nodes, connections, settings}' "${rendered}" > "${clean}"
119128

120129
if [[ "${DRY_RUN}" == "1" ]]; then
@@ -146,6 +155,79 @@ for pair in "${WORKFLOWS[@]}"; do
146155
rm -f "${rendered}" "${clean}"
147156
done
148157

158+
SKIP_ORCHESTRATOR="${SKIP_ORCHESTRATOR:-0}"
159+
160+
# ---------------------------------------------------------------------------
161+
# W0 — Sync Orchestrator
162+
# Sequences W2 → W3 → W1 every 15 min via executeWorkflow(wait=true).
163+
# Imported last because it needs the real W1/W2/W3 workflow IDs assigned by
164+
# n8n in the loop above (or from a DRY_RUN placeholder).
165+
# ---------------------------------------------------------------------------
166+
167+
if [[ "${SKIP_ORCHESTRATOR}" == "1" ]]; then
168+
echo "[install-workflows] SKIP_ORCHESTRATOR=1 — skipping W0 orchestrator import."
169+
else
170+
ORCH_SRC="${WF_DIR}/W0-orchestrator-sync-sequencer.json"
171+
if [[ ! -f "${ORCH_SRC}" ]]; then
172+
echo "[install-workflows] ERROR: missing ${ORCH_SRC}" >&2
173+
exit 1
174+
fi
175+
176+
W1_ID="${CREATED_IDS[0]}"
177+
W2_ID="${CREATED_IDS[1]}"
178+
W3_ID="${CREATED_IDS[2]}"
179+
180+
W1_ID_E="$(escape_sed "${W1_ID}")"
181+
W2_ID_E="$(escape_sed "${W2_ID}")"
182+
W3_ID_E="$(escape_sed "${W3_ID}")"
183+
184+
orch_rendered="$(mktemp -t W0.rendered.XXXXXX).json"
185+
: > "${orch_rendered}"
186+
sed \
187+
-e "s|{{W1_WORKFLOW_ID}}|${W1_ID_E}|g" \
188+
-e "s|{{W2_WORKFLOW_ID}}|${W2_ID_E}|g" \
189+
-e "s|{{W3_WORKFLOW_ID}}|${W3_ID_E}|g" \
190+
"${ORCH_SRC}" > "${orch_rendered}"
191+
192+
if grep -q '{{[A-Z_]*}}' "${orch_rendered}"; then
193+
echo "[install-workflows] ERROR: unreplaced placeholders in orchestrator:" >&2
194+
grep -oE '\{\{[A-Z_]+\}\}' "${orch_rendered}" | sort -u >&2
195+
exit 1
196+
fi
197+
198+
orch_clean="$(mktemp -t W0.clean.XXXXXX).json"
199+
jq '{name, nodes, connections, settings}' "${orch_rendered}" > "${orch_clean}"
200+
201+
if [[ "${DRY_RUN}" == "1" ]]; then
202+
echo "[install-workflows] DRY RUN — would POST W0 orchestrator from ${orch_clean}"
203+
CREATED_IDS+=("dry-run-W0")
204+
CREATED_NAMES+=("$(jq -r .name "${orch_clean}")")
205+
else
206+
echo "[install-workflows] POST W0 → ${N8N_BASE_URL}/api/v1/workflows"
207+
response="$(curl -sS -X POST "${N8N_BASE_URL}/api/v1/workflows" \
208+
-H "X-N8N-API-KEY: ${N8N_API_KEY}" \
209+
-H "Content-Type: application/json" \
210+
--data-binary @"${orch_clean}")"
211+
212+
wf_id="$(printf '%s' "${response}" | jq -r '.id // empty')"
213+
wf_name="$(printf '%s' "${response}" | jq -r '.name // empty')"
214+
215+
if [[ -z "${wf_id}" ]]; then
216+
echo "[install-workflows] ERROR: no id in response for W0:" >&2
217+
printf '%s\n' "${response}" >&2
218+
exit 1
219+
fi
220+
221+
echo "[install-workflows] W0 id=${wf_id} name='${wf_name}'"
222+
CREATED_IDS+=("${wf_id}")
223+
CREATED_NAMES+=("${wf_name}")
224+
225+
rm -f "${orch_rendered}" "${orch_clean}"
226+
fi
227+
228+
WORKFLOWS+=("W0:W0-orchestrator-sync-sequencer.json")
229+
fi
230+
149231
echo
150232
echo "====================================================================="
151233
echo " Workflows created:"
@@ -154,10 +236,14 @@ for i in "${!CREATED_IDS[@]}"; do
154236
printf " %-4s %s (%s)\n" "${label}" "${CREATED_IDS[$i]}" "${CREATED_NAMES[$i]}"
155237
done
156238
echo
157-
echo " NEXT: activate workflows in the n8n UI in this order:"
158-
echo " W1 → W3 → W2"
159-
echo
160-
echo " Why this order? W1 is the fast Obsidian→Notion/Morgen path, W3 is"
161-
echo " the reverse guard for Notion-originated edits, and W2 is the slow"
162-
echo " Morgen-completion sweeper that depends on both being live."
239+
if [[ "${SKIP_ORCHESTRATOR}" == "1" ]]; then
240+
echo " NEXT: SKIP_ORCHESTRATOR=1 — you asked for bare W1/W2/W3 with their own"
241+
echo " schedule triggers. Activate them in this order: W1 → W3 → W2"
242+
echo " (W1 is the fast push-based path, W3 guards Notion edits, W2 sweeps Morgen.)"
243+
else
244+
echo " NEXT: activate ONLY the W0-Sync-Orchestrator in the n8n UI."
245+
echo " Leave W1/W2/W3 inactive — the orchestrator triggers them directly"
246+
echo " via executeWorkflow so they never race each other on the shared"
247+
echo " .sync-state.json file."
248+
fi
163249
echo "====================================================================="

‎workflows/W0-orchestrator-sync-sequencer.json‎

Lines changed: 131 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,131 @@
1+
{
2+
"name": "W0-Sync-Orchestrator",
3+
"nodes": [
4+
{
5+
"id": "b7c9a0f1-1a23-4b56-9f8e-1a2b3c4d5e6f",
6+
"name": "Every 15 Minutes",
7+
"type": "n8n-nodes-base.scheduleTrigger",
8+
"typeVersion": 1.3,
9+
"position": [
10+
100,
11+
300
12+
],
13+
"parameters": {
14+
"rule": {
15+
"interval": [
16+
{
17+
"field": "minutes",
18+
"minutesInterval": 15
19+
}
20+
]
21+
}
22+
}
23+
},
24+
{
25+
"id": "c0d1e2f3-4a56-4b78-9c0d-1e2f3a4b5c6d",
26+
"name": "Run W2 (Morgen → Obsidian)",
27+
"type": "n8n-nodes-base.executeWorkflow",
28+
"typeVersion": 1.2,
29+
"position": [
30+
340,
31+
300
32+
],
33+
"parameters": {
34+
"source": "database",
35+
"workflowId": {
36+
"__rl": true,
37+
"mode": "id",
38+
"value": "{{W2_WORKFLOW_ID}}"
39+
},
40+
"mode": "each",
41+
"options": {
42+
"waitForSubWorkflow": true
43+
}
44+
}
45+
},
46+
{
47+
"id": "d1e2f3a4-5b67-4c89-9d0e-1f2a3b4c5d6e",
48+
"name": "Run W3 (Notion → Obsidian)",
49+
"type": "n8n-nodes-base.executeWorkflow",
50+
"typeVersion": 1.2,
51+
"position": [
52+
580,
53+
300
54+
],
55+
"parameters": {
56+
"source": "database",
57+
"workflowId": {
58+
"__rl": true,
59+
"mode": "id",
60+
"value": "{{W3_WORKFLOW_ID}}"
61+
},
62+
"mode": "each",
63+
"options": {
64+
"waitForSubWorkflow": true
65+
}
66+
}
67+
},
68+
{
69+
"id": "e2f3a4b5-6c78-4d90-9e0f-1a2b3c4d5e6f",
70+
"name": "Run W1 (Obsidian → Notion + Morgen)",
71+
"type": "n8n-nodes-base.executeWorkflow",
72+
"typeVersion": 1.2,
73+
"position": [
74+
820,
75+
300
76+
],
77+
"parameters": {
78+
"source": "database",
79+
"workflowId": {
80+
"__rl": true,
81+
"mode": "id",
82+
"value": "{{W1_WORKFLOW_ID}}"
83+
},
84+
"mode": "each",
85+
"options": {
86+
"waitForSubWorkflow": true
87+
}
88+
}
89+
}
90+
],
91+
"connections": {
92+
"Every 15 Minutes": {
93+
"main": [
94+
[
95+
{
96+
"node": "Run W2 (Morgen → Obsidian)",
97+
"type": "main",
98+
"index": 0
99+
}
100+
]
101+
]
102+
},
103+
"Run W2 (Morgen → Obsidian)": {
104+
"main": [
105+
[
106+
{
107+
"node": "Run W3 (Notion → Obsidian)",
108+
"type": "main",
109+
"index": 0
110+
}
111+
]
112+
]
113+
},
114+
"Run W3 (Notion → Obsidian)": {
115+
"main": [
116+
[
117+
{
118+
"node": "Run W1 (Obsidian → Notion + Morgen)",
119+
"type": "main",
120+
"index": 0
121+
}
122+
]
123+
]
124+
}
125+
},
126+
"settings": {
127+
"executionOrder": "v1",
128+
"availableInMCP": true,
129+
"callerPolicy": "workflowsFromSameOwner"
130+
}
131+
}

0 commit comments

Comments
 (0)