Skip to content

Commit 5e48a9a

Browse files
do-meclaude
andcommitted
chore(release): prep v0.0.8
- Bump apps/desktop/package.json to 0.0.8. - README links the latest release via /releases/latest/download/ instead of pinning to a specific tag — the badges age gracefully and the README no longer needs an edit on every bump. - CHANGELOG entry for v0.0.8 covers the 322 M scaling chapter: three compounding leaks plugged at filter time, auto-cleanup of DuckDB spill, u32 packed wire, spatial-sort tooltip, Metal watchdog chunking, and the V8 budget bump. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
1 parent 2d2f0b8 commit 5e48a9a

3 files changed

Lines changed: 105 additions & 18 deletions

File tree

CHANGELOG.md

Lines changed: 97 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -7,6 +7,95 @@ releases used the `app-vX.Y.Z` prefix). Other streams (Python package,
77
static web viewer) have their own changelogs / tag prefixes — see
88
[`docs/RELEASING.md`](docs/RELEASING.md).
99

10+
## v0.0.8 — 2026-04-29
11+
12+
**322 M-row scaling.** Verified end-to-end on the eubucco buildings
13+
parquet (322,562,870 points, 7.8 GiB on disk) on a 32 GiB Apple
14+
Silicon laptop. Full color-by + filter interactivity holds at this
15+
scale; memory pipeline hardened against three independent leaks
16+
that compounded to OOM the renderer at filter time on v0.0.7.
17+
18+
### Performance — 322 M points
19+
20+
| stage | result |
21+
| --------------------------------- | ----------------------------------- |
22+
| 322 M cold scatter | 2461 MB JS heap, 520 ms `toArray` |
23+
| 322 M with category column | 2769 MB JS heap, 1.8 s `toArray` |
24+
| Filter click → 158 M residential | 1205 MB JS heap, 338 ms `toArray` |
25+
| Tooltip pick at 322 M | ~50 ms (was ~10 s pre-spatial-sort) |
26+
| Cold load (parquet → first frame) | 12–15 s end-to-end |
27+
28+
### Fixed
29+
30+
- **Color-by + filter clicks no longer OOM at 322 M.** Three leaks
31+
combined to exhaust V8's ArrayBuffer pool: Mosaic's `QueryManager`
32+
LRU pinned every 2.5 GB Arrow Table for 3 hours, the renderer kept
33+
`lastXPacked` / `lastYPacked` alive past consumer null-out, and
34+
`queryResult` allocated the new `toArray()` copy in the same
35+
microtask that nulled the old buffers. Each survivable alone;
36+
together they OOM'd at 322 M. Now: cache disabled, renderer refs
37+
released on null-input, `queryResult` is async with a microtask
38+
yield + a gated forced GC.
39+
- **`categoryCount` no longer hardcoded to 1.** Previously poisoned
40+
the WGSL category uniform whenever `categoryColors.length > 1`,
41+
rendering all points as a single category.
42+
- **Auto-cleanup of DuckDB spill.** Sessions at this scale spill
43+
80–120 GB to `$TMPDIR/duckdb_gsa_*`; previously these leaked until
44+
macOS swept them on reboot. Now wiped on `atexit` and on
45+
SIGTERM/SIGINT, with stale orphans (>24 h) reclaimed at startup.
46+
- **`Response.arrayBuffer()` ~2 GB ceiling unblocked.** Custom
47+
`streamingRestConnector` reads the response body chunk-by-chunk,
48+
bypassing Chrome's fetch-pipeline cap; the wire size limit is now
49+
V8's, not the fetch pipeline's.
50+
- **u32 packed wire format.** u16 over a world-scale bbox quantised
51+
to ~5 km — visible coordinate grid at city zoom. u32 quantum is
52+
~8 mm; sub-pixel at any zoom level.
53+
- **Metal `MTLCommandBuffer` watchdog at 322 M.** Chunked
54+
`drawPoints` + WG-local atomic reduction in `viewport_cull` keep
55+
every command buffer under the macOS 5 s timeout.
56+
- **GPU buffer destroy is now device-lost-safe.** Wrapped in
57+
`device.queue.onSubmittedWorkDone()` to defer past in-flight
58+
command buffers; rapid pan-release no longer poisons the
59+
`MTLDevice`.
60+
- **Tooltip latency at 322 M: ~10 s → ~50 ms.** Background-
61+
materialise CTAS now appends `ORDER BY` on the precomputed u32
62+
quantised columns; row-group min/max stats become tight enough to
63+
prune >99 % of groups for any small-radius tooltip.
64+
- **Frontend-mode side panel unblocks on plain URLs** — no
65+
`?perf=1` required.
66+
67+
### Changed
68+
69+
- **Electron V8 budget: 12 GB → 16 GB**, with `--expose-gc` so the
70+
renderer can force a major sweep between releasing the prior
71+
batch's typed arrays and allocating the next batch's. Pairs with
72+
the sidecar `memory_limit` clamp (50 % RAM, max 64 GB) to keep
73+
V8 and DuckDB from contesting RAM.
74+
- **Mosaic LRU query cache disabled** in both `Viewer.svelte` and
75+
`FileViewer.svelte`. Histogram / count re-fetches on filter
76+
change cost <50 ms; pinning a 2.5 GB scatter Table for 3 h costs
77+
the entire ArrayBuffer pool.
78+
- **Render every point at world view** — no automatic cap, no
79+
sampling. The watchdog is per-buffer, not per-frame; chunking
80+
splits the work, never drops it.
81+
- **DuckDB `con.sql(...).fetch_arrow_table()`** replaces
82+
`cursor.execute(...)` for any large arrow export. ~10× faster on
83+
multi-GB results (the relation API streams; cursor materialises a
84+
full result-set buffer first).
85+
86+
### Added
87+
88+
- **[`docs/optimization_history.md`](docs/optimization_history.md)**
89+
field journal of the 322 M scaling chapter: what worked, what did
90+
not, common bananaskins, current bottleneck (synchronous
91+
`Arrow.Vector.toArray()` on the category column at ~1.8 s), and
92+
prioritised open follow-ups.
93+
- **README** — replaces hardcoded version-pinned download URLs with
94+
`/releases/latest/download/`, adds a Performance table per distro,
95+
Troubleshooting (V8 OOM workaround for vanilla Chrome, Metal
96+
watchdog freeze, stale `viewer-state.json`, leftover spill), and a
97+
Roadmap section organised by layer.
98+
1099
## v0.0.7 — 2026-04-25
11100

12101
**300 M-row scaling.** The loader, wire transport, and WebGPU renderer
@@ -16,13 +105,13 @@ on a 300 M-row synthetic Europe parquet (6.6 GiB on disk).
16105

17106
### Performance — 300 M points
18107

19-
| stage | result |
20-
| ------------------------------- | ----------------------------------- |
21-
| `fast_load_parquet` (cold) | 2.4 s |
22-
| Initial scatter (Arrow IPC) | 9.4 s for 1.20 GB on the wire |
23-
| Color-by category fetch | 11.9 s for 1.50 GB on the wire |
24-
| 10 s scripted pan inside window | 332 CSS-pans : 3 GPU re-renders |
25-
| Zoom re-render | 1.0 – 1.5 s |
108+
| stage | result |
109+
| ------------------------------- | ------------------------------- |
110+
| `fast_load_parquet` (cold) | 2.4 s |
111+
| Initial scatter (Arrow IPC) | 9.4 s for 1.20 GB on the wire |
112+
| Color-by category fetch | 11.9 s for 1.50 GB on the wire |
113+
| 10 s scripted pan inside window | 332 CSS-pans : 3 GPU re-renders |
114+
| Zoom re-render | 1.0 – 1.5 s |
26115

27116
### Changed
28117

@@ -191,7 +280,7 @@ upstream work:
191280
- **Unsigned build.** First launch:
192281
- macOS: right-click → Open, or System Settings → Privacy & Security
193282
→ "Open Anyway".
194-
- Windows: SmartScreen warning → *More info → Run anyway*.
283+
- Windows: SmartScreen warning → _More info → Run anyway_.
195284
- Linux: no prompt; `chmod +x` the `.AppImage` before launching.
196285
- **Only Apple Silicon has been smoke-tested.** The x64 / Linux /
197286
Windows builds are produced by CI but have not been manually verified.

README.md

Lines changed: 7 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -12,27 +12,25 @@ You can load the data from a remote URL too! Clicking this link, you download 10
1212

1313
[LinkedIn Post for more context](https://www.linkedin.com/posts/dominik-weckm%C3%BCller_geospatial-atlas-is-born-explore-100m-points-activity-7411826555179429890-CiHX)
1414

15-
## Desktop App Download (v0.0.7)
15+
## Desktop App Download
1616

17-
One-click download of the latest desktop app. All bundles are
18-
**unsigned**, so every platform needs a one-off bypass step after
19-
install — shown per-row below.
17+
All links below resolve to the [latest tagged release](https://github.com/do-me/geospatial-atlas/releases/latest) — no need to update them on every bump. See the [CHANGELOG](CHANGELOG.md) for what's in each version. All bundles are **unsigned**, so every platform needs a one-off bypass step after install — shown per-row below.
2018

21-
- **Windows (MSI)**[geospatial-atlas-windows-x64.msi](https://github.com/do-me/geospatial-atlas/releases/download/v0.0.7/geospatial-atlas-windows-x64.msi)
19+
- **Windows (MSI)**[geospatial-atlas-windows-x64.msi](https://github.com/do-me/geospatial-atlas/releases/latest/download/geospatial-atlas-windows-x64.msi)
2220
Double-click; on the SmartScreen warning click **More info → Run anyway**.
23-
- **Windows (NSIS setup)**[geospatial-atlas-windows-x64-setup.exe](https://github.com/do-me/geospatial-atlas/releases/download/v0.0.7/geospatial-atlas-windows-x64-setup.exe)
21+
- **Windows (NSIS setup)**[geospatial-atlas-windows-x64-setup.exe](https://github.com/do-me/geospatial-atlas/releases/latest/download/geospatial-atlas-windows-x64-setup.exe)
2422
Double-click; same **More info → Run anyway** SmartScreen step.
25-
-**macOS (Apple Silicon)**[geospatial-atlas-macos-arm64.dmg](https://github.com/do-me/geospatial-atlas/releases/download/v0.0.7/geospatial-atlas-macos-arm64.dmg)
23+
-**macOS (Apple Silicon)**[geospatial-atlas-macos-arm64.dmg](https://github.com/do-me/geospatial-atlas/releases/latest/download/geospatial-atlas-macos-arm64.dmg)
2624
Open the dmg, drag to `/Applications`, then strip the Gatekeeper
2725
quarantine flag once:
2826
```bash
2927
xattr -cr "/Applications/Geospatial Atlas.app"
3028
```
31-
- 🐧 **Linux (Debian / Ubuntu)**[geospatial-atlas-linux-x64.deb](https://github.com/do-me/geospatial-atlas/releases/download/v0.0.7/geospatial-atlas-linux-x64.deb)
29+
- 🐧 **Linux (Debian / Ubuntu)**[geospatial-atlas-linux-x64.deb](https://github.com/do-me/geospatial-atlas/releases/latest/download/geospatial-atlas-linux-x64.deb)
3230
```bash
3331
sudo dpkg -i geospatial-atlas-linux-x64.deb
3432
```
35-
- 🐧 **Linux (Fedora / RHEL)**[geospatial-atlas-linux-x64.rpm](https://github.com/do-me/geospatial-atlas/releases/download/v0.0.7/geospatial-atlas-linux-x64.rpm)
33+
- 🐧 **Linux (Fedora / RHEL)**[geospatial-atlas-linux-x64.rpm](https://github.com/do-me/geospatial-atlas/releases/latest/download/geospatial-atlas-linux-x64.rpm)
3634
```bash
3735
sudo rpm -i geospatial-atlas-linux-x64.rpm
3836
```

apps/desktop/package.json

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,7 +1,7 @@
11
{
22
"name": "geospatial-atlas-desktop",
33
"private": true,
4-
"version": "0.0.7",
4+
"version": "0.0.8",
55
"description": "Desktop viewer for geospatial & embedding data — Electron shell wrapping the Geospatial Atlas viewer + Python sidecar.",
66
"homepage": "https://github.com/do-me/geospatial-atlas",
77
"author": {

0 commit comments

Comments
 (0)