Repository navigation
Feat/feature point cloud export - #106
Merged
Merged
Conversation
added 4 commits
September 18, 2026 15:55
Snapshot the loaded map's feature identifiers and positions at deserialization, before the session runs, then diff against the map captured at save: which points the map inherited, which were minted this session, how far inherited ones moved, and where dropped ones sat. Cluster each category and write a colored point cloud sidecar so the spatial pattern is inspectable off-device. Device runs establish that identifiers survive a save/load/save round trip intact, so inherited-vs-new is decidable per point, while ARKit's own culling of contradicted features is opportunistic and cannot carry a staleness signal. Inherited-point drift doubles as a relocalization quality meter. Gated on PerfDiag. Two O(n) passes — map load and save — nothing per frame.
The Scan4D bundle carried relocalization.worldmap, an NSKeyedArchiver blob of
ARKit's ARWorldMap. It decodes only on-device with ARKit loaded, so no server
or Python consumer could read a byte of it, and it dominated bundle size. Drop
it from the bundle; the app still keeps the map in the scan directory for its
own rescan and extend flows.
In its place, serialize that same map's sparse feature cloud at save time as
arkit_features.bin — a 128-byte ASCII header over packed little-endian
{id: u8, x/y/z: f4} records, which numpy reads in one line. ARKit's feature
identifiers survive a map's save/load/save cycle, so clouds from successive
scans of one location join on id rather than being re-matched geometrically,
which is what makes them usable for measuring change between epochs.
The sidecar is written unconditionally beside the map archive and promoted into
the scan directory by saveScan. When it is lost — a failed write, a failed
promotion, or a world-map export that produced no map — the name lands in
incomplete_artifacts, so absence with no entry means the scan predates the
format and nothing else.
The point cloud sidecar, drift-bucket colouring and connected-components clustering existed to answer whether ARKit reuses feature identifiers across a map's save/load/save cycle, and to locate where culled points had been. Device runs settled both: identifiers persist, and ARKit's culling of contradicted features is opportunistic rather than reliable, so a dropped point carries no staleness signal worth plotting. Keep the part that still earns its place — the baseline captured at map load and the one summary line, whose inherited-point drift doubles as a relocalization quality meter across rescan generations. Its format is unchanged, so logs stay comparable with the runs already collected.
The cloud is serialized from the live ARWorldMap at save time, so scans captured before that code existed have none, and no amount of reprocessing produced one — the postprocess pass re-derives colour and mesh artifacts but never touched the map. Their exports silently omitted the file. Nothing was actually lost: every scan persists that same map as arworldmap.map, and NSKeyedUnarchiver restores it without an ARSession, so the cloud is recoverable offline. Process now re-derives it for any scan that has a map and no cloud, which puts the whole existing library on the same footing as new captures for change-over-time comparison. The step is deliberately excluded from needsPostprocess. That gate blocks rescan, connect and upload on pending structural work, and a derived artifact appearing there would make every legacy scan in the library block those flows at once — the same reasoning that keeps the equirect masks out of it. Decoding runs inside an autoreleasepool over a memory-mapped archive so a 50 MB map and its decoded form are never both resident, and every failure path leaves the step pending for a later pass rather than failing the scan. A recovered cloud sits in the raw capture frame, exactly like one written at save: the world map is never re-based, even when registration has been baked into the mesh.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Instead of the serialized ARWorldMap we export the sparse feature point cloud (annotated with IDs of features) instead. What you can do with this downsteam...tbd, but at minimum you have a sparse point cloud, and perhaps more interesting is ability to make some inferences based on diffs over scan epochs: