Skip to content

fix: harden cross-platform test infrastructure and runtime paths - #1003

Open
skobe79 wants to merge 194 commits into
craft-ai-agents:mainfrom
skobe79:fix/test-infra-and-prod
Open

fix: harden cross-platform test infrastructure and runtime paths#1003
skobe79 wants to merge 194 commits into
craft-ai-agents:mainfrom
skobe79:fix/test-infra-and-prod

Conversation

@skobe79

@skobe79 skobe79 commented Aug 3, 2026

Copy link
Copy Markdown

Summary

This updates the existing fix/test-infra-and-prod branch PR with the cross-platform test-infrastructure hardening from this work:

  • Keep the exact Bun test command and isolated runner enforced by CI on Ubuntu, Windows, and macOS.
  • Route child-process environments through the shared sanitizer so undefined values and the literal "undefined" sentinel cannot leak into spawned processes.
  • Add the reusable cross-platform smoke runner and artifact-based OS divergence reporting.
  • Make missing smoke reports fail closed inside --diff --expected-os, so a partial artifact set cannot print a misleading green “no divergence” result.
  • Source the test matrix and expected report set from one os-matrix workflow output to prevent OS-list drift.
  • Preserve renderer regression, motion, and accessibility coverage already accumulated on this branch.

This PR is based on the existing branch and therefore also contains the earlier product and renderer work already present in its history; the bullets above identify the test-infrastructure/runtime hardening added and verified in this thread.

Validation

  • bun test scripts/__tests__/cross-platform-smoke.test.ts — 4 passed, 0 failed.
  • bun scripts/cross-platform-smoke.mjs — 20/20 probes passed on Windows.
  • validate.yml parsed successfully and its fromJSON(needs.os-matrix.outputs.os_list) / --expected-os wiring was checked.
  • Commit hooks passed: test discovery, build-leak, icon fingerprint, renderer Node-import, and IPC inventory checks.

CI behavior

  • test-suite runs the full suite on ubuntu-latest, windows-latest, and macos-latest.
  • cross-platform-divergence compares the uploaded reports and fails with named missing OSes or probe divergences.
  • The heavy validate job remains PR-only.

Skobez and others added 30 commits July 24, 2026 04:39
- Restore omitted tsconfig.base.json from historical commit 0e84b1c
- Fix cross-platform document-tool test runner (python3 unavailable on Windows)
- Sanitize PYTHONPATH/PYTHONHOME/VIRTUAL_ENV case-insensitively in
  production script runtime and document-tool harness
- Restore and harden i18n coverage checker with fail-closed traversal
- Add CRLF-aware locale sort comparison to avoid line-ending-only rewrites
- Add regression tests for all repaired behaviors

Reviewed-by: independent-subagent (deleg_f9c7ba88, deleg_7511037f)
Closes: inherited validation blockers from v0.11.2 baseline
…e prompts

- Add 9-panel LayoutShell component with sidebar navigation, theme toggle, breadcrumbs, topbar slot
- Wire LayoutShell into renderer entrypoint as the new app shell
- Fix memory/database.ts to use db.run() instead of db.pragma()
- Fix memory/repository.ts type exports, duplicate property, and optional previous diff
- Add memory integration tests covering schema bootstrap, CRUD, archive isolation, audit
- Exclude __tests__ from packages/shared tsconfig to avoid test-runner type noise
- Update system prompt personas/memory references
- Add MemoryPanel with list/graph views, search, filters, detail pane
- Update LayoutShell to render MemoryPanel on memory view
- Use AnyMemory types from shared package
- Add HomeHero matching ARCHstudio dark + lime/purple accent style
- Add CommandPanel with run/stop, history, output, and terminal styling
- Wire HomeHero onto the command shell view in LayoutShell
…tion

- Add MemoryGraph component with interactive nodes/edges
- Wire graph view into MemoryPanel with selection sync
- Make MemoryPanel selection state self-contained
…t sync

- Add memory:* RPC channels (list, get, create, update, archive, restore, delete, search)
- Build MemoryRepository handler in main process
- Add channel map entries and ElectronAPI types for memory methods
- Convert MemoryPanel.tsx from mock to live data via window.electronAPI.listMemories()
- Add database-compat.ts: bun:sqlite on Bun, node:sqlite on Node/Electron
- Add obsidian-sync.ts: writes every memory change to D:\OwnerAgent\vault as .md
- Mark bun:sqlite as external in esbuild so main bundle resolves at runtime
- Add playground mocks for memory API calls
- Add bun:sqlite to MAIN_BUNDLE_EXTERNALS in electron-dev.ts
- Add --external:bun:sqlite to electron-build-main.ts
- Add --external:bun:sqlite to package.json build:main/build:main:win
- Clean up database-compat.ts eval require pattern
- Add validate-assets.ts script for build verification
- Disable code signing in electron-builder.yml (local builds)
- Mark bun:sqlite as external in all esbuild configs
…ommand/state/mode/media/z-scale, elevation, motion, typography, derived from existing theme vars
…on); fix red ipc-channels manifest (archCommand + memory channels)
…1.3)

- Extend NavigationState union (shared/types.ts) with Runs/Memory/MediaLab states + type guards, matching the existing Sources/Skills/Automations/Projects pattern
- Bridge route strings to the new states in route-parser.ts (convertCompoundToNavigationState)
- Add sidebar entries (Activity/Brain/Clapperboard icons) with active-state highlighting, wired through routes.view.runs/memory/mediaLab()
- MainContentPanel now renders the real panels/runs, panels/memory (SQLite-backed), panels/media-lab components for these navigators -- these were previously only reachable from the isolated playground.html harness and invisible in the actual app

Note: working tree contains unrelated in-progress rebrand/icon changes (not from this commit) - left untouched and unstaged.
…eal-time health polling to ProvidersPanel

IntegrationsPanel rewritten with tabbed Sources/Messaging layout:
- Messaging tab shows platform status cards for Telegram, WhatsApp, Lark
- Each card displays connection state with animated status dot, identity,
  error info, and connect/reconnect/forget actions
- Quick-add wizard modal with platform-specific forms: Telegram token
  input + test/save, WhatsApp phone entry + QR/pairing-code display,
  Lark app credentials + test/save
- Real-time status via getMessagingConfig() + onMessagingPlatformStatus()
- Sources tab preserves original source-integration grid unchanged

ProvidersPanel gains real-time health polling:
- 5-second setInterval fetches providers silently (no spinner flash)
- Live toggle button with pulsing green dot — click to pause/resume
- Existing onLlmConnectionsChanged event listener kept as instant-update
  supplement (belt + suspenders)

🤖 Generated with Codebuff
Co-Authored-By: Codebuff <noreply@codebuff.com>
…arkline, and histogram

- Add LlmInferenceStore with per-slug rolling window (turn + tool_call events)
- Record turn completions and tool-call outcomes in both ClaudeAgent and PiAgent
- Emit wall-clock latencyMs with every turn completion for the ProvidersPanel sparkline
- Persist rolling average latency to localStorage across sessions
- Auto-tune slow-threshold per provider from historical averages (avg * 1.5, floor 500ms)
- Add latency histogram (3-bucket <500ms / 500-2000ms / >2000ms) to each provider card
- Replace hardcoded 2000ms threshold with calcSlowThreshold() at all 4 display sites

🤖 Generated with Codebuff
Co-Authored-By: Codebuff <noreply@codebuff.com>
Skobez and others added 29 commits August 4, 2026 06:46
New ESLint rule flagging raw {isOpen &&}/{expanded &&} conditional
renders that mount with no transition, with 24 tests and wiring in
both apps/electron and packages/ui lint configs.
Add motion-safe:active:scale-[0.97] so top bar buttons compress
slightly on press, matching the shared Button base class.
Use useReducedMotion() to swap the spring for a zero-duration transition
when reduced motion is preferred, so the collapsible mounts/unmounts as
an instant state change while keeping AnimatePresence behavior.
Add a happy-dom component test for the webhook detail box: hidden when
collapsed, renders method/url/status/duration on expand, hides again on
collapse — closing the runtime coverage gap for the AnimatedCollapsibleContent
adoption.
…mocks

Remove dead Tailwind animation classes as a no-teleporting-state exemption and
complete the worker-wide motion/react mock so renderer tests remain isolated.

Generated with Codebuff 🤖
Co-Authored-By: Codebuff <noreply@codebuff.com>
buildWebhookEnv only filtered `value !== undefined`, so the literal
"undefined" sentinel (Windows WSL leak class) passed straight into
webhook URLs/headers. Route ARCHSTUDIO_WH_* through the shared
sanitizeChildProcessEnv so the leak class is stripped here too, and add
a regression test covering the webhook env builder.

🤖 Generated with Codebuff
Co-Authored-By: Codebuff <noreply@codebuff.com>
…motion kill

Wire tailwindcss-animate (the shadcn v4 provider of animate-in /
fade-in-0 / zoom-in-95 / slide-in-from-* utilities) into the renderer
and shared UI CSS entries, and drop the blanket [data-radix-*]
animation:none block that previously suppressed every overlay entrance.

Add a prefers-reduced-motion kill rule that matches the real
variant-prefixed class tokens (data-[state=open]:animate-in) via
substring selectors — bare `.animate-in` never matched them, so the
entrances were playing for reduced-motion users too.

🤖 Generated with Codebuff
Co-Authored-By: Codebuff <noreply@codebuff.com>
…rage

Swap the happy-dom test environment for jsdom so focus tracking and
accessibility states match a real browser, and wire vitest-axe's
toHaveNoViolations onto bun:test via a preload so violations are caught
at test time. The new axe-a11y test surfaced a genuine defect: the
workspace tabs claimed role="tab" without a tablist parent, so they are
now plain toggle buttons conveying state via .is-active.

🤖 Generated with Codebuff
Co-Authored-By: Codebuff <noreply@codebuff.com>
Centralize the hostile-env filter (undefined / literal "undefined" /
empty-string values) in sanitizeChildProcessEnv and apply it at every
spawn site — CLI server spawner, electron main spawns and shell-env,
MCP stdio transports, agent subprocess builders, automation hooks, and
the dev/build scripts — so the Windows WSL literal-"undefined" leak can
no longer reach a subprocess through any path.

Add an ESLint rule (env-sanitizer-audit) that flags any Bun.spawn /
child_process call whose env option spreads process.env without passing
through the sanitizer, so future spawn sites get review feedback at
lint time instead of leaking silently.

🤖 Generated with Codebuff
Co-Authored-By: Codebuff <noreply@codebuff.com>
Compose the aj-geddes cross-platform-compatibility Quick Start fixture
with the existing 3-OS test-suite matrix: each OS runs the probe suite
(real repo path/env utilities plus node:path invariants) and uploads a
JSON report; a follow-up job diffs the three reports so a probe that
passes on linux but fails on windows surfaces as named workflow noise
instead of a postmortem.

Also expose bun run cross-platform-smoke and cross-platform-smoke:diff
and cover the runner with a unit test.

🤖 Generated with Codebuff
Co-Authored-By: Codebuff <noreply@codebuff.com>
SimpleDropdown and ZoomControls carried the animate-in/fade-in-0/
zoom-in-95 utilities that compile to zero CSS in this build (the
tailwindcss-animate plugin was never enabled for these), so their
dropdown/menu panels appeared instantly with no transition. Adopt the
same motion/react pattern used in ProfileMenu and SearchPanel:
AnimatePresence + motion.div with initial/animate/exit opacity+scale,
100ms easeOut, and useReducedMotion collapsing to an instant swap.

🤖 Generated with Codebuff
Co-Authored-By: Codebuff <noreply@codebuff.com>
Adds the manually-installed electron-best-practices skill (from
jwynia/agent-skills, installed by git-clone into .agents/skills/) to
the tracked skill tree, matching the existing 16-skill layout.

skills-lock.json is intentionally untouched: as documented in the prior
skills install commit, its computedHash scheme is not reproducible from
skill contents and nothing in this repo reads the lock file.

🤖 Generated with Codebuff
Co-Authored-By: Codebuff <noreply@codebuff.com>
buildDarwinSandboxProfile is only ever called under process.platform ===
'darwin' (see applyFilesystemIsolation), where resolve() never emits a
backslash. The normalization added for a Windows-run unit test could
therefore only ever corrupt a backslash that is a legitimate part of a
macOS filename: a session dir /Users/x/a\b/sessions/s became
/Users/x/a/b/sessions/s, so the sandboxed command was denied writes to
its own session directory while possibly being granted them to an
unrelated subtree. It also made escapeSandboxPath's backslash branch
dead code. Reverted, and the test now derives its expectation from the
same resolve() call instead of hardcoding a POSIX separator, so it
stays meaningful on both hosts.

Same class of bug in llm-tool's attachment basename: splitting on both
separators mangles legal POSIX filenames containing a backslash, so the
<file name="..."> tag and every error message reported a name that does
not exist on disk. path.basename is already platform-correct and was
imported here anyway.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Drill mode is documented as ephemeral — in-memory only, reset on session
switch. But the gate-forward branch called saveExpandedPaths(next) with
the drilled path included, so the expansion it produced was written to
localStorage even though the marker was not. On the next launch the
[sessionId] effect restored that past-cap path into expandedPaths and
reset drilledPaths to empty, leaving a tree expanded one level past the
cap with no data-drilled marker, no gate banner, and a depth selector
still reading N.

Filter drilled paths out inside saveExpandedPaths rather than at the one
call site, so every writer is covered — including the count-cap trim.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Four tests had been edited into passing without still proving anything.

session-validation: the join()-safety case asserted
expect(result).toBe(join(root, 'sessions', sanitized)) where `result`
was that same expression one line above — x === x. A regression letting
sanitizeSessionId return '../tmp' would pass it, and the companion
startsWith(rootNorm) check too. Pins the literal 'tmp' again.

count-cap-bfs: the fixture was flattened to 60 dirs at one depth, making
trimExpandedByCount's `a.depth - b.depth` sort key constant while the
title still claimed "leaf-most-dropped". Verified by mutation: inverting
the comparator to `b.depth - a.depth` leaves all 5 tests green. The flat
fixture is kept (the old two-level one could not work — the depth-cap
invariant evicts depth >= expandDepth before the count cap runs) and its
comments now say what it actually covers. The depth term gets a new
integration test with a two-level fixture at expandDepth=3, which does
go red under the inverted comparator.

permissions-config-migration: the stderr diagnostic sat after the
expect() that throws on the very failure it explains.

check-test-discovery: bun treats the roots in the `test` script as
substring filters and exits 0 when only *some* miss — verified on 1.3.10
that `bun test apps/cli/src zzz-nope` is silently green. Renaming a
directory would drop its tests from the suite with no signal. The gate
now parses those roots from package.json and fails if any of the 443
first-party test files is unmatched.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
- resolve npx via Bun.which() to avoid Windows ENOENT race
- make PATH env assertion case-insensitive for Windows
- bump factory.test timeout to 20s
End-to-end media:comfy-stop RPC channel: protocol channel, routing
allowlist, transport map, ElectronAPI type, server handler
(Windows-only netstat+taskkill tree-kill, 30s health poll),
UI Stop engine button with stopping state, mock + smoke test,
and regenerated IPC channel snapshot.
Agent + local Python tools architecture:
- Demucs stem splitter (tools/audio/stem_splitter.py) — 4-stem
  separation (vocals/drums/bass/other) with GPU support
- Audio processor (tools/audio/audio_processor.py) — ffmpeg wrapper for
  stretch, transpose, trim, normalize, mix, and beat rendering
- Server handler (audio.ts) spawns Python tools as child processes,
  tracks jobs via in-memory map with progress polling
- 4 new RPC channels: media:stem-split, media:beat-render,
  media:audio-process, media:audio-job-status
- StemSplitterPanel — file picker, split button, per-stem playback
- BeatMakerPanel — 16-step Web Audio sequencer with synthesized
  kick/snare/hihat/clap/bass, BPM, volume per track, WAV render
- RemixTimelinePanel — multitrack loader with per-track gain/pan,
  play-all, mix-and-export via server
- New tabs in Media Lab: Stems, Beats, Remix alongside Create/Library
- Full protocol wiring: channels, routing, DTOs, channel-map, types, mock
- IPC snapshot regenerated (393 channels)
…rubbing

Agent session tools (4 new, registry mode):
- split_stems: Demucs stem separation via Python tool
- render_beat: beat pattern to WAV via Python tool
- mix_tracks: ffmpeg multi-track mixing
- audio_process: stretch/transpose/trim/normalize single-file ops
All spawn Python tools directly (like script_sandbox), return file paths.

UI enhancements:
- WaveformDisplay component (canvas, AudioContext decode, progress bar)
- StemSplitter: per-stem waveform with color-coded playback progress
- BeatMaker: custom sample loading (upload WAV, decode via Web Audio,
  play in the step sequencer alongside synthesized drums)
- RemixTimeline: per-track waveform with click-to-seek scrubbing,
  live playhead progress
Dashboard:
- New DashboardPanel as default landing view (first nav item)
- Stats row: sessions, processing, audio, memories, sources, automations
- 'While you were away' activity feed from recent sessions
- Quick actions grid (new chat, sessions, media lab, knowledge)
- Recent sessions list with live-processing pulse
- Latest audio output with click-to-open
- Pulls real data from existing RPC channels (sessions, media, memory, sources, automations)
- Auto-refreshes every 30s, manual refresh button
- Responsive grid layout

Music Studio improvements:
- MusicPlayer component replaces ComfyUI workflow picker in audio mode
  (play/pause, skip, seek, volume, shuffle, repeat, waveform, playlist)
- Stems/Beats/Remix tabs only show when Music Studio is active
- Library filtered to audio-only when in Music Studio
- Header swaps ComfyUI engine signals for audio engine info in music mode
- 'Create' tab becomes 'Player' in audio mode
Centralize the OS matrix output and pass it to smoke diffing so missing
artifacts are reported by the script and cannot produce a misleading green
workflow step.

Generated with Codebuff 🤖
Co-Authored-By: Codebuff <noreply@codebuff.com>
@skobe79 skobe79 changed the title Fix/test infra and prod fix: harden cross-platform test infrastructure and runtime paths Aug 18, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant