Skip to content

ChaosEngineAI v0.7.4

Choose a tag to compare

View all tags
@github-actions github-actions released this 06 May 18:47
· 306 commits to staging since this release
v0.7.4
cfa1d53
This commit was created on GitHub.com and signed with GitHub’s verified signature.
GPG key ID: B5690EEEBB952194
Verified
Learn about vigilant mode.

v0.7.4 — chat uplift + image/video gen polish

Chat experience (the headline)

Phase 1 — UX foundations

  • Syntax highlighting in code blocks, in-thread search, conversation export, real cancel (mid-stream abort), reasoning-effort levels.
  • Reasoning panel: collapsible streaming preview, fixed first-paragraph gap.

Phase 2.0 — perf surface

  • Prompt-processing feedback + TTFT (time-to-first-token) live indicator.
  • Watchdogs: prompt-eval timeout, memory gate, runaway guards (token rate floor, repetition guard), panic + thermal banners, image/video gates that block kicking off a generation when VRAM/RAM headroom is unsafe.

Phase 2.1 — refactor

  • Decomposed monolithic ChatTab.tsx into ChatSidebar / ChatHeader / ChatThread / ChatComposer.

Phase 2.2 — sampler control

  • Full sampler exposure: top_p, top_k, min_p, repeat_penalty, seed, mirostat, reasoning_effort.
  • JSON-schema constrained-output opt-in (json_schema field).

Phase 2.4–2.5 — message-tree workflows

  • Conversation branching: fork from any assistant message into a sibling thread.
  • In-thread compare: render sibling variants side-by-side under the assistant bubble.

Phase 2.6–2.7 — context & prompts

  • Cross-platform RAG: semantic embedding via llama-embedding + cosine retrieval over local docs.
  • Prompt presets + variables: fill-form before "Use in Chat" so reusable prompts can take inputs.

Phase 2.8 — structured tool output

  • Tool call results render as table / code / markdown / image based on returned shape, not raw JSON.

Phase 2.10 — MCP client

  • Stdio JSON-RPC transport + tool adapter so any local MCP server is callable from chat. Provenance shown per tool result.

Phase 2.11–2.12 — model-aware composer

  • Typed capability declarations (vision / tools / json_schema / reasoning) surface as badges in every model picker.
  • Composer auto-gating (e.g. attach-image button hidden when active model has no vision).
  • Mid-thread model swap with one-turn override (try a different model for a single response, then revert).

Phase 2.13 — OpenAI-compatible server

  • Full sampler chain + embeddings parity. Apps that talk to /v1/chat/completions no longer lose advanced sampler params on the way through.

Phase 2.14 — catalog browser

  • VRAM-fit hints on every Discover variant card so you see at a glance what'll actually run on your machine.

Phase 3.x — substrate transparency

  • KV strategy chip in composer: per-turn cache override (native / chaosengine / rotorquant / turboquant / triattention) without touching launch settings.
  • DDTree accepted-token overlay: substrate truth view of which speculative draft tokens were accepted.
  • Logprobs viz (advanced-mode gated): per-message confidence summary, MLX logprobs streaming passthrough.
  • Substrate routing inspector: per-turn badge above the metrics row showing which engine + binary served the response.
  • Per-turn host strip: cross-platform perf telemetry (CPU / GPU / RAM / temp).
  • Delve mode: critic-pass on assistant messages.
  • Workspace knowledge stacks: shared RAG corpus across sessions.
  • Chat-template inspection: detect Gemma + ChatML quirks, llama.cpp chat-template fix.

Image generation

  • First Block Cache cross-platform diffusion cache hook (diffusers.hooks.apply_first_block_cache). Default threshold 0.12, ≈1.8× speedup on FLUX.1-dev with imperceptible drift. Replaces the per-model TeaCache vendoring deferral.
  • TaylorSeer / MagCache / PyramidAttentionBroadcast / FasterCache strategies wired against diffusers 0.38 native API.
  • SDXL VAE fp16 fix on MPS / CUDA — keeps SDXL on Apple Silicon in fp16 instead of the slow fp32 fallback.
  • Distill LoRA support — Hyper-SD-8step + Turbo-Alpha for FLUX.1-dev.
  • AYS (Align Your Steps) sampler for SD/SDXL.
  • CFG decay parity with the video runtime (opt-in cfgDecay field).
  • Live denoise thumbnails via callback_on_step_end — TAESD/TAEHV preview VAE swap decodes per-step latents into ≤192 px PNG thumbnails streamed to the UI. Handles 4D (B, C, H, W) and FLUX's packed 3D (B, seq_len, 64) shapes.
  • MLX-native LLM prompt enhancer (Apple Silicon) — mlx-community/Qwen2.5-0.5B-Instruct-4bit rewrites your prompt into the active DiT's training distribution. Per-family system prompts for FLUX / Wan / LTX / HunyuanVideo / SDXL / SD3.
  • Vision attach gating — visionEnabled flag gates image attach across all runtimes; --mmproj wired for llama.cpp vision with sibling detection.

Video generation

  • mlx-video Wan runtime end-to-end (Apple Silicon):
    • One-shot convert pipeline for Wan-AI/Wan2.{1-T2V-1.3B,1-T2V-14B,2-TI2V-5B,2-T2V-A14B,2-I2V-A14B} — wraps python -m mlx_video.models.wan_2.convert subprocess.
    • Runtime routing through mlx_video_runtime.py with Wan-shaped CLI (--model-dir, --guide-scale, --scheduler).
    • GUI install panel under Video Discover with per-repo install buttons + live install log.
    • Live Wan2.1 MLX smoke validated: 19.6s end-to-end at 480×272, 5 frames, 4 steps.
  • Distill transformer support for Wan 2.2 A14B I2V (lightx2v 4-step, bf16 + fp8_e4m3 variants) — full transformer swap via _swap_distill_transformers.
  • STG (Spatial Temporal Guidance) slider wired through to mlx-video subprocess for LTX-2.
  • CogVideoX footprints right-sized + T5EncoderModel error diagnosed.

CUDA quantization (Windows / Linux foundation)

  • Nunchaku / SVDQuant transformer load (FLUX.1-dev + FLUX.1-schnell svdq-int4 variants). Preferred over NF4/int8wo when nunchaku>=1.2.1 is installed.
  • FP8 layerwise casting for non-FLUX DiTs (Wan / Qwen-Image / SD3 / LTX). Family-correct fp8 dtype (E5M2 for HunyuanVideo, E4M3 elsewhere). Compute capability gate refuses pre-Ada GPUs (SM <8.9).
  • NVIDIA/kvpress install action staged (kvpress>=0.5.3 registered).
  • Studio FP8 layerwise toggle in both Image + Video Studio.

Speculative decoding

  • dflash-mlx pin bump f825ffb → 8d8545d (v0.1.4.1 → v0.1.5.1) — target_ops adapter pattern, draft model quantization with Metal MMA kernels, branchless Metal kernels, fused draft KV projections.

Windows / CUDA stability

  • PowerShell ports of build-llama-turbo + build-sdcpp.
  • MSVC + CUDA detection helpers: accept VS Build Tools installs that report isComplete=0, append version= to CMAKE_GENERATOR_INSTANCE for unregistered installs, fix CUDA-integration elevated copy + invalidate stale CMake cache, auto-sync CUDA VS integration before cmake configure.
  • Install CUDA torch: self-debugging button with expandable per-attempt log + Restart prompt.
  • Windows CUDA detection fix + post-install runtime probe.
  • Preserve Windows GPU runtime on uninstall + lock extras path.
  • Video Studio dropping GPU warning now surfaces inline Install button.
  • T5 lazy-import diagnostic on generate paths (catches missing-dep failures before kicking off long generations).

Studio polish

  • Restored pre-aec1975 card layout for Image / Video Discover + My Models. Dropped the duplicate Wan panel.
  • KV cache chip filter harmonized with launch-settings modal so toggle states stay consistent across surfaces.
  • Chat cache-fit warning VRAM-aware on CUDA hosts.
  • Surfaced CPU torch on CUDA host. Raised chat default maxTokens to 4096.
  • Fixed Studio cache preview returning 0 GB on chat model selection.
  • Hide MLX-only catalog variants on non-Apple platforms.
  • Qwen 3.6 catalog entry.

Test infrastructure

  • backend_service/runtime_paths.py — append extras to sys.path instead of insert(1, ...). Repo-local adapter shims (notably turboquant_mlx) keep import authority across pytest, dev .venv, and Tauri-bundled launches. Was also a latent runtime bug masking the shim's adapter hooks after Setup → Install turboquant-mlx-full on the desktop app.

Bundles below: macOS aarch64 dmg + Linux x64 AppImage / deb + Windows x64 setup.exe. latest.json for Tauri auto-updater.

Assets 13
Loading