AI Presence Kit

AI Presence Kit is a renderer-agnostic presence state layer for AI interfaces, with a reference facial controller system. It turns runtime signals like typing, pausing, waiting, thinking, streaming, speaking, interruption, and error into inspectable interaction posture.

The simple SVG face in this repo is the proof surface. It exists to prove that gaze, blink, brows, mouth, posture, and motion can make an AI interface feel attentive before, during, and after model output.

Question

Can an AI interface make many small facial movement decisions from runtime state quickly enough that it feels co-present before the model responds?

Why This Exists

AI interfaces should not feel frozen until text appears. They can show presence while reading, waiting, thinking, streaming, speaking, or recovering from interruption without pretending to know a user's private emotional state.

AI Presence Kit explores that middle layer: runtime signals become interaction posture, then independent micro-controllers turn that posture into legible interface motion.

Goal Statement

Build a minimal, renderer-agnostic presence state layer and reference facial controller system for expressive AI interaction: a low-latency web prototype that maps input, response, voice, and latency signals into a conservative presence state model, then lets independent facial controllers make parallel micro-decisions for gaze, blink, brows, mouth, posture, and motion.

The finished prototype should feel less like an avatar and more like a living interface: attentive before it speaks, elegant when idle, responsive under pressure, coherent without pose swaps, and adjustable in expressiveness without pretending to infer private emotion.

Collaborating

This project is looking for collaborators interested in AI interfaces, expressive systems, interaction design, SVG/rendering, or low-latency UI behavior.

Good places to help include the reference face, runtime adapters, React/browser proof paths, package docs, release gates, and non-face renderers that consume the same presence state layer. See CONTRIBUTING.md for the current collaboration map.

Product Direction

The GitHub-facing package should be framed around a renderer-agnostic presence runtime and reference face proof, not a static avatar library:

• @ai-presence/core: presence state machine and event model.
• @ai-presence/react: React hooks and components for AI apps.
• @ai-presence/face: default SVG reference face and facial micro-controller proof.
• @ai-presence/adapters: optional adapters for common AI runtimes.

The default face should remain charming and immediately legible, but it should increasingly be driven by independent micro-decisions instead of expression pose swaps. The shared state contract starts with:

idle
user-typing
reading
waiting
thinking
streaming
speaking
interrupted
ready
error

This repo now starts that split with a small browser-safe core runtime at:

packages/core/src/presence-core.js

The prototype loads that core runtime first, then lets the SVG face interpret the canonical state as a renderer-specific expression. In the metrics panel, Presence state is the package-facing state contract; Renderer is the face's current expression. Core also exposes a small transition trace primitive so integrations can inspect what happened before the first visible token and whether an interruption posture occurred.

The shortest pitch:

Replace spinners with presence.

The differentiating wedge:

• Static avatar libraries give an agent identity.
• Character and lip-sync tools animate rendered characters, especially while speaking.
• AI chat frameworks build threads, messages, tools, composers, and streams.
• AI Presence Kit sits in the missing middle: it turns runtime state into human-readable interface presence.

It is not emotion recognition. It should express interaction posture: reading, waiting, thinking, streaming, speaking, interrupted, ready, or error.

See docs/GOAL_LOOP.md for the active build loop and next milestones.

Demo

The default app opens on the reference face and the runtime controls that drive it: still, attentive, expressive, voice, speaker, local face-tracking, metrics, and compare. This is the public proof surface: the face stays renderer-owned while the underlying presence state, adapter signals, and controller decisions remain inspectable.

The comparison harness remains a validation route rather than the lead visual. It runs the same simulated latency on both sides and records DOM timing evidence: data-generic-first-token-ms="1400", data-presence-first-state-ms="0", data-presence-frame-before-token-ms="0", and data-presence-decision-trace-lead-ms="1400".

The React browser demo runs the provider, snapshot hook, renderer slot, adapter path, and face expression mapping with actual React and ReactDOM. The actual renderer slot exposes six-channel trace DOM evidence: data-face-decision-trace="complete", data-face-decision-trace-channels="gaze blink brows mouth posture motion", data-face-decision-trace-decisions="6", data-face-decision-trace-warnings="0", data-face-decision-trace-renderer-safe="true", and data-face-latency-phase="before-output" during the pre-output turn. The same slot also drives a non-face status surface with data-nonface-renderer="status-surface", data-nonface-state, data-nonface-phase, data-nonface-attention, data-nonface-event, data-nonface-frame-time, and data-nonface-before-output="true" before response text appears. It also mirrors renderer-agnostic trace-summary evidence through data-react-trace-summary="complete", data-react-trace-first-output-ms, data-react-trace-lead-ms, and data-react-trace-final-state. It also mirrors transition context from the rendered SVG as data-face-transition-context="thinking stream-open 0", data-face-transition-controller-reads="gaze blink brows mouth posture motion", data-face-transition-controller-reads-event="true", and data-face-transition-controller-reads-age="true" so adapter-driven pre-output cues stay visible without recomputing face internals.

The React composer-lane route reuses the same provider, renderer slot, and Vercel AI SDK adapter path without loading @ai-presence/face. It shows the real-app adoption pattern in browser DOM: before assistant text appears, the lane exposes data-renderer="composer-lane", data-presence-state="waiting", data-presence-phase="before-output", data-composer-lock="true", data-assistant-text-empty="true", data-progress-step="stream-open", data-stream-open-ms="420ms", data-first-output-ms="none", and positive data-lead-ms.

The vanilla status-surface route loads only @ai-presence/core and @ai-presence/adapters browser globals. It proves the same before-output handoff without React or the reference face package: before assistant text appears, the plain DOM surface exposes data-renderer="status-surface", data-presence-state="waiting", data-presence-phase="before-output", data-presence-attention="response", data-presence-event="stream-open", data-assistant-text-empty="true", data-stream-open-ms="420ms", data-first-output-ms="none", and positive data-lead-ms.

The controller gallery route (?controllerGallery=1) exposes transition-cue proof for fresh submit, stream-open, token, and interrupt events. Its DOM evidence includes data-transition-events="submit stream-open token interrupt", data-transition-decision-trace="complete", and data-transition-controller-reads="gaze blink brows mouth posture motion" so all six controller decisions are inspectable against recent transition context.

Package Shape

Current prototype packages:

packages/core/src/presence-core.js
packages/core/dist/index.mjs
packages/face/src/presence-face.js
packages/face/dist/index.mjs
packages/adapters/src/runtime-adapter.js
packages/adapters/dist/index.mjs
packages/react/src/presence-react.js
packages/react/dist/index.mjs
examples/react-browser.html

Each package now has a local npm-style manifest, TypeScript declarations, and dual package entrypoints:

packages/core/package.json
packages/core/src/presence-core.d.ts
packages/face/package.json
packages/face/src/presence-face.d.ts
packages/adapters/package.json
packages/adapters/src/runtime-adapter.d.ts
packages/react/package.json
packages/react/src/presence-react.d.ts

Node/CommonJS consumers can use require. ESM consumers can import from the package export map:

import { createPresenceRuntime } from "@ai-presence/core";
import { createOpenAIResponsesAdapter } from "@ai-presence/adapters";
import { faceExpressionForPresence, renderPresenceFaceSvg } from "@ai-presence/face";
import { createPresenceReactBindings } from "@ai-presence/react";

const presence = createPresenceRuntime();
const responsesPresence = createOpenAIResponsesAdapter(presence);

responsesPresence.handleEvent({ type: "response.created" });
responsesPresence.handleEvent({ type: "response.output_item.added" });
responsesPresence.handleEvent({ type: "response.output_text.delta", delta: "Hello" });

const expression = faceExpressionForPresence(presence.getSnapshot());
const renderedFace = renderPresenceFaceSvg(presence.getSnapshot(), { timeMs: Date.now() });
console.log(renderedFace.decisionTrace.decisionCount); // 6
console.log(renderedFace.attributes.decisionTrace); // "complete"

Trace usage:

import {
  PresenceEvent,
  createPresenceRuntime,
  createPresenceTrace,
  summarizePresenceTrace,
} from "@ai-presence/core";

const trace = createPresenceTrace({ limit: 32 });
const presence = createPresenceRuntime();
trace.attach(presence);

presence.send(PresenceEvent.SUBMIT);
presence.send(PresenceEvent.STREAM_OPEN);
presence.send(PresenceEvent.TOKEN);

const summary = summarizePresenceTrace(trace);
console.log(trace.getEntries().map((entry) => entry.state));
console.log(summary.firstOutputMs);
console.log(summary.interruptMs, summary.interrupted);

Intended public packages:

@ai-presence/core
@ai-presence/react
@ai-presence/face
@ai-presence/adapters

First-time integration path: see docs/INTEGRATION_QUICKSTART.md for the minimal copyable path from published packages to runtime events, trace evidence, adapter mapping, and optional renderer handoff. For a design-partner trial, use docs/ADOPTER_TRIAL.md to ask a React/Vercel AI SDK app developer for a 10-minute integration and feedback on API friction. For no-network local proofs, run node examples/quickstart-presence.mjs to see generic chat lifecycle events become before-output trace evidence, node examples/status-surface-presence.mjs to see the same core/adapters path drive a framework-free non-face status surface, node examples/composer-lane-presence.mjs to see a real-app-style composer lane replace an empty-output wait with status, progress, and timeline evidence, node examples/vercel-ai-sdk-presence.mjs to see the documented Vercel AI SDK useChat status/message shape become framework=vercel-ai-sdk before-output evidence, node examples/assistant-lifecycle-presence.mjs to see a thread/run/message lifecycle surface expose the open run before assistant text exists, or node examples/assistant-ui-external-store-presence.mjs to see a documented assistant-ui ExternalStoreRuntime route map onNew, isRunning, and assistant message status.type into the same before-output proof.

Minimal core usage:

const presence = AIPresenceCore.createPresenceRuntime();

presence.send(AIPresenceCore.PresenceEvent.USER_INPUT, { text: "What should I build" });
presence.send(AIPresenceCore.PresenceEvent.USER_PAUSE, { text: "What should I build", completion: 0.4 });
presence.send(AIPresenceCore.PresenceEvent.SUBMIT);
presence.send(AIPresenceCore.PresenceEvent.TOKEN);

console.log(presence.getSnapshot().state);

Adapter usage:

const adapter = AIPresenceAdapters.createRuntimeSignalAdapter(presence);

adapter.send({ type: AIPresenceAdapters.RuntimeSignal.STREAM_OPEN });
adapter.send({ type: AIPresenceAdapters.RuntimeSignal.TOKEN });
adapter.send({ type: AIPresenceAdapters.RuntimeSignal.RESPONSE_COMPLETE });

OpenAI Responses adapter usage:

const responsesPresence = AIPresenceAdapters.createOpenAIResponsesAdapter(presence);

responsesPresence.handleEvent({ type: "response.created" });
responsesPresence.handleEvent({ type: "response.output_item.added" });
responsesPresence.handleEvent({ type: "response.output_text.delta", delta: "Hello" });
responsesPresence.handleEvent({ type: "response.completed" });

Vercel AI SDK adapter usage:

const aiSdkPresence = AIPresenceAdapters.createVercelAISDKAdapter(presence);

aiSdkPresence.update({ status: "submitted" });
aiSdkPresence.update({ status: "streaming", messages: [] });
aiSdkPresence.update({
  status: "streaming",
  messages: [{ role: "assistant", parts: [{ type: "text", text: "Hello" }] }],
});
aiSdkPresence.onFinish({ finishReason: "stop" });

Assistant lifecycle adapter usage:

const assistantPresence = AIPresenceAdapters.createAssistantLifecycleAdapter(presence);

assistantPresence.handleEvent({ type: "run-created", threadId, runId });
assistantPresence.handleEvent({ type: "message-created", threadId, runId, messageId });
assistantPresence.handleEvent({ type: "text-delta", threadId, runId, messageId, delta: "Hello" });
assistantPresence.handleEvent({ type: "run-completed", threadId, runId });

Realtime adapter usage:

const realtimePresence = AIPresenceAdapters.createOpenAIRealtimeAdapter(presence);

realtimePresence.handleEvent({ type: "input_audio_buffer.speech_started" });
realtimePresence.handleEvent({ type: "response.output_audio.delta" });
realtimePresence.handleEvent({ type: "response.done" });

React binding usage:

const {
  PresenceProvider,
  PresenceRenderer,
  PresenceRendererSlot,
  usePresenceFrameTime,
  usePresenceSnapshot,
  useVercelAIPresence,
} = AIPresenceReact.createPresenceReactBindings(React);

function ChatPresence({ chat }) {
  const presence = useVercelAIPresence(chat);
  return React.createElement("section", presence.evidenceAttributes, chat.status);
}

useVercelAIPresence(chat) is the shortest production-shaped React path for Vercel AI SDK useChat: pass the plain status and messages shape, attach presence.evidenceAttributes to your status/composer surface, and inspect data-presence-before-output="true" with data-first-output-ms="none" while assistant text is still empty.

PresenceRendererSlot gives custom React renderers the snapshot, shared control inputs, live frame time, and runtime in one render-prop payload while keeping the face renderer outside @ai-presence/react.

usePresenceFrameTime() gives React renderers a small live millisecond clock. It is renderer-agnostic: a face, badge, waveform, or other surface can use it to advance temporal frames between presence state transitions.

Reference renderer usage:

const expression = AIPresenceFace.faceExpressionForPresence(presence.getSnapshot());

Setup

Server-backed web prototype:

• index.html
• styles.css
• app.js
• packages/core/src/presence-core.js
• packages/face/src/presence-face.js
• packages/adapters/src/runtime-adapter.js
• packages/react/src/presence-react.js
• packages/*/package.json
• packages/*/src/*.d.ts
• packages/*/dist/index.mjs
• server.mjs
• .env.example
• FACIAL_EXPRESSION_RESEARCH.md

Run from this folder:

npm install
cp .env.example .env
# edit .env and set OPENAI_API_KEY
node server.mjs

Then open:

http://127.0.0.1:8058

Useful local checks:

npm run validate

Individual validation steps:

npm run check
npm run perf:core
npm run perf:face
npm test
npm run demo:adapters
npm run demo:quickstart
npm run demo:status-surface
npm run demo:react
npm run demo:composer-lane
npm run demo:vercel-ai-sdk
npm run demo:assistant-lifecycle
npm run demo:assistant-ui-external-store
npm run pack:dry-run

npm run demo:quickstart runs the no-network adoption proof from examples/quickstart-presence.mjs, mapping generic chat lifecycle events through @ai-presence/adapters and printing statePath, eventPath, firstOutputMs, leadMs, finalState, hasOutput, complete, and interrupted.

npm run demo:status-surface runs a framework-free non-face consumer proof from examples/status-surface-presence.mjs. It maps the same adapter-driven lifecycle into a plain status surface with data-renderer="status-surface", data-presence-state="waiting", data-presence-phase="before-output", data-presence-attention="response", data-presence-event="stream-open", and data-presence-before-output="true" before the first output.

npm run demo:composer-lane runs a real-app-style composer lane proof from examples/composer-lane-presence.mjs. It uses the Vercel AI SDK adapter path for submitted, streaming, and ready updates, then maps the renderer-agnostic snapshot and trace summary into a status bar, locked message composer, progress lane, and trace timeline. It prints renderer=composer-lane, statePath, eventPath, phasePath, streamOpenMs, firstOutputMs, leadMs, finalState, hasOutput, complete, and interrupted, while the pre-output lane exposes data-renderer="composer-lane", data-presence-state="waiting", data-presence-phase="before-output", data-composer-lock="true", data-assistant-text-empty="true", and data-progress-step="stream-open" without importing the SVG face renderer.

npm run demo:vercel-ai-sdk runs a named-runtime proof from examples/vercel-ai-sdk-presence.mjs. It follows primary AI SDK docs for useChat status, messages, assistant text parts, onFinish.isAbort, and onError without importing Vercel packages, then maps submitted, streaming, ready, error, and aborted finish into createVercelAISDKAdapter. It prints framework=vercel-ai-sdk, statusPath, streamOpenMs, firstOutputMs, leadMs, presenceBeforeOutputMs, finalState=ready, hasOutput=true, complete=true, interrupted=false, abortState=interrupted, and errorState=error.

npm run demo:assistant-lifecycle runs an assistant app lifecycle proof from examples/assistant-lifecycle-presence.mjs. It simulates a thread/run/message opening before visible assistant text, then prints surface=assistant-lifecycle, statePath, eventPath, frameworkEventPath, streamOpenMs, firstOutputMs, leadMs, presenceBeforeOutputMs, finalState, hasOutput, complete, and interrupted, while the pre-output surface exposes data-presence-state="waiting", data-presence-phase="before-output", data-assistant-output-empty="true", and data-presence-before-output="true".

npm run demo:assistant-ui-external-store runs a named-framework proof from examples/assistant-ui-external-store-presence.mjs. It follows the documented assistant-ui ExternalStoreRuntime route without importing assistant-ui, then maps onNew, isRunning, an empty assistant message with status.type="running", the first text delta, and status.type="complete" into createAssistantLifecycleAdapter. It prints surface=assistant-ui-external-store, framework=assistant-ui, route=ExternalStoreRuntime, frameworkEventPath, frameworkStatusPath, streamOpenMs, firstOutputMs, leadMs, presenceBeforeOutputMs, finalState, hasOutput, complete, and interrupted.

npm run demo:adapters prints Vercel AI SDK, assistant lifecycle, OpenAI Responses, OpenAI Realtime, and generic chat transitions with reference face frame evidence plus bounded six-channel decision-trace evidence such as trace=complete, decisions=6, safe=true, and warnings=0. Its trace summaries include interruptMs and interrupted so interruption posture is visible without coupling the core package to the face renderer.

npm run perf:core runs a local package-level smoke benchmark for the renderer-agnostic runtime path. It drives createPresenceRuntime().send(...), Vercel AI SDK and generic chat adapters, createPresenceTrace().record(...), and summarizePresenceTrace(...) through completed traces with thinking and waiting before the first token, final ready, hasOutput=true, and complete=true. It is local core/adapters/trace latency evidence, not a browser latency probe, OpenAI call, face-renderer benchmark, or release-blocking CI gate.

npm run perf:face runs a local package-level smoke benchmark for the face pipeline. It creates shared presence snapshots across all canonical states, measures faceControllerFrameForPresence -> faceControllerDecisionTraceForFrame, then measures full renderPresenceFaceSvg output. It fails if complete six-channel renderer-safe trace evidence is missing, warning-bearing, or over the conservative average budgets: 0.25ms for frame+trace and 0.75ms for SVG renderer evidence. This is local face-pipeline evidence, not a browser latency probe, OpenAI call, or release-blocking CI gate.

Adapter source assumptions are tracked in packages/adapters/README.md. Release-readiness gates are tracked in docs/RELEASE_READINESS.md. Versioning, changelog, and npm package-name policy are tracked in docs/RELEASE_POLICY.md.

The app can still be opened directly as a static file, but OpenAI-backed speculation, streaming responses, and speech require server.mjs.

For repeatable local browser evidence across the documented routes, run:

npm run browser:smoke

This starts server.mjs on a temporary local port with OpenAI disabled, drives local headless Chrome through the Chrome DevTools Protocol, verifies rendered DOM evidence for the reference, metrics/controller, controller gallery, comparison, React browser, face-free composer-lane, and vanilla status-surface routes, then stops the server. It is a local release gate, not part of CI or the default npm run validate gate.

For visual QA without touching the default first screen, the app accepts quiet view-only query params:

http://127.0.0.1:8058/?metrics=1&presence=expressive

These only set the initial metrics visibility and presence level; they do not send text or trigger OpenAI calls.

The A/B comparison harness is available at:

http://127.0.0.1:8058/?compare=1&autorunCompare=1

It runs the same simulated first-token latency on both sides; the presence side exposes state, frame, and decision-trace evidence before the first token while the generic side stays in a loading posture. The DOM dataset records the 1400ms first-token latency and the 0ms presence-before-output lead edge.

The browser React demo is available at:

http://127.0.0.1:8058/examples/react-browser.html

It uses the installed React and ReactDOM UMD builds, then drives @ai-presence/react, @ai-presence/adapters, the reference face mapping, and a non-face status surface in a real rendered React tree. The browser-smoke gate confirms the renderer slot reports state=waiting, data-face-latency-phase="before-output", complete six-channel decision-trace evidence, 6 decisions, 0 warnings, and data-face-decision-trace-renderer-safe="true" before output appears. It also confirms the sibling status surface reports data-nonface-renderer="status-surface", data-nonface-state="waiting", data-nonface-phase="before-output", and data-nonface-before-output="true" while response text is still empty.

The face-free React composer-lane route is available at:

http://127.0.0.1:8058/examples/react-browser-composer-lane.html

It uses the same React bindings and adapter path but does not load the face package. The browser-smoke gate confirms data-renderer="composer-lane", data-presence-state="waiting", data-presence-phase="before-output", data-composer-lock="true", data-assistant-text-empty="true", data-progress-step="stream-open", data-stream-open-ms="420ms", data-first-output-ms="none", and positive data-lead-ms before response text appears.

The framework-free browser status-surface route is available at:

http://127.0.0.1:8058/examples/vanilla-status-surface.html

It loads @ai-presence/core and @ai-presence/adapters directly from browser globals, without React or @ai-presence/face. The browser-smoke gate confirms data-renderer="status-surface", data-presence-state="waiting", data-presence-phase="before-output", data-presence-attention="response", data-presence-event="stream-open", data-assistant-text-empty="true", data-stream-open-ms="420ms", data-first-output-ms="none", and positive data-lead-ms before response text appears.

Expected .env keys:

OPENAI_API_KEY=
OPENAI_MODEL=gpt-5.4-mini
OPENAI_SPECULATE_MODEL=gpt-5.4-nano
OPENAI_RESPONSE_REASONING_EFFORT=none
OPENAI_SPECULATE_REASONING_EFFORT=none
OPENAI_TEXT_VERBOSITY=low
OPENAI_RESPONSE_MAX_OUTPUT_TOKENS=260
OPENAI_SPECULATE_MAX_OUTPUT_TOKENS=120
OPENAI_REALTIME_MODEL=gpt-realtime-2
OPENAI_REALTIME_VOICE=marin
OPENAI_REALTIME_CALL_TIMEOUT_MS=14000
OPENAI_TTS_MODEL=gpt-4o-mini-tts
OPENAI_TTS_VOICE=marin
OPENAI_TTS_FORMAT=wav
PORT=8058

Default rationale:

• OPENAI_SPECULATE_MODEL=gpt-5.4-nano: speed- and cost-sensitive classification-style work while the user is typing.
• OPENAI_MODEL=gpt-5.4-mini: lower-latency final responses without paying flagship-model cost on every turn.
• OPENAI_*_REASONING_EFFORT=none: lowest-latency setting for these short, non-tool, UI-facing turns.
• OPENAI_TEXT_VERBOSITY=low: concise responses and fewer output tokens.
• OPENAI_REALTIME_MODEL=gpt-realtime-2: direct speech-to-speech model for the Voice control.
• OPENAI_REALTIME_VOICE=marin: realtime output voice for speech-to-speech mode.
• OPENAI_TTS_MODEL=gpt-4o-mini-tts: current Speech API TTS model; not the deprecated general gpt-4o chat/audio-preview path.
• OPENAI_TTS_VOICE=marin: OpenAI's Speech docs recommend marin or cedar for best quality on the current TTS path.
• OPENAI_TTS_FORMAT=wav: browser-friendly low-latency audio format. pcm may be faster, but would need custom playback code.

Prototype Scope

Included:

• Reusable core presence state runtime with canonical states and transition events.
• Conservative AI presence state model, rendered by a minimal SVG face.
• Text input as the primary mode.
• Presence levels: Still, Attentive, and Expressive.
• Voice toggle, default off, using OpenAI Realtime speech-to-speech over WebRTC when the server has an API key.
• Speaker toggle, default off.
• Metrics toggle, default hidden.
• Live input event stream from every typed change.
• Local ultra-fast motor-reflex analysis from typing mechanics and sentence shape.
• Background probes use a pure analysis path so measurement does not overwrite live attention or completion state.
• Immediate face state updates through a render-only presence gain layer.
• Presence-scaled micro-presence: subtle blink and gaze drift that respect reduced-motion settings.
• Source-aware expression arbitration: local reflexes, mic changes, and speaking states render immediately; speculative refinements get a short presence-tuned settle window.
• Debounced and cancellable OpenAI speculation lane via /api/speculate.
• Prepared speculative reads are passed forward as weak context for /api/respond when the user submits matching text.
• Guarded response prefetch starts after high-completion speculation and is reused only for exact matching submitted text.
• Streaming OpenAI response lane via /api/respond.
• Direct speech-to-speech lane via /api/realtime/call when Voice is on; browser microphone audio streams to OpenAI Realtime over WebRTC and model speech returns as a remote audio stream.
• Optional OpenAI speech lane via /api/speech for typed responses when speaker is on, with sentence-level early TTS queueing during streamed responses.
• Stale-turn cancellation when the user starts typing over an in-flight response or speech queue.
• Local fallback if the server is running without OPENAI_API_KEY.

Not included:

• Persistent memory.
• Realistic avatar rendering.
• Production lip sync.
• A claim to infer the user's true emotions.

What To Watch

See FACIAL_EXPRESSION_RESEARCH.md for the research-backed motion grammar behind the reference face: gaze aversion, blink timing, eye movement, expression caution, and implementation candidates.

Core latency measures:

• Time to first expression.
• Keystroke to local read.
• Keystroke to usable OpenAI speculation.
• Submit to stream open.
• Time to first token after submit.
• Submit to completed response stream.
• Time to first audio when speaker is on.
• Rolling p50/p90 first-token latency for the current session.
• Input events processed.
• Stale speculative jobs canceled.
• Interrupted response/speech work canceled on new input.
• Compact event trace for the current input/turn.
• Optional three-run latency probe for local read, speculation, stream open, first token, and completion.
• Local package-level core runtime, adapter, trace-recording, and trace-summary average from npm run perf:core.
• Local package-level face controller frame+decision-trace and SVG renderer averages from npm run perf:face.
• Early TTS request and first-audio timing when Speaker is enabled.
• Attention, arousal, and commit signals from the local reflex layer.
• Presence level, expression source, and active response lane.

Core feel measures:

• Does the presence layer feel attentive while the user is still typing?
• Are the expressions readable without becoming distracting?
• Does muted mode still feel alive?
• Does idle presence feel alive without becoming decorative or distracting?
• Does each presence level feel like the same mind at a different expressive gain?
• Does the metrics panel reveal useful leading-edge behavior?
• Do local reflexes stay reversible, avoiding confident emotion claims before OpenAI speculation returns?

Interaction Oversight Loop

Use this loop after changing any input, submit, response, or metrics behavior:

1. Action contract: after each user action, verify what should happen to the visible composer, response text, face state, focus, metrics, and async work.
2. Submit path: Enter and click-send should capture the draft once, clear the visible composer, preserve the submitted text for the active response, and leave focus ready for the next draft.
3. Text editing path: Shift+Enter should add a newline, IME composition Enter should not submit, and typing over a response should cancel stale response/speech work before starting a new local reflex read.
4. Async path: prefetch may be reused only for exact submitted text; stale speculation, response, speech, and prefetch jobs must not write back into the current turn.
5. Controls path: mic, speaker, presence, and metrics toggles should update only their own lane and should not silently change the agent's underlying response behavior.
6. Instrumentation path: metrics and probes may reveal latency state, but they should stay optional, visually quiet, and avoid mutating the face or live response.

Current Result

Initial static prototype created and browser-tested on 2026-05-28.

Validation notes:

• Desktop browser pass loaded the app, toggled metrics, typed input, and submitted a response.
• Mobile pass at 390x844 showed no horizontal overflow and kept controls usable.
• Observed expression timing was single-digit milliseconds during sequential typing and roughly one frame during programmatic fill.
• Local feature read measured as 0ms in-browser for the current heuristic lane.
• Text response streaming produced first token around 1ms in the static local path.
• Speaker remained off by default; first audio stayed unset while muted.
• Node syntax checks passed for app.js and server.mjs.
• Server health endpoint returned openaiConfigured: false without a local .env, as expected.
• Missing-key API calls return 503 with a clear .env setup message.
• Browser-tested the server-backed app at http://127.0.0.1:8058; without a key, it stayed usable through local fallback.
• With .env configured, browser-tested real OpenAI speculation and streaming response calls.
• Optional metrics now include active lane, stream-open latency, and a compact trace of input, local reflex, speculation, token, completion, and fallback events.
• Response trace marks when a prepared typing-time read is used for the submitted text.
• Trace marks prefetch start, readiness, hit/wait, and reuse when response work completed before submit.
• Speaker-on mode can now queue completed response sentences for TTS before the full response finishes.
• New input can interrupt stale response and speech work, returning control to the local reflex lane.
• Metrics includes a quiet Run probe for repeatable local/OpenAI latency snapshots without moving the face.
• Headless Chrome visual QA captured the quiet default screen plus metrics-open desktop and mobile states.
• Metrics-open desktop view gives the face room instead of covering it; mobile view stacks metrics under the face with visible label/value rows.
• Oversight audit caught and fixed stale composer state after submit, click-send focus loss, and accidental Enter-submit during IME composition.
• Browser validation covered Shift+Enter newline behavior, click-send clear/focus, prefetch reuse after submit, and typing-to-interrupt stale OpenAI response work.
• Package-shaped no-build surfaces now exist for core state, face renderer mapping, and generic runtime-signal adapters.
• A/B comparison harness now contrasts generic loading against AI Presence Kit with the same simulated first-token latency.
• Framework-facing starter adapters now cover assistant lifecycle events, Vercel AI SDK statuses, OpenAI Responses streaming events, OpenAI Realtime server events, and generic chat lifecycle events.
• Core runtime subscriptions and the first React binding factory now support provider, snapshot hook, state hook, and renderer-slot patterns without adding a build step.
• Each package now has npm-style manifests and TypeScript declaration files.
• npm run demo:adapters prints adapter-to-presence traces, reference face frame evidence, and six-channel decision-trace evidence for the five starter adapter paths.
• npm run perf:face prints compact package-level face-pipeline timing evidence across all canonical states while validating complete, renderer-safe, warning-free six-channel decision traces for both the controller frame path and full SVG renderer path.
• npm pack --dry-run passes for @ai-presence/core, @ai-presence/face, @ai-presence/adapters, and @ai-presence/react when using a writable npm cache.
• React usage is covered by examples/react-presence-demo.js, examples/react-browser.html, examples/react-browser-composer-lane.html, npm run demo:react, tests/react-example.test.mjs, and tests/react-browser-example.test.mjs.
• ESM import entrypoints now sit beside the CommonJS/browser-global source files for all four packages.
• A browser React demo now uses real React and ReactDOM runtime builds to exercise the provider, snapshot hook, renderer slot, adapter path, and face expression mapping.
• README media now shows the A/B comparison harness and the real React browser demo.
• npm run validate now mirrors the GitHub Actions CI gate: syntax checks, tests, adapter demo, React demo, and package dry-runs.
• docs/RELEASE_POLICY.md records the first public release policy and the current npm registry availability check for the intended package names.
• Release readiness and initial changes are captured in docs/RELEASE_READINESS.md and CHANGELOG.md.
• Browser validation covered the comparison route, pre-token presence cues, equal first-token timing, same response text, no state leak between panes, desktop two-column layout, and mobile no-overflow layout.
• npm run check, npm test, and git diff --check pass for the current prototype.

Current adoption slice:

• The real-app adoption slice is represented by examples/composer-lane-presence.mjs, examples/react-browser-composer-lane.html, and examples/vanilla-status-surface.html: framework-free and React/browser routes that use package-shaped core/adapters APIs and prove before-output trace evidence without depending on the reference SVG face.
• The named Vercel AI SDK adoption proof is represented by examples/vercel-ai-sdk-presence.mjs: a no-dependency useChat status/message route that maps documented submitted, streaming, ready, error, assistant text parts, and aborted finish into createVercelAISDKAdapter.
• The assistant lifecycle adoption slice is represented by createAssistantLifecycleAdapter and examples/assistant-lifecycle-presence.mjs: a framework-package-free thread/run/message lifecycle path that proves an open assistant run and message shell can show waiting before visible text.
• The named assistant-ui adoption proof is represented by examples/assistant-ui-external-store-presence.mjs: a no-dependency ExternalStoreRuntime route that maps documented onNew, isRunning, and assistant message status.type values into the assistant lifecycle adapter and proves the same before-output trace evidence.
• The release consumer smoke now repeats the OpenAI Responses adapter path, Vercel AI SDK useChat status/message path, assistant lifecycle adapter path, assistant-ui ExternalStoreRuntime route, vanilla status-surface pattern, and composer-lane pattern in a fresh temp consumer with installed @ai-presence/core and @ai-presence/adapters, proving the runtime adapter, named-runtime, status-surface, composer/progress, and timeline handoffs are publishable package surface rather than repo-local source.

Next iteration:

• Keep validating named framework routes only when exact current lifecycle names can be verified from primary docs, preserving no-dependency adapter proofs and before-output trace evidence.
• Use npm run release:preflight before broad release-readiness claims, and use npm run release:publish -- X.Y.Z when package source, package README, package metadata, examples, release media, public package behavior, or other published artifacts change. Private docs-only milestones still go through public gate, validation, CI, and PR merge without publishing.