Skip to content

Commit c97fe3e

Browse files
ozgesolidkeyclaude
andcommitted
Feature: MF4 Signals viz P1 — multi-signal overlay on a shared time axis
Stage-2 P1 of the flexible MF4 visualization (design: docs/MF4_VISUALIZATION.md). New "Signals" bottom-panel tab: left = searchable checkbox list of numeric channels (via trendDiscoverFields); right = an overlay time-series of the checked signals on a shared x axis, with per-signal colors, a legend, a hover readout, normalize toggle (0–1 per signal vs shared real-Y), and click-a-point → jump to that record's line. Backend: extractSignalSeries() in trendEngine.ts reads the file ONCE and returns all selected signals aligned on the master `t` (falling back to record index when absent), index-bucketed and min/max/avg downsampled to ~maxPoints — so a multi-million-record MF4 stays smooth (peak memory O(maxPoints × signals), not O(records)). New TREND_SIGNAL_SERIES IPC wired through shared/types, preload, main handler, and the renderer Api type. Tests: 3 for extractSignalSeries (t-alignment, downsample min/max preservation, index fallback). 317 pass, build + typecheck green. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
1 parent 5feec51 commit c97fe3e

9 files changed

Lines changed: 581 additions & 1 deletion

File tree

src/main/index.ts

Lines changed: 18 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -25,7 +25,7 @@ import { loadDatadogConfig, saveDatadogConfig, clearDatadogConfig, fetchDatadogL
2525
import { startApiServer, stopApiServer, ApiContext, addChatMessage, getChatMessages, getSseClientCount, getAgentName, loadPersistedSession, API_PORT } from './api-server';
2626
import { runRecipe, RecipeOptions } from '../mcp-server/recipes';
2727
import { BaselineStore, buildFingerprint } from './baselineStore';
28-
import { discoverFields, extractSeries, detectTransitions, correlate } from './trendEngine';
28+
import { discoverFields, extractSeries, extractSignalSeries, detectTransitions, correlate } from './trendEngine';
2929
// Native-dependent modules — lazy-loaded to prevent SIGSEGV if bindings aren't built
3030
let SerialHandler: any = null;
3131
let LogcatHandler: any = null;
@@ -4334,6 +4334,23 @@ ipcMain.handle(IPC.TREND_SERIES, async (_, options) => {
43344334
}
43354335
});
43364336

4337+
ipcMain.handle(IPC.TREND_SIGNAL_SERIES, async (_, options) => {
4338+
const handler = getFileHandler();
4339+
if (!handler) return { success: false, error: 'No file open' };
4340+
if (!options?.fields?.length) return { success: false, error: 'fields required' };
4341+
try {
4342+
const result = extractSignalSeries(handler, options.fields, {
4343+
startLine: options.startLine,
4344+
endLine: options.endLine,
4345+
xField: options.xField,
4346+
maxPoints: options.maxPoints,
4347+
});
4348+
return { success: true, ...result };
4349+
} catch (error) {
4350+
return { success: false, error: String(error) };
4351+
}
4352+
});
4353+
43374354
ipcMain.handle(IPC.TREND_TRANSITIONS, async (_, options) => {
43384355
const handler = getFileHandler();
43394356
if (!handler) return { success: false, error: 'No file open' };

src/main/trendEngine.ts

Lines changed: 141 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -340,6 +340,147 @@ function topN(rec: Record<string, number>, n: number): Record<string, number> {
340340
return Object.fromEntries(sorted);
341341
}
342342

343+
// ── 2b. Multi-signal aligned series (Signals overlay viz) ────────────────────
344+
345+
export interface SignalSeries {
346+
field: string;
347+
type: FieldType;
348+
values: (number | null)[]; // avg per emitted bucket (null = no sample in bucket)
349+
min: (number | null)[]; // per-bucket min (spike-preserving band)
350+
max: (number | null)[]; // per-bucket max
351+
viewerLines: number[]; // representative 1-based line per bucket (click→line)
352+
globalMin: number; // for normalize / autoscale
353+
globalMax: number;
354+
present: number; // total records that had this field
355+
}
356+
357+
export interface SignalSeriesResult {
358+
x: { field: string; values: number[]; isIndex: boolean }; // shared axis (t, or record index fallback)
359+
series: SignalSeries[];
360+
totalRecords: number; // lines that carried an x value
361+
buckets: number; // emitted bucket count (≤ maxPoints)
362+
truncated: boolean;
363+
}
364+
365+
/**
366+
* Read the file ONCE and return several fields ALIGNED on a shared x axis
367+
* (default the `t` master), downsampled to ~maxPoints buckets so a multi-million
368+
* record MF4 stays smooth. Records are file-ordered and `t` is monotonic, so we
369+
* bucket by record index (cheap, no second pass) and keep min/max/avg per bucket
370+
* to preserve spikes. If a line has no `t`, the record index is used as x.
371+
*/
372+
export function extractSignalSeries(
373+
handler: FileHandler,
374+
fields: string[],
375+
opts: ScanRange & { xField?: string; maxPoints?: number } = {},
376+
): SignalSeriesResult {
377+
const xField = opts.xField ?? 't';
378+
const maxPoints = Math.min(Math.max(opts.maxPoints ?? 4000, 100), 20000);
379+
const total = handler.getTotalLines();
380+
const start = Math.max(0, opts.startLine ?? 0);
381+
const end = Math.min(total - 1, opts.endLine ?? total - 1);
382+
const span = Math.max(1, end - start + 1);
383+
const bucketSize = Math.max(1, Math.ceil(span / maxPoints));
384+
const nBuckets = Math.ceil(span / bucketSize);
385+
386+
// Per-bucket accumulators for the x axis.
387+
const xSum = new Float64Array(nBuckets);
388+
const xCnt = new Int32Array(nBuckets);
389+
const repLine = new Int32Array(nBuckets); // representative 1-based line per bucket
390+
let sawRealX = false;
391+
392+
// Per-signal accumulators.
393+
const accs = fields.map(() => ({
394+
sum: new Float64Array(nBuckets),
395+
cnt: new Int32Array(nBuckets),
396+
min: new Float64Array(nBuckets).fill(Infinity),
397+
max: new Float64Array(nBuckets).fill(-Infinity),
398+
type: null as FieldType | null,
399+
present: 0,
400+
}));
401+
402+
let totalRecords = 0;
403+
const { truncated } = scanLines(
404+
handler,
405+
{ startLine: start, endLine: end, maxScan: opts.maxScan ?? 50_000_000 },
406+
(lineNumber, text) => {
407+
const map = extractFields(text);
408+
if (map.size === 0) return;
409+
const relIdx = lineNumber - start;
410+
let bucket = Math.floor(relIdx / bucketSize);
411+
if (bucket < 0) bucket = 0; else if (bucket >= nBuckets) bucket = nBuckets - 1;
412+
413+
// x value: the master field if numeric, else the record index.
414+
const xRaw = map.get(xField);
415+
const xNum = xRaw !== undefined ? toNum(xRaw) : null;
416+
const x = xNum !== null ? (sawRealX = true, xNum) : relIdx;
417+
xSum[bucket] += x;
418+
xCnt[bucket]++;
419+
if (repLine[bucket] === 0) repLine[bucket] = lineNumber + 1;
420+
totalRecords++;
421+
422+
for (let f = 0; f < fields.length; f++) {
423+
const raw = map.get(fields[f]);
424+
if (raw === undefined) continue;
425+
const n = toNum(raw);
426+
if (n === null) continue;
427+
const a = accs[f];
428+
if (a.type === null) a.type = classifyValue(raw);
429+
a.sum[bucket] += n;
430+
a.cnt[bucket]++;
431+
if (n < a.min[bucket]) a.min[bucket] = n;
432+
if (n > a.max[bucket]) a.max[bucket] = n;
433+
a.present++;
434+
}
435+
},
436+
);
437+
438+
// Compact to buckets that actually have an x sample, preserving order.
439+
const xValues: number[] = [];
440+
const keep: number[] = [];
441+
for (let b = 0; b < nBuckets; b++) {
442+
if (xCnt[b] > 0) { keep.push(b); xValues.push(xSum[b] / xCnt[b]); }
443+
}
444+
445+
const series: SignalSeries[] = fields.map((field, f) => {
446+
const a = accs[f];
447+
const values: (number | null)[] = [];
448+
const minA: (number | null)[] = [];
449+
const maxA: (number | null)[] = [];
450+
const viewerLines: number[] = [];
451+
let gMin = Infinity, gMax = -Infinity;
452+
for (const b of keep) {
453+
viewerLines.push(repLine[b] || 1);
454+
if (a.cnt[b] > 0) {
455+
const avg = a.sum[b] / a.cnt[b];
456+
values.push(avg);
457+
minA.push(a.min[b]);
458+
maxA.push(a.max[b]);
459+
if (a.min[b] < gMin) gMin = a.min[b];
460+
if (a.max[b] > gMax) gMax = a.max[b];
461+
} else {
462+
values.push(null); minA.push(null); maxA.push(null);
463+
}
464+
}
465+
return {
466+
field,
467+
type: a.type ?? 'numeric',
468+
values, min: minA, max: maxA, viewerLines,
469+
globalMin: gMin === Infinity ? 0 : gMin,
470+
globalMax: gMax === -Infinity ? 0 : gMax,
471+
present: a.present,
472+
};
473+
});
474+
475+
return {
476+
x: { field: sawRealX ? xField : 'index', values: xValues, isIndex: !sawRealX },
477+
series,
478+
totalRecords,
479+
buckets: xValues.length,
480+
truncated,
481+
};
482+
}
483+
343484
// ── 3. Transition ("flip") detection ─────────────────────────────────────────
344485

345486
export interface TransitionResult {

src/preload/index.ts

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -84,6 +84,7 @@ const IPC = {
8484
// Trends notebook
8585
TREND_DISCOVER_FIELDS: 'trend-discover-fields',
8686
TREND_SERIES: 'trend-series',
87+
TREND_SIGNAL_SERIES: 'trend-signal-series',
8788
TREND_TRANSITIONS: 'trend-transitions',
8889
TREND_CORRELATE: 'trend-correlate',
8990
// Guided triage
@@ -691,6 +692,8 @@ const api = {
691692
ipcRenderer.invoke(IPC.TREND_DISCOVER_FIELDS, options),
692693
trendSeries: (options: { field: string; startLine?: number; endLine?: number; bucketCount?: number; maxPoints?: number; pattern?: string; patternFlags?: string }): Promise<{ success: boolean; [key: string]: any }> =>
693694
ipcRenderer.invoke(IPC.TREND_SERIES, options),
695+
signalSeries: (options: { fields: string[]; xField?: string; startLine?: number; endLine?: number; maxPoints?: number }): Promise<{ success: boolean; [key: string]: any }> =>
696+
ipcRenderer.invoke(IPC.TREND_SIGNAL_SERIES, options),
694697
trendTransitions: (options: { field: string; startLine?: number; endLine?: number; maxTransitions?: number; pattern?: string; patternFlags?: string }): Promise<{ success: boolean; [key: string]: any }> =>
695698
ipcRenderer.invoke(IPC.TREND_TRANSITIONS, options),
696699
trendCorrelate: (options: { field: string; event: string; startLine?: number; endLine?: number; pattern?: string; patternFlags?: string }): Promise<{ success: boolean; [key: string]: any }> =>

src/renderer/index.html

Lines changed: 20 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -387,6 +387,7 @@ <h2>LOGAN</h2>
387387
<button class="bottom-tab-btn" data-bottom-tab="analysis" data-help="Log level distribution, crash detection, component breakdown, and timestamp analysis.&#10;Click Run Analysis to scan the entire file.">Analysis</button>
388388
<button class="bottom-tab-btn" data-bottom-tab="investigate" data-help="Guided root-cause finding. Pick a symptom (it crashed, it froze, it's slow…) and LOGAN runs a recipe that searches, checks time gaps, and trends fields — then pins clickable findings.&#10;No expertise needed.">Investigate</button>
389389
<button class="bottom-tab-btn" data-bottom-tab="trends" data-help="Trend any field over time. Click Discover to auto-find log variables (key=value, JSON), then chart a field as value-over-time, value flips, or correlation.&#10;Click a chart point to jump to that line.">Trends</button>
390+
<button class="bottom-tab-btn" data-bottom-tab="signals" data-help="Overlay multiple numeric signals (e.g. MF4 channels) on one shared time axis.&#10;Check signals on the left to plot them; click a point to jump to that record.">Signals</button>
390391
<button class="bottom-tab-btn" data-bottom-tab="time-gaps" data-help="Find gaps in log timestamps where no events were recorded.&#10;Useful for detecting hangs, freezes, or missing data periods.">Time Gaps</button>
391392
<button class="bottom-tab-btn" data-bottom-tab="search-results" data-help="Results from Ctrl+F search. Click a result to jump to that line.&#10;Supports regex, wildcard, case-sensitive, and whole-word modes.">Search Results</button>
392393
<button class="bottom-tab-btn" data-bottom-tab="search-configs" data-help="Save and reuse complex search patterns across files.&#10;Each config stores pattern, regex/case/word flags, and a label.">Search Configs</button>
@@ -456,6 +457,25 @@ <h2>LOGAN</h2>
456457
<p class="placeholder">Click <strong>Discover Fields</strong> to scan the log for variables, then add a cell to chart one over time.</p>
457458
</div>
458459
</div>
460+
<!-- Signals tab (multi-signal overlay viz) -->
461+
<div class="bottom-tab-view" data-bottom-tab="signals">
462+
<div class="bottom-tab-sub-header signals-toolbar">
463+
<button id="btn-signals-discover" class="secondary-btn small">Discover Signals</button>
464+
<input id="signals-search" class="trends-input signals-search" type="text" placeholder="filter signals…" disabled>
465+
<label class="signals-toggle" title="Scale every signal to 0–1 so different ranges compare by shape"><input type="checkbox" id="signals-normalize" checked> Normalize</label>
466+
<span id="signals-summary" class="signals-summary"></span>
467+
</div>
468+
<div class="signals-body">
469+
<div id="signals-list" class="signals-list">
470+
<p class="placeholder">Click <strong>Discover Signals</strong> to list numeric channels, then check signals to overlay them.</p>
471+
</div>
472+
<div class="signals-plot-wrap">
473+
<canvas id="signals-canvas" class="signals-canvas"></canvas>
474+
<div id="signals-readout" class="signals-readout"></div>
475+
<div id="signals-legend" class="signals-legend"></div>
476+
</div>
477+
</div>
478+
</div>
459479
<!-- Time Gaps tab -->
460480
<div class="bottom-tab-view" data-bottom-tab="time-gaps">
461481
<div class="time-gap-controls">

0 commit comments

Comments
 (0)