Skip to content

Commit d9958d1

Browse files
kyasbalkyasbal
authored
feat(web): add v6 parser foundation (errors and interfaces) (#628)
1 parent 353dfc7 commit d9958d1

3 files changed

Lines changed: 290 additions & 0 deletions

File tree

‎web/src/app/parser/README.md‎

Lines changed: 153 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,153 @@
1+
# KHI Frontend Parser Architecture
2+
3+
This directory contains the core logic for parsing KHI (Kubernetes History Inspector) inspection files.
4+
5+
## Streamed Data Assembly Architecture
6+
7+
KHI inspection files (v6+) can be extremely large, making it impossible to parse the entire file into memory at once or rely purely on JSON. To handle this efficiently, KHI uses a **Streamed Data Assembly** architecture.
8+
9+
This architecture explicitly separates the **Wire Format** (Protobuf binary chunks) from the **Application Model** (Highly optimized domain stores used by the UI).
10+
11+
### The Three-Tier Data Model
12+
13+
To maintain a clean separation of concerns and high performance, the KHI frontend utilizes a three-tier data architecture:
14+
15+
1. **DTO (Data Transfer Object) / Wire Format:** The raw Protobuf bindings generated by `@bufbuild/protobuf`. These match the file schema exactly (e.g., holding `string_id` instead of actual strings) and are not optimized for fast querying or UI rendering.
16+
2. **Domain Models & Stores (Application Model):** The `IDataAssembler` converts DTOs into **Domain Models** by resolving references (e.g., looking up `string_id` from the String Pool). These models are then stored in highly optimized repositories (e.g., `LogStore`, `TimelineStore`) built by the `InspectionDataBuilder`. These stores keep data sorted by time and provide blazing-fast binary search queries (`getLogsInRange`, `getLatestRevisionAt`).
17+
3. **ViewModel:** The data structures explicitly crafted for the Dumb Components to render. **Smart Components** observe user interactions, query the Domain Stores to fetch the necessary Domain Models, and dynamically assemble these ViewModels on the fly.
18+
19+
### Execution Flow & Data Lifecycle
20+
21+
The data lifecycle is divided into two distinct phases: **Phase 1 (Parsing)** which happens once when the file is loaded, and **Phase 2 (Querying)** which happens continuously as the user interacts with the UI.
22+
23+
```text
24+
===============================================================================
25+
PHASE 1: File Parsing & Data Assembly (One-time execution during file load)
26+
===============================================================================
27+
28+
[ KHI File (ArrayBuffer) ]
29+
|
30+
v
31+
( 1. BinaryReader ) <--- Extracts chunk binary sequentially
32+
|
33+
v
34+
( 2. KHIFileStreamer ) <--- Resolves Version (e.g. V6_BLUEPRINT)
35+
|
36+
v
37+
+-------------------------------------+
38+
| Chunk Ingestion Phase (Streaming) |
39+
| |
40+
| [Chunk 1: InterningPool] ------> | Assembler A `.ingest(proto DTO)`
41+
| [Chunk 2: Log] ------> | Assembler B `.ingest(proto DTO)`
42+
| [Chunk 3: Timeline] ------> | Assembler C `.ingest(proto DTO)`
43+
+-------------------------------------+
44+
|
45+
| (EOF Reached)
46+
v
47+
+-------------------------------------------+
48+
| 3. Assembly Phase (Sorted by Priority) |
49+
| |
50+
| 1st: InterningPoolAssembler (Priority 10) |
51+
| => Mutates builder.stringPool |
52+
| |
53+
| 2nd: LogAssembler (Priority 100) |
54+
| => Reads builder.stringPool |
55+
| => Resolves IDs to actual strings |
56+
| => Mutates builder.logStore |
57+
| (Stores 'DomainLog' objects) |
58+
| |
59+
| 3rd: TimelineAssembler (Priority 200) |
60+
| => Reads builder.stringPool & Logs |
61+
| => Resolves IDs to actual strings |
62+
| => Mutates builder.timelineStore |
63+
| (Stores 'DomainTimeline' objects) |
64+
+-------------------------------------------+
65+
|
66+
v
67+
( 4. builder.build() )
68+
|
69+
v
70+
+=======================================================+
71+
| [ ParsedKHIFile (Domain Stores) ] | <--- Kept in memory
72+
| |
73+
| - stringPool |
74+
| - logStore (Provides Binary Search) |
75+
| - timelineStore (Provides Binary Search) |
76+
+=======================================================+
77+
78+
79+
===============================================================================
80+
PHASE 2: UI Rendering & Interaction (Continuous, driven by user actions)
81+
===============================================================================
82+
83+
+=======================================================+
84+
| [ ParsedKHIFile (Domain Stores) ] |
85+
+=======================================================+
86+
|
87+
| 2. Query stores for specific time range / filters
88+
| (e.g., logStore.getLogsInRange(start, end))
89+
v
90+
+-------------------------------------------+
91+
| Smart Components | <--- 1. Intercept user event
92+
| (e.g., LogPane, TimelineView) | (Scroll, Click, Zoom)
93+
+-------------------------------------------+
94+
|
95+
| 3. Construct rendering-friendly 'ViewModel'
96+
| from the queried 'Domain Models'
97+
v
98+
[ Dumb Components (ViewModel) ] <--- 4. Render to DOM
99+
```
100+
101+
### Core Concepts
102+
103+
1. **`BinaryReader` and `KHIFileStreamer` (Orchestrator)**
104+
- The file is read chunk by chunk directly from the `ArrayBuffer` to avoid memory spikes.
105+
- The streamer identifies the chunk type and looks up how to handle it in the version-specific registry.
106+
107+
2. **`ParserBlueprint` and `ChunkDefinition`**
108+
- Each file version (e.g., v6) defines a blueprint that maps chunk IDs to their corresponding Protobuf decoding logic and `IDataAssembler`.
109+
110+
3. **`IDataAssembler`**
111+
- Receives decoded Protobuf objects (`ingest`) as the file streams.
112+
- Extracts necessary information, cross-references data, and constructs Domain Models.
113+
- Assemblers are executed in a strict **Priority Order** during the assembly phase. For example, the `InterningPoolAssembler` runs first so that resolved strings are available when the `LogAssembler` runs.
114+
115+
4. **`InspectionDataBuilder`**
116+
- Acts as the temporary mutable context during the parsing phase.
117+
- Assemblers push their constructed Domain Models into this builder.
118+
119+
5. **Optimized Domain Stores (`/domain`)**
120+
- Stores pre-sort Domain Models by timestamp and provide high-performance query methods (like binary search) to ensure the UI remains fast and responsive even when exploring massive datasets.
121+
122+
## Directory Structure
123+
124+
To maintain the Three-Tier Data Model, this directory is organized as follows:
125+
126+
```text
127+
web/src/app/
128+
├── store/ # Application Model (Domain Stores and Models)
129+
│ └── domain/ # Highly optimized domain stores built by assemblers
130+
│ ├── string-pool-store.ts
131+
│ ├── log-store.ts
132+
│ └── timeline-store.ts
133+
│
134+
└── parser/ # Core logic for KHI file parsing
135+
├── core/ # Version-agnostic orchestrator and interfaces
136+
│ ├── interfaces.ts # IDataAssembler, ParserBlueprint, ChunkDefinition
137+
│ ├── binary-reader.ts # Utility to read chunks from ArrayBuffer
138+
│ ├── file-streamer.ts # KHIFileStreamer (Main orchestrator)
139+
│ └── builder.ts # InspectionDataBuilder
140+
│
141+
├── errors/ # Custom error definitions for the parsing phase
142+
│ └── parser-errors.ts # KHIInvalidFileError, KHIDataAssemblyError, etc.
143+
│
144+
├── blueprints/ # Version-specific blueprints and the main registry
145+
│ ├── registry.ts # VERSION_REGISTRY
146+
│ └── v6-blueprint.ts # Blueprint mapping chunk IDs to v6 assemblers
147+
│
148+
└── assemblers/ # Version-specific chunk assemblers
149+
└── v6/
150+
├── interning-pool-assembler.ts
151+
├── log-assembler.ts
152+
└── ...
153+
```

‎web/src/app/parser/core/interfaces.ts‎

Lines changed: 56 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,56 @@
1+
/**
2+
* Copyright 2026 Google LLC
3+
*
4+
* Licensed under the Apache License, Version 2.0 (the "License");
5+
* you may not use this file except in compliance with the License.
6+
* You may obtain a copy of the License at
7+
*
8+
* http://www.apache.org/licenses/LICENSE-2.0
9+
*
10+
* Unless required by applicable law or agreed to in writing, software
11+
* distributed under the License is distributed on an "AS IS" BASIS,
12+
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
13+
* See the License for the specific language governing permissions and
14+
* limitations under the License.
15+
*/
16+
17+
import { InspectionDataBuilder } from './builder';
18+
19+
/**
20+
* Stateful assembler that collects decoded Protobufs and mutates the final model.
21+
*/
22+
export interface IDataAssembler<TProto = unknown> {
23+
/**
24+
* Ingests a decoded Protobuf chunk. Called multiple times if chunks are split.
25+
*/
26+
ingest(proto: TProto): void;
27+
28+
/**
29+
* Integrates the ingested data into the final InspectionData model.
30+
*/
31+
assembleInto(builder: InspectionDataBuilder): void;
32+
}
33+
34+
/**
35+
* Defines how a specific chunk type is handled for a specific version.
36+
*/
37+
export interface ChunkDefinition<TProto = unknown> {
38+
readonly typeId: number;
39+
/**
40+
* Stateless function to decode raw bytes into a Protobuf object.
41+
*/
42+
readonly decode: (bytes: Uint8Array) => TProto;
43+
/**
44+
* Factory method for the stateful assembler.
45+
*/
46+
readonly createAssembler: () => IDataAssembler<TProto>;
47+
/**
48+
* Execution priority for dependency resolution (Lower number = executed first).
49+
*/
50+
readonly priority: number;
51+
}
52+
53+
/**
54+
* A version-specific registry of chunk definitions.
55+
*/
56+
export type ParserBlueprint = Map<number, ChunkDefinition>;

‎web/src/app/parser/errors/parser-errors.ts‎

Lines changed: 81 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,81 @@
1+
/**
2+
* Copyright 2026 Google LLC
3+
*
4+
* Licensed under the Apache License, Version 2.0 (the "License");
5+
* you may not use this file except in compliance with the License.
6+
* You may obtain a copy of the License at
7+
*
8+
* http://www.apache.org/licenses/LICENSE-2.0
9+
*
10+
* Unless required by applicable law or agreed to in writing, software
11+
* distributed under the License is distributed on an "AS IS" BASIS,
12+
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
13+
* See the License for the specific language governing permissions and
14+
* limitations under the License.
15+
*/
16+
17+
/**
18+
* Thrown when the file does not have the expected KHI magic bytes.
19+
*/
20+
export class KHIInvalidFileError extends Error {
21+
constructor(message: string) {
22+
super(message);
23+
this.name = 'KHIInvalidFileError';
24+
}
25+
}
26+
27+
/**
28+
* Thrown when the parsed file version is not supported by any registered blueprint.
29+
*/
30+
export class KHIVersionMismatchError extends Error {
31+
constructor(message: string) {
32+
super(message);
33+
this.name = 'KHIVersionMismatchError';
34+
}
35+
}
36+
37+
/**
38+
* Context provided when a chunk fails to decode.
39+
*/
40+
export interface ChunkErrorContext {
41+
readonly version: number;
42+
readonly typeId: number;
43+
readonly chunkIndex: number;
44+
readonly offset: number;
45+
readonly cause: unknown;
46+
}
47+
48+
/**
49+
* Thrown when a specific chunk fails to decode from Protobuf binary.
50+
*/
51+
export class KHIChunkDecodeError extends Error {
52+
constructor(public readonly context: ChunkErrorContext) {
53+
super(
54+
`Failed to decode chunk (typeId: ${context.typeId}, index: ${context.chunkIndex}, offset: ${context.offset}) in version ${context.version}.`,
55+
{ cause: context.cause },
56+
);
57+
this.name = 'KHIChunkDecodeError';
58+
}
59+
}
60+
61+
/**
62+
* Context provided when the data assembly phase fails.
63+
*/
64+
export interface AssemblyErrorContext {
65+
readonly version: number;
66+
readonly typeId: number;
67+
readonly cause: unknown;
68+
}
69+
70+
/**
71+
* Thrown when an assembler fails to mutate the final model.
72+
*/
73+
export class KHIDataAssemblyError extends Error {
74+
constructor(public readonly context: AssemblyErrorContext) {
75+
super(
76+
`Failed to assemble data for chunk type ${context.typeId} in version ${context.version}.`,
77+
{ cause: context.cause },
78+
);
79+
this.name = 'KHIDataAssemblyError';
80+
}
81+
}

0 commit comments

Comments
 (0)