fix(k8saudit): preserve immutable identity metadata after truncated logs - #979
Conversation
When Kubernetes audit logs contain truncated responses (audit.k8s.io/truncated: "true"), groupManifestGenerator reset prevRevisionReader to nil. Subsequent patch requests merged into an empty map reader, causing metadata.uid to be lost. Later tasks interpreted the missing UID as a new resource creation, incorrectly emitting a ChangeEventTypeCreation event and overwriting the initial revision at creationTimestamp. This commit updates manifest_generator_task to preserve immutable identity metadata (apiVersion, kind, metadata.name, metadata.namespace, metadata.uid, metadata.creationTimestamp) upon encountering truncated logs so that subsequent patches retain the resource identity.
There was a problem hiding this comment.
Code Review
This pull request modifies the manifest generator task to preserve immutable resource identity metadata (such as apiVersion, kind, name, namespace, uid, and creationTimestamp) when handling truncated logs, rather than completely clearing the previous revision. It introduces a new helper function extractResourceIdentity and adds comprehensive unit tests to verify this behavior. However, a critical compilation error was identified: the variables pathMetadataUID and pathMetadataCreationTimestamp are referenced in the new helper function but are not defined in the global variable block.
| pathAPIVersion = structured.CompileFieldPath("apiVersion") | ||
| pathKind = structured.CompileFieldPath("kind") | ||
| pathItems = structured.CompileFieldPath("items") | ||
| pathMetadataName = structured.CompileFieldPath("metadata.name") | ||
| pathMetadataNamespace = structured.CompileFieldPath("metadata.namespace") |
There was a problem hiding this comment.
The variables pathMetadataUID and pathMetadataCreationTimestamp are used in extractResourceIdentity but are not defined in the global var block. This will cause a compilation error. Please define them here.
pathAPIVersion = structured.CompileFieldPath("apiVersion")
pathKind = structured.CompileFieldPath("kind")
pathItems = structured.CompileFieldPath("items")
pathMetadataName = structured.CompileFieldPath("metadata.name")
pathMetadataNamespace = structured.CompileFieldPath("metadata.namespace")
pathMetadataUID = structured.CompileFieldPath("metadata.uid")
pathMetadataCreationTimestamp = structured.CompileFieldPath("metadata.creationTimestamp")There was a problem hiding this comment.
Thank you for the review. Both pathMetadataUID and pathMetadataCreationTimestamp are package-level variables defined in resource_helpers.go (lines 30 and 33) within the same package (commonlogk8saudit_impl).
In Go, package-level identifiers are shared across all files within the same package, so they are accessible here without compilation error. Declaring them again in manifest_generator_task.go would cause a Go compilation error (redeclared in this block).
…ed log body When a log is truncated, groupManifestGenerator now returns the preserved immutable identity reader as ResourceBodyReader rather than nil. This allows the truncated log itself to retain its immutable identity metadata in the generated timeline revision, preventing the resource body from appearing completely empty in the UI and downstream tasks.
5fa939f
* chore(github): reorganize issue and PR label taxonomy (#971) * chore(github): reorganize issue and PR label taxonomy Reorganize GitHub issue and PR label system to improve discoverability, triage ergonomics, and release automation alignment: - Update .github/release.yml changelog categories to use new label names (type:feature, type:bug, type:perf, type:refactor, area:cicd, area:devenv, type:docs, type:chore). - Configure .github/ISSUE_TEMPLATE/bug-report.md with default labels "type:bug, status:needs-triage". - Configure .github/ISSUE_TEMPLATE/feature-request.md with default labels "type:feature, status:needs-triage". * fix(github): separate maintenance and dependencies categories in release notes * feat(k8sevent): replace resource UIDs with readable names in event summaries (#973) * feat(k8sevent): replace resource UIDs with readable names in event summaries - Add ResourceIdentity.SummaryTag() method to format resources as readable tags - Replace matched resource UIDs in Kubernetes event messages with readable tags in both GKE and OSS log ingesters - Map matched resource UIDs to corresponding resource timelines in ProcessLogByGroup - Add HasEventCount assertion helper to TimelineChangeSetAsserter * fix(review): address review comments on deduplicating formatEventSummary and HasEventCount * docs(agent): clarify cmp.Diff usage and test assertion examples in go-coding-rule (#974) * docs(agent): clarify cmp.Diff usage and test assertion examples in go-coding-rule * fix(agent): address review comments on collections comparison in go-coding-rule * feat(googlecloudk8scommon): default kind filter to all kinds except leases (#975) * feat(timeline): allow inline editing of search filter chips on click (#977) * feat(timeline): allow inline editing of search filter chips on click Enable in-place editing of search filter chips in ChipSearchBarComponent when clicked. Clicking a chip renders an inline text input with text pre-selected and auto-focused. Changes can be committed with Enter or blur, or cancelled with Escape. Delimiter splitting and whitespace removal are supported. * fix(review): address review comments on blur race condition and focus preservation * fix(inspection): resolve timezone offset fallback to UTC by unifying to timezoneShiftHours (#978) Following the Connect-RPC migration (#908), the timezone shift parameter was updated in Protobuf and backend RPC handlers to timezoneShiftHours (float64). However, TimeZoneShiftInputTask and the frontend InspectionClient retained references to the legacy timezoneShift key, causing inspections to always fall back to UTC time. This change: - Unifies the context key to TaskInputKeyTimezoneShiftHours in inspectioncore - Updates TimeZoneShiftInputTask to read TaskInputKeyTimezoneShiftHours - Updates backend-api.service.ts and its test to use timezoneShiftHours - Updates parser_test.go to float64(9) - Adds unit tests for TimeZoneShiftInputTask * fix(k8saudit): preserve immutable identity metadata after truncated logs (#979) * fix(k8saudit): preserve immutable identity metadata after truncated logs When Kubernetes audit logs contain truncated responses (audit.k8s.io/truncated: "true"), groupManifestGenerator reset prevRevisionReader to nil. Subsequent patch requests merged into an empty map reader, causing metadata.uid to be lost. Later tasks interpreted the missing UID as a new resource creation, incorrectly emitting a ChangeEventTypeCreation event and overwriting the initial revision at creationTimestamp. This commit updates manifest_generator_task to preserve immutable identity metadata (apiVersion, kind, metadata.name, metadata.namespace, metadata.uid, metadata.creationTimestamp) upon encountering truncated logs so that subsequent patches retain the resource identity. * fix(k8saudit): return preserved immutable identity reader for truncated log body When a log is truncated, groupManifestGenerator now returns the preserved immutable identity reader as ResourceBodyReader rather than nil. This allows the truncated log itself to retain its immutable identity metadata in the generated timeline revision, preventing the resource body from appearing completely empty in the UI and downstream tasks. * feat(cicd): support manual tag and release draft generation via workflow_dispatch (#980) * feat(cicd): support manual tag creation and release draft generation via workflow_dispatch Add a workflow_dispatch trigger to .github/workflows/release.yaml, allowing maintainers to initiate tag creation and release draft generation directly from the GitHub Actions Web UI without local terminal operations. - Accept a required tag_name input parameter (e.g., v1.2.3 or v1.2.3-beta-1) - Validate tag format and verify uniqueness via git ls-remote before starting builds - Create and push the Git tag after successful binary compilation - Automatically generate release notes from categorised commit history * fix(cicd): pass step outputs via env to prevent template injection * refactor(inspection): remove legacy LabelKeyInspectionTypes and InspectionTypeLabel (#981) Remove deprecated LabelKeyInspectionTypes, InspectionTypeLabel, and legacy fallback logic from inspection runner. Update tests and documentation to use InspectionTypeLabelSelector. * feat(inspection): set default environment name in composer form (#982) Add WithDefaultValueFunc to InputComposerEnvironmentNameTask so that when Composer environments are fetched via autocomplete, the first environment is automatically selected as the default value if no previous selection was made. --------- Co-authored-by: kept1994 <keigof@google.com>
Summary
This PR fixes an issue where Kubernetes audit logs with truncated responses (
audit.k8s.io/truncated: "true") caused the manifest generator to lose resource identity metadata, leading subsequent patch logs to be misinterpreted as newly created resources.apiVersion,kind,metadata.name,metadata.namespace,metadata.uid,metadata.creationTimestamp) from the previous revision when processing truncated logs, rather than resetting to an empty reader.ResourceBodyReaderon truncated logs (instead ofnil), ensuring the truncated log itself retains its immutable identity manifest in the timeline revision and does not appear empty in the UI.Process(), maintainingprevRevisionReaderasniluntil a revision is observed, and only extracting identity metadata when a prior revision actually exists.prevRevisionReaderisnilprior to a patch merge.TestExtractResourceIdentityverifying field extraction for empty maps, non-identity fields, type-only metadata, full identity metadata, and cluster-scoped resources.TestGroupManifestGeneratorverifying that immutable identity metadata is preserved on truncated logs and across single/consecutive truncated logs into subsequent patches, that initial truncated logs merge patches cleanly into empty bodies, and that DeleteOptions after truncated logs return preserved identity readers.