Skip to content

fix(k8saudit): preserve immutable identity metadata after truncated logs - #979

Merged
kyasbal merged 2 commits into
GoogleCloudPlatform:mainfrom
kyasbal:push-srmzyxyptort
Sep 10, 2026
Merged

fix(k8saudit): preserve immutable identity metadata after truncated logs#979
kyasbal merged 2 commits into
GoogleCloudPlatform:mainfrom
kyasbal:push-srmzyxyptort

Conversation

@kyasbal

@kyasbal kyasbal commented Sep 10, 2026

Copy link
Copy Markdown
Member

Summary

This PR fixes an issue where Kubernetes audit logs with truncated responses (audit.k8s.io/truncated: "true") caused the manifest generator to lose resource identity metadata, leading subsequent patch logs to be misinterpreted as newly created resources.

  • Manifest Generator:
    • Preserves immutable identity metadata (apiVersion, kind, metadata.name, metadata.namespace, metadata.uid, metadata.creationTimestamp) from the previous revision when processing truncated logs, rather than resetting to an empty reader.
    • Returns the preserved immutable identity reader as ResourceBodyReader on truncated logs (instead of nil), ensuring the truncated log itself retains its immutable identity manifest in the timeline revision and does not appear empty in the UI.
  • Nil Revision Handling:
    • Avoids eager empty map initialization at the start of Process(), maintaining prevRevisionReader as nil until a revision is observed, and only extracting identity metadata when a prior revision actually exists.
    • Safely falls back to an empty map node if prevRevisionReader is nil prior to a patch merge.
  • Unit & Integration Tests:
    • Added unit tests in TestExtractResourceIdentity verifying field extraction for empty maps, non-identity fields, type-only metadata, full identity metadata, and cluster-scoped resources.
    • Added and updated test cases in TestGroupManifestGenerator verifying that immutable identity metadata is preserved on truncated logs and across single/consecutive truncated logs into subsequent patches, that initial truncated logs merge patches cleanly into empty bodies, and that DeleteOptions after truncated logs return preserved identity readers.

When Kubernetes audit logs contain truncated responses (audit.k8s.io/truncated: "true"),
groupManifestGenerator reset prevRevisionReader to nil. Subsequent patch requests merged
into an empty map reader, causing metadata.uid to be lost. Later tasks interpreted the
missing UID as a new resource creation, incorrectly emitting a ChangeEventTypeCreation
event and overwriting the initial revision at creationTimestamp.

This commit updates manifest_generator_task to preserve immutable identity metadata
(apiVersion, kind, metadata.name, metadata.namespace, metadata.uid, metadata.creationTimestamp)
upon encountering truncated logs so that subsequent patches retain the resource identity.
@kyasbal
kyasbal requested a review from RyuSA as a code owner September 10, 2026 06:26
@kyasbal kyasbal added type:bug Something isn't working as expected area:backend-parser Log parsers and log timeline mapper tasks labels Sep 10, 2026

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request modifies the manifest generator task to preserve immutable resource identity metadata (such as apiVersion, kind, name, namespace, uid, and creationTimestamp) when handling truncated logs, rather than completely clearing the previous revision. It introduces a new helper function extractResourceIdentity and adds comprehensive unit tests to verify this behavior. However, a critical compilation error was identified: the variables pathMetadataUID and pathMetadataCreationTimestamp are referenced in the new helper function but are not defined in the global variable block.

Comment on lines +43 to +47
pathAPIVersion = structured.CompileFieldPath("apiVersion")
pathKind = structured.CompileFieldPath("kind")
pathItems = structured.CompileFieldPath("items")
pathMetadataName = structured.CompileFieldPath("metadata.name")
pathMetadataNamespace = structured.CompileFieldPath("metadata.namespace")

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

critical

The variables pathMetadataUID and pathMetadataCreationTimestamp are used in extractResourceIdentity but are not defined in the global var block. This will cause a compilation error. Please define them here.

	pathAPIVersion                = structured.CompileFieldPath("apiVersion")
	pathKind                      = structured.CompileFieldPath("kind")
	pathItems                     = structured.CompileFieldPath("items")
	pathMetadataName              = structured.CompileFieldPath("metadata.name")
	pathMetadataNamespace         = structured.CompileFieldPath("metadata.namespace")
	pathMetadataUID               = structured.CompileFieldPath("metadata.uid")
	pathMetadataCreationTimestamp = structured.CompileFieldPath("metadata.creationTimestamp")

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you for the review. Both pathMetadataUID and pathMetadataCreationTimestamp are package-level variables defined in resource_helpers.go (lines 30 and 33) within the same package (commonlogk8saudit_impl).

In Go, package-level identifiers are shared across all files within the same package, so they are accessible here without compilation error. Declaring them again in manifest_generator_task.go would cause a Go compilation error (redeclared in this block).

…ed log body

When a log is truncated, groupManifestGenerator now returns the preserved immutable
identity reader as ResourceBodyReader rather than nil. This allows the truncated log
itself to retain its immutable identity metadata in the generated timeline revision,
preventing the resource body from appearing completely empty in the UI and downstream
tasks.
@kyasbal
kyasbal added this pull request to the merge queue Sep 10, 2026
Merged via the queue into GoogleCloudPlatform:main with commit 5fa939f Sep 10, 2026
17 checks passed
@kyasbal
kyasbal deleted the push-srmzyxyptort branch September 10, 2026 06:48
kyasbal added a commit that referenced this pull request Sep 11, 2026
* chore(github): reorganize issue and PR label taxonomy (#971)

* chore(github): reorganize issue and PR label taxonomy

Reorganize GitHub issue and PR label system to improve discoverability,
triage ergonomics, and release automation alignment:
- Update .github/release.yml changelog categories to use new label names (type:feature, type:bug, type:perf, type:refactor, area:cicd, area:devenv, type:docs, type:chore).
- Configure .github/ISSUE_TEMPLATE/bug-report.md with default labels "type:bug, status:needs-triage".
- Configure .github/ISSUE_TEMPLATE/feature-request.md with default labels "type:feature, status:needs-triage".

* fix(github): separate maintenance and dependencies categories in release notes

* feat(k8sevent): replace resource UIDs with readable names in event summaries (#973)

* feat(k8sevent): replace resource UIDs with readable names in event summaries

- Add ResourceIdentity.SummaryTag() method to format resources as readable tags
- Replace matched resource UIDs in Kubernetes event messages with readable tags in both GKE and OSS log ingesters
- Map matched resource UIDs to corresponding resource timelines in ProcessLogByGroup
- Add HasEventCount assertion helper to TimelineChangeSetAsserter

* fix(review): address review comments on deduplicating formatEventSummary and HasEventCount

* docs(agent): clarify cmp.Diff usage and test assertion examples in go-coding-rule (#974)

* docs(agent): clarify cmp.Diff usage and test assertion examples in go-coding-rule

* fix(agent): address review comments on collections comparison in go-coding-rule

* feat(googlecloudk8scommon): default kind filter to all kinds except leases (#975)

* feat(timeline): allow inline editing of search filter chips on click (#977)

* feat(timeline): allow inline editing of search filter chips on click

Enable in-place editing of search filter chips in ChipSearchBarComponent
when clicked. Clicking a chip renders an inline text input with text
pre-selected and auto-focused. Changes can be committed with Enter or blur,
or cancelled with Escape. Delimiter splitting and whitespace removal are
supported.

* fix(review): address review comments on blur race condition and focus preservation

* fix(inspection): resolve timezone offset fallback to UTC by unifying to timezoneShiftHours (#978)

Following the Connect-RPC migration (#908), the timezone shift parameter
was updated in Protobuf and backend RPC handlers to timezoneShiftHours
(float64). However, TimeZoneShiftInputTask and the frontend InspectionClient
retained references to the legacy timezoneShift key, causing inspections
to always fall back to UTC time.

This change:
- Unifies the context key to TaskInputKeyTimezoneShiftHours in inspectioncore
- Updates TimeZoneShiftInputTask to read TaskInputKeyTimezoneShiftHours
- Updates backend-api.service.ts and its test to use timezoneShiftHours
- Updates parser_test.go to float64(9)
- Adds unit tests for TimeZoneShiftInputTask

* fix(k8saudit): preserve immutable identity metadata after truncated logs (#979)

* fix(k8saudit): preserve immutable identity metadata after truncated logs

When Kubernetes audit logs contain truncated responses (audit.k8s.io/truncated: "true"),
groupManifestGenerator reset prevRevisionReader to nil. Subsequent patch requests merged
into an empty map reader, causing metadata.uid to be lost. Later tasks interpreted the
missing UID as a new resource creation, incorrectly emitting a ChangeEventTypeCreation
event and overwriting the initial revision at creationTimestamp.

This commit updates manifest_generator_task to preserve immutable identity metadata
(apiVersion, kind, metadata.name, metadata.namespace, metadata.uid, metadata.creationTimestamp)
upon encountering truncated logs so that subsequent patches retain the resource identity.

* fix(k8saudit): return preserved immutable identity reader for truncated log body

When a log is truncated, groupManifestGenerator now returns the preserved immutable
identity reader as ResourceBodyReader rather than nil. This allows the truncated log
itself to retain its immutable identity metadata in the generated timeline revision,
preventing the resource body from appearing completely empty in the UI and downstream
tasks.

* feat(cicd): support manual tag and release draft generation via workflow_dispatch (#980)

* feat(cicd): support manual tag creation and release draft generation via workflow_dispatch

Add a workflow_dispatch trigger to .github/workflows/release.yaml, allowing
maintainers to initiate tag creation and release draft generation directly from
the GitHub Actions Web UI without local terminal operations.

- Accept a required tag_name input parameter (e.g., v1.2.3 or v1.2.3-beta-1)
- Validate tag format and verify uniqueness via git ls-remote before starting builds
- Create and push the Git tag after successful binary compilation
- Automatically generate release notes from categorised commit history

* fix(cicd): pass step outputs via env to prevent template injection

* refactor(inspection): remove legacy LabelKeyInspectionTypes and InspectionTypeLabel (#981)

Remove deprecated LabelKeyInspectionTypes, InspectionTypeLabel, and
legacy fallback logic from inspection runner. Update tests and
documentation to use InspectionTypeLabelSelector.

* feat(inspection): set default environment name in composer form (#982)

Add WithDefaultValueFunc to InputComposerEnvironmentNameTask so that
when Composer environments are fetched via autocomplete, the first
environment is automatically selected as the default value if no
previous selection was made.

---------

Co-authored-by: kept1994 <keigof@google.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area:backend-parser Log parsers and log timeline mapper tasks type:bug Something isn't working as expected

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant