Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
47 changes: 23 additions & 24 deletions docs/en/task-system/03-advanced-and-form-tasks.md
Original file line number Diff line number Diff line change
Expand Up @@ -31,48 +31,47 @@ func Register(registry coreinspection.InspectionTaskRegistry) error {

On the "New Inspection" screen in KHI, the system dynamically determines which tasks to include and run in the graph based on the selected environment and log types. To control this behavior, you can attach special labels to inspection tasks.

### 2.1 Filtering with General Label Selectors (`LabelSelector`)
### 2.1 Filtering Tasks with `InspectionTypeLabelSelector`

In current KHI versions, you can attach arbitrary key-value metadata labels to tasks and filter them flexibly using **`LabelSelector`**, which evaluates boolean logic expressions (AND, OR, NOT, etc.) to enable tasks only in specific environments or modes.
In KHI, each `InspectionType` defines a set of key-value labels indicating its target environment, log source, and platform:

- `inspectioncore_contract.InspectionTypeLabelKeyEnvironment` (`"khi.google.com/environment"`)
- `inspectioncore_contract.InspectionTypeLabelKeyLogSource` (`"khi.google.com/log_source"`)
- `inspectioncore_contract.InspectionTypeLabelKeyBasePlatform` (`"khi.google.com/base_platform"`)

To restrict a task so that it only runs for compatible inspection types, attach an `InspectionTypeLabelSelector` label option specifying the required label key-value pairs:

```go
// Set task labels using the general LabelValue option
var AdvancedTask = task.NewTask(AdvancedTaskID, []taskid.UntypedTaskReference{}, func(ctx context.Context) (any, error) {
var AdvancedTask = coretask.NewTask(AdvancedTaskID, []taskid.UntypedTaskReference{}, func(ctx context.Context) (any, error) {
return nil, nil
},
coretask.LabelValue("environment", "gcp"),
coretask.LabelValue("feature-stage", "beta"),
inspectioncore_contract.InspectionTypeLabelSelector(map[string]string{
inspectioncore_contract.InspectionTypeLabelKeyEnvironment: "googlecloud",
inspectioncore_contract.InspectionTypeLabelKeyBasePlatform: "kubernetes",
}),
)
```

During server initialization or inspection configuration, KHI evaluates expressions like the following to select tasks:

```go
selector, _ := labelselector.Parse("environment=gcp && !feature-stage=deprecated")
compatibleTasks := taskSet.Select(selector)
```
When an inspection starts, the runner checks that all key-value pairs in the selector match the selected `InspectionType.Labels`. Tasks without an `InspectionTypeLabelSelector` are treated as global tasks and are included for all inspection types.

### 2.2 Legacy Inspection Type Labels (`InspectionTypeLabel`)

For backward compatibility, you can still use traditional `InspectionTypeLabel` declarations.
This enables the task only for the Inspection Types listed in the label (e.g., GCP Cloud Logging, local log files, etc.).
You can also apply an `InspectionTypeLabelSelector` to all tasks registered in a package by wrapping the registry with `coreinspection.NewScopedRegistry`:

```go
var MyTask = task.NewTask(MyTaskID, []taskid.UntypedTaskReference{}, func(ctx context.Context) (any, error) {
return nil, nil
}, inspectioncore_contract.InspectionTypeLabel(
"example.khi.google.com/inspection-type-1",
"example.khi.google.com/inspection-type-2",
))
func Register(registry coreinspection.InspectionTaskRegistry) error {
scoped := coreinspection.NewScopedRegistry(registry, inspectioncore_contract.InspectionTypeLabelSelector(map[string]string{
inspectioncore_contract.InspectionTypeLabelKeyEnvironment: "googlecloud",
}))
return coretask.RegisterTasks(scoped, TaskA, TaskB)
}
```

### 2.3 FeatureTask Labels
### 2.2 FeatureTask Labels

The FeatureTask label is a special label that exposes a task as a toggleable feature on KHI's "New Inspection" screen.
By specifying this label on main feature tasks such as mappers, you allow users to enable or disable the feature.

```go
inspectioncore_contract.FeatureTaskLabel("my-feature", "Feature label", "Detailed description of the feature", true, "gcp-gke")
inspectioncore_contract.FeatureTaskLabel("Feature label", "Detailed description of the feature", 1000, true)
```

## 3. Task Utilities for Discovering Information from Logs (`Inventory` and `Discovery` Tasks)
Expand Down
47 changes: 23 additions & 24 deletions docs/ja/task-system/03-advanced-and-form-tasks.md
Original file line number Diff line number Diff line change
Expand Up @@ -31,48 +31,47 @@ func Register(registry coreinspection.InspectionTaskRegistry) error {

KHI の「New Inspection」画面では、選択された環境やログ種別に応じて、どのタスクをグラフに含めて実行するかが動的に決定されます。これらを制御するために、インスペクションタスクには特別なラベルを付与できます。

### 2.1 汎用ラベルセレクタによるフィルタリング (`LabelSelector`)
### 2.1 `InspectionTypeLabelSelector` によるタスクの絞り込み

現在の KHI では、タスクに対して任意のキー・バリュー形式のメタデータラベルを付与し、それらをブール論理 (AND / OR / NOT 等) の式で表現した **`LabelSelector`** によって、特定の環境やモードでのみ有効化する柔軟なタスク絞り込みを行います。
KHI では、各 `InspectionType` が対象環境やログソース、プラットフォームを表すキー・バリュー形式のラベルを保持しています。代表的なキーは以下の通りです。

- `inspectioncore_contract.InspectionTypeLabelKeyEnvironment` (`"khi.google.com/environment"`)
- `inspectioncore_contract.InspectionTypeLabelKeyLogSource` (`"khi.google.com/log_source"`)
- `inspectioncore_contract.InspectionTypeLabelKeyBasePlatform` (`"khi.google.com/base_platform"`)

特定のインスペクションタイプでのみタスクを実行可能にするには、タスク定義時に `InspectionTypeLabelSelector` ラベルオプションを指定して必要なキー・バリューのペアを設定します。

```go
// 汎用の LabelValue オプションを利用したタスクラベル設定
var AdvancedTask = task.NewTask(AdvancedTaskID, []taskid.UntypedTaskReference{}, func(ctx context.Context) (any, error) {
var AdvancedTask = coretask.NewTask(AdvancedTaskID, []taskid.UntypedTaskReference{}, func(ctx context.Context) (any, error) {
return nil, nil
},
coretask.LabelValue("environment", "gcp"),
coretask.LabelValue("feature-stage", "beta"),
inspectioncore_contract.InspectionTypeLabelSelector(map[string]string{
inspectioncore_contract.InspectionTypeLabelKeyEnvironment: "googlecloud",
inspectioncore_contract.InspectionTypeLabelKeyBasePlatform: "kubernetes",
}),
)
```

これに対して、サーバー初期化時やインスペクション構成時に以下のような式を評価してタスクを抽出します:

```go
selector, _ := labelselector.Parse("environment=gcp && !feature-stage=deprecated")
compatibleTasks := taskSet.Select(selector)
```
インスペクション開始時、ランナーはセレクタに含まれるすべてのキー・バリューが選択された `InspectionType.Labels` に一致するかを検証します。`InspectionTypeLabelSelector` が指定されていないタスクはグローバルタスクとして扱われ、すべてのインスペクションタイプで利用可能になります。

### 2.2 レガシー Inspection Type ラベル (`InspectionTypeLabel`)

互換性のため、従来の `InspectionTypeLabel` も引き続き利用可能です。
このラベルにリストされている Inspection Type (例: GCP Cloud Logging, ローカルログファイル等) でのみタスクを有効化します。
また、パッケージ内で登録する全タスクに一括してセレクタを適用する場合は、`coreinspection.NewScopedRegistry` でレジストリをラップして登録できます。

```go
var MyTask = task.NewTask(MyTaskID, []taskid.UntypedTaskReference{}, func(ctx context.Context) (any, error) {
return nil, nil
}, inspectioncore_contract.InspectionTypeLabel(
"example.khi.google.com/inspection-type-1",
"example.khi.google.com/inspection-type-2",
))
func Register(registry coreinspection.InspectionTaskRegistry) error {
scoped := coreinspection.NewScopedRegistry(registry, inspectioncore_contract.InspectionTypeLabelSelector(map[string]string{
inspectioncore_contract.InspectionTypeLabelKeyEnvironment: "googlecloud",
}))
return coretask.RegisterTasks(scoped, TaskA, TaskB)
}
```

### 2.3 FeatureTask ラベル
### 2.2 FeatureTask ラベル

FeatureTask ラベルは、そのタスクを KHI の「New Inspection」画面におけるトグル可能な機能として公開するための特別なラベルです。
マッパータスクなどの主機能となるタスクに指定することで、ユーザーは機能の有効/無効を選択できます。

```go
inspectioncore_contract.FeatureTaskLabel("my-feature", "機能ラベル", "機能詳細の説明文", true, "gcp-gke")
inspectioncore_contract.FeatureTaskLabel("機能ラベル", "機能詳細の説明文", 1000, true)
```

## 3. ログから情報を発見するためのタスクユーティリティ (`Inventory` と `Discovery` タスク)
Expand Down
19 changes: 5 additions & 14 deletions pkg/core/inspection/runner.go
Original file line number Diff line number Diff line change
Expand Up @@ -184,7 +184,7 @@ func (i *InspectionTaskRunner) SetInspectionType(inspectionType string) error {

filteredTasks := []coretask.UntypedTask{}
for _, task := range i.inspectionServer.RootTaskSet.GetAll() {
if i.isTaskCompatible(task, currentType) {
if isTaskCompatible(task, currentType) {
filteredTasks = append(filteredTasks, task)
}
}
Expand All @@ -205,24 +205,15 @@ func (i *InspectionTaskRunner) SetInspectionType(inspectionType string) error {
return i.SetFeatureList(defaultFeatureIds)
}

func (i *InspectionTaskRunner) isTaskCompatible(task coretask.UntypedTask, currentType *InspectionType) bool {
func isTaskCompatible(task coretask.UntypedTask, inspectionType *InspectionType) bool {
labels := task.Labels()

// 1. Evaluate with new Label Selector if present
// 1. Evaluate with Label Selector if present.
if selector, ok := typedmap.Get(labels, inspectioncore_contract.LabelKeyInspectionTypeLabelSelector); ok {
return selector.Match(currentType.Labels)
return selector.Match(inspectionType.Labels)
}

// 2. Fallback to legacy list
if legacyList, ok := typedmap.Get(labels, inspectioncore_contract.LabelKeyInspectionTypes); ok {
if slices.Contains(legacyList, currentType.Id) {
slog.Warn("Legacy inspection type list is used for task. Please migrate to label-selector approach.", "taskID", task.UntypedID().String())
return true
}
return false
}

// 3. Defaults to true if neither is defined (global tasks)
// 2. Defaults to true if no selector is defined (global tasks).
return true
}

Expand Down
89 changes: 47 additions & 42 deletions pkg/core/inspection/runner_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -57,7 +57,6 @@ func TestInspectionTaskRunner_Interceptor(t *testing.T) {
func(ctx context.Context) (any, error) {
return "success", nil
},
coretask.WithLabelValue(inspectioncore_contract.LabelKeyInspectionTypes, []string{inspectionType.Id}),
coretask.WithLabelValue(inspectioncore_contract.LabelKeyInspectionDefaultFeatureFlag, true),
coretask.WithLabelValue(inspectioncore_contract.LabelKeyInspectionFeatureFlag, true),
coretask.NewSubsequentTaskRefsTaskLabel(inspectioncore_contract.SerializerTaskID.Ref()),
Expand Down Expand Up @@ -113,60 +112,77 @@ func TestInspectionTaskRunner_Interceptor(t *testing.T) {
}

func TestIsTaskCompatible(t *testing.T) {
runner := &InspectionTaskRunner{}

tests := []struct {
name string
taskLabels map[string]any // simplified setup
currentType *InspectionType
want bool
name string
labelOpts []coretask.LabelOpt
inspectionType *InspectionType
want bool
}{
{
name: "Selector matches target labels",
taskLabels: map[string]any{
inspectioncore_contract.LabelKeyInspectionTypeLabelSelector.Key(): inspectioncore_contract.LabelSelector{"platform": "gke"},
labelOpts: []coretask.LabelOpt{
inspectioncore_contract.InspectionTypeLabelSelector(inspectioncore_contract.LabelSelector{"platform": "gke"}),
},
currentType: &InspectionType{
inspectionType: &InspectionType{
Id: "some-env",
Labels: map[string]string{"platform": "gke", "provider": "google"},
},
want: true,
},
{
name: "Selector does not match target labels",
taskLabels: map[string]any{
inspectioncore_contract.LabelKeyInspectionTypeLabelSelector.Key(): inspectioncore_contract.LabelSelector{"platform": "gke"},
name: "Selector does not match target labels due to value mismatch",
labelOpts: []coretask.LabelOpt{
inspectioncore_contract.InspectionTypeLabelSelector(inspectioncore_contract.LabelSelector{"platform": "gke"}),
},
currentType: &InspectionType{
inspectionType: &InspectionType{
Id: "some-env",
Labels: map[string]string{"platform": "gdc"},
},
want: false,
},
{
name: "Fallback to legacy list - match",
taskLabels: map[string]any{
inspectioncore_contract.LabelKeyInspectionTypes.Key(): []string{"legacy-env", "other-env"},
name: "Selector does not match target labels due to missing key in target",
labelOpts: []coretask.LabelOpt{
inspectioncore_contract.InspectionTypeLabelSelector(inspectioncore_contract.LabelSelector{"platform": "gke"}),
},
currentType: &InspectionType{
Id: "legacy-env",
inspectionType: &InspectionType{
Id: "some-env",
Labels: map[string]string{"provider": "google"},
},
want: false,
},
{
name: "Multi-key selector matches when all keys match",
labelOpts: []coretask.LabelOpt{
inspectioncore_contract.InspectionTypeLabelSelector(inspectioncore_contract.LabelSelector{
"platform": "gke",
"provider": "google",
}),
},
inspectionType: &InspectionType{
Id: "some-env",
Labels: map[string]string{"platform": "gke", "provider": "google", "region": "us-central1"},
},
want: true,
},
{
name: "Fallback to legacy list - no match",
taskLabels: map[string]any{
inspectioncore_contract.LabelKeyInspectionTypes.Key(): []string{"other-env"},
name: "Multi-key selector fails when only some keys match",
labelOpts: []coretask.LabelOpt{
inspectioncore_contract.InspectionTypeLabelSelector(inspectioncore_contract.LabelSelector{
"platform": "gke",
"provider": "aws",
}),
},
currentType: &InspectionType{
Id: "legacy-env",
inspectionType: &InspectionType{
Id: "some-env",
Labels: map[string]string{"platform": "gke", "provider": "google"},
},
want: false,
},
{
name: "No selector, no legacy list (Global task)",
taskLabels: map[string]any{},
currentType: &InspectionType{
name: "No selector (Global task)",
labelOpts: []coretask.LabelOpt{},
inspectionType: &InspectionType{
Id: "any-env",
},
want: true,
Expand All @@ -175,26 +191,16 @@ func TestIsTaskCompatible(t *testing.T) {

for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
// Create task with labels
opts := []coretask.LabelOpt{}
for k, v := range tt.taskLabels {
if k == inspectioncore_contract.LabelKeyInspectionTypeLabelSelector.Key() {
opts = append(opts, inspectioncore_contract.InspectionTypeLabelSelector(v.(inspectioncore_contract.LabelSelector)))
} else if k == inspectioncore_contract.LabelKeyInspectionTypes.Key() {
opts = append(opts, coretask.WithLabelValue(inspectioncore_contract.LabelKeyInspectionTypes, v.([]string)))
}
}

task := coretask.NewTask(
taskid.NewDefaultImplementationID[any]("test-task"),
nil,
func(ctx context.Context) (any, error) { return nil, nil },
opts...,
tt.labelOpts...,
)

got := runner.isTaskCompatible(task, tt.currentType)
if diff := cmp.Diff(tt.want, got); diff != "" {
t.Errorf("isTaskCompatible() mismatch (-want +got):\n%s", diff)
got := isTaskCompatible(task, tt.inspectionType)
if got != tt.want {
t.Errorf("isTaskCompatible() = %v, want %v", got, tt.want)
}
})
}
Expand Down Expand Up @@ -481,7 +487,6 @@ func TestInspectionTaskRunner_Cancel(t *testing.T) {
}
return "success", nil
},
coretask.WithLabelValue(inspectioncore_contract.LabelKeyInspectionTypes, []string{inspectionType.Id}),
coretask.WithLabelValue(inspectioncore_contract.LabelKeyInspectionDefaultFeatureFlag, true),
coretask.WithLabelValue(inspectioncore_contract.LabelKeyInspectionFeatureFlag, true),
coretask.NewSubsequentTaskRefsTaskLabel(inspectioncore_contract.SerializerTaskID.Ref()),
Expand Down
2 changes: 1 addition & 1 deletion pkg/core/task/label.go
Original file line number Diff line number Diff line change
Expand Up @@ -94,7 +94,7 @@ func (r *requiredTaskLabelImpl) Write(label *typedmap.TypedMap) {
typedmap.Set(label, LabelKeyRequiredTask, true)
}

// InspectionTypeLabel returns a LabelOpt to mark the task is always included in the result task graph.
// NewRequiredTaskLabel returns a LabelOpt to mark the task is always included in the result task graph.
func NewRequiredTaskLabel() *requiredTaskLabelImpl {
return &requiredTaskLabelImpl{}
}
Expand Down
1 change: 0 additions & 1 deletion pkg/server/api/v1/workbench_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -62,7 +62,6 @@ func createTestInspectionServerForWorkbench(t *testing.T) (*coreinspection.Inspe
func(ctx context.Context) (any, error) {
return "success", nil
},
coretask.WithLabelValue(inspectioncore_contract.LabelKeyInspectionTypes, []string{inspectionType.Id}),
coretask.WithLabelValue(inspectioncore_contract.LabelKeyInspectionDefaultFeatureFlag, true),
coretask.WithLabelValue(inspectioncore_contract.LabelKeyInspectionFeatureFlag, true),
coretask.NewSubsequentTaskRefsTaskLabel(inspectioncore_contract.SerializerTaskID.Ref()),
Expand Down
1 change: 0 additions & 1 deletion pkg/server/workbench/manager_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -58,7 +58,6 @@ func createTestInspectionServer(t *testing.T) (*coreinspection.InspectionTaskSer
func(ctx context.Context) (any, error) {
return "success", nil
},
coretask.WithLabelValue(inspectioncore_contract.LabelKeyInspectionTypes, []string{inspectionType.Id}),
coretask.WithLabelValue(inspectioncore_contract.LabelKeyInspectionDefaultFeatureFlag, true),
coretask.WithLabelValue(inspectioncore_contract.LabelKeyInspectionFeatureFlag, true),
coretask.NewSubsequentTaskRefsTaskLabel(inspectioncore_contract.SerializerTaskID.Ref()),
Expand Down
Loading
Loading