Skip to content

feat: azure maintenance events #382

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
jtschelling wants to merge 23 commits into
base: main
Choose a base branch
from
Open

feat: azure maintenance events #382

Show file tree
Hide file tree
Changes from 18 commits
Commits
Show all changes
23 commits
Select commit Hold shift + click to select a range
0779af6
feat: azure csp-health-monitor skeleton
jtschelling Nov 17, 2025
777a141
feat: get an azure VMClient
jtschelling Nov 18, 2025
d3a6e4b
feat: add mise to gitignore
jtschelling Nov 18, 2025
a50b21a
feat: start implementing StartMonitoring with cursor
jtschelling Nov 18, 2025
c2f84bc
feat: implement normalizer
jtschelling Nov 19, 2025
fe02771
feat: import maintenance pkg
jtschelling Nov 19, 2025
b467ba2
feat: use maintenance api instead of vm client
jtschelling Nov 19, 2025
1cbc5aa
fix: remove vmClient
jtschelling Nov 20, 2025
d2b5c59
feat: clean up
jtschelling Nov 20, 2025
2dd91af
fix: var name
jtschelling Nov 20, 2025
b9362f9
feat: unit test happy path no maintenance updates
jtschelling Nov 20, 2025
403d72f
ci: add test for a maintenance event to push to the channel
jtschelling Nov 20, 2025
681da0a
ci: use testify
jtschelling Nov 20, 2025
b2a6b10
ci: unit test azure normalize
jtschelling Nov 20, 2025
29d0490
fix: break up the large poll func
jtschelling Nov 20, 2025
783679a
ci: lint
jtschelling Nov 20, 2025
dc50464
ci: go mod tidy
jtschelling Nov 20, 2025
6244c8a
Merge branch 'main' into jtschelling/124-azure-maintenance-events
jtschelling Nov 20, 2025
bbb25fd
fix: go mod tidy
jtschelling Dec 1, 2025
4c8415e
fix: clean up bad rebase
jtschelling Dec 1, 2025
cbfffc7
fix: respond to pr comments
jtschelling Dec 1, 2025
cfebb54
feat: get informer working and use envtest
jtschelling Dec 2, 2025
e2888b3
fix: lint
jtschelling Dec 2, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 7 additions & 1 deletion .gitignore
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,12 @@
# Language-Specific Build Artifacts & Dependencies
# ============================================================================

### Version Managers ###
mise.toml

### Version Managers ###
mise.toml

### Go ###
# Binaries for programs and plugins
*.exe
Expand Down Expand Up @@ -451,4 +457,4 @@ labeler/labeler
metadata-collector/metadata-collector
node-drainer/node-drainer
platform-connectors/platform-connectors
event-exporter/event-exporter
event-exporter/event-exporter
20 changes: 19 additions & 1 deletion health-monitors/csp-health-monitor/cmd/csp-health-monitor/main.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -34,6 +34,7 @@ import (
"github.com/nvidia/nvsentinel/health-monitors/csp-health-monitor/pkg/config"
"github.com/nvidia/nvsentinel/health-monitors/csp-health-monitor/pkg/csp"
awsclient "github.com/nvidia/nvsentinel/health-monitors/csp-health-monitor/pkg/csp/aws"
azureclient "github.com/nvidia/nvsentinel/health-monitors/csp-health-monitor/pkg/csp/azure"
gcpclient "github.com/nvidia/nvsentinel/health-monitors/csp-health-monitor/pkg/csp/gcp"
"github.com/nvidia/nvsentinel/health-monitors/csp-health-monitor/pkg/datastore"
eventpkg "github.com/nvidia/nvsentinel/health-monitors/csp-health-monitor/pkg/event"
Expand Down Expand Up @@ -256,7 +257,24 @@ func initActiveMonitor(
return awsMonitor
}

slog.Info("No CSP is explicitly enabled in the configuration (GCP or AWS).")
if cfg.Azure.Enabled {
slog.Info("Azure configuration is enabled.")

azureMonitor, err := azureclient.NewClient(ctx, cfg.Azure, cfg.ClusterName, kubeconfigPath)
if err != nil {
metrics.CSPMonitorErrors.WithLabelValues(string(model.CSPAzure), "init_error").Inc()
slog.Error("Failed to initialize Azure monitor. Azure will not be monitored.", "error", err)

return nil
}

slog.Info("Azure monitor initialized",
"subscriptionID", cfg.Azure.SubscriptionID)

return azureMonitor
}

slog.Info("No CSP is explicitly enabled in the configuration (GCP, AWS, or Azure).")

return nil
}
Expand Down
74 changes: 43 additions & 31 deletions health-monitors/csp-health-monitor/go.mod
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -7,49 +7,57 @@ toolchain go1.25.3
require (
cloud.google.com/go/compute v1.38.0
cloud.google.com/go/logging v1.13.1
github.com/Azure/azure-sdk-for-go/sdk/azcore v1.20.0
github.com/Azure/azure-sdk-for-go/sdk/azidentity v1.13.0
github.com/Azure/azure-sdk-for-go/sdk/resourcemanager/maintenance/armmaintenance v1.3.0
github.com/Azure/azure-sdk-for-go/sdk/azcore v1.20.0
github.com/Azure/azure-sdk-for-go/sdk/azidentity v1.13.0
github.com/Azure/azure-sdk-for-go/sdk/resourcemanager/maintenance/armmaintenance v1.3.0
github.com/BurntSushi/toml v1.5.0
github.com/aws/aws-sdk-go-v2 v1.40.0
github.com/aws/aws-sdk-go-v2/config v1.32.1
github.com/aws/aws-sdk-go-v2/service/health v1.35.1
github.com/aws/aws-sdk-go-v2 v1.39.6
github.com/aws/aws-sdk-go-v2/config v1.31.18
github.com/aws/aws-sdk-go-v2/service/health v1.34.10
github.com/hashicorp/go-multierror v1.1.1
github.com/nvidia/nvsentinel/commons v0.0.0
github.com/nvidia/nvsentinel/data-models v0.0.0
github.com/nvidia/nvsentinel/store-client v0.0.0
github.com/prometheus/client_golang v1.23.2
github.com/stretchr/testify v1.11.1
golang.org/x/sync v0.18.0
google.golang.org/api v0.256.0
google.golang.org/api v0.254.0
google.golang.org/genproto v0.0.0-20250603155806-513f23925822
google.golang.org/genproto/googleapis/api v0.0.0-20251111163417-95abcf5c77ba
google.golang.org/genproto/googleapis/rpc v0.0.0-20251111163417-95abcf5c77ba
google.golang.org/genproto/googleapis/api v0.0.0-20250818200422-3122310a409c
google.golang.org/genproto/googleapis/rpc v0.0.0-20251022142026-3a174f9686a8
google.golang.org/grpc v1.76.0
google.golang.org/protobuf v1.36.10
k8s.io/api v0.34.2
k8s.io/apimachinery v0.34.2
k8s.io/client-go v0.34.2
k8s.io/api v0.34.1
k8s.io/apimachinery v0.34.1
k8s.io/client-go v0.34.1
sigs.k8s.io/controller-runtime v0.22.4
)

require (
cel.dev/expr v0.25.1 // indirect
cloud.google.com/go v0.121.6 // indirect
cloud.google.com/go/auth v0.17.0 // indirect
cloud.google.com/go/auth/oauth2adapt v0.2.8 // indirect
cloud.google.com/go/compute/metadata v0.9.0 // indirect
cloud.google.com/go/iam v1.5.2 // indirect
cloud.google.com/go/longrunning v0.6.7 // indirect
github.com/Azure/azure-sdk-for-go/sdk/internal v1.11.2 // indirect
github.com/AzureAD/microsoft-authentication-library-for-go v1.6.0 // indirect
github.com/Azure/azure-sdk-for-go/sdk/internal v1.11.2 // indirect
github.com/AzureAD/microsoft-authentication-library-for-go v1.6.0 // indirect
github.com/Masterminds/semver/v3 v3.4.0 // indirect
github.com/aws/aws-sdk-go-v2/credentials v1.19.1 // indirect
github.com/aws/aws-sdk-go-v2/feature/ec2/imds v1.18.14 // indirect
github.com/aws/aws-sdk-go-v2/internal/configsources v1.4.14 // indirect
github.com/aws/aws-sdk-go-v2/internal/endpoints/v2 v2.7.14 // indirect
github.com/aws/aws-sdk-go-v2/credentials v1.18.22 // indirect
github.com/aws/aws-sdk-go-v2/feature/ec2/imds v1.18.13 // indirect
github.com/aws/aws-sdk-go-v2/internal/configsources v1.4.13 // indirect
github.com/aws/aws-sdk-go-v2/internal/endpoints/v2 v2.7.13 // indirect
github.com/aws/aws-sdk-go-v2/internal/ini v1.8.4 // indirect
github.com/aws/aws-sdk-go-v2/service/internal/accept-encoding v1.13.3 // indirect
github.com/aws/aws-sdk-go-v2/service/internal/presigned-url v1.13.14 // indirect
github.com/aws/aws-sdk-go-v2/service/signin v1.0.1 // indirect
github.com/aws/aws-sdk-go-v2/service/sso v1.30.4 // indirect
github.com/aws/aws-sdk-go-v2/service/ssooidc v1.35.9 // indirect
github.com/aws/aws-sdk-go-v2/service/sts v1.41.1 // indirect
github.com/aws/aws-sdk-go-v2/service/internal/presigned-url v1.13.13 // indirect
github.com/aws/aws-sdk-go-v2/service/sso v1.30.1 // indirect
github.com/aws/aws-sdk-go-v2/service/ssooidc v1.35.5 // indirect
github.com/aws/aws-sdk-go-v2/service/sts v1.40.0 // indirect
github.com/aws/smithy-go v1.23.2 // indirect
github.com/beorn7/perks v1.0.1 // indirect
github.com/cespare/xxhash/v2 v2.3.0 // indirect
Expand All @@ -61,7 +69,7 @@ require (
github.com/go-logr/logr v1.4.3 // indirect
github.com/go-logr/stdr v1.2.2 // indirect
github.com/go-openapi/jsonpointer v0.22.1 // indirect
github.com/go-openapi/jsonreference v0.21.3 // indirect
github.com/go-openapi/jsonreference v0.21.2 // indirect
github.com/go-openapi/swag v0.25.1 // indirect
github.com/go-openapi/swag/cmdutils v0.25.1 // indirect
github.com/go-openapi/swag/conv v0.25.1 // indirect
Expand All @@ -75,33 +83,37 @@ require (
github.com/go-openapi/swag/typeutils v0.25.1 // indirect
github.com/go-openapi/swag/yamlutils v0.25.1 // indirect
github.com/gogo/protobuf v1.3.2 // indirect
github.com/golang-jwt/jwt/v5 v5.3.0 // indirect
github.com/golang-jwt/jwt/v5 v5.3.0 // indirect
github.com/golang/snappy v1.0.0 // indirect
github.com/google/gnostic-models v0.7.0 // indirect
github.com/google/go-cmp v0.7.0 // indirect
github.com/google/s2a-go v0.1.9 // indirect
github.com/google/uuid v1.6.0 // indirect
github.com/googleapis/enterprise-certificate-proxy v0.3.7 // indirect
github.com/googleapis/enterprise-certificate-proxy v0.3.6 // indirect
github.com/googleapis/gax-go/v2 v2.15.0 // indirect
github.com/hashicorp/errwrap v1.1.0 // indirect
github.com/json-iterator/go v1.1.12 // indirect
github.com/klauspost/compress v1.18.0 // indirect
github.com/lib/pq v1.10.9 // indirect
github.com/kylelemons/godebug v1.1.0 // indirect
github.com/kylelemons/godebug v1.1.0 // indirect
github.com/modern-go/concurrent v0.0.0-20180306012644-bacd9c7ef1dd // indirect
github.com/modern-go/reflect2 v1.0.3-0.20250322232337-35a7c28c31ee // indirect
github.com/montanaflynn/stats v0.7.1 // indirect
github.com/munnerz/goautoneg v0.0.0-20191010083416-a7dc8b61c822 // indirect
github.com/pkg/browser v0.0.0-20240102092130-5ac0b6a4141c // indirect
github.com/pmezard/go-difflib v1.0.1-0.20181226105442-5d4384ee4fb2 // indirect
github.com/prometheus/client_model v0.6.2 // indirect
github.com/prometheus/common v0.67.2 // indirect
github.com/prometheus/procfs v0.19.2 // indirect
github.com/prometheus/procfs v0.17.0 // indirect
github.com/spf13/pflag v1.0.10 // indirect
github.com/stretchr/objx v0.5.2 // indirect
github.com/x448/float16 v0.8.4 // indirect
github.com/xdg-go/pbkdf2 v1.0.0 // indirect
github.com/xdg-go/scram v1.1.2 // indirect
github.com/xdg-go/stringprep v1.0.4 // indirect
github.com/youmark/pkcs8 v0.0.0-20240726163527-a2c0da244d78 // indirect
go.mongodb.org/mongo-driver v1.17.6 // indirect
go.mongodb.org/mongo-driver v1.17.4 // indirect
go.opentelemetry.io/auto/sdk v1.2.1 // indirect
go.opentelemetry.io/contrib/instrumentation/google.golang.org/grpc/otelgrpc v0.61.0 // indirect
go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp v0.61.0 // indirect
Expand All @@ -111,14 +123,14 @@ require (
go.uber.org/automaxprocs v1.6.0 // indirect
go.yaml.in/yaml/v2 v2.4.3 // indirect
go.yaml.in/yaml/v3 v3.0.4 // indirect
golang.org/x/crypto v0.44.0 // indirect
golang.org/x/net v0.47.0 // indirect
golang.org/x/oauth2 v0.33.0 // indirect
golang.org/x/sys v0.38.0 // indirect
golang.org/x/term v0.37.0 // indirect
golang.org/x/text v0.31.0 // indirect
golang.org/x/crypto v0.43.0 // indirect
golang.org/x/mod v0.29.0 // indirect
golang.org/x/net v0.46.0 // indirect
golang.org/x/oauth2 v0.32.0 // indirect
golang.org/x/sys v0.37.0 // indirect
golang.org/x/term v0.36.0 // indirect
golang.org/x/text v0.30.0 // indirect
golang.org/x/time v0.14.0 // indirect
gomodules.xyz/jsonpatch/v2 v2.5.0 // indirect
gopkg.in/evanphx/json-patch.v4 v4.13.0 // indirect
gopkg.in/inf.v0 v0.9.1 // indirect
gopkg.in/yaml.v3 v3.0.1 // indirect
Expand Down
Loading
Loading