Skip to content

Conversation

@ljluestc
Copy link

Description

This PR adds comprehensive symlink support to the k0s Manifest Deployer, enabling users to store manifests in external locations (like git repositories) and symlink them into /var/lib/k0s/manifests for automatic deployment and reconciliation. This directly addresses the feature request in issue #4402 where users want to maintain manifests in a central git repository and use symlinks for automatic deployment.

Problem Solved:
Previously, the Manifest Deployer did not resolve symlinks in the manifests directory, did not listen for filesystem events behind symlinks, and could not detect symlinked directories as valid stacks or process symlinked manifest files.

Solution Implemented:

  • Enhanced file discovery to resolve symlinks and gracefully handle broken ones
  • Added support for symlinked directories as valid stacks
  • Implemented dynamic watching of symlink targets for automatic reconciliation
  • Added comprehensive error handling and deduplication

Fixes #4402

Type of change

  • New feature (non-breaking change which adds functionality)
  • Bug fix (non-breaking change which fixes an issue)
  • Breaking change (fix or feature that would cause existing functionality to not work as expected)
  • Documentation update

How Has This Been Tested?

  • Auto test added

Automated Testing:

Added 12 comprehensive test cases across 3 test files:

  • File Discovery Tests (pkg/applier/symlink_test.go):

    • TestFindManifestFilesInDir_WithSymlinks - Tests symlinked directory resolution
    • TestFindManifestFilesInDir_WithSymlinkedFiles - Tests symlinked manifest file detection
    • TestFindManifestFilesInDir_BrokenSymlink - Tests graceful handling of broken symlinks
    • TestFindManifestFilesInDir_MixedFilesAndSymlinks - Tests mixed regular and symlinked files
  • Manager Integration Tests (pkg/applier/manager_symlink_test.go):

    • TestManager_WithSymlinkedStacks - Tests symlinked stack directory detection and application
    • TestManager_WithSymlinkedStackAfterStart - Tests dynamic symlink creation after manager startup
    • TestManager_WithBrokenSymlink - Tests manager tolerance of broken symlinks
    • TestManager_WithSymlinkedManifestFile - Tests symlinked manifest file processing
  • Stack Applier Tests (pkg/applier/stackapplier_symlink_test.go):

    • TestStackApplier_WithSymlinkedDirectory - Tests stack applier with symlinked directories
    • TestStackApplier_WithSymlinkedFiles - Tests stack applier with symlinked files
    • TestStackApplier_WithBrokenSymlink - Tests error handling for broken symlinks
    • TestStackApplier_WithMixedFilesAndSymlinks - Tests mixed scenarios

To run tests:

# Run all symlink tests
go test ./pkg/applier -v -run "TestSymlink|TestManager.*Symlink|TestStackApplier.*Symlink"

# Run full test suite to ensure no regressions
go test ./pkg/applier -v

- Add KubeSchedulerStatus and KubeControllerManagerStatus to K0sStatus
- Implement lease checking in status handler to verify component health
- Update k0s status command output to show component status
- Fixes k0sproject#3435
- Enhanced FindManifestFilesInDir to resolve symlinks and skip broken ones
- Added getAllStackDirectories to detect symlinked directories as stacks
- Implemented symlink watching in StackApplier with dynamic event handling
- Added comprehensive test coverage with 12 new test cases
- Enables git-based manifest workflows with automatic reconciliation

Fixes k0sproject#4402
Copy link
Member

@twz123 twz123 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for working on this! Had a very quick first pass on this.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you merge this into the existing manager_test.go file?

Comment on lines +117 to +122

fmt.Fprintln(w, "Kube-scheduler probing successful:", status.KubeSchedulerStatus.Success)
fmt.Fprintln(w, "Kube-scheduler probing last error: ", status.KubeSchedulerStatus.Message)

fmt.Fprintln(w, "Kube-controller-manager probing successful:", status.KubeControllerManagerStatus.Success)
fmt.Fprintln(w, "Kube-controller-manager probing last error: ", status.KubeControllerManagerStatus.Message)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Although this seems useful, it doesn't belong to this PR. Would you mind to shove this into another one?

Comment on lines +27 to +57
// First, try to resolve any symlinks in the directory path
resolvedDir, err := filepath.EvalSymlinks(dir)
if err != nil {
// If we can't resolve the directory path, use the original directory
// This can happen if the directory contains broken symlinks
resolvedDir = dir
}

// Get all yaml files in the resolved directory
files, err := filepath.Glob(filepath.Join(resolvedDir, manifestFilePattern))
if err != nil {
return nil, err
}

for _, file := range files {
// Check if the file is a symlink and if it's broken
info, err := os.Lstat(file)
if err != nil {
continue // Skip if we can't even stat the file
}

if info.Mode()&os.ModeSymlink != 0 {
// It's a symlink, check if it's broken
_, err := os.Stat(file)
if err != nil {
continue // Skip broken symlinks
}
}

fileMap[file] = true
}
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I wonder if this is necessary? Wouldn't the OS / file system handle this automatically?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Manifest-Deployer] allow to use soft links in /var/lib/k0s/manifests

2 participants