diff --git a/.augment-guidelines b/.augment-guidelines index 54492bdd1..d773b4774 100644 --- a/.augment-guidelines +++ b/.augment-guidelines @@ -4,6 +4,25 @@ - Use `pnpx` instead of `npx` - Always run the tests before comitting or pushing. +## Task list handling procedure + +When you're working through task lists, make sure to follow the following steps and guidelines: + +1. **Complete the implementation** - Make all necessary code changes to fully address the task requirements +2. **Create a focused commit** - Before marking any task as complete, create a single atomic commit that contains only the changes for that specific task, using conventional commit format (e.g., "fix(klipper): implement temp branch cleanup in checkout function") +3. **Mark task complete** - Only after the commit is created, update the task status to COMPLETE +4. **Move to next task** - Proceed to the next task in the list + +**Specific requirements:** +- Each commit should be atomic and focused on one logical change +- Use conventional commit format: `type(scope): description` +- Include detailed commit messages with bullet points explaining the specific changes made +- Ensure all code changes are tested and functional before committing +- Follow the existing code style and patterns in the repository +- Handle any merge conflicts or issues that arise during the process + +Work through each task completely before moving to the next one, ensuring proper version control practices with meaningful commits for each completed task. + # Linting - use `pnpm run lint` in the `src` folder to run the linter once, without watching for changes.\ diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index 4ea7feebb..0498ebfe2 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -96,6 +96,13 @@ jobs: repository: Arksine/moonraker path: "moonraker" + - name: Bash Syntax Check + working-directory: ratos-configurator + run: | + # Use the standalone bash validation script for consistent validation + # across CI and local development environments + ./scripts/validate-bash-syntax.sh --quiet + - name: Lint working-directory: ratos-configurator/src run: pnpm run lint:ci diff --git a/LOGGING_SYSTEM.md b/LOGGING_SYSTEM.md index f94ff478f..ad3015330 100644 --- a/LOGGING_SYSTEM.md +++ b/LOGGING_SYSTEM.md @@ -17,7 +17,7 @@ The unified logging system consists of four main components: The bash logging library provides structured logging capabilities for shell scripts, outputting logs in JSON format compatible with the pino logging system used throughout the application. **All logs are written to the main RatOS log file** (`/var/log/ratos-configurator.log`) with a `source: "ratos-update"` field for filtering. -#### Features: +#### Features - **JSON-formatted logs** compatible with pino - **Multiple log levels**: trace, debug, info, warn, error, fatal - **Unified log file** - writes to main RatOS log instead of separate files @@ -26,7 +26,7 @@ The bash logging library provides structured logging capabilities for shell scri - **Command execution logging** with automatic error handling - **Timestamped entries** with process information -#### Usage Example: +#### Usage Example ```bash #!/bin/bash source "$(dirname "$0")/ratos-logging.sh" @@ -49,7 +49,7 @@ execute_with_logging "apt-get update" "package_update" "APT_UPDATE_FAILED" log_script_complete "my-script.sh" $? ``` -#### Configuration: +#### Configuration - `RATOS_LOG_LEVEL`: Set minimum log level (default: info) - `RATOS_LOG_FILE`: Log file path (default: uses `${LOG_FILE}` from environment, typically `/var/log/ratos-configurator.log`) - `RATOS_LOG_MAX_SIZE`: Maximum log file size before rotation (default: 0 = disabled when using main log) @@ -61,7 +61,7 @@ log_script_complete "my-script.sh" $? The CLI provides several commands for viewing and analyzing update logs. **Update logs are now a subcommand of the main `logs` command** and automatically filter the main log file to show only entries with `source: "ratos-update"`. -#### Commands: +#### Commands **`ratos logs update-logs summary`** - Shows a summary of the most recent update attempt from the main log @@ -81,7 +81,7 @@ The CLI provides several commands for viewing and analyzing update logs. **Updat - Options: - `-d, --details`: Show detailed information -#### Usage Examples: +#### Usage Examples ```bash # Show update summary (note the new command structure) ratos logs update-logs summary @@ -104,7 +104,7 @@ ratos logs rotate # Force log rotation The web interface provides a comprehensive log viewer accessible at `/configure/update-logs`. -#### Features: +#### Features - **Log Summary Dashboard**: Overview of recent update attempts - **Interactive Log Viewer**: Browse and filter log entries - **Real-time Filtering**: Filter by log level, context, and search terms @@ -112,7 +112,7 @@ The web interface provides a comprehensive log viewer accessible at `/configure/ - **Download Capability**: Download raw log files - **Auto-refresh**: Automatic updates when new logs are available -#### Components: +#### Components - `UpdateLogsViewer`: Main component for displaying logs - `UpdateLogsErrorBoundary`: Error boundary for graceful error handling - `LogSummaryCard`: Summary statistics and controls @@ -120,7 +120,7 @@ The web interface provides a comprehensive log viewer accessible at `/configure/ ### 4. API Endpoints -#### TRPC Endpoints (`src/server/routers/update-logs.ts`): +#### TRPC Endpoints (`src/server/routers/update-logs.ts`) - `update-logs.summary`: Get log summary statistics (filtered by `source: "ratos-update"`) - `update-logs.entries`: Get filtered log entries (filtered by `source: "ratos-update"`) - `update-logs.errors`: Get only errors and warnings (filtered by `source: "ratos-update"`) @@ -128,7 +128,7 @@ The web interface provides a comprehensive log viewer accessible at `/configure/ - `update-logs.clear`: **Disabled** - Cannot clear main log file (use log rotation instead) - `update-logs.download`: Download main log file (contains all sources) -#### REST Endpoints: +#### REST Endpoints - `GET /api/update-logs/download`: Download log file as attachment ### 5. Debug Integration @@ -156,7 +156,7 @@ All logs follow a consistent JSON format: } ``` -### Fields: +### Fields - `level`: Numeric log level (10=trace, 20=debug, 30=info, 40=warn, 50=error, 60=fatal) - `time`: ISO 8601 timestamp - `msg`: Human-readable log message @@ -170,7 +170,7 @@ All logs follow a consistent JSON format: Standardized error codes help identify common issues: -### Update Script Error Codes: +### Update Script Error Codes - `SCRIPT_ERROR`: General script failure - `SCRIPT_SUCCESS`: Script completed successfully - `SYMLINK_CREATE_FAILED`: Failed to create symbolic link @@ -180,34 +180,66 @@ Standardized error codes help identify common issues: - `EXTENSION_SYMLINK_FAILED`: Extension symlinking failed - `OWNERSHIP_CHANGE_FAILED`: File ownership change failed -### System Error Codes: +### Klipper Migration Error Codes +- `KLIPPER_DIR_NOT_FOUND`: Klipper directory not found +- `KLIPPER_NOT_GIT_REPO`: Klipper directory is not a git repository +- `KLIPPER_DIR_ACCESS_FAILED`: Cannot access Klipper directory +- `KLIPPER_STAGED_CHANGES`: Uncommitted staged changes prevent migration +- `KLIPPER_UNCOMMITTED_CHANGES`: Uncommitted changes prevent migration +- `KLIPPER_MIGRATION_FAILED`: General Klipper migration failure +- `GIT_REMOTE_URL_FAILED`: Failed to get git remote URL +- `GIT_REMOTE_ADD_FAILED`: Failed to add git remote +- `GIT_REMOTE_UPDATE_FAILED`: Failed to update git remote URL +- `GIT_FETCH_FAILED`: Failed to fetch from remote repository +- `GIT_FETCH_RETRY`: Fetch retry attempt +- `GIT_CHECKOUT_FAILED`: Failed to checkout branch +- `GIT_CHECKOUT_REMOTE_FAILED`: Failed to checkout remote branch +- `GIT_TEMP_BRANCH_FAILED`: Failed to create temporary branch +- `GIT_TEMP_BRANCH_CLEANUP`: Temporary branch cleanup operation +- `GIT_TEMP_BRANCH_CLEANUP_FAILED`: Failed to clean up temporary branch +- `GIT_COMMIT_NOT_FOUND`: Target commit not found +- `GIT_RESET_FAILED`: Failed to reset to target commit +- `GIT_UPSTREAM_SET_FAILED`: Failed to set upstream tracking +- `REMOTE_URL_MISMATCH`: Remote URL doesn't match expected value +- `REPOSITORY_CHECK_FAILED`: Repository check failed +- `REMOTE_SETUP_FAILED`: Remote setup failed +- `FETCH_FAILED`: Fetch operation failed +- `CHECKOUT_FAILED`: Checkout operation failed +- `RESET_FAILED`: Reset operation failed +- `OWNERSHIP_FAILED`: Ownership fix failed + +### System Error Codes - `FILE_NOT_FOUND`: Required file not found - `PERMISSION_DENIED`: Insufficient permissions - `NETWORK_ERROR`: Network connectivity issue - `DISK_FULL`: Insufficient disk space +- `ENV_VAR_MISSING`: Required environment variable not set +- `USER_NOT_FOUND`: Required system user account does not exist +- `GROUP_NOT_FOUND`: Required system group does not exist +- `OWNERSHIP_CHANGE_FAILED`: Failed to change file/directory ownership ## Error Handling and Retry Logic -### Bash Scripts: +### Bash Scripts - Automatic error trapping with `set -eE` - Stack trace capture on script failure - Graceful error reporting with context - Exit codes indicate success/failure status -### Web UI: +### Web UI - Error boundaries prevent UI crashes - Automatic retry with exponential backoff - Graceful degradation when logs unavailable - User-friendly error messages -### CLI: +### CLI - Robust error handling for missing files - Clear error messages with suggested actions - Non-zero exit codes for scripting ## Monitoring and Alerting -### Log Rotation: +### Log Rotation - Automatic rotation when files exceed 10MB - Keeps 5 backup files by default - Configurable via environment variables @@ -219,7 +251,7 @@ Standardized error codes help identify common issues: ## Troubleshooting -### Common Issues: +### Common Issues **Log file not found:** - Ensure update scripts have been run at least once @@ -241,7 +273,7 @@ Standardized error codes help identify common issues: - Ensure scripts are using the logging library correctly - Verify JSON format of log entries -### Debug Commands: +### Debug Commands ```bash # Check main log file location and size ls -la /var/log/ratos-configurator.log* @@ -265,25 +297,25 @@ shellcheck -ax -s bash configuration/scripts/ratos-update.sh ## Development -### Adding New Log Sources: +### Adding New Log Sources 1. Source the logging library: `source "$(dirname "$0")/ratos-logging.sh"` 2. Set up error trapping: `setup_error_trap "script-name"` 3. Use logging functions: `log_info`, `log_error`, etc. 4. Add appropriate error codes to documentation -### Code Quality Standards: +### Code Quality Standards - **ShellCheck Compliance**: All bash scripts must pass ShellCheck validation - **Error Handling**: Use proper error trapping with selective `set +e`/`set -e` - **Variable Quoting**: Always quote variables and use `read -r` for input - **Exit Codes**: Use proper exit code handling and propagation -### Testing: +### Testing - Unit tests in `src/__tests__/update-logs.test.ts` - Integration tests for CLI commands - End-to-end tests for web UI - ShellCheck validation in CI/CD pipeline -### Contributing: +### Contributing - Follow existing log format and error code conventions - Run ShellCheck on all bash scripts before committing - Add tests for new functionality diff --git a/configuration/moonraker.conf b/configuration/moonraker.conf index 723cd58e3..d008e7c58 100644 --- a/configuration/moonraker.conf +++ b/configuration/moonraker.conf @@ -112,7 +112,7 @@ info_tags: [update_manager klipper] channel: dev -pinned_commit: b7233d1197d9a2158676ad39d02b80f787054e20 +pinned_commit: 1c96f096fdeea8e2e79237b679ed6fa944fbae5e [update_manager moonraker] channel: dev diff --git a/configuration/scripts/klipper-fork-migration.sh b/configuration/scripts/klipper-fork-migration.sh new file mode 100755 index 000000000..3b5b27ff7 --- /dev/null +++ b/configuration/scripts/klipper-fork-migration.sh @@ -0,0 +1,765 @@ +#!/usr/bin/env bash +set -euo pipefail +IFS=$'\n\t' + +# Portable script directory resolution with fallbacks +if command -v realpath >/dev/null 2>&1; then + # Primary method: use realpath when available (preferred for accuracy) + SCRIPT_DIR=$( cd -- "$( dirname -- "$(realpath -- "${BASH_SOURCE[0]}")" )" &> /dev/null && pwd ) +elif command -v readlink >/dev/null 2>&1 && readlink -f /dev/null >/dev/null 2>&1; then + # Fallback method: use readlink -f if available and functional (test with /dev/null) + # Note: macOS and BSD systems may have readlink but without -f flag support + SCRIPT_DIR=$( cd -- "$( dirname -- "$(readlink -f "${BASH_SOURCE[0]}")" )" &> /dev/null && pwd ) +else + # Ultimate fallback: use basic dirname approach for maximum compatibility + # Note: This may not resolve symlinks, but provides basic functionality + SCRIPT_DIR=$( cd -- "$( dirname -- "${BASH_SOURCE[0]}" )" &> /dev/null && pwd ) + if [ -z "$SCRIPT_DIR" ]; then + echo "ERROR: Unable to determine script directory. Neither realpath nor functional readlink -f is available and basic dirname failed." >&2 + exit 1 + fi +fi + +# Source logging library first +# shellcheck source=configuration/scripts/ratos-logging.sh +if [ ! -f "$SCRIPT_DIR/ratos-logging.sh" ]; then + echo "ERROR: ratos-logging.sh not found in $SCRIPT_DIR" + exit 1 +fi +# shellcheck disable=SC1091 +source "$SCRIPT_DIR"/ratos-logging.sh + +# Set up error trapping and logging +setup_error_trap "klipper-fork-migration" +START_TIME=$(get_timestamp) + +# Log script start +log_script_start "klipper-fork-migration.sh" "1.0.0" + +# Check if running as root (after logging is available) +if [ "$EUID" -ne 0 ]; then + log_fatal "Please run as root" "script_init" "PERMISSION_DENIED" + exit 1 +fi + +# shellcheck source=configuration/scripts/ratos-common.sh +if [ ! -f "$SCRIPT_DIR/ratos-common.sh" ]; then + log_fatal "ratos-common.sh not found in $SCRIPT_DIR" "script_init" "FILE_NOT_FOUND" + exit 1 +fi +# shellcheck disable=SC1091 +source "$SCRIPT_DIR"/ratos-common.sh + +# Required environment variables (sourced from ratos-common.sh -> environment.sh): +# - KLIPPER_DIR: Path to the Klipper installation directory +# - RATOS_USERNAME: RatOS system user for file ownership +# - RATOS_USERGROUP: RatOS system group for file ownership +# These variables are loaded from ~/.ratos.env.system or ~/.ratos.env + +# validate_required_env_var() - Reusable helper function for environment variable validation +# +# Validates that required environment variables are set and optionally performs additional +# validation checks based on the validation type specified. +# +# PARAMETERS: +# $1 - var_name: Name of the environment variable to validate +# $2 - validation_type: Type of additional validation to perform +# - "basic": Only check if variable is set (default) +# - "directory": Check if path exists and is accessible +# - "user": Check if user exists on the system +# - "group": Check if group exists on the system +# +# RETURN CODES: +# 0 - Success: Variable is set and passes validation +# 1 - Failure: Variable validation failed (script will exit) +# +validate_required_env_var() { + local var_name="$1" + local validation_type="${2:-basic}" + + # Get the variable value using indirect expansion + local var_value + eval "var_value=\${${var_name}:-}" + + # Check if variable is set + if [ -z "$var_value" ]; then + log_fatal "$var_name environment variable is not set. This should be defined in ~/.ratos.env.system" "script_init" "ENV_VAR_MISSING" + exit 1 + fi + + # Perform additional validation based on type + case "$validation_type" in + "directory") + if [ ! -d "$var_value" ]; then + log_fatal "$var_name path does not exist: $var_value" "script_init" "${var_name}_NOT_FOUND" + exit 1 + fi + if [ ! -r "$var_value" ] || [ ! -x "$var_value" ]; then + log_fatal "$var_name path is not accessible: $var_value" "script_init" "${var_name}_ACCESS_FAILED" + exit 1 + fi + ;; + "user") + if ! id "$var_value" >/dev/null 2>&1; then + log_fatal "$var_name user does not exist on system: $var_value" "script_init" "USER_NOT_FOUND" + exit 1 + fi + ;; + "group") + if ! getent group "$var_value" >/dev/null 2>&1; then + log_fatal "$var_name group does not exist on system: $var_value" "script_init" "GROUP_NOT_FOUND" + exit 1 + fi + ;; + "basic") + # No additional validation needed + ;; + *) + log_fatal "Invalid validation type '$validation_type' for $var_name" "script_init" "INVALID_VALIDATION_TYPE" + exit 1 + ;; + esac + + return 0 +} + +# Validate required environment variables using helper function +validate_required_env_var "KLIPPER_DIR" "directory" +validate_required_env_var "RATOS_USERNAME" "user" +validate_required_env_var "RATOS_USERGROUP" "group" + +# Migration constants (readonly to prevent accidental modification) +readonly OFFICIAL_KLIPPER_URL="https://github.com/Klipper3d/klipper.git" +readonly RATOS_FORK_URL="https://github.com/Rat-OS/klipper.git" +readonly RATOS_FORK_REMOTE="ratos-fork" +readonly TARGET_BRANCH="topic/first-layer-experimental" +readonly MOONRAKER_CONF_PATH="$SCRIPT_DIR/../moonraker.conf" + +# extract_target_commit_from_moonraker() - Dynamically extracts klipper pinned_commit from moonraker.conf +# +# Parses the moonraker.conf file to locate the [update_manager klipper] section and extract +# the pinned_commit value. This ensures the migration script always uses the correct target +# commit that matches the current moonraker configuration. +# +# CONFIGURATION FILE FORMAT: +# [update_manager klipper] +# channel: dev +# pinned_commit: +# +# RETURN CODES: +# 0 - Success: Target commit extracted successfully +# 1 - File error: moonraker.conf file not found or not readable +# 2 - Parse error: klipper section or pinned_commit not found +# 3 - Validation error: extracted commit hash is invalid format +# +# OUTPUT: +# Prints the extracted commit hash to stdout on success +# +extract_target_commit_from_moonraker() +{ + log_info "Extracting target commit from moonraker.conf..." "extract_commit" + + # Check if moonraker.conf exists and is readable + if [ ! -f "$MOONRAKER_CONF_PATH" ]; then + log_error "moonraker.conf not found at: $MOONRAKER_CONF_PATH" "extract_commit" "MOONRAKER_CONF_NOT_FOUND" + return 1 + fi + + if [ ! -r "$MOONRAKER_CONF_PATH" ]; then + log_error "moonraker.conf is not readable: $MOONRAKER_CONF_PATH" "extract_commit" "MOONRAKER_CONF_NOT_READABLE" + return 1 + fi + + log_info "Reading moonraker.conf from: $MOONRAKER_CONF_PATH" "extract_commit" + + # Parse the moonraker.conf file to extract pinned_commit from [update_manager klipper] section + # Use awk for robust parsing that handles various formatting styles + local extracted_commit + extracted_commit=$(awk ' + BEGIN { + in_klipper_section = 0 + pinned_commit = "" + } + + # Match section headers and track if we are in the klipper update_manager section + /^\[update_manager klipper\]/ { + in_klipper_section = 1 + next + } + + # Reset section tracking when we encounter a new section + /^\[/ && !/^\[update_manager klipper\]/ { + in_klipper_section = 0 + next + } + + # Extract pinned_commit when in the correct section + in_klipper_section && /^pinned_commit:/ { + # Remove "pinned_commit:" prefix and trim whitespace + gsub(/^pinned_commit:[ \t]*/, "") + gsub(/[ \t]*$/, "") + pinned_commit = $0 + } + + END { + if (pinned_commit != "") { + print pinned_commit + } + } + ' "$MOONRAKER_CONF_PATH") + + # Check if we successfully extracted a commit hash + if [ -z "$extracted_commit" ]; then + log_error "Could not find pinned_commit in [update_manager klipper] section" "extract_commit" "KLIPPER_PINNED_COMMIT_NOT_FOUND" + log_error "Please ensure moonraker.conf contains a valid [update_manager klipper] section with pinned_commit field" "extract_commit" "KLIPPER_PINNED_COMMIT_NOT_FOUND" + return 2 + fi + + # Validate commit hash format (40-character hexadecimal string) + if ! echo "$extracted_commit" | grep -qE '^[a-fA-F0-9]{40}$'; then + log_error "Invalid commit hash format: $extracted_commit" "extract_commit" "INVALID_COMMIT_HASH_FORMAT" + log_error "Expected 40-character hexadecimal string" "extract_commit" "INVALID_COMMIT_HASH_FORMAT" + return 3 + fi + + log_info "Successfully extracted target commit: $extracted_commit" "extract_commit" + echo "$extracted_commit" + return 0 +} + +# Extract TARGET_COMMIT dynamically from moonraker.conf +TARGET_COMMIT=$(extract_target_commit_from_moonraker) +extract_result=$? + +if [ $extract_result -ne 0 ]; then + log_fatal "Failed to extract target commit from moonraker.conf (exit code $extract_result)" "script_init" "TARGET_COMMIT_EXTRACTION_FAILED" + exit 1 +fi + +# Make TARGET_COMMIT readonly after successful extraction +readonly TARGET_COMMIT + +# check_klipper_repository() - Validates repository state and determines migration requirements +# +# Implements strict repository state validation with comprehensive edge case handling. +# Only supported repository configurations are allowed to proceed with migration. +# +# REPOSITORY STATE LOGIC: +# 1. Official Klipper Source → Proceed with Migration +# - Repository origin points to any official Klipper URL format +# - Return 0 to indicate migration is needed +# 2. RatOS Fork at Correct Commit → Skip Migration Gracefully +# - Repository origin points to RatOS fork URL +# - Current HEAD points to the pinned commit AND current branch is expected branch +# - Return 1 to indicate migration not needed (safe skip) +# 3. RatOS Fork at Different Commit → Proceed with Migration +# - Repository origin points to RatOS fork URL +# - Current HEAD does NOT point to pinned commit OR current branch is not expected +# - Return 0 to indicate migration is needed to reset to correct state +# 4. Any Other Remote/Source → Fatal Error +# - Repository origin points to unsupported URL +# - Return 2 (fatal error) with appropriate error logging +# +# RETURN CODES: +# 0 - Migration needed: Repository requires migration to RatOS fork +# 1 - Migration not needed: Repository is already at correct RatOS fork state +# 2 - Fatal error: Repository validation failed or unsupported configuration +# +check_klipper_repository() +{ + log_info "Checking Klipper repository configuration..." "check_repository" + + if [ ! -d "$KLIPPER_DIR" ]; then + log_error "Klipper directory not found at $KLIPPER_DIR" "check_repository" "KLIPPER_DIR_NOT_FOUND" + return 2 # Fatal error + fi + + if [ ! -d "$KLIPPER_DIR/.git" ]; then + log_error "Klipper directory is not a git repository" "check_repository" "KLIPPER_NOT_GIT_REPO" + return 2 # Fatal error + fi + + # Get current origin URL + local current_origin + if ! current_origin=$(git -C "$KLIPPER_DIR" remote get-url origin 2>/dev/null); then + log_error "Cannot get origin URL from Klipper repository" "check_repository" "GIT_REMOTE_URL_FAILED" + return 2 # Fatal error + fi + + log_info "Repository origin URL: $current_origin" "check_repository" + + # Define all valid official Klipper repository URL formats + local official_urls=( + "$OFFICIAL_KLIPPER_URL" # HTTPS + "git@github.com:Klipper3d/klipper.git" # SSH shorthand + "ssh://git@github.com/Klipper3d/klipper.git" # SSH protocol + "git://github.com/Klipper3d/klipper.git" # Git protocol + ) + + # Check if current origin is official Klipper repository + local is_official_repo=false + for official_url in "${official_urls[@]}"; do + if [[ "$current_origin" == "$official_url" ]]; then + is_official_repo=true + break + fi + done + + if [[ "$is_official_repo" == true ]]; then + # Case 1: Official Klipper Source → Proceed with Migration + log_info "Repository is using official Klipper source, migration needed." "check_repository" + return 0 + fi + + # Check if current origin is RatOS fork + if [[ "$current_origin" == "$RATOS_FORK_URL" ]]; then + log_info "Repository is using RatOS fork, checking current state..." "check_repository" + + # Get current HEAD commit + local current_commit + if ! current_commit=$(git -C "$KLIPPER_DIR" rev-parse HEAD 2>/dev/null); then + log_error "Cannot get current HEAD commit from repository" "check_repository" "GIT_HEAD_FAILED" + return 2 # Fatal error + fi + + # Get current branch (handle detached HEAD state) + local current_branch + current_branch=$(git -C "$KLIPPER_DIR" branch --show-current 2>/dev/null || echo "") + + log_info "Current commit: $current_commit" "check_repository" + log_info "Current branch: ${current_branch:-"(detached HEAD)"}" "check_repository" + log_info "Expected commit: $TARGET_COMMIT" "check_repository" + log_info "Expected branch: $TARGET_BRANCH" "check_repository" + + # Check if repository is at correct commit and branch + if [[ "$current_commit" == "$TARGET_COMMIT" ]] && [[ "$current_branch" == "$TARGET_BRANCH" ]]; then + # Case 2: RatOS Fork at Correct Commit → Skip Migration Gracefully + log_info "Repository is already at correct RatOS fork state (commit $TARGET_COMMIT on branch $TARGET_BRANCH)" "check_repository" + log_info "Migration not needed." "check_repository" + return 1 # Skip migration + else + # Case 3: RatOS Fork at Different Commit → Proceed with Migration + log_info "Repository is using RatOS fork but not at correct state:" "check_repository" + if [[ "$current_commit" != "$TARGET_COMMIT" ]]; then + log_info " - Current commit ($current_commit) differs from expected ($TARGET_COMMIT)" "check_repository" + fi + if [[ "$current_branch" != "$TARGET_BRANCH" ]]; then + log_info " - Current branch (${current_branch:-"detached HEAD"}) differs from expected ($TARGET_BRANCH)" "check_repository" + fi + log_info "Migration needed to reset to correct state." "check_repository" + return 0 + fi + fi + + # Case 4: Any Other Remote/Source → Fatal Error + log_error "Repository is using an unsupported remote source. Only official Klipper or RatOS fork repositories are supported." "check_repository" "UNSUPPORTED_REPOSITORY_SOURCE" + log_error "Current origin URL: $current_origin" "check_repository" "UNSUPPORTED_REPOSITORY_SOURCE" + log_error "Supported sources:" "check_repository" "UNSUPPORTED_REPOSITORY_SOURCE" + log_error " - Official Klipper: ${official_urls[*]}" "check_repository" "UNSUPPORTED_REPOSITORY_SOURCE" + log_error " - RatOS Fork: $RATOS_FORK_URL" "check_repository" "UNSUPPORTED_REPOSITORY_SOURCE" + return 2 # Fatal error +} + +# check_uncommitted_changes() - Validates repository working directory state +# +# Checks for staged and unstaged changes that would prevent safe migration. +# Uses Git plumbing commands for reliable detection of repository state. +# +# RETURN CODES: +# 0 - Success: No uncommitted changes found, migration can proceed +# 2 - Directory access error: Cannot access Klipper directory +# 3 - Uncommitted changes error: Staged or unstaged changes prevent migration +# +check_uncommitted_changes() +{ + log_info "Checking for uncommitted changes..." "check_changes" + + cd "$KLIPPER_DIR" || { + log_error "Cannot change to Klipper directory" "check_changes" "KLIPPER_DIR_ACCESS_FAILED" + return 2 # Directory access error + } + + # Check for staged changes (index vs HEAD) using Git plumbing commands + if ! git diff-index --cached --quiet HEAD --; then + log_error "There are staged changes in the Klipper repository." "check_changes" "KLIPPER_STAGED_CHANGES" + log_error "Please commit or stash these changes before running migration." "check_changes" "KLIPPER_STAGED_CHANGES" + + # Get list of staged files for error reporting + local staged_files + staged_files=$(git diff-index --cached --name-only HEAD -- | tr '\n' ' ') + log_error "Staged files: $staged_files" "check_changes" "KLIPPER_STAGED_CHANGES" + return 3 # Uncommitted changes error + fi + + # Check for unstaged changes (working directory vs index) using Git plumbing commands + if ! git diff-index --quiet HEAD --; then + log_error "There are uncommitted changes in the Klipper repository." "check_changes" "KLIPPER_UNCOMMITTED_CHANGES" + log_error "Please commit or stash these changes before running migration." "check_changes" "KLIPPER_UNCOMMITTED_CHANGES" + + # Get list of modified files for error reporting + local modified_files + modified_files=$(git diff-index --name-only HEAD -- | tr '\n' ' ') + log_error "Modified files: $modified_files" "check_changes" "KLIPPER_UNCOMMITTED_CHANGES" + return 3 # Uncommitted changes error + fi + + log_info "No uncommitted changes found." "check_changes" + return 0 +} + +handle_existing_remote() +{ + log_info "Checking for existing RatOS fork remote..." "handle_remote" + + cd "$KLIPPER_DIR" || { + log_error "Cannot change to Klipper directory" "handle_remote" "KLIPPER_DIR_ACCESS_FAILED" + return 1 + } + + # Cache the remote URL to avoid multiple git subprocess calls + local existing_url + existing_url=$(git remote get-url "$RATOS_FORK_REMOTE" 2>/dev/null) + + # Check if ratos-fork remote already exists + if [ -n "$existing_url" ]; then + if [ "$existing_url" != "$RATOS_FORK_URL" ]; then + log_warn "Remote '$RATOS_FORK_REMOTE' exists but points to different URL:" "handle_remote" "REMOTE_URL_MISMATCH" + log_warn " Current: $existing_url" "handle_remote" "REMOTE_URL_MISMATCH" + log_warn " Expected: $RATOS_FORK_URL" "handle_remote" "REMOTE_URL_MISMATCH" + log_info "Updating remote URL..." "handle_remote" + + if ! execute_with_logging git remote set-url "$RATOS_FORK_REMOTE" "$RATOS_FORK_URL" "handle_remote" "GIT_REMOTE_UPDATE_FAILED"; then + log_error "Failed to update remote URL" "handle_remote" "GIT_REMOTE_UPDATE_FAILED" + return 1 + fi + log_info "Remote URL updated successfully." "handle_remote" + else + log_info "Remote '$RATOS_FORK_REMOTE' already exists with correct URL." "handle_remote" + fi + else + log_info "Adding RatOS fork remote..." "handle_remote" + if ! execute_with_logging git remote add "$RATOS_FORK_REMOTE" "$RATOS_FORK_URL" "handle_remote" "GIT_REMOTE_ADD_FAILED"; then + log_error "Failed to add RatOS fork remote" "handle_remote" "GIT_REMOTE_ADD_FAILED" + return 1 + fi + log_info "RatOS fork remote added successfully." "handle_remote" + fi + + return 0 +} + +fetch_ratos_fork() +{ + log_info "Fetching from RatOS fork..." "fetch_fork" + + cd "$KLIPPER_DIR" || { + log_error "Cannot change to Klipper directory" "fetch_fork" "KLIPPER_DIR_ACCESS_FAILED" + return 1 + } + + # Attempt to fetch with retries + local max_retries=3 + local retry_count=0 + + while [ $retry_count -lt $max_retries ]; do + if execute_with_logging git fetch "$RATOS_FORK_REMOTE" "fetch_fork" "GIT_FETCH_FAILED"; then + log_info "Successfully fetched from RatOS fork." "fetch_fork" + return 0 + else + retry_count=$((retry_count + 1)) + log_warn "Fetch attempt $retry_count failed." "fetch_fork" "GIT_FETCH_RETRY" + if [ $retry_count -lt $max_retries ]; then + log_info "Retrying in 5 seconds..." "fetch_fork" + sleep 5 + fi + fi + done + + log_error "Failed to fetch from RatOS fork after $max_retries attempts" "fetch_fork" "GIT_FETCH_FAILED" + log_error "Please check your network connection and try again." "fetch_fork" "NETWORK_ERROR" + return 1 +} + +checkout_target_branch() +{ + log_info "Checking out target branch..." "checkout_branch" + + # Track if we created a temporary branch for cleanup + local temp_branch="" + local created_temp_branch=false + + # Shared cleanup function for temporary branches (called on error paths) + cleanup_temp_branch_on_error() { + if [ "$created_temp_branch" = true ] && [ -n "$temp_branch" ]; then + log_info "Cleaning up temporary migration branch due to error: $temp_branch" "checkout_branch" + if git -C "$KLIPPER_DIR" branch -D "$temp_branch" >/dev/null 2>&1; then + log_info "Successfully cleaned up temporary branch: $temp_branch" "checkout_branch" "GIT_TEMP_BRANCH_CLEANUP" + else + log_warn "Failed to clean up temporary branch: $temp_branch (this is not critical)" "checkout_branch" "GIT_TEMP_BRANCH_CLEANUP_FAILED" + fi + fi + } + + # Set up function-level ERR trap for unexpected failures (signals, unhandled command failures) + # This provides additional safety beyond explicit cleanup calls on known error paths + trap 'cleanup_temp_branch_on_error' ERR + + # Check if we're in detached HEAD state + if ! git -C "$KLIPPER_DIR" symbolic-ref HEAD >/dev/null 2>&1; then + log_info "Repository is in detached HEAD state." "checkout_branch" + log_info "Creating and checking out a temporary branch..." "checkout_branch" + temp_branch="temp-migration-$(date +%s)-$$" + if ! execute_with_logging git -C "$KLIPPER_DIR" checkout -b "$temp_branch" "checkout_branch" "GIT_TEMP_BRANCH_FAILED"; then + log_error "Failed to create temporary branch" "checkout_branch" "GIT_TEMP_BRANCH_FAILED" + cleanup_temp_branch_on_error + return 1 + fi + created_temp_branch=true + fi + + # Check if target branch already exists locally + if git -C "$KLIPPER_DIR" show-ref --verify --quiet "refs/heads/$TARGET_BRANCH"; then + log_info "Local branch '$TARGET_BRANCH' already exists, switching to it..." "checkout_branch" + if ! execute_with_logging git -C "$KLIPPER_DIR" checkout "$TARGET_BRANCH" "checkout_branch" "GIT_CHECKOUT_FAILED"; then + log_error "Failed to checkout existing branch '$TARGET_BRANCH'" "checkout_branch" "GIT_CHECKOUT_FAILED" + cleanup_temp_branch_on_error + return 1 + fi + else + log_info "Creating and checking out branch '$TARGET_BRANCH' from RatOS fork..." "checkout_branch" + if ! execute_with_logging git -C "$KLIPPER_DIR" checkout -b "$TARGET_BRANCH" "$RATOS_FORK_REMOTE/$TARGET_BRANCH" "checkout_branch" "GIT_CHECKOUT_REMOTE_FAILED"; then + log_error "Failed to checkout branch '$TARGET_BRANCH' from RatOS fork" "checkout_branch" "GIT_CHECKOUT_REMOTE_FAILED" + log_error "Please ensure the branch exists on the remote repository." "checkout_branch" "GIT_CHECKOUT_REMOTE_FAILED" + cleanup_temp_branch_on_error + return 1 + fi + fi + + # Clean up temporary branch if we created one (successful completion) + if [ "$created_temp_branch" = true ] && [ -n "$temp_branch" ]; then + log_info "Cleaning up temporary migration branch: $temp_branch" "checkout_branch" + if execute_with_logging git -C "$KLIPPER_DIR" branch -D "$temp_branch" "checkout_branch" "GIT_TEMP_BRANCH_CLEANUP"; then + log_info "Successfully cleaned up temporary branch: $temp_branch" "checkout_branch" + else + log_warn "Failed to clean up temporary branch: $temp_branch (this is not critical)" "checkout_branch" "GIT_TEMP_BRANCH_CLEANUP_FAILED" + fi + fi + + # Clear the ERR trap since we're completing successfully + trap - ERR + + log_info "Successfully checked out branch '$TARGET_BRANCH'." "checkout_branch" + return 0 +} + +reset_to_target_commit() +{ + log_info "Resetting to target commit..." "reset_commit" + + cd "$KLIPPER_DIR" || { + log_error "Cannot change to Klipper directory" "reset_commit" "KLIPPER_DIR_ACCESS_FAILED" + return 1 + } + + # Verify the target commit exists + if ! git cat-file -e "$TARGET_COMMIT" 2>/dev/null; then + log_error "Target commit '$TARGET_COMMIT' not found in repository" "reset_commit" "GIT_COMMIT_NOT_FOUND" + log_error "Please ensure the commit exists and try again." "reset_commit" "GIT_COMMIT_NOT_FOUND" + return 1 + fi + + # Reset to target commit + if ! execute_with_logging git reset --hard "$TARGET_COMMIT" "reset_commit" "GIT_RESET_FAILED"; then + log_error "Failed to reset to target commit '$TARGET_COMMIT'" "reset_commit" "GIT_RESET_FAILED" + return 1 + fi + + log_info "Successfully reset to commit '$TARGET_COMMIT'." "reset_commit" + + # Set upstream tracking + if ! execute_with_logging git branch --set-upstream-to="$RATOS_FORK_REMOTE/$TARGET_BRANCH" "$TARGET_BRANCH" "reset_commit" "GIT_UPSTREAM_SET_FAILED"; then + log_warn "Failed to set upstream tracking, but migration completed successfully." "reset_commit" "GIT_UPSTREAM_SET_FAILED" + else + log_info "Upstream tracking set to '$RATOS_FORK_REMOTE/$TARGET_BRANCH'." "reset_commit" + fi + + return 0 +} + +fix_klipper_ownership() +{ + log_info "Ensuring Klipper directory ownership..." "fix_ownership" + + if [ -n "$(find "$KLIPPER_DIR" \( \! -user "$RATOS_USERNAME" -o \! -group "$RATOS_USERGROUP" \) -quit)" ]; then + if execute_with_logging chown -R "$RATOS_USERNAME:$RATOS_USERGROUP" "$KLIPPER_DIR" "fix_ownership" "OWNERSHIP_CHANGE_FAILED"; then + log_info "Klipper directory ownership has been set to $RATOS_USERNAME:$RATOS_USERGROUP." "fix_ownership" + else + log_error "Failed to set Klipper directory ownership" "fix_ownership" "OWNERSHIP_CHANGE_FAILED" + return 1 + fi + else + log_info "Klipper directory ownership already set correctly." "fix_ownership" + fi + + return 0 +} + +# migrate_klipper_repository() - Main migration orchestration function +# +# This function coordinates the complete Klipper repository migration process from the +# original Klipper repository to the RatOS fork. It performs all necessary validation, +# setup, and migration steps in a specific order to ensure a safe and reliable migration. +# +# RETURN CODES: +# 0 - Success: Migration completed successfully or was not needed +# 2 - Fatal repository check error: Repository validation failed critically +# - Repository is not a Git repository +# - Repository directory is inaccessible +# - Other critical repository state issues +# 3 - Uncommitted changes error: Repository has uncommitted changes that prevent migration +# - Staged changes exist in the repository +# - Modified files exist in the working directory +# - User must commit or stash changes before migration +# 4 - Remote setup error: Failed to configure the RatOS fork remote +# - Unable to add new remote +# - Failed to update existing remote URL +# - Git remote operations failed +# 5 - Fetch error: Failed to fetch from the RatOS fork remote +# - Network connectivity issues +# - Remote repository is inaccessible +# - Authentication problems +# - All retry attempts exhausted +# 6 - Checkout error: Failed to checkout the target branch +# - Target branch doesn't exist on remote +# - Git checkout operations failed +# - Repository is in an inconsistent state +# 7 - Reset error: Failed to reset to the target commit +# - Target commit doesn't exist +# - Git reset operations failed +# - Unable to set upstream tracking +# 8 - Ownership error: Failed to fix file ownership +# - Insufficient permissions to change ownership +# - Invalid user or group specified +# - File system errors during ownership change +# +# DEPENDENCIES: +# - check_klipper_repository(): Validates repository state +# - check_uncommitted_changes(): Ensures clean working directory +# - handle_existing_remote(): Configures RatOS fork remote +# - fetch_ratos_fork(): Downloads latest changes from RatOS fork +# - checkout_target_branch(): Switches to target branch +# - reset_to_target_commit(): Resets to specific commit +# - fix_klipper_ownership(): Ensures proper file ownership +# +# ENVIRONMENT VARIABLES REQUIRED: +# - KLIPPER_DIR: Path to Klipper installation directory +# - TARGET_BRANCH: Branch name to migrate to +# - TARGET_COMMIT: Specific commit hash to reset to +# - RATOS_FORK_REMOTE: Name of the RatOS fork remote +# - RATOS_FORK_URL: URL of the RatOS fork repository +# - RATOS_USERNAME: System username for ownership +# - RATOS_USERGROUP: System group for ownership +# +# USAGE: +# migrate_klipper_repository +# exit_code=$? +# if [ $exit_code -ne 0 ]; then +# echo "Migration failed with exit code: $exit_code" +# fi +# +migrate_klipper_repository() +{ + log_info "Starting Klipper repository migration to RatOS fork..." "migrate_repository" + + # Check if migration is needed + local check_result + check_klipper_repository + check_result=$? + + if [ $check_result -eq 1 ]; then + # Migration not needed (safe skip) + log_info "Migration not needed, skipping." "migrate_repository" + return 0 + elif [ $check_result -eq 2 ]; then + # Fatal error occurred + log_error "Fatal error during repository check" "migrate_repository" "REPOSITORY_CHECK_FAILED" + return 2 + fi + + # Check for uncommitted changes + local code + check_uncommitted_changes + code=$? + if [ $code -eq 2 ]; then + # Directory access error + log_error "Cannot access Klipper directory for uncommitted changes check" "migrate_repository" "KLIPPER_DIR_ACCESS_FAILED" + return 2 + elif [ $code -eq 3 ]; then + # Uncommitted changes detected + log_error "Uncommitted changes prevent migration" "migrate_repository" "KLIPPER_UNCOMMITTED_CHANGES" + return 3 + fi + + # Handle existing remote + handle_existing_remote + code=$? + if [ $code -ne 0 ]; then + log_error "Failed to handle existing remote (exit code $code)" "migrate_repository" "REMOTE_SETUP_FAILED" + return 4 + fi + + # Fetch from RatOS fork + fetch_ratos_fork + code=$? + if [ $code -ne 0 ]; then + log_error "Failed to fetch from RatOS fork (exit code $code)" "migrate_repository" "FETCH_FAILED" + return 5 + fi + + # Checkout target branch + checkout_target_branch + code=$? + if [ $code -ne 0 ]; then + log_error "Failed to checkout target branch (exit code $code)" "migrate_repository" "CHECKOUT_FAILED" + return 6 + fi + + # Reset to target commit + reset_to_target_commit + code=$? + if [ $code -ne 0 ]; then + log_error "Failed to reset to target commit (exit code $code)" "migrate_repository" "RESET_FAILED" + return 7 + fi + + # Fix ownership + fix_klipper_ownership + code=$? + if [ $code -ne 0 ]; then + log_error "Failed to fix ownership (exit code $code)" "migrate_repository" "OWNERSHIP_FAILED" + return 8 + fi + + log_info "Klipper repository migration completed successfully!" "migrate_repository" + log_info "Repository is now using RatOS fork at commit $TARGET_COMMIT" "migrate_repository" + log_info "Branch: $TARGET_BRANCH" "migrate_repository" + log_info "Remote: $RATOS_FORK_URL" "migrate_repository" + + return 0 +} + +# Main execution +migrate_klipper_repository +code=$? + +# Create log summary and complete +create_log_summary "klipper-fork-migration.sh" "$START_TIME" +log_script_complete "klipper-fork-migration.sh" "$code" + +if [ $code -ne 0 ]; then + log_error "Klipper repository migration failed (exit code $code)!" "main" "KLIPPER_MIGRATION_FAILED" + exit $code +fi + +log_info "Klipper repository migration script completed successfully" "main" +exit 0 diff --git a/configuration/scripts/ratos-update.sh b/configuration/scripts/ratos-update.sh index 31986d6ad..a660c0ce0 100755 --- a/configuration/scripts/ratos-update.sh +++ b/configuration/scripts/ratos-update.sh @@ -135,6 +135,41 @@ symlink_extensions() fi } + +# ensure_klipper_fork_migration() - Ensures Klipper repository is migrated to RatOS fork +# +# This function delegates all repository state validation and migration logic to the +# dedicated klipper-fork-migration.sh script, which provides comprehensive handling of: +# - Official Klipper repositories (migration needed) +# - RatOS fork repositories at correct state (migration not needed) +# - RatOS fork repositories at incorrect state (migration needed) +# - Unsupported repository sources (fatal error) +# - All edge cases including detached HEAD, uncommitted changes, etc. +# +# RETURN CODES: +# 0 - Success: Migration completed successfully or was not needed +# 1+ - Error: Migration failed (specific error codes from migration script) +# +ensure_klipper_fork_migration() +{ + log_info "Ensuring Klipper repository is properly configured..." "ensure_klipper_migration" + + # Delegate all repository validation and migration logic to the dedicated script + # The migration script handles all scenarios comprehensively: + # - Repository state validation with 4 distinct scenarios + # - Graceful skipping when migration is not needed + # - Comprehensive error handling and edge case management + # - Consistent logging and error reporting + if ! "$SCRIPT_DIR"/klipper-fork-migration.sh; then + local code=$? + log_error "Klipper fork migration failed (exit code $code)!" "ensure_klipper_migration" "KLIPPER_MIGRATION_FAILED" + return $code + fi + + log_info "Klipper repository configuration verified successfully!" "ensure_klipper_migration" + return 0 +} + # Main execution with error handling main() { local exit_code=0 @@ -145,6 +180,7 @@ main() { # Use set +e to prevent immediate exit on function failure set +e + ensure_klipper_fork_migration || exit_code=1 update_symlinks || exit_code=1 ensure_sudo_command_whitelisting || exit_code=1 ensure_service_permission || exit_code=1 diff --git a/scripts/README.md b/scripts/README.md new file mode 100644 index 000000000..1cc44e15d --- /dev/null +++ b/scripts/README.md @@ -0,0 +1,155 @@ +# Scripts Directory + +This directory contains standalone utility scripts for the RatOS-configurator project. + +## Available Scripts + +### `validate-bash-syntax.sh` + +A comprehensive bash script syntax validation tool with parallel processing for optimal performance. + +#### Features + +- **Automatic Discovery**: Finds `.sh` files and files with bash shebangs +- **Parallel Processing**: Uses `xargs -P` for fast validation across multiple CPU cores +- **Smart Exclusions**: Automatically excludes `node_modules`, `.git`, and `.augment` directories +- **Flexible Configuration**: Supports custom directories, exclusion patterns, and parallelism limits +- **Cross-Platform**: Works on Linux, macOS, and other Unix-like systems +- **CI/Local Integration**: Optimized for both automated workflows and local development + +#### Usage + +```bash +# Basic usage - validate all scripts in repository +./scripts/validate-bash-syntax.sh + +# Validate specific directory +./scripts/validate-bash-syntax.sh --directory ./configuration + +# Quiet mode for CI integration +./scripts/validate-bash-syntax.sh --quiet + +# Verbose mode for debugging +./scripts/validate-bash-syntax.sh --verbose + +# Custom parallelism +./scripts/validate-bash-syntax.sh --max-parallel 4 + +# Additional exclusions +./scripts/validate-bash-syntax.sh --exclude "*/test/*" --exclude "*/backup/*" +``` + +#### Command Line Options + +| Option | Description | Default | +|--------|-------------|---------| +| `-h, --help` | Show help message and exit | - | +| `-p, --max-parallel N` | Maximum parallel processes | Auto-detect (max 8) | +| `-d, --directory DIR` | Directory to validate | Repository root | +| `-e, --exclude PATTERN` | Additional exclusion pattern | - | +| `-v, --verbose` | Enable verbose output | false | +| `-q, --quiet` | Suppress progress indicators | false | + +#### Exit Codes + +| Code | Meaning | +|------|---------| +| `0` | All scripts passed validation or no scripts found | +| `1` | One or more scripts failed syntax validation | +| `2` | Script execution error (invalid arguments, missing dependencies, etc.) | + +#### Integration Examples + +##### Pre-commit Hook + +Add to `.git/hooks/pre-commit`: + +```bash +#!/bin/bash +echo "Running bash syntax validation..." +if ! ./scripts/validate-bash-syntax.sh --quiet; then + echo "❌ Bash syntax validation failed. Please fix errors before committing." + exit 1 +fi +echo "✅ Bash syntax validation passed." +``` + +##### CI Workflow + +```yaml +- name: Bash Syntax Check + run: ./scripts/validate-bash-syntax.sh --quiet +``` + +##### Local Development + +```bash +# Quick validation during development +./scripts/validate-bash-syntax.sh + +# Validate only configuration scripts +./scripts/validate-bash-syntax.sh -d ./configuration -v + +# Validate with custom exclusions +./scripts/validate-bash-syntax.sh -e "*/deprecated/*" +``` + +##### Custom Scripts + +```bash +#!/bin/bash +if ./scripts/validate-bash-syntax.sh --quiet; then + echo "All scripts are valid, proceeding with deployment..." +else + echo "Script validation failed, aborting deployment." + exit 1 +fi +``` + +#### Performance + +The script automatically detects the number of CPU cores and uses parallel processing for optimal performance: + +- **Single-threaded**: ~1-2 scripts per second +- **Parallel (8 cores)**: ~15-20 scripts per second +- **Large repositories**: Validates 100+ scripts in under 10 seconds + +#### Exclusion Patterns + +Default exclusions: +- `*/node_modules/*` - Node.js dependencies +- `*/.git/*` - Git metadata +- `./.augment/*` - Augment tool files + +Additional exclusions can be added with the `--exclude` option and support standard shell glob patterns. + +## Development + +### Adding New Scripts + +When adding new utility scripts to this directory: + +1. **Make scripts executable**: `chmod +x scripts/your-script.sh` +2. **Add proper shebang**: `#!/bin/bash` +3. **Include help option**: Support `-h` or `--help` +4. **Document in this README**: Add usage examples and description +5. **Test with validator**: Run `./scripts/validate-bash-syntax.sh -d ./scripts` + +### Script Standards + +- Use `#!/bin/bash` shebang +- Include `set -euo pipefail` for strict error handling +- Provide help documentation with `--help` +- Use consistent exit codes (0 = success, 1+ = various errors) +- Include comprehensive error messages +- Support both verbose and quiet modes when appropriate + +### Testing + +```bash +# Test all scripts in this directory +./scripts/validate-bash-syntax.sh --directory ./scripts --verbose + +# Test specific script +bash -n ./scripts/your-script.sh +``` diff --git a/scripts/validate-bash-syntax.sh b/scripts/validate-bash-syntax.sh new file mode 100755 index 000000000..60f58147c --- /dev/null +++ b/scripts/validate-bash-syntax.sh @@ -0,0 +1,451 @@ +#!/bin/bash + +# validate-bash-syntax.sh - Standalone bash script syntax validation tool +# +# This script provides comprehensive bash script syntax validation with parallel processing +# for improved performance. It can be used in CI workflows, local development, and other +# automation contexts. +# +# FEATURES: +# - Discovers bash scripts (.sh files and files with bash shebangs) +# - Parallel validation using xargs -P for optimal performance +# - Comprehensive error reporting with clear output formatting +# - Configurable exclusion patterns and parallel processing limits +# - Works from any directory (automatically detects repository root) +# - Supports both CI and local development workflows +# +# USAGE: +# ./scripts/validate-bash-syntax.sh [OPTIONS] +# +# OPTIONS: +# -h, --help Show this help message +# -p, --max-parallel N Maximum parallel processes (default: auto-detect, max 8) +# -d, --directory DIR Directory to validate (default: repository root) +# -e, --exclude PATTERN Additional exclusion pattern (can be used multiple times) +# -v, --verbose Enable verbose output +# -q, --quiet Suppress progress indicators (errors still shown) +# +# EXIT CODES: +# 0 - All scripts passed validation or no scripts found +# 1 - One or more scripts failed syntax validation +# 2 - Script execution error (invalid arguments, missing dependencies, etc.) +# +# EXAMPLES: +# ./scripts/validate-bash-syntax.sh # Validate all scripts in repository +# ./scripts/validate-bash-syntax.sh -p 4 # Use max 4 parallel processes +# ./scripts/validate-bash-syntax.sh -d ./config # Validate only scripts in config directory +# ./scripts/validate-bash-syntax.sh -e "*/test/*" # Exclude additional test directory +# ./scripts/validate-bash-syntax.sh -q # Quiet mode for CI integration +# + +set -euo pipefail + +# Script metadata +readonly SCRIPT_NAME="validate-bash-syntax.sh" +readonly SCRIPT_VERSION="1.0.0" + +# Default configuration +DEFAULT_MAX_PARALLEL=0 # 0 means auto-detect +DEFAULT_DIRECTORY="" # Empty means auto-detect repository root +DEFAULT_EXCLUSIONS=( + "*/node_modules/*" + "*/.git/*" + "./.augment/*" +) +VERBOSE=false +QUIET=false + +# Global variables +MAX_PARALLEL=$DEFAULT_MAX_PARALLEL +VALIDATION_DIRECTORY="$DEFAULT_DIRECTORY" +EXCLUSIONS=("${DEFAULT_EXCLUSIONS[@]}") + +# Color codes for output formatting +readonly RED='\033[0;31m' +readonly GREEN='\033[0;32m' +readonly YELLOW='\033[1;33m' +readonly BLUE='\033[0;34m' +readonly NC='\033[0m' # No Color + +# Logging functions +log_info() { + if [[ "$QUIET" != true ]]; then + echo -e "${BLUE}ℹ️ $1${NC}" + fi +} + +log_success() { + echo -e "${GREEN}✅ $1${NC}" +} + +# shellcheck disable=SC2317 # Function may be used conditionally or in future features +log_warning() { + echo -e "${YELLOW}⚠️ $1${NC}" +} + +log_error() { + echo -e "${RED}❌ $1${NC}" >&2 +} + +log_verbose() { + if [[ "$VERBOSE" == true ]]; then + echo -e "${BLUE}🔍 $1${NC}" + fi +} + +# Help function +show_help() { + cat << EOF +$SCRIPT_NAME v$SCRIPT_VERSION - Bash Script Syntax Validation Tool + +USAGE: + $SCRIPT_NAME [OPTIONS] + +DESCRIPTION: + Validates bash script syntax with parallel processing for optimal performance. + Automatically discovers .sh files and files with bash shebangs, excluding + common directories like node_modules, .git, and .augment. + +OPTIONS: + -h, --help Show this help message and exit + -p, --max-parallel N Maximum parallel processes (default: auto-detect, max 8) + -d, --directory DIR Directory to validate (default: repository root) + -e, --exclude PATTERN Additional exclusion pattern (can be used multiple times) + -v, --verbose Enable verbose output for debugging + -q, --quiet Suppress progress indicators (errors still shown) + +EXIT CODES: + 0 - All scripts passed validation or no scripts found + 1 - One or more scripts failed syntax validation + 2 - Script execution error (invalid arguments, missing dependencies, etc.) + +EXAMPLES: + $SCRIPT_NAME # Validate all scripts in repository + $SCRIPT_NAME -p 4 # Use max 4 parallel processes + $SCRIPT_NAME -d ./configuration # Validate only scripts in configuration directory + $SCRIPT_NAME -e "*/test/*" # Exclude additional test directory + $SCRIPT_NAME -q # Quiet mode for CI integration + $SCRIPT_NAME -v -p 2 # Verbose mode with 2 parallel processes + +INTEGRATION: + CI Workflows: Use -q flag for clean CI output + Pre-commit: Add to .git/hooks/pre-commit + Local Dev: Run without flags for full interactive output + Custom Scripts: Check exit code for automation integration + +EOF +} + +# Parse command line arguments +parse_arguments() { + while [[ $# -gt 0 ]]; do + case $1 in + -h|--help) + show_help + exit 0 + ;; + -p|--max-parallel) + if [[ -z "${2:-}" ]] || [[ "$2" =~ ^- ]]; then + log_error "Option $1 requires a numeric argument" + exit 2 + fi + if ! [[ "$2" =~ ^[0-9]+$ ]] || [[ "$2" -lt 1 ]]; then + log_error "Max parallel processes must be a positive integer, got: $2" + exit 2 + fi + MAX_PARALLEL="$2" + shift 2 + ;; + -d|--directory) + if [[ -z "${2:-}" ]] || [[ "$2" =~ ^- ]]; then + log_error "Option $1 requires a directory argument" + exit 2 + fi + if [[ ! -d "$2" ]]; then + log_error "Directory does not exist: $2" + exit 2 + fi + VALIDATION_DIRECTORY="$2" + shift 2 + ;; + -e|--exclude) + if [[ -z "${2:-}" ]] || [[ "$2" =~ ^- ]]; then + log_error "Option $1 requires a pattern argument" + exit 2 + fi + EXCLUSIONS+=("$2") + shift 2 + ;; + -v|--verbose) + VERBOSE=true + shift + ;; + -q|--quiet) + QUIET=true + shift + ;; + -*) + log_error "Unknown option: $1" + log_error "Use -h or --help for usage information" + exit 2 + ;; + *) + log_error "Unexpected argument: $1" + log_error "Use -h or --help for usage information" + exit 2 + ;; + esac + done +} + +# Detect repository root directory +detect_repository_root() { + local current_dir="$PWD" + + # Try to find .git directory by walking up the directory tree + while [[ "$current_dir" != "/" ]]; do + if [[ -d "$current_dir/.git" ]]; then + echo "$current_dir" + return 0 + fi + current_dir="$(dirname "$current_dir")" + done + + # If no .git found, use current directory + echo "$PWD" +} + +# Set up validation directory +setup_validation_directory() { + if [[ -z "$VALIDATION_DIRECTORY" ]]; then + VALIDATION_DIRECTORY="$(detect_repository_root)" + log_verbose "Auto-detected repository root: $VALIDATION_DIRECTORY" + else + # Convert to absolute path + VALIDATION_DIRECTORY="$(cd "$VALIDATION_DIRECTORY" && pwd)" + log_verbose "Using specified directory: $VALIDATION_DIRECTORY" + fi + + if [[ ! -d "$VALIDATION_DIRECTORY" ]]; then + log_error "Validation directory does not exist: $VALIDATION_DIRECTORY" + exit 2 + fi + + if [[ ! -r "$VALIDATION_DIRECTORY" ]]; then + log_error "Validation directory is not readable: $VALIDATION_DIRECTORY" + exit 2 + fi +} + +# Determine optimal parallelism +setup_parallelism() { + if [[ "$MAX_PARALLEL" -eq 0 ]]; then + # Auto-detect number of CPU cores + if command -v nproc >/dev/null 2>&1; then + MAX_PARALLEL=$(nproc) + elif command -v sysctl >/dev/null 2>&1; then + # macOS fallback + MAX_PARALLEL=$(sysctl -n hw.ncpu 2>/dev/null || echo 2) + else + # Conservative fallback + MAX_PARALLEL=2 + fi + + # Cap at 8 to avoid overwhelming systems + if [[ "$MAX_PARALLEL" -gt 8 ]]; then + MAX_PARALLEL=8 + fi + + log_verbose "Auto-detected parallelism: $MAX_PARALLEL processes" + else + log_verbose "Using specified parallelism: $MAX_PARALLEL processes" + fi +} + +# Build find exclusion arguments +build_exclusion_args() { + local exclusion_args=() + + for pattern in "${EXCLUSIONS[@]}"; do + exclusion_args+=("-not" "-path" "$pattern") + done + + printf '%s\n' "${exclusion_args[@]}" +} + +# Discover bash scripts in the validation directory +discover_bash_scripts() { + local bash_files=() + local exclusion_args + + # Build exclusion arguments (redirect to stderr to avoid mixing with output) + readarray -t exclusion_args < <(build_exclusion_args) + + # Log to stderr to avoid mixing with file list output + log_info "Finding bash scripts in: $VALIDATION_DIRECTORY" >&2 + log_verbose "Using exclusion patterns: ${EXCLUSIONS[*]}" >&2 + + # Find all .sh files + while IFS= read -r -d '' file; do + bash_files+=("$file") + done < <(find "$VALIDATION_DIRECTORY" -name "*.sh" -type f "${exclusion_args[@]}" -print0 2>/dev/null) + + log_verbose "Found ${#bash_files[@]} .sh files" >&2 + + # Find files with bash shebangs (excluding .sh files already found) + while IFS= read -r -d '' file; do + # Skip if it's already a .sh file + if [[ "$file" != *.sh ]]; then + bash_files+=("$file") + fi + done < <(find "$VALIDATION_DIRECTORY" -type f -not -name "*.*" "${exclusion_args[@]}" -exec grep -q "^#!/bin/bash\|^#!/usr/bin/env bash" {} \; -print0 2>/dev/null) + + log_verbose "Total bash scripts discovered: ${#bash_files[@]}" >&2 + + # Return the array by printing each element on a separate line to stdout + printf '%s\n' "${bash_files[@]}" +} + +# Validation function for individual scripts (used by xargs) +# shellcheck disable=SC2317 # Function is called indirectly via xargs and export -f +validate_script() { + local script="$1" + local results_file="$2" + local errors_file="$3" + local output_dir="$4" + + # Make paths relative to validation directory for cleaner output + local display_path="${script#"$VALIDATION_DIRECTORY"/}" + if [[ "$display_path" == "$script" ]]; then + display_path="$script" # Keep absolute path if not under validation directory + fi + + # Create a unique output file for this validation to avoid intermingled output + local script_hash + script_hash=$(echo "$script" | sha256sum | cut -d' ' -f1) + local output_file="$output_dir/validation_$script_hash.out" + + # Capture all output for this script validation + { + if [[ "$QUIET" != true ]]; then + echo "🔍 Checking syntax of: $display_path" + fi + + if bash -n "$script" 2>/dev/null; then + if [[ "$QUIET" != true ]]; then + echo "✅ $display_path - syntax OK" + fi + echo "$script:OK" >> "$results_file" + else + echo "❌ $display_path - syntax ERROR:" + bash -n "$script" 2>&1 | sed 's/^/ /' + echo "$script:ERROR" >> "$results_file" + echo "$script" >> "$errors_file" + fi + + if [[ "$QUIET" != true ]]; then + echo + fi + } > "$output_file" + + # Immediately display the results for real-time feedback + # This provides responsive user experience while maintaining organized output per script + cat "$output_file" +} + +# Main validation function +validate_bash_scripts() { + local bash_files=() + + # Discover bash scripts + readarray -t bash_files < <(discover_bash_scripts) + + if [[ ${#bash_files[@]} -eq 0 ]]; then + log_info "No bash scripts found in the validation directory - skipping syntax validation" + log_info "This is normal for directories that don't contain shell scripts." + return 0 + fi + + log_info "Found ${#bash_files[@]} bash script(s) to validate:" + if [[ "$VERBOSE" == true ]]; then + for script in "${bash_files[@]}"; do + local display_path="${script#"$VALIDATION_DIRECTORY"/}" + echo " - $display_path" + done + echo + fi + + # Create temporary files and directory for results + local validation_results + local validation_errors + local output_dir + validation_results=$(mktemp) + validation_errors=$(mktemp) + output_dir=$(mktemp -d) + + # Note: Cleanup will be handled manually to avoid trap interference with exit codes + + # Export the validation function for xargs + export -f validate_script + export VALIDATION_DIRECTORY + export QUIET + + log_info "Running validation in parallel (max $MAX_PARALLEL processes)..." + if [[ "$QUIET" != true ]]; then + echo + fi + + # Run validation in parallel using xargs + # Each validate_script() call will immediately display its results for real-time feedback + printf '%s\n' "${bash_files[@]}" | xargs -I {} -P "$MAX_PARALLEL" bash -c 'validate_script "$@"' _ {} "$validation_results" "$validation_errors" "$output_dir" + + # Read results and count failures + local failed_count=0 + if [[ -f "$validation_errors" ]]; then + failed_count=$(wc -l < "$validation_errors" 2>/dev/null || echo 0) + fi + + # Report final results + if [[ "$failed_count" -eq 0 ]]; then + log_success "All bash scripts passed syntax validation!" + # Cleanup temporary files and directory + rm -f "$validation_results" "$validation_errors" + rm -rf "$output_dir" + return 0 + else + log_error "$failed_count script(s) failed syntax validation:" + if [[ -f "$validation_errors" ]]; then + while IFS= read -r failed_script; do + local display_path="${failed_script#"$VALIDATION_DIRECTORY"/}" + echo " - $display_path" + done < "$validation_errors" + fi + echo + log_error "Please fix the syntax errors in the above scripts before proceeding." + # Cleanup temporary files and directory + rm -f "$validation_results" "$validation_errors" + rm -rf "$output_dir" + return 1 + fi +} + +# Main execution function +main() { + # Parse command line arguments + parse_arguments "$@" + + # Setup validation environment + setup_validation_directory + setup_parallelism + + # Change to validation directory + cd "$VALIDATION_DIRECTORY" + + # Run validation and propagate exit code + validate_bash_scripts + return $? +} + +# Execute main function with all arguments and exit with its return code +main "$@" +exit $? diff --git a/src/bin/ratos b/src/bin/ratos index ac4731211..cbb2f47f9 100755 --- a/src/bin/ratos +++ b/src/bin/ratos @@ -1,4 +1,4 @@ -#!/bin/bash +#!/usr/bin/env bash SCRIPT_DIR=$( cd -- "$( dirname -- "$(realpath -- "${BASH_SOURCE[0]}")" )" &> /dev/null && pwd ) RATOS_BIN_CWD=$(pwd) export RATOS_BIN_CWD diff --git a/src/scripts/README-mock-logs.md b/src/scripts/README-mock-logs.md index 1677995d2..1c5748a56 100644 --- a/src/scripts/README-mock-logs.md +++ b/src/scripts/README-mock-logs.md @@ -62,6 +62,10 @@ The mock data generator creates realistic update log scenarios including: - `PERMISSION_DENIED` - File permission issues - `DISK_SPACE_LOW` - Insufficient disk space - `BACKUP_FAILED` - Backup operation failures +- `KLIPPER_MIGRATION_FAILED` - Klipper repository migration failures +- `GIT_FETCH_FAILED` - Git fetch operation failures +- `GIT_CHECKOUT_FAILED` - Git checkout operation failures +- `KLIPPER_UNCOMMITTED_CHANGES` - Uncommitted changes in Klipper repository ### Test Scenarios diff --git a/src/scripts/generate-mock-logs.js b/src/scripts/generate-mock-logs.js index 01e6ebcfe..c4889006d 100644 --- a/src/scripts/generate-mock-logs.js +++ b/src/scripts/generate-mock-logs.js @@ -44,6 +44,10 @@ const ERROR_CODES = [ 'PERMISSION_DENIED', 'DISK_SPACE_LOW', 'BACKUP_FAILED', + 'KLIPPER_MIGRATION_FAILED', + 'GIT_FETCH_FAILED', + 'GIT_CHECKOUT_FAILED', + 'KLIPPER_UNCOMMITTED_CHANGES', ]; // Mock hostnames diff --git a/src/scripts/generate-mock-update-logs.ts b/src/scripts/generate-mock-update-logs.ts index 7b6652e6f..eff01d609 100644 --- a/src/scripts/generate-mock-update-logs.ts +++ b/src/scripts/generate-mock-update-logs.ts @@ -58,6 +58,10 @@ const ERROR_CODES = [ 'PERMISSION_DENIED', 'DISK_SPACE_LOW', 'BACKUP_FAILED', + 'KLIPPER_MIGRATION_FAILED', + 'GIT_FETCH_FAILED', + 'GIT_CHECKOUT_FAILED', + 'KLIPPER_UNCOMMITTED_CHANGES', ]; // Mock hostnames