Skip to content
Open
Show file tree
Hide file tree
Changes from 34 commits
Commits
Show all changes
45 commits
Select commit Hold shift + click to select a range
82b9f8a
feat: implement Klipper repository migration to RatOS fork
miklschmidt Jun 21, 2025
9fdf7eb
fix: address CodeRabbit AI feedback and ShellCheck warnings
miklschmidt Jun 21, 2025
0ee8fc4
revert: roll back exponential backoff implementation
miklschmidt Jun 21, 2025
79c5978
feat: implement exit code capture and display for better debugging
miklschmidt Jun 21, 2025
94a6bfd
fix: add ShellCheck disable directive for sourced file
miklschmidt Jun 21, 2025
2366f9b
feat: add strict error handling with set -euo pipefail
miklschmidt Jun 21, 2025
3aa2a21
feat(klipper-migration): integrate with unified logging system
miklschmidt Jun 22, 2025
a13c0c5
feat(ratos-update): update klipper migration integration with unified…
miklschmidt Jun 22, 2025
19d24eb
docs(logging): add Klipper migration error codes to unified logging s…
miklschmidt Jun 22, 2025
873c05a
test(logging): update mock data with Klipper migration error codes
miklschmidt Jun 22, 2025
a08bcbc
fix(klipper-migration): standardize error code naming consistency
miklschmidt Jun 22, 2025
8cda3cf
feat(klipper-migration): enhance constants section with validation an…
miklschmidt Jun 22, 2025
e2be0e3
refactor(klipper-migration): optimize uncommitted changes detection w…
miklschmidt Jun 22, 2025
9718010
feat(klipper-migration): add automatic cleanup for temporary migratio…
miklschmidt Jun 22, 2025
9505fab
docs(logging): remove trailing punctuation from headings for MD026 co…
miklschmidt Jun 22, 2025
afe811c
fix(klipper): implement temp branch cleanup in checkout function
miklschmidt Jun 22, 2025
ed5b4f1
fix(klipper): update shebang to use portable env bash
miklschmidt Jun 22, 2025
0dd355d
fix(klipper): standardize error handling for root user check
miklschmidt Jun 22, 2025
8a6a394
feat(klipper): enhance environment variable validation robustness
miklschmidt Jun 22, 2025
33e8a67
refactor(klipper): optimize uncommitted changes detection with Git pl…
miklschmidt Jun 22, 2025
c96cb9c
perf(klipper): optimize remote management logic for better performance
miklschmidt Jun 22, 2025
39b2431
docs(logging): add new error codes for enhanced validation
miklschmidt Jun 22, 2025
c79bdb9
docs(klipper): add comprehensive documentation for migrate_klipper_re…
miklschmidt Jun 22, 2025
f0a85f8
chore(ai): add guidelines for processing task lists
miklschmidt Jun 22, 2025
178fe6f
fix(klipper): fix temporary branch cleanup trap mechanism in checkout…
miklschmidt Jun 22, 2025
71f767a
fix(cli): use /usr/bin/env bash instead of /bin/bash
miklschmidt Jun 22, 2025
84a65a2
ci: Add bash script syntax check to CI
miklschmidt Jun 22, 2025
3068cbd
feat(klipper): enhance temporary branch cleanup with function-level E…
miklschmidt Jun 22, 2025
96ea2b9
feat(klipper): enhance URL detection and error code differentiation
miklschmidt Jun 22, 2025
e9b1950
fix(ci): improve node_modules exclusion pattern in bash script valida…
miklschmidt Jun 22, 2025
f0024d6
fix(ci): correct grep usage with find -print0 to eliminate duplicate …
miklschmidt Jun 22, 2025
cc4f7d8
feat(ci): improve robustness when no bash scripts are found
miklschmidt Jun 22, 2025
70623c5
feat(ci): optimize shell script validation with parallel processing
miklschmidt Jun 22, 2025
39e1997
ci: Improve .git directory exclusion in bash linter
miklschmidt Jun 22, 2025
d60c201
ci: Refine detection of extensionless bash scripts
miklschmidt Jun 22, 2025
b921b93
feat(klipper): improve portability of path resolution with command fa…
miklschmidt Jun 22, 2025
0ccde04
refactor(klipper): optimize git command execution by eliminating dire…
miklschmidt Jun 22, 2025
3ffe6cc
feat(klipper): enhance repository state validation with comprehensive…
miklschmidt Jun 22, 2025
936b35e
feat(klipper): enhance script portability, maintainability, and corre…
miklschmidt Jun 22, 2025
a35ca1b
refactor(update): eliminate redundant repository validation in ensure…
miklschmidt Jun 22, 2025
cd8ebe0
feat(scripts): extract bash validation logic into standalone reusable…
miklschmidt Jun 22, 2025
e6e23c3
fix(scripts): resolve shellcheck SC2317 warnings in validate-bash-syn…
miklschmidt Jun 22, 2025
c95a8bc
feat(scripts): organize parallel validation output to eliminate inter…
miklschmidt Jun 22, 2025
84b7d8e
feat(scripts): add real-time feedback to parallel bash validation
miklschmidt Jun 22, 2025
e6550f4
feat(klipper): dynamically extract target commit from moonraker.conf
miklschmidt Jun 22, 2025
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
19 changes: 19 additions & 0 deletions .augment-guidelines
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,25 @@
- Use `pnpx` instead of `npx`
- Always run the tests before comitting or pushing.

## Task list handling procedure

When you're working through task lists, make sure to follow the following steps and guidelines:

1. **Complete the implementation** - Make all necessary code changes to fully address the task requirements
2. **Create a focused commit** - Before marking any task as complete, create a single atomic commit that contains only the changes for that specific task, using conventional commit format (e.g., "fix(klipper): implement temp branch cleanup in checkout function")
3. **Mark task complete** - Only after the commit is created, update the task status to COMPLETE
4. **Move to next task** - Proceed to the next task in the list

**Specific requirements:**
- Each commit should be atomic and focused on one logical change
- Use conventional commit format: `type(scope): description`
- Include detailed commit messages with bullet points explaining the specific changes made
- Ensure all code changes are tested and functional before committing
- Follow the existing code style and patterns in the repository
- Handle any merge conflicts or issues that arise during the process

Work through each task completely before moving to the next one, ensuring proper version control practices with meaningful commits for each completed task.

# Linting

- use `pnpm run lint` in the `src` folder to run the linter once, without watching for changes.\
Expand Down
96 changes: 96 additions & 0 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -96,6 +96,102 @@ jobs:
repository: Arksine/moonraker
path: "moonraker"

- name: Bash Syntax Check
working-directory: ratos-configurator
run: |
echo "🔍 Finding bash scripts in the repository..."

# Find all .sh files and files with bash shebangs
bash_files=()

# Add all .sh files
while IFS= read -r -d '' file; do
bash_files+=("$file")
done < <(find . -name "*.sh" -type f -not -path "*/node_modules/*" -not -path "*/.git/*" -not -path "./.augment/*" -print0)

# Add files with bash shebangs (excluding .sh files already found)
while IFS= read -r -d '' file; do
# Skip if it's already a .sh file
if [[ "$file" != *.sh ]]; then
bash_files+=("$file")
fi
done < <(find . -path "*/node_modules/*" -prune -o -type f -not -path "*/.git/*" -not -path "./.augment/*" -exec grep -q "^#!/bin/bash\|^#!/usr/bin/env bash" {} \; -print0 2>/dev/null)

if [ ${#bash_files[@]} -eq 0 ]; then
echo "ℹ️ No bash scripts found in the repository - skipping bash syntax validation"
echo "This is normal for branches/forks that don't contain shell scripts."
exit 0
fi

echo "📋 Found ${#bash_files[@]} bash script(s) to validate:"
printf ' - %s\n' "${bash_files[@]}"
echo

# Create a temporary file to store validation results
validation_results=$(mktemp)
validation_errors=$(mktemp)

# Function to validate a single script (will be used by xargs)
validate_script() {
local script="$1"
local results_file="$2"
local errors_file="$3"

echo "🔍 Checking syntax of: $script"
if bash -n "$script" 2>/dev/null; then
echo "✅ $script - syntax OK"
echo "$script:OK" >> "$results_file"
else
echo "❌ $script - syntax ERROR:"
bash -n "$script" 2>&1 | sed 's/^/ /'
echo "$script:ERROR" >> "$results_file"
echo "$script" >> "$errors_file"
fi
echo
}

# Export the function so xargs can use it
export -f validate_script

# Determine optimal parallelism (use number of CPU cores, max 8 to avoid overwhelming GitHub Actions)
max_parallel=$(nproc)
if [ "$max_parallel" -gt 8 ]; then
max_parallel=8
fi

echo "🚀 Running validation in parallel (max $max_parallel processes)..."
echo

# Run validation in parallel using xargs
printf '%s\n' "${bash_files[@]}" | xargs -I {} -P "$max_parallel" bash -c 'validate_script "$@"' _ {} "$validation_results" "$validation_errors"

# Read results and count failures
failed_count=0
if [ -f "$validation_errors" ]; then
failed_count=$(wc -l < "$validation_errors" 2>/dev/null || echo 0)
fi

# Report results
if [ "$failed_count" -eq 0 ]; then
echo "🎉 All bash scripts passed syntax validation!"
else
echo "💥 $failed_count script(s) failed syntax validation:"
if [ -f "$validation_errors" ]; then
while IFS= read -r failed_script; do
echo " - $failed_script"
done < "$validation_errors"
fi
echo
echo "Please fix the syntax errors in the above scripts before merging."

# Cleanup temporary files
rm -f "$validation_results" "$validation_errors"
exit 1
fi

# Cleanup temporary files
rm -f "$validation_results" "$validation_errors"

- name: Lint
working-directory: ratos-configurator/src
run: pnpm run lint:ci
Expand Down
76 changes: 54 additions & 22 deletions LOGGING_SYSTEM.md
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,7 @@ The unified logging system consists of four main components:

The bash logging library provides structured logging capabilities for shell scripts, outputting logs in JSON format compatible with the pino logging system used throughout the application. **All logs are written to the main RatOS log file** (`/var/log/ratos-configurator.log`) with a `source: "ratos-update"` field for filtering.

#### Features:
#### Features
- **JSON-formatted logs** compatible with pino
- **Multiple log levels**: trace, debug, info, warn, error, fatal
- **Unified log file** - writes to main RatOS log instead of separate files
Expand All @@ -26,7 +26,7 @@ The bash logging library provides structured logging capabilities for shell scri
- **Command execution logging** with automatic error handling
- **Timestamped entries** with process information

#### Usage Example:
#### Usage Example
```bash
#!/bin/bash
source "$(dirname "$0")/ratos-logging.sh"
Expand All @@ -49,7 +49,7 @@ execute_with_logging "apt-get update" "package_update" "APT_UPDATE_FAILED"
log_script_complete "my-script.sh" $?
```

#### Configuration:
#### Configuration
- `RATOS_LOG_LEVEL`: Set minimum log level (default: info)
- `RATOS_LOG_FILE`: Log file path (default: uses `${LOG_FILE}` from environment, typically `/var/log/ratos-configurator.log`)
- `RATOS_LOG_MAX_SIZE`: Maximum log file size before rotation (default: 0 = disabled when using main log)
Expand All @@ -61,7 +61,7 @@ log_script_complete "my-script.sh" $?

The CLI provides several commands for viewing and analyzing update logs. **Update logs are now a subcommand of the main `logs` command** and automatically filter the main log file to show only entries with `source: "ratos-update"`.

#### Commands:
#### Commands

**`ratos logs update-logs summary`**
- Shows a summary of the most recent update attempt from the main log
Expand All @@ -81,7 +81,7 @@ The CLI provides several commands for viewing and analyzing update logs. **Updat
- Options:
- `-d, --details`: Show detailed information

#### Usage Examples:
#### Usage Examples
```bash
# Show update summary (note the new command structure)
ratos logs update-logs summary
Expand All @@ -104,31 +104,31 @@ ratos logs rotate # Force log rotation

The web interface provides a comprehensive log viewer accessible at `/configure/update-logs`.

#### Features:
#### Features
- **Log Summary Dashboard**: Overview of recent update attempts
- **Interactive Log Viewer**: Browse and filter log entries
- **Real-time Filtering**: Filter by log level, context, and search terms
- **Error Highlighting**: Visual distinction for different log levels
- **Download Capability**: Download raw log files
- **Auto-refresh**: Automatic updates when new logs are available

#### Components:
#### Components
- `UpdateLogsViewer`: Main component for displaying logs
- `UpdateLogsErrorBoundary`: Error boundary for graceful error handling
- `LogSummaryCard`: Summary statistics and controls
- `LogEntryComponent`: Individual log entry display

### 4. API Endpoints

#### TRPC Endpoints (`src/server/routers/update-logs.ts`):
#### TRPC Endpoints (`src/server/routers/update-logs.ts`)
- `update-logs.summary`: Get log summary statistics (filtered by `source: "ratos-update"`)
- `update-logs.entries`: Get filtered log entries (filtered by `source: "ratos-update"`)
- `update-logs.errors`: Get only errors and warnings (filtered by `source: "ratos-update"`)
- `update-logs.contexts`: Get available log contexts (filtered by `source: "ratos-update"`)
- `update-logs.clear`: **Disabled** - Cannot clear main log file (use log rotation instead)
- `update-logs.download`: Download main log file (contains all sources)

#### REST Endpoints:
#### REST Endpoints
- `GET /api/update-logs/download`: Download log file as attachment

### 5. Debug Integration
Expand Down Expand Up @@ -156,7 +156,7 @@ All logs follow a consistent JSON format:
}
```

### Fields:
### Fields
- `level`: Numeric log level (10=trace, 20=debug, 30=info, 40=warn, 50=error, 60=fatal)
- `time`: ISO 8601 timestamp
- `msg`: Human-readable log message
Expand All @@ -170,7 +170,7 @@ All logs follow a consistent JSON format:

Standardized error codes help identify common issues:

### Update Script Error Codes:
### Update Script Error Codes
- `SCRIPT_ERROR`: General script failure
- `SCRIPT_SUCCESS`: Script completed successfully
- `SYMLINK_CREATE_FAILED`: Failed to create symbolic link
Expand All @@ -180,34 +180,66 @@ Standardized error codes help identify common issues:
- `EXTENSION_SYMLINK_FAILED`: Extension symlinking failed
- `OWNERSHIP_CHANGE_FAILED`: File ownership change failed

### System Error Codes:
### Klipper Migration Error Codes
- `KLIPPER_DIR_NOT_FOUND`: Klipper directory not found
- `KLIPPER_NOT_GIT_REPO`: Klipper directory is not a git repository
- `KLIPPER_DIR_ACCESS_FAILED`: Cannot access Klipper directory
- `KLIPPER_STAGED_CHANGES`: Uncommitted staged changes prevent migration
- `KLIPPER_UNCOMMITTED_CHANGES`: Uncommitted changes prevent migration
- `KLIPPER_MIGRATION_FAILED`: General Klipper migration failure
- `GIT_REMOTE_URL_FAILED`: Failed to get git remote URL
- `GIT_REMOTE_ADD_FAILED`: Failed to add git remote
- `GIT_REMOTE_UPDATE_FAILED`: Failed to update git remote URL
- `GIT_FETCH_FAILED`: Failed to fetch from remote repository
- `GIT_FETCH_RETRY`: Fetch retry attempt
- `GIT_CHECKOUT_FAILED`: Failed to checkout branch
- `GIT_CHECKOUT_REMOTE_FAILED`: Failed to checkout remote branch
- `GIT_TEMP_BRANCH_FAILED`: Failed to create temporary branch
- `GIT_TEMP_BRANCH_CLEANUP`: Temporary branch cleanup operation
- `GIT_TEMP_BRANCH_CLEANUP_FAILED`: Failed to clean up temporary branch
- `GIT_COMMIT_NOT_FOUND`: Target commit not found
- `GIT_RESET_FAILED`: Failed to reset to target commit
- `GIT_UPSTREAM_SET_FAILED`: Failed to set upstream tracking
- `REMOTE_URL_MISMATCH`: Remote URL doesn't match expected value
- `REPOSITORY_CHECK_FAILED`: Repository check failed
- `REMOTE_SETUP_FAILED`: Remote setup failed
- `FETCH_FAILED`: Fetch operation failed
- `CHECKOUT_FAILED`: Checkout operation failed
- `RESET_FAILED`: Reset operation failed
- `OWNERSHIP_FAILED`: Ownership fix failed

### System Error Codes
- `FILE_NOT_FOUND`: Required file not found
- `PERMISSION_DENIED`: Insufficient permissions
- `NETWORK_ERROR`: Network connectivity issue
- `DISK_FULL`: Insufficient disk space
- `ENV_VAR_MISSING`: Required environment variable not set
- `USER_NOT_FOUND`: Required system user account does not exist
- `GROUP_NOT_FOUND`: Required system group does not exist
- `OWNERSHIP_CHANGE_FAILED`: Failed to change file/directory ownership

## Error Handling and Retry Logic

### Bash Scripts:
### Bash Scripts
- Automatic error trapping with `set -eE`
- Stack trace capture on script failure
- Graceful error reporting with context
- Exit codes indicate success/failure status

### Web UI:
### Web UI
- Error boundaries prevent UI crashes
- Automatic retry with exponential backoff
- Graceful degradation when logs unavailable
- User-friendly error messages

### CLI:
### CLI
- Robust error handling for missing files
- Clear error messages with suggested actions
- Non-zero exit codes for scripting

## Monitoring and Alerting

### Log Rotation:
### Log Rotation
- Automatic rotation when files exceed 10MB
- Keeps 5 backup files by default
- Configurable via environment variables
Expand All @@ -219,7 +251,7 @@ Standardized error codes help identify common issues:

## Troubleshooting

### Common Issues:
### Common Issues

**Log file not found:**
- Ensure update scripts have been run at least once
Expand All @@ -241,7 +273,7 @@ Standardized error codes help identify common issues:
- Ensure scripts are using the logging library correctly
- Verify JSON format of log entries

### Debug Commands:
### Debug Commands
```bash
# Check main log file location and size
ls -la /var/log/ratos-configurator.log*
Expand All @@ -265,25 +297,25 @@ shellcheck -ax -s bash configuration/scripts/ratos-update.sh

## Development

### Adding New Log Sources:
### Adding New Log Sources
1. Source the logging library: `source "$(dirname "$0")/ratos-logging.sh"`
2. Set up error trapping: `setup_error_trap "script-name"`
3. Use logging functions: `log_info`, `log_error`, etc.
4. Add appropriate error codes to documentation

### Code Quality Standards:
### Code Quality Standards
- **ShellCheck Compliance**: All bash scripts must pass ShellCheck validation
- **Error Handling**: Use proper error trapping with selective `set +e`/`set -e`
- **Variable Quoting**: Always quote variables and use `read -r` for input
- **Exit Codes**: Use proper exit code handling and propagation

### Testing:
### Testing
- Unit tests in `src/__tests__/update-logs.test.ts`
- Integration tests for CLI commands
- End-to-end tests for web UI
- ShellCheck validation in CI/CD pipeline

### Contributing:
### Contributing
- Follow existing log format and error code conventions
- Run ShellCheck on all bash scripts before committing
- Add tests for new functionality
Expand Down
2 changes: 1 addition & 1 deletion configuration/moonraker.conf
Original file line number Diff line number Diff line change
Expand Up @@ -112,7 +112,7 @@ info_tags:

[update_manager klipper]
channel: dev
pinned_commit: b7233d1197d9a2158676ad39d02b80f787054e20
pinned_commit: 1c96f096fdeea8e2e79237b679ed6fa944fbae5e

[update_manager moonraker]
channel: dev
Expand Down
Loading
Loading