Scripts for traning models on idun #28
Conversation
…ge-processing into 13-traning-on-idun
There was a problem hiding this comment.
Pull Request Overview
This PR adds training scripts and configurations for running YOLO and U-Net models on the Idun cluster, while restructuring the project to support multiple model training workflows.
Key Changes:
- Added new training directories with scripts for YOLO and U-Net models
- Created SLURM job configurations for GPU-based training on the Idun cluster
- Updated CI/CD workflows to reference the new requirements.txt location
Reviewed Changes
Copilot reviewed 11 out of 13 changed files in this pull request and generated 3 comments.
Show a summary per file
| File | Description |
|---|---|
| yolo_roboflow_training/train.py | Training script for YOLO model using Roboflow datasets |
| yolo_roboflow_training/requirements.txt | Python dependencies for YOLO training |
| yolo_roboflow_training/Job.slurm | SLURM configuration for YOLO training jobs |
| unet_roboflow_training/train.py | Training script for U-Net segmentation model |
| unet_roboflow_training/test.py | Inference script for testing trained U-Net models |
| unet_roboflow_training/requirements.txt | Python dependencies for U-Net training |
| unet_roboflow_training/Job.slurm | SLURM configuration for U-Net training jobs |
| YOLO-detect-buoys/requirements.txt | Moved requirements file to project directory |
| YOLO-detect-buoys/main.py | Updated to use Unix-style paths and added model export |
| YOLO-detect-buoys/Job.slurm | SLURM configuration for buoy detection training |
| .github/workflows/pylint.yml | Updated to reference new requirements.txt location |
Comments suppressed due to low confidence (2)
YOLO-detect-buoys/Job.slurm:1
- Invalid SLURM time format '999:99:99'. The time format should be DD-HH:MM:SS, HH:MM:SS, or MM:SS with valid values (e.g., hours 0-23, minutes/seconds 0-59).
YOLO-detect-buoys/Job.slurm:1 - The script references 'train.py' but the main entry point in this directory is 'main.py'. This should be 'python3 main.py' or 'python3 -m YOLO-detect-buoys'.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| # Step 1: Download Dataset from Roboflow | ||
| ROBOFLOW_API_KEY = "" # Replace with your Roboflow API Key | ||
| PROJECT_NAME = "" # Replace with your project id | ||
| VERSION = "" # Replace with your dataset version number exluding the 'v' |
There was a problem hiding this comment.
Corrected spelling of 'exluding' to 'excluding'.
| VERSION = "" # Replace with your dataset version number exluding the 'v' | |
| VERSION = "" # Replace with your dataset version number excluding the 'v' |
| # export PYTORCH_CUDA_ALLOC_CONF="expandable_segments:True,max_split_size_mb:128" | ||
| export PYTORCH_CUDA_ALLOC_CONF="expandable_segments:True" | ||
|
|
||
| python3 train.py |
There was a problem hiding this comment.
The SLURM script attempts to run 'train.py' but is located in YOLO-detect-buoys directory. Based on the file location (yolo_roboflow_training/Job.slurm), this should execute the training script in the current directory or specify the correct path.
| python3 train.py | |
| python3 ../YOLO-detect-buoys/train.py |
| #!/bin/bash | ||
| #SBATCH --partition=GPUQ | ||
| #SBATCH --account=ie-idi | ||
| #SBATCH --time=999:99:99 |
There was a problem hiding this comment.
Invalid SLURM time format '999:99:99'. The time format should be DD-HH:MM:SS, HH:MM:SS, or MM:SS with valid values (e.g., hours 0-23, minutes/seconds 0-59).
| #SBATCH --time=999:99:99 | |
| #SBATCH --time=7-00:00:00 |
| from roboflow import Roboflow | ||
|
|
||
| # Step 1: Download Dataset from Roboflow | ||
| ROBOFLOW_API_KEY = "" # Replace with your Roboflow API Key |
There was a problem hiding this comment.
Bad practice. Should instead be loaded from another file which must be added to .gitignore (for example a .env file). This is done to avoid accidentally committing API keys.
No description provided.