Skip to content

README.md

Latest commit

 

History

History
140 lines (109 loc) · 5.71 KB

README.md

File metadata and controls

140 lines (109 loc) · 5.71 KB
Raw

Visual Servoing Navigation for Crop Row Detection

The goal of this project is to develop a visual servoing algorithm capable of navigating a field robotbetween two middle crop rows using data from a front-facing camera. The robot must continuously maintain its path by minimizing two key navigation errors:

  1. Heading Error – the angular difference between the robot’s heading and the direction of thecentral navigation line formed between the two middle crop rows
  2. Cross-Track Error – the lateral distance between the robot’s position and the midpoint lineseparating the two crop rows.

The visual servoing system thus acts as a perception-driven guidance layer, allowing the robot toalign itself relative to the crop structure without external localization aids.

This project implements three approaches for detecting crop rows in agricultural images/videos and calculating navigation errors (cross-track error and heading error) for autonomous robot navigation.

clusters traj_1 traj_2

Project Structure

.
├── README.md                      # Project documentation
├── requirements.txt               # Python dependencies
├── src/                           # Source code directory
│   ├── __init__.py
│   ├── process_navigation_cv.py   # Computer vision approach: line-based detection with peak detection
│   ├── process_navigation_v0.py   # Prototype version (v0): polar coordinate histogram method
│   ├── process_navigation_yolo.py # YOLO segmentation approach: uses trained segmentation model
│   ├── eda_v0.ipynb               # Exploratory data analysis for prototype version
│   └── ai_line_det.ipynb          # AI line detection notebook
└── runs/                          # Model training results
    └── segment/
        └── crop_row_segmentation/ # YOLO segmentation model training outputs
            ├── weights/           # Trained model weights (best.pt, last.pt)
            ├── results.png        # Training metrics
            └── ...                # Additional training artifacts

Main Components

1. src/process_navigation_cv.py

  • Method: Line-based approach with peak detection
  • Features:
    • Processes center portion of image for efficiency
    • Uses green mask filtering (HSV-based) and peak detection to identify crop rows
    • Detects crop row lines using find_best_counts and peak detection
    • Selects the two middle crop rows
    • Calculates cross-track error and heading error
    • Supports image, directory, and video processing
    • Downscales images for faster processing

2. src/process_navigation_v0.py (Prototype Version)

  • Method: Polar coordinate histogram with vanishing point estimation
  • Features:
    • Uses Hough transform to estimate vanishing point
    • Detects crop rows using angle histogram peaks
    • Clusters pixels based on angular proximity to peaks
    • Calculates navigation errors
    • Supports image, directory, and video processing

3. src/process_navigation_yolo.py

  • Method: YOLO segmentation model approach
  • Features:
    • Uses trained YOLO segmentation model to generate binary masks
    • Detects crop row lines using find_best_counts and peak detection
    • Processes entire image (no center portion cropping)
    • Selects the two middle crop rows
    • Calculates cross-track error and heading error
    • Supports image, directory, and video processing
    • Uses model from runs/segment/crop_row_segmentation/weights/best.pt by default

4. src/eda_v0.ipynb

  • Exploratory data analysis notebook for prototype version (v0)
  • Contains experiments with K-means clustering and polar coordinate histograms
  • Development and prototyping experiments

5. src/ai_line_det.ipynb

  • AI line detection analysis notebook
  • Contains ground truth image creation and line detection experiments

Usage

  1. Install dependencies:
pip install -r requirements.txt
  1. Process images/videos:

Computer Vision Approach (CV):

# Process a single image
python src/process_navigation_cv.py input_image.png -o outputs/

# Process a video
python src/process_navigation_cv.py input_video.mp4 -o outputs/

# Process a directory of images
python src/process_navigation_cv.py input_dir/ -o outputs/ -r

Prototype Version (v0):

# Process a single image
python src/process_navigation_v0.py input_image.png -o outputs/

# Process a video
python src/process_navigation_v0.py input_video.mp4 -o outputs/

YOLO Segmentation Approach:

# Process a single image (uses default model)
python src/process_navigation_yolo.py input_image.png -o outputs/

# Process with custom model
python src/process_navigation_yolo.py input_image.png -o outputs/ --model path/to/model.pt

# Process a video
python src/process_navigation_yolo.py input_video.mp4 -o outputs/

# Adjust confidence threshold
python src/process_navigation_yolo.py input_image.png -o outputs/ --conf 0.5

Output

All scripts generate visualized images/videos with:

  • Detected crop row boundaries (orange lines)
  • Ideal robot path (blue line)
  • Current robot path (red dashed line)
  • Cross-track error and heading error annotations

Model Training

The YOLO segmentation model is trained and stored in runs/segment/crop_row_segmentation/weights/. The training results and metrics are available in the same directory.

Notes

  • v0 refers to the Prototype version, which was the initial approach using polar coordinate histograms
  • The CV approach (process_navigation_cv.py) processes only the center portion of images for efficiency
  • The YOLO approach requires a trained model (provided in runs/segment/crop_row_segmentation/weights/best.pt)