Skip to content

CLAUDE.md

Latest commit

 

History

History
99 lines (74 loc) · 4.54 KB

CLAUDE.md

File metadata and controls

99 lines (74 loc) · 4.54 KB
Raw

CLAUDE.md

This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.

Project Overview

Domain-specific video segmentation research applying SAM 2 (Segment Anything Model 2) to materials science datasets. Developed by ORNLxUTK (Oak Ridge National Laboratory × University of Tennessee, Knoxville). Targets welding (TIG), machining (MAZAK), plasma, and polymer (visible + infrared) video datasets.

Repository Structure

This is a monorepo with 5 git submodules:

  • sam2/ — Meta's SAM 2 framework (forked from facebookresearch/sam2). The core model library.
  • SAM2inference/ — Inference, fine-tuning evaluation, and metrics pipelines. Main entry points: baseline_inference.py, lora_inference.py, fullfinetune_inference.py, metrics.py.
  • Datasets/ — Converts Roboflow/COCO annotations to standard VOC-style segmentation masks.
  • DatasetVariants/ — Creates cross-validation dataset splits and preprocessing variants from the 5 base datasets.
  • irPOLYMERpreprocess/ — Specialized preprocessing for infrared polymer imaging (BM3D denoising, adaptive histogram equalization, normalization strategies).
  • WAAMlabeledDataset/ — WAAM (Wire Arc Additive Manufacturing) MAZAK dataset with annotation conversion tools and interactive prompt creation. Contains the prepared MAZAK_SAM2_Roboflow_Frames/ dataset in VOC-style layout (JPEGImages + Annotations, test split with 6 videos). Two segmentation categories: melt pool (green) and feed wire (white). Annotation ordering uses an area-based heuristic (larger region = melt pool).

Package Management

Uses uv as the package manager. Each submodule has its own pyproject.toml and uv.lock.

# Root project setup
uv sync

# Submodule-specific setup (run from submodule directory)
cd SAM2inference && uv sync
cd irPOLYMERpreprocess && uv sync

The root project installs sam-2 as an editable local dependency from sam2/. SAM2inference does the same via ../sam2.

Python Version Requirements

  • Root: >=3.11
  • SAM2inference: ==3.11.11 (pinned exactly)
  • irPOLYMERpreprocess: >=3.12
  • WAAMlabeledDataset: >=3.12
  • sam2: >=3.10

Key Commands

Inference (from SAM2inference/)

python baseline_inference.py    # Run SAM 2.1 baseline checkpoints (tiny/small/base-plus/large)
python lora_inference.py        # Run LoRA fine-tuned models
python fullfinetune_inference.py # Run fully fine-tuned models
python metrics.py               # Evaluate segmentation results (DAVIS/VOS-style metrics)
python create_prompts.py        # Interactive point prompt creation (mouse clicks)

Dataset Preparation (from DatasetVariants/)

python datasetcombos.py         # Create cross-validation dataset combinations
python preprocess.py            # Apply preprocessing to dataset variants

IR Preprocessing (from irPOLYMERpreprocess/)

python main.py                  # Run full preprocessing pipeline
python global.py --plot         # Global normalization with visualization

WAAM Dataset Preparation (from WAAMlabeledDataset/)

python makeannotationimages.py      # Convert Roboflow COCO JSON → PNG segmentation masks
python createpeftsam2ftdir.py       # Build MAZAK_SAM2_Roboflow_Frames/ directory structure
python create_prompts.py            # Interactive prompt creation (1 click per object default)
python create_all_prompts.py        # Batch prompt creation for 1, 3, and 5 clicks

Linting

ruff check                      # Available in SAM2inference and irPOLYMERpreprocess

Architecture: Model Fine-Tuning Pipeline

Raw Data → Datasets/ (annotation conversion) → DatasetVariants/ (splits + preprocessing)
→ SAM 2 training (sam2/training/) → SAM2inference/ (inference + evaluation)

Three fine-tuning strategies are compared:

  1. Baseline — Pre-trained SAM 2.1 checkpoints, no adaptation
  2. LoRA — Low-rank adaptation via PEFT library (ranks: 2, 4, 16, 32)
  3. Full fine-tune — All model weights updated

Model sizes tested: tiny, small, base-plus, large. Each dataset uses 5-fold cross-validation.

Key Conventions

  • LoRA checkpoint directories follow the naming pattern: {dataset}_{LoRA_rank}_{baseline_size}
  • Segmentation mask colors: 0 (white) = wire/nozzle, 1 (green) = material, background = black
  • SAM2inference uses pydantic for configuration validation
  • Metrics evaluation is based on DAVIS 2017 / VOS benchmark standards
  • Datasets: TIG (01-05), PLASMA (01-05), visPOLYMER (01-05), MAZAK (01-05), irPOLYMER (01-05) plus normalized variants