Cell Tracking in Microscopy Videos

A computer vision project implementing deep learning-based cell detection and Kalman filter tracking for microscopy videos.

Overview

This project addresses the challenge of detecting and tracking individual cells in time-lapse microscopy videos. Using the BF-C2DL-HSC dataset from the Cell Tracking Challenge, we implement:

1. Cell Segmentation: A U-Net architecture with an EfficientNet-B0 backbone, trained using Focal Tversky Loss for accurate cell detection and segmentation, particularly effective in scenarios with class imbalance.
2. Cell Tracking: The DeepSORT algorithm is employed for robust multi-object tracking, associating segmented cells across frames to maintain their identities over time.

The solution enables analysis of cell behavior (movement patterns, division events, etc.) for biomedical research applications.

Features

• Deep Learning for Segmentation: Utilizes a U-Net model with an EfficientNet-B0 encoder pre-trained on ImageNet.
• Advanced Loss Function: Employs Focal Tversky Loss to handle class imbalance effectively during segmentation model training.
• Robust Tracking: Implements DeepSORT, combining appearance features (via a MobileNet-based embedder) and motion information (via Kalman filtering) for reliable cell tracking.
• Data Augmentation: Leverages Albumentations for comprehensive image augmentation to improve model generalization.
• CTC Compatibility: Prepares outputs in the format required by the Cell Tracking Challenge evaluation tools and includes steps for running the official evaluation scripts.
• Modular Pipeline: Presented in a Jupyter notebook (cv.ipynb) with clear sections for each stage of the workflow: setup, data processing, model training, inference, tracking, and evaluation.
• Visualization: Includes generation of videos overlaying predicted masks on raw images.

Dataset

This project uses the BF-C2DL-HSC dataset from the Cell Tracking Challenge:

• Mouse hematopoietic stem cells in hydrogel microwells
• Bright Field microscopy imaging
• Low contrast with challenging cell movements
• ~2,400 frames (1.6GB)

Installation

# Clone repository
git clone https://github.com/yourusername/cell_tracking.git
cd cell_tracking

# Create and activate conda environment
conda create -n cell_tracking python=3.10
conda activate cell_tracking

# Install dependencies
pip install -r requirements.txt

# Download dataset
./scripts/download_dataset.sh

# (Optional) Download CTC Evaluation Software
# mkdir Official_CTC_EvaluationSoftware && cd Official_CTC_EvaluationSoftware
# wget https://public.celltrackingchallenge.net/software/EvaluationSoftware.zip
# unzip EvaluationSoftware.zip
# cd ..

Project Structure

cell_tracking/
├── data/                     # Data directory (gitignored, create manually or via script)
│   ├── raw/                  # Raw image sequences (e.g., BF-C2DL-HSC/01/)
│   │   └── BF-C2DL-HSC/
│   │       ├── 01/           # Image files (t000.tif, t001.tif, ...)
│   │       └── 01_GT/        # Ground truth (SEG, TRA)
│   └── processed/            # Processed data (if any)
├── outputs_foolproof/        # Output directory (gitignored)
│   ├── masks_pred/           # Predicted segmentation masks
│   ├── tracks_pred/          # Tracking results (res_track.txt)
│   ├── visualizations/       # Output videos and plots
│   └── 01_RES/               # Results formatted for CTC evaluation
│   └── 01_GT/                # Copied GT for CTC evaluation
├── scripts/                  # Helper scripts (e.g., download_dataset.sh)
├── Official_CTC_EvaluationSoftware/ # CTC evaluation tools (gitignored, download manually)
├── .gitignore                # Specifies intentionally untracked files
├── cv.ipynb                  # Main Jupyter notebook with the pipeline
├── README.md                 # This file
└── requirements.txt          # Python dependencies

Usage

1. Prerequisites: Ensure you have Conda installed and the required hardware (GPU recommended for training).
2. Installation: Follow the steps in the Installation section to set up the environment and download the dataset.
3. Download Evaluation Software (Optional but Recommended):
   • Download the official CTC evaluation software from the Cell Tracking Challenge website.
   • Extract it into the Official_CTC_EvaluationSoftware directory (or update the path in the notebook).
4. Run the Pipeline:
   • Open and run the Jupyter notebook cv.ipynb.
   • Execute the cells sequentially. You may need to adjust paths in the "Configuration: Paths and Hyperparameters" section, especially PATH_TO_CTC_EVALUATION_EXECUTABLES.
5. View Results:
   • Predicted masks will be saved in outputs_foolproof/masks_pred/.
   • Tracking results will be in outputs_foolproof/tracks_pred/.
   • Visualizations (videos, plots) will be in outputs_foolproof/visualizations/.
   • CTC evaluation scores will be printed in the notebook and saved in outputs_foolproof/01_RES/TRA_log.txt.

Contributing

Contributions are welcome! Please feel free to submit a pull request or open an issue for any bugs, suggestions, or enhancements.

License

This project is licensed under the MIT License