GSOC 2025 || Discussion on Proposal 3 - Airborne Wildlife Benchmark Dataset

[Abhishek-Dimri]

Hi @bw4sz, @henrykironde, and @ethanwhite,

I’m excited to work on Proposal 3: The Airborne Wildlife Benchmark Dataset and would love to discuss my approach before finalizing my proposal. Below is an outline of my understanding, project breakdown, and a few key questions to ensure alignment with project goals.

---

Understanding the Problem & Approach

Airborne wildlife datasets are fragmented, inconsistently annotated, and lack standardization, making it difficult to train general-purpose animal detectors. Inspired by MillionTrees, this project aims to create a MillionAnimals benchmark—organizing datasets, training a general wildlife detector, and integrating it with DeepForest.

🔹 Proposed Workflow:

1. Dataset Collection – Identify and standardize publicly available airborne wildlife datasets.
2. Benchmark Creation – Adapt MillionTrees’ framework to define dataset structure, evaluation metrics, and annotation formats.
3. Model Training – Train a baseline animal detector and experiment with transfer learning.
4. Integration & Documentation – Ensure compatibility with DeepForest and VisionAgent while providing detailed documentation.

---

Key Discussion Points & Questions

🔸 Dataset Standardization

• Are there any pre-selected datasets, or should I independently gather sources?
• I noticed that Pascal VOC is the preferred annotation format for DeepForest. Should we stick to this format for MillionAnimals, or is there a need to explore other formats?

🔸 Baseline Model Development

• Should the baseline model focus solely on detecting animals in general, or should it also include coarse species-level classification (e.g., birds vs. mammals)?
• I was unable to find a specific benchmark evaluation metric in the MillionTrees documentation. Based on common practices, I assume we will use mAP, F1-score, and recall per species for evaluation. Could you confirm if these align with our goals, or if any other metric is preferred?

🔸 DeepForest & VisionAgent Integration

• Would VisionAgent primarily assist with active learning for better dataset annotation, or should it be leveraged for model deployment?
• What are the best practices for ensuring efficient dataset loading and model training within DeepForest, especially for large-scale datasets like MillionAnimals?

---

Proposed Deliverables

📌 Minimal Deliverables:
✅ Standardized MillionAnimals benchmark dataset
✅ Baseline wildlife detection model
✅ DeepForest integration for reproducible training
✅ Complete documentation for dataset and model usage

🚀 Stretch Goals (If Time Permits):

• Explore semi-supervised learning to enhance model performance
• Add support for additional datasets or species-based classification

---

Next Steps

📌 1. Feedback on Approach: Does this plan align with the project’s objectives? Any suggestions to refine it?
📌 2. Resources & References: Any specific repositories or datasets I should explore before finalizing my proposal?
📌 3. Getting Started: To validate feasibility, I could start by standardizing a small subset of airborne datasets and training a minimal model. Would this be a useful starting point?

Looking forward to your insights!

Best,
Abhishek Dimri

[naxatra2]

I am not the mentor so IDK the actual parts of this project. But, I guess this issue is related to it #915. It looks kind of related.

[bw4sz]

This is all sounds good, but just dropping any of the vision agent or LLM integration that is a whole other, very exploratory project that shouldn't involve this.

I think the community has moved more to COCO annotation, but more importantly the entire idea behind wrapping datasets into pytorch datasets is to avoid any of that from the user side, the user just sees

from MillionAnimals.datasets import dataset
ds = dataset()
dataloader = ds.get_train_loader()
for batch in dataloader:
    image, label, metadata
``

The entire idea is based on the https://github.com/p-lambda/wilds benchmark which inspired us to move in this directly, a bit of the torchgeo datasets as well https://torchgeo.readthedocs.io/en/latest/api/datasets.html, have a look at those and study the overall idea and workflow. Wilds doesn't have many object detection sets, so you have to imagine some of those changes that are in MillionTrees.