Segmentation

docs/getting_started/focused_getting_starteds/Segmentation/index.md

@@ -0,0 +1,37 @@
+# Getting Started with Segmentation in FiftyOne
+
+## Who this is for
+
+This page is for those new to FiftyOne looking to get started with segmentation workflows! 
+We will cover how to load, visualize, enrich, and evaluate segmentation datasets with FiftyOne.
+
+This tutorial is ideal for computer vision engineers and AI researchers working with instance and semantic segmentation tasks. 
+Some basic knowledge of Python and computer vision is assumed.
+
+## Assumed Knowledge
+
+We assume familiarity with common segmentation tasks (semantic and instance), dataset formats (e.g., COCO), and how masks or polygons are used in visual tasks.
+
+## Time to complete
+
+20–30 minutes
+
+## Required packages
+
+To follow along, you’ll need the following packages:
+
+```bash
+pip install fiftyone opencv-python-headless pillow matplotlib
+pip install torch torchvision
+```
+
+## Content
+
+### [Step 1: Loading Segmentation Datasets](./step1.ipynb)
+Learn how to load semantic and instance segmentation datasets from FiftyOne’s zoo and from custom formats like COCO or segmentation masks.
+
+### [Step 2: Adding Instance Segmentations](./step2.ipynb)
+Enrich your dataset by adding segmentation predictions using both built-in models (e.g., SAM2) and your own custom models, with polygon and bounding box support.
+
+### [Step 3: Segment Anything 2 (SAM2) in FiftyOne](./step3.ipynb)
+Explore SAM 2’s groundbreaking capabilities for image and video segmentation. Use bounding boxes, keypoints, or zero prompts, and run video mask propagation from a single frame.

docs/getting_started/focused_getting_starteds/Segmentation/step1.ipynb

@@ -0,0 +1,170 @@
+{
+ "cells": [
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "# Step 1: Loading a Segmentation Dataset in FiftyOne"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "In this first step, we will explore how to load **segmentation datasets** into FiftyOne. Segmentation datasets may be of two types: **semantic segmentation** (pixel-wise class labels) and **instance segmentation** (individual object masks). \n",
+    "\n",
+    "FiftyOne makes it easy to load both types using its Dataset Zoo or from custom formats like COCO or FiftyOne format. Let's start by loading a common format instance segmentation dataset."
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## Loading a Common Format Segmentation Dataset\n",
+    "Segmentation datasets are often provided in standard formats such as COCO, VOC, YOLO, KITTI, and FiftyOne format. FiftyOne supports direct ingestion of these datasets with just a few lines of code.\n",
+    "\n",
+    "Make sure your dataset follows the folder structure and file naming conventions required by the specific format (e.g., COCO JSON annotations or class mask folders for semantic segmentation)."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "import fiftyone as fo\n",
+    "\n",
+    "# Create the dataset\n",
+    "name = \"my-dataset\"\n",
+    "dataset_dir = \"/path/to/segmentation-dataset\"\n",
+    "\n",
+    "# Create the dataset\n",
+    "dataset = fo.Dataset.from_dir(\n",
+    "    dataset_dir=dataset_dir,\n",
+    "    dataset_type=fo.types.COCODetectionDataset, # Change with your type\n",
+    "    name=name,\n",
+    ")\n",
+    "\n",
+    "# View summary info about the dataset\n",
+    "print(dataset)\n",
+    "\n",
+    "# Print the first few samples in the dataset\n",
+    "print(dataset.head())"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "Check out the docs for each format to find optional parameters you can pass for things like train/test split, subfolders, or label paths: https://voxel51.com/docs/fiftyone/user_guide/dataset_creation/dataset_types.html"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "# FiftyOne with a Coffee-Beans Dataset\n",
+    "We will walk through how to use [FiftyOne](https://voxel51.com/docs/fiftyone) to build better segmentation datasets and models. \n",
+    "\n",
+    "- Load your own dataset [into FiftyOne](https://voxel51.com/docs/fiftyone/user_guide/dataset_creation/index.html). For this example, we use a [Coffee-Beans Dataset](https://huggingface.co/datasets/pjramg/colombian_coffee) in COCO format.\n",
+    "- Use FiftyOne [in a notebook](https://voxel51.com/docs/fiftyone/environments/index.html#notebooks)\n",
+    "- Explore your segmentation dataset using [views](https://voxel51.com/docs/fiftyone/user_guide/using_views.html) and the [FiftyOne App](https://voxel51.com/docs/fiftyone/user_guide/app.html)\n",
+    "\n",
+    "*Note: For manually adding segmentations, refer to the `step_x` notebook.*"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "import fiftyone as fo\n",
+    "\n",
+    "dataset = fo.Dataset.from_dir(\n",
+    "    dataset_type=fo.types.COCODetectionDataset,\n",
+    "    dataset_dir=\"./colombian_coffee\",\n",
+    "    data_path=\"images/default\",\n",
+    "    labels_path=\"annotations/instances_default.json\",\n",
+    "    label_types=\"segmentations\",\n",
+    "    label_field=\"categories\",\n",
+    "    name=\"coffee\",\n",
+    "    include_id=True,\n",
+    "    overwrite=True\n",
+    ")\n",
+    "\n",
+    "# View summary info about the dataset\n",
+    "print(dataset)\n",
+    "\n",
+    "# Print the first few samples in the dataset\n",
+    "print(dataset.head())"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "We can see our images have loaded in the App, but no segmentation masks are shown yet. Next, we’ll ensure annotations are properly loaded."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "session = fo.launch_app(dataset)"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## Using the App\n",
+    "\n",
+    "With the FiftyOne App, you can visualize your samples and their segmentation masks in an interactive UI. Double-click any sample to enter the expanded view, where you can study individual samples with overlayed masks.\n",
+    "\n",
+    "The [view bar](https://voxel51.com/docs/fiftyone/user_guide/app.html#using-the-view-bar) lets you filter and search your dataset to analyze specific classes or objects.\n",
+    "\n",
+    "You can seamlessly move between Python and the App. For example, create a filtered view using the `Shuffle()` and `Limit()` stages in Python or directly in the App UI."
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "Once your annotations are loaded correctly, you can confirm that your **segmentation masks** (not detections!) are present and visualized correctly. 🎉"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "session.show()"
+   ]
+  }
+ ],
+ "metadata": {
+  "kernelspec": {
+   "display_name": "OSS310",
+   "language": "python",
+   "name": "python3"
+  },
+  "language_info": {
+   "codemirror_mode": {
+    "name": "ipython",
+    "version": 3
+   },
+   "file_extension": ".py",
+   "mimetype": "text/x-python",
+   "name": "python",
+   "nbconvert_exporter": "python",
+   "pygments_lexer": "ipython3",
+   "version": "3.10.16"
+  }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 2
+}