Skip to content

Commit

Permalink
Initial commit
Browse files Browse the repository at this point in the history
  • Loading branch information
Schmiddi-75 committed Jan 17, 2021
0 parents commit 14996bd
Show file tree
Hide file tree
Showing 212 changed files with 5,675 additions and 0 deletions.
Binary file added Carvana/Input/0cdf5b5d0ce1_01.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added Carvana/Input/0cdf5b5d0ce1_02.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added Carvana/Input/0cdf5b5d0ce1_03.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added Carvana/Input/0cdf5b5d0ce1_04.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added Carvana/Input/0cdf5b5d0ce1_05.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added Carvana/Input/0cdf5b5d0ce1_06.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added Carvana/Input/0cdf5b5d0ce1_07.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added Carvana/Input/0cdf5b5d0ce1_08.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added Carvana/Input/0cdf5b5d0ce1_09.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added Carvana/Input/0cdf5b5d0ce1_10.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added Carvana/Input/0cdf5b5d0ce1_11.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added Carvana/Input/0cdf5b5d0ce1_12.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added Carvana/Input/0cdf5b5d0ce1_13.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added Carvana/Input/0cdf5b5d0ce1_14.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added Carvana/Input/0cdf5b5d0ce1_15.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added Carvana/Input/0cdf5b5d0ce1_16.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added Carvana/Input/0ce66b539f52_01.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added Carvana/Input/0ce66b539f52_02.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added Carvana/Input/0ce66b539f52_03.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added Carvana/Input/0ce66b539f52_04.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added Carvana/Input/0ce66b539f52_05.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added Carvana/Input/0ce66b539f52_06.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added Carvana/Input/0ce66b539f52_07.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added Carvana/Input/0ce66b539f52_08.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added Carvana/Input/0ce66b539f52_09.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added Carvana/Input/0ce66b539f52_10.png
Binary file added Carvana/Input/0ce66b539f52_11.png
Binary file added Carvana/Input/0ce66b539f52_12.png
Binary file added Carvana/Input/0ce66b539f52_13.png
Binary file added Carvana/Input/0ce66b539f52_14.png
Binary file added Carvana/Input/0ce66b539f52_15.png
Binary file added Carvana/Input/0ce66b539f52_16.png
Binary file added Carvana/Input/0d1a9caf4350_01.png
Binary file added Carvana/Input/0d1a9caf4350_02.png
Binary file added Carvana/Input/0d1a9caf4350_03.png
Binary file added Carvana/Input/0d1a9caf4350_04.png
Binary file added Carvana/Input/0d1a9caf4350_05.png
Binary file added Carvana/Input/0d1a9caf4350_06.png
Binary file added Carvana/Input/0d1a9caf4350_07.png
Binary file added Carvana/Input/0d1a9caf4350_08.png
Binary file added Carvana/Input/0d1a9caf4350_09.png
Binary file added Carvana/Input/0d1a9caf4350_10.png
Binary file added Carvana/Input/0d1a9caf4350_11.png
Binary file added Carvana/Input/0d1a9caf4350_12.png
Binary file added Carvana/Input/0d1a9caf4350_13.png
Binary file added Carvana/Input/0d1a9caf4350_14.png
Binary file added Carvana/Input/0d1a9caf4350_15.png
Binary file added Carvana/Input/0d1a9caf4350_16.png
Binary file added Carvana/Input/0d3adbbc9a8b_01.png
Binary file added Carvana/Input/0d3adbbc9a8b_02.png
Binary file added Carvana/Input/0d3adbbc9a8b_03.png
Binary file added Carvana/Input/0d3adbbc9a8b_04.png
Binary file added Carvana/Input/0d3adbbc9a8b_05.png
Binary file added Carvana/Input/0d3adbbc9a8b_06.png
Binary file added Carvana/Input/0d3adbbc9a8b_07.png
Binary file added Carvana/Input/0d3adbbc9a8b_08.png
Binary file added Carvana/Input/0d3adbbc9a8b_09.png
Binary file added Carvana/Input/0d3adbbc9a8b_10.png
Binary file added Carvana/Input/0d3adbbc9a8b_11.png
Binary file added Carvana/Input/0d3adbbc9a8b_12.png
Binary file added Carvana/Input/0d3adbbc9a8b_13.png
Binary file added Carvana/Input/0d3adbbc9a8b_14.png
Binary file added Carvana/Input/0d3adbbc9a8b_15.png
Binary file added Carvana/Input/0d3adbbc9a8b_16.png
Binary file added Carvana/Input/0d53224da2b7_01.png
Binary file added Carvana/Input/0d53224da2b7_02.png
Binary file added Carvana/Input/0d53224da2b7_03.png
Binary file added Carvana/Input/0d53224da2b7_04.png
Binary file added Carvana/Input/0d53224da2b7_05.png
Binary file added Carvana/Input/0d53224da2b7_06.png
Binary file added Carvana/Input/0d53224da2b7_07.png
Binary file added Carvana/Input/0d53224da2b7_08.png
Binary file added Carvana/Input/0d53224da2b7_09.png
Binary file added Carvana/Input/0d53224da2b7_10.png
Binary file added Carvana/Input/0d53224da2b7_11.png
Binary file added Carvana/Input/0d53224da2b7_12.png
Binary file added Carvana/Input/0d53224da2b7_13.png
Binary file added Carvana/Input/0d53224da2b7_14.png
Binary file added Carvana/Input/0d53224da2b7_15.png
Binary file added Carvana/Input/0d53224da2b7_16.png
Binary file added Carvana/Input/0de66245f268_01.png
Binary file added Carvana/Input/0de66245f268_02.png
Binary file added Carvana/Input/0de66245f268_03.png
Binary file added Carvana/Input/0de66245f268_04.png
Binary file added Carvana/Input/0de66245f268_05.png
Binary file added Carvana/Input/0de66245f268_06.png
Binary file added Carvana/Input/0de66245f268_07.png
Binary file added Carvana/Input/0de66245f268_08.png
Binary file added Carvana/Input/0de66245f268_09.png
Binary file added Carvana/Input/0de66245f268_10.png
Binary file added Carvana/Input/0de66245f268_11.png
Binary file added Carvana/Input/0de66245f268_12.png
Binary file added Carvana/Input/0de66245f268_13.png
Binary file added Carvana/Input/0de66245f268_14.png
Binary file added Carvana/Input/0de66245f268_15.png
Binary file added Carvana/Input/0de66245f268_16.png
Binary file added Carvana/Target/0cdf5b5d0ce1_01.png
Binary file added Carvana/Target/0cdf5b5d0ce1_02.png
Binary file added Carvana/Target/0cdf5b5d0ce1_03.png
Binary file added Carvana/Target/0cdf5b5d0ce1_04.png
Binary file added Carvana/Target/0cdf5b5d0ce1_05.png
Binary file added Carvana/Target/0cdf5b5d0ce1_06.png
Binary file added Carvana/Target/0cdf5b5d0ce1_07.png
Binary file added Carvana/Target/0cdf5b5d0ce1_08.png
Binary file added Carvana/Target/0cdf5b5d0ce1_09.png
Binary file added Carvana/Target/0cdf5b5d0ce1_10.png
Binary file added Carvana/Target/0cdf5b5d0ce1_11.png
Binary file added Carvana/Target/0cdf5b5d0ce1_12.png
Binary file added Carvana/Target/0cdf5b5d0ce1_13.png
Binary file added Carvana/Target/0cdf5b5d0ce1_14.png
Binary file added Carvana/Target/0cdf5b5d0ce1_15.png
Binary file added Carvana/Target/0cdf5b5d0ce1_16.png
Binary file added Carvana/Target/0ce66b539f52_01.png
Binary file added Carvana/Target/0ce66b539f52_02.png
Binary file added Carvana/Target/0ce66b539f52_03.png
Binary file added Carvana/Target/0ce66b539f52_04.png
Binary file added Carvana/Target/0ce66b539f52_05.png
Binary file added Carvana/Target/0ce66b539f52_06.png
Binary file added Carvana/Target/0ce66b539f52_07.png
Binary file added Carvana/Target/0ce66b539f52_08.png
Binary file added Carvana/Target/0ce66b539f52_09.png
Binary file added Carvana/Target/0ce66b539f52_10.png
Binary file added Carvana/Target/0ce66b539f52_11.png
Binary file added Carvana/Target/0ce66b539f52_12.png
Binary file added Carvana/Target/0ce66b539f52_13.png
Binary file added Carvana/Target/0ce66b539f52_14.png
Binary file added Carvana/Target/0ce66b539f52_15.png
Binary file added Carvana/Target/0ce66b539f52_16.png
Binary file added Carvana/Target/0d1a9caf4350_01.png
Binary file added Carvana/Target/0d1a9caf4350_02.png
Binary file added Carvana/Target/0d1a9caf4350_03.png
Binary file added Carvana/Target/0d1a9caf4350_04.png
Binary file added Carvana/Target/0d1a9caf4350_05.png
Binary file added Carvana/Target/0d1a9caf4350_06.png
Binary file added Carvana/Target/0d1a9caf4350_07.png
Binary file added Carvana/Target/0d1a9caf4350_08.png
Binary file added Carvana/Target/0d1a9caf4350_09.png
Binary file added Carvana/Target/0d1a9caf4350_10.png
Binary file added Carvana/Target/0d1a9caf4350_11.png
Binary file added Carvana/Target/0d1a9caf4350_12.png
Binary file added Carvana/Target/0d1a9caf4350_13.png
Binary file added Carvana/Target/0d1a9caf4350_14.png
Binary file added Carvana/Target/0d1a9caf4350_15.png
Binary file added Carvana/Target/0d1a9caf4350_16.png
Binary file added Carvana/Target/0d3adbbc9a8b_01.png
Binary file added Carvana/Target/0d3adbbc9a8b_02.png
Binary file added Carvana/Target/0d3adbbc9a8b_03.png
Binary file added Carvana/Target/0d3adbbc9a8b_04.png
Binary file added Carvana/Target/0d3adbbc9a8b_05.png
Binary file added Carvana/Target/0d3adbbc9a8b_06.png
Binary file added Carvana/Target/0d3adbbc9a8b_07.png
Binary file added Carvana/Target/0d3adbbc9a8b_08.png
Binary file added Carvana/Target/0d3adbbc9a8b_09.png
Binary file added Carvana/Target/0d3adbbc9a8b_10.png
Binary file added Carvana/Target/0d3adbbc9a8b_11.png
Binary file added Carvana/Target/0d3adbbc9a8b_12.png
Binary file added Carvana/Target/0d3adbbc9a8b_13.png
Binary file added Carvana/Target/0d3adbbc9a8b_14.png
Binary file added Carvana/Target/0d3adbbc9a8b_15.png
Binary file added Carvana/Target/0d3adbbc9a8b_16.png
Binary file added Carvana/Target/0d53224da2b7_01.png
Binary file added Carvana/Target/0d53224da2b7_02.png
Binary file added Carvana/Target/0d53224da2b7_03.png
Binary file added Carvana/Target/0d53224da2b7_04.png
Binary file added Carvana/Target/0d53224da2b7_05.png
Binary file added Carvana/Target/0d53224da2b7_06.png
Binary file added Carvana/Target/0d53224da2b7_07.png
Binary file added Carvana/Target/0d53224da2b7_08.png
Binary file added Carvana/Target/0d53224da2b7_09.png
Binary file added Carvana/Target/0d53224da2b7_10.png
Binary file added Carvana/Target/0d53224da2b7_11.png
Binary file added Carvana/Target/0d53224da2b7_12.png
Binary file added Carvana/Target/0d53224da2b7_13.png
Binary file added Carvana/Target/0d53224da2b7_14.png
Binary file added Carvana/Target/0d53224da2b7_15.png
Binary file added Carvana/Target/0d53224da2b7_16.png
Binary file added Carvana/Target/0de66245f268_01.png
Binary file added Carvana/Target/0de66245f268_02.png
Binary file added Carvana/Target/0de66245f268_03.png
Binary file added Carvana/Target/0de66245f268_04.png
Binary file added Carvana/Target/0de66245f268_05.png
Binary file added Carvana/Target/0de66245f268_06.png
Binary file added Carvana/Target/0de66245f268_07.png
Binary file added Carvana/Target/0de66245f268_08.png
Binary file added Carvana/Target/0de66245f268_09.png
Binary file added Carvana/Target/0de66245f268_10.png
Binary file added Carvana/Target/0de66245f268_11.png
Binary file added Carvana/Target/0de66245f268_12.png
Binary file added Carvana/Target/0de66245f268_13.png
Binary file added Carvana/Target/0de66245f268_14.png
Binary file added Carvana/Target/0de66245f268_15.png
Binary file added Carvana/Target/0de66245f268_16.png
Binary file added Carvana/Test/Input/3f3e362dea23_03.png
Binary file added Carvana/Test/Input/4f0397cf7937_14.png
Binary file added Carvana/Test/Input/6c3470c34408_10.png
Binary file added Carvana/Test/Target/3f3e362dea23_03.png
Binary file added Carvana/Test/Target/4f0397cf7937_14.png
Binary file added Carvana/Test/Target/6c3470c34408_10.png
213 changes: 213 additions & 0 deletions Part1.ipynb
Original file line number Diff line number Diff line change
@@ -0,0 +1,213 @@
{
"cells": [
{
"cell_type": "code",
"execution_count": 1,
"metadata": {
"pycharm": {
"name": "#%% Carvana dataset example\n"
}
},
"outputs": [],
"source": [
"from transformations import Compose, Resize, DenseTarget\n",
"from transformations import MoveAxis, Normalize01\n",
"from customdatasets import SegmentationDataSet\n",
"from torch.utils.data import DataLoader\n",
"from sklearn.model_selection import train_test_split\n",
"import pathlib\n",
"\n",
"# root directory\n",
"root = pathlib.Path.cwd() / 'Carvana'\n",
"def get_filenames_of_path(path: pathlib.Path, ext: str = '*'):\n",
" \"\"\"Returns a list of files in a directory/path. Uses pathlib.\"\"\"\n",
" filenames = [file for file in path.glob(ext) if file.is_file()]\n",
" return filenames\n",
"\n",
"# input and target files\n",
"inputs = get_filenames_of_path(root / 'Input')\n",
"targets = get_filenames_of_path(root / 'Target')\n",
"\n",
"# training transformations and augmentations\n",
"transforms = Compose([\n",
" DenseTarget(),\n",
" MoveAxis(),\n",
" Normalize01()\n",
"])\n",
"\n",
"# random seed\n",
"random_seed = 42\n",
"\n",
"# split dataset into training set and validation set\n",
"train_size = 0.8 # 80:20 split\n",
"\n",
"inputs_train, inputs_valid = train_test_split(\n",
" inputs,\n",
" random_state=random_seed,\n",
" train_size=train_size,\n",
" shuffle=True)\n",
"\n",
"targets_train, targets_valid = train_test_split(\n",
" targets,\n",
" random_state=random_seed,\n",
" train_size=train_size,\n",
" shuffle=True)\n",
"\n",
"# dataset training\n",
"dataset_train = SegmentationDataSet(inputs=inputs_train,\n",
" targets=targets_train,\n",
" transform=transforms)\n",
"\n",
"# dataset validation\n",
"dataset_valid = SegmentationDataSet(inputs=inputs_valid,\n",
" targets=targets_valid,\n",
" transform=transforms)\n",
"\n",
"# dataloader training\n",
"dataloader_training = DataLoader(dataset=dataset_train,\n",
" batch_size=2,\n",
" shuffle=True)\n",
"\n",
"# dataloader validation\n",
"dataloader_validation = DataLoader(dataset=dataset_valid,\n",
" batch_size=2,\n",
" shuffle=True)\n"
]
},
{
"cell_type": "code",
"execution_count": 2,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"x = shape: torch.Size([2, 3, 1280, 1918]); type: torch.float32\n",
"x = min: 0.0; max: 1.0\n",
"y = shape: torch.Size([2, 1280, 1918]); class: tensor([0, 1]); type: torch.int64\n"
]
}
],
"source": [
"x, y = next(iter(dataloader_training))\n",
"\n",
"print(f'x = shape: {x.shape}; type: {x.dtype}')\n",
"print(f'x = min: {x.min()}; max: {x.max()}')\n",
"print(f'y = shape: {y.shape}; class: {y.unique()}; type: {y.dtype}')"
]
},
{
"cell_type": "code",
"execution_count": 3,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"<napari.viewer.Viewer at 0x21663894a00>"
]
},
"execution_count": 3,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"%gui qt\n",
"from visual import Input_Target_Pair_Generator\n",
"from visual import show_input_target_pair_napari\n",
"gen = Input_Target_Pair_Generator(dataloader_training, rgb=True)\n",
"show_input_target_pair_napari(gen)"
]
},
{
"cell_type": "code",
"execution_count": 4,
"metadata": {},
"outputs": [],
"source": [
"import albumentations\n",
"from transformations import Compose, AlbuSeg2d, DenseTarget, MoveAxis, Normalize01\n",
"# training transformations and augmentations\n",
"transforms_training = Compose([\n",
" # Resize(input_size=(128, 128, 3), target_size=(128, 128)),\n",
" AlbuSeg2d(albu=albumentations.HorizontalFlip(p=0.5)),\n",
" DenseTarget(),\n",
" MoveAxis(),\n",
" Normalize01()\n",
"])\n",
"\n",
"# validation transformations\n",
"transforms_validation = Compose([\n",
" # Resize(input_size=(128, 128, 3), target_size=(128, 128)),\n",
" DenseTarget(),\n",
" MoveAxis(),\n",
" Normalize01()\n",
"])"
]
},
{
"cell_type": "code",
"execution_count": 5,
"metadata": {},
"outputs": [],
"source": [
"# dataset training\n",
"dataset_train = SegmentationDataSet(inputs=inputs_train,\n",
" targets=targets_train,\n",
" transform=transforms_training)\n",
"\n",
"# dataset validation\n",
"dataset_valid = SegmentationDataSet(inputs=inputs_valid,\n",
" targets=targets_valid,\n",
" transform=transforms_validation)\n",
"\n",
"# dataloader training\n",
"dataloader_training = DataLoader(dataset=dataset_train,\n",
" batch_size=2,\n",
" shuffle=True)\n",
"\n",
"# dataloader validation\n",
"dataloader_validation = DataLoader(dataset=dataset_valid,\n",
" batch_size=2,\n",
" shuffle=True)"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"%gui qt\n",
"from visual import Input_Target_Pair_Generator\n",
"from visual import show_input_target_pair_napari\n",
"gen_train = Input_Target_Pair_Generator(dataloader_training, rgb=True)\n",
"gen_valid = Input_Target_Pair_Generator(dataloader_validation, rgb=True)\n",
"show_input_target_pair_napari(gen_train, gen_valid)"
]
}
],
"metadata": {
"kernelspec": {
"display_name": "Python 3",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.8.5"
}
},
"nbformat": 4,
"nbformat_minor": 1
}
Loading

0 comments on commit 14996bd

Please sign in to comment.