Skip to content

Latest commit

 

History

History
81 lines (36 loc) · 3.27 KB

Training.md

File metadata and controls

81 lines (36 loc) · 3.27 KB

Training

Below are the steps to run

Megadepth

  1. Download the datasets

From the link below (download the MegaDepth v1 Dataset & MegaDepth v1 SfM models): Megadepth

Unzip the tar files and rename the MegaDepth v1 SfM folder to "Undistorted SfM"

  1. Move datasets to correct folder

Move both the v1 and SfM datasets into "/LoFTR-in-Tensorflow/src/training/datasets/megadepth"

  1. Download Indicies

Download the indices from the original LoFTR paper.

Megadepth Indices

After opening the link, the indices are located in "LoFTR/train-data/megadepth_indices.tar"

  1. Move indices to correct folder

Untar the indices folder and also move it to "/LoFTR-in-Tensorflow/src/training/datasets/megadepth"

  1. Run Training File

Currently the training file is a CPU only training file. We are working on a GPU training file that will utilise multiple GPU's

The weights for this training will be saved into "./weights/megadepth/cp_megadepth.ckpt"

Scannet

scannet (please request access via their github: https://github.com/ScanNet/ScanNet)

  1. Using the provided download file, download all scenes, using the --type .sens optional parameter to retrieve the binary files for the model. A user should have suffecient space to store roughly 1.5TB of images.

  2. Create a "scannet" folder in the "src/training/datasets/" filepath. Include the "scans" folder from step 1 in this folder.

  3. Download and extract the "scannet_indices" file provided in the original LoFTR paper, found here: https://drive.google.com/drive/folders/1SrIn9WJ1IuG08yh2nEvIsLftXHLrrIwh.

  4. Place the extracted "train/" folder and "intrinsics.npz" files into the same "scannet" folder from step 2.

  5. create a folder for the weights in the "weights/" folder in the main directory called "scannet/"

  6. Run the train_scannet.py using the terminal or from an sbatch file. A minimum of 256GB RAM is recommended.

NYU Depth V2

  1. The NYU Depth V2 is provided in a) labeled format b) raw format. In the project, we downloaded several scenes from the raw format as it suits better our needs for the final data structure. the download link for the dataset is provided below. https://cs.nyu.edu/~silberman/datasets/nyu_depth_v2.html

  2. After downloading the scenes, we select the corresponding ppm and pgm files from the index.txt file.

  3. The depth files (pgm) files are not provided in the frame of the RGB camera, and need to undergo transformation to the same frame. The toolbox provided with the dataset contains MATLAB scripts to transform the depth maps to align with the RGB frames. The download link for NYU MATLAB toolbox is provided below

http://cs.nyu.edu/~silberman/code/toolbox_nyu_depth_v2.zip

  1. After transforming the depth maps, we needed to select corresponding pairs of images that respect the condition of the covisibility being between 0.4 and 0.7. Python code was developed by using SIFT features detector to select such pairs of images and the corresponding depth maps.

  2. The output from the python file generates a list of dictionaries containing batch of 4 pairs of images, depth, transformation and corresponding intrinsic matrices.