Skip to content

Gated Fusion Network for Joint Image Deblurring and Super-Resolution(BMVC 2018 Oral)

Notifications You must be signed in to change notification settings

jacquelinelala/GFN

Folders and files

NameName
Last commit message
Last commit date

Latest commit

3b80f53 · Feb 22, 2021

History

60 Commits
Oct 10, 2018
Nov 19, 2018
Oct 10, 2018
Oct 10, 2018
Oct 10, 2018
Nov 19, 2018
Feb 22, 2021
Mar 12, 2019
Mar 12, 2019

Repository files navigation

GFN

"Gated Fusion Network for Joint Image Deblurring and Super-Resolution" by Xinyi Zhang, Hang Dong, Zhe Hu, Wei-Sheng Lai, Fei Wang, Ming-Hsuan Yang(oral presentation on BMVC2018).

There are more details you can find on Project Website : http://xinyizhang.tech/bmvc2018.

Archi heatmap

Improve the training process

In order to obtain a more stable training process, now we adopt a three-step training strategy, which differs from our paper and improves PSNR from 27.74dB to 27.81dB on LR-GOPRO 4x dataset.

Model LR-GOPRO 4x PSNR(dB) Time(s)
SCGAN 22.74 0.66
SRResNet 24.40 0.07
ED-DSRN 26.44 0.10
DeepDeblur + EDSR 25.09 2.70
EDSR + DeepDeblur 26.35 8.10
GFN(BMVC paper) 27.74 0.07
GFN(Now) 27.81 0.07

Dependencies

  • Python 3.6
  • PyTorch >= 0.4.0
  • torchvision
  • numpy
  • skimage
  • h5py
  • MATLAB

How to test:

Test on LR-GOPRO Validation

Test on the latest trained model

This model is the result of the third step with 55 epoch.

  1. Git clone this repository.
$git clone https://github.com/jacquelinelala/GFN.git
$cd GFN
  1. Download the original GOPRO_Large dataset from Google Drive.
  2. Generate the validation images of LR-GOPRO dataset: Run matlab function GFN/h5_generator/gopro_val_generator.m. The generated test images will be stored in your_downloads_directory/GOPRO_Large/Validation_4x.

(If you don't have access to MATLAB, we offer a validation dataset for testing. You can download it from GoogleDrive or Pan Baidu.)

>> folder = 'your_downloads_directory/GOPRO_Large'; # You should replace the your_downloads_directory by your GOPRO_Large's directory.
>> gopro_val_generator(folder)
  1. Download the trained model GFN_epoch_55.pkl from here, then unzip and move the GFN_epoch_55.pkl to GFN/models folder.

  2. Run the GFN/test_GFN_x4.py with cuda on command line:

GFN/$python test_GFN_x4.py --dataset your_downloads_directory/GOPRO_Large/Validation_4x

Then the deblurring and super-solving images ending with GFN_4x.png are in the directory of GOPRO_Large/Validation/Results.

  1. Calculate the PSNR using Matlab function GFN/evaluation/test_RGB.m. The output of the average PSNR is 27.810232 dB. You can also use the GFN/evaluation/test_bicubic.m to calculate the bicubic method.
>> folder = 'your_downloads_directory/GOPRO_Large';
>> test_RGB(folder)

How to train

Train on LR-GOPRO dataset

You should accomplish the first two steps in Test on LR-GOPRO Validation before the following steps.

Train from scratch

  1. Generate the train hdf5 files of LR-GOPRO dataset: Run the matlab function gopro_hdf5_generator.m which is in the directory of GFN/h5_generator. The generated hdf5 files are stored in the your_downloads_directory/GOPRO_Large/GOPRO_train256_4x_HDF5.
>> folder = 'your_downloads_directory/GOPRO_Large';
>> gopro_hdf5_generator(folder)
  1. Run the GFN/train_GFN_4x.py with cuda on command line:
GFN/$python train_GFN_4x.py --dataset your_downloads_directory/GOPRO_Large/GOPRO_train256_4x_HDF5
  1. The three step intermediate models will be respectively saved in models/1/ models/2 and models/3. You can also use the following command to test the intermediate results during the training process. Run the GFN/test_GFN_x4.py with cuda on command line:
GFN/$python test_GFN_x4.py --dataset your_downloads_directory/GOPRO_Large/Validation_4x --intermediate_process models/1/GFN_epoch_30.pkl # We give an example of step1 epoch30. You can replace another pkl file in models/.

Resume training from breakpoints

Since the training process will take 3 or 4 days, you can use the following command to resume the training process from any breakpoints. Run the GFN/train_GFN_4x.py with cuda on command line:

GFN/$python train_GFN_4x.py --dataset your_downloads_directory/GOPRO_Large/GOPRO_train256_4x_HDF5 --resume models/1/GFN_epoch_30.pkl # Just an example of step1 epoch30.

Citation

If you use these models in your research, please cite:

@conference{Zhang2018,
	author = {Xinyi Zhang and Hang Dong and Zhe Hu and Wei-Sheng Lai and Fei Wang and Ming-Hsuan Yang},
	title = {Gated Fusion Network for Joint Image Deblurring and Super-Resolution},
	booktitle = {BMVC},
	year = {2018}
}

About

Gated Fusion Network for Joint Image Deblurring and Super-Resolution(BMVC 2018 Oral)

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published