Added HardNet-PS, HardNet-6Br and guide what to use :)

README.md

@@ -6,23 +6,26 @@ HardNet model implementation in PyTorch for NIPS 2017 paper ["Working hard to kn
 ## Important update
 After two major updates of PyTorch library, we were not able to reproduce the results. Therefore, we did a hyperparameter search and we found that increasing the learning rate to 10 and dropout rate to 0.3 leads to better results, see figure below. We have obtained the same results on different machines
 . We will update arXiv version soon.  
-Pretrained weights in PyTorch format are update. 
+Pretrained weights in PyTorch format are updated.
+
+We also release version, trained on all subsets of Brown dataset, called [HardNet6Brown]('pretrained/6Brown).
 
-We also are planning to release better version of HardNet, trained on several datasets, stay tuned :)
 
 ## Benchmark on [HPatches](https://github.com/hpatches/hpatches-benchmark), mAP
-![HPatches-results](img/hpatches_results.png)
+
+![HPatches-results](img/hardnet_hpatches.png)
 
 
-## Retrieval on Oxford5k, mAP
+## Retrieval on Oxford5k, mAP, Hessian-Affine detector
 
 | Descriptor    | BoW  |  BoW + SV | BoW + SV + QE | HQE + MA |
 | -----    | ----  |  ---- | ---- | ---- |
 | [TFeatLib](https://github.com/vbalnt/tfeat)   | 46.7  |  55.6 | 72.2 | n/a |
 | [RootSIFT](http://ieeexplore.ieee.org/document/6248018/)    | 55.1  |  63.0 | 78.4 | 88.0 |
 | [L2NetLib+](https://github.com/yuruntian/L2-Net)   | 59.8  |  67.7 | 80.4 | n/a |
-| HardNetLib+   | 59.8  |  68.6 | 83.0 |  88.2 |
+| HardNetLibNIPS+   | 59.8  |  68.6 | 83.0 |  88.2 |
 | HardNet++   | **60.8**  |  **69.6** | **84.5** | **88.3** |
+| [HesAffNet](https://github.com/ducha-aiki/affnet) + HardNet++ | **68.3**  |  **77.8** | **89.0** | **89.5** |
 
 
 
@@ -43,11 +46,21 @@ Logs are stored in tensorboard format in directory logs/
 
 ## Pre-trained models
 
-Pre-trained models can be found in folder pretrained:  train_liberty and train_liberty_with_aug
+Pre-trained models can be found in folder [pretrained](pretrained).
 
 ## 3rd party pre-trained models
 
-Rahul Mitra presented new [large-scale patch PS-dataset](https://github.com/rmitra/PS-Dataset) and trained even better HardNet on it. Weights are [here](https://www.dropbox.com/s/q89g5kfuke6c348/hardnet-PS.t7?dl=1) here, unfortunately in Torch format, not PyTorch.
+Rahul Mitra presented new [large-scale patch PS-dataset](https://github.com/rmitra/PS-Dataset) and trained even better HardNet on it. Original weights in torch format are [here](https://www.dropbox.com/s/q89g5kfuke6c348/hardnet-PS.t7?dl=1).
+
+Converted PyTorch version is [here](pretrained/3rd_party/HardNetPS/).
+
+## Which weights should I use?
+
+For practical applications, we recommend [HardNet++](pretrained/pretrained_all_datasets/HardNet++.pth').
+
+For comparison with other descriptors, which are trained on Liberty Brown dataset, we recommend [HardNetLib+](pretrained/train_liberty_with_aug/checkpoint_liberty_with_aug.pth).
+
+For the best descriptor, which is NOT trained on HPatches dataset, we recommend model by Mitra et.al., link in section above.
 
 ## Usage example
 
@@ -62,6 +75,11 @@ cd examples/caffe
 python extract_hardnetCaffe_desc_from_hpatches_file.py ../imgs/ref.png hardnet_caffe.txt
 ```
 
+## Projects, which use HardNet
+
+[AffNet](https://github.com/ducha-aiki/affnet) -- learned local affine shape estimator.
+
+
 ## Citation
 
 Please cite us if you use this code:

examples/caffe/convert_weights_to_caffe.py

@@ -114,13 +114,13 @@ def copy_weights(m,):
 
 mws = [
 "../../pretrained/train_liberty_with_aug/checkpoint_liberty_with_aug.pth",
-"../../pretrained/train_yosemite/checkpoint_yosemite_no_aug.pth",
+#"../../pretrained/train_yosemite/checkpoint_yosemite_no_aug.pth",
 "../../pretrained/train_liberty/checkpoint_liberty_no_aug.pth",
-"../../pretrained/train_notredame_with_aug/checkpoint_notredame_with_aug.pth",
-"../../pretrained/train_notredame/checkpoint_notredame_no_aug.pth",
-"../../pretrained/train_yosemite_with_aug/checkpoint_yosemite_with_aug.pth",
-"../../pretrained/pretrained_all_datasets/HardNet++.pth"
-"../../pretrained/pretrained_all_datasets/HardNetFullBrown.pth"
+#"../../pretrained/train_notredame_with_aug/checkpoint_notredame_with_aug.pth",
+#"../../pretrained/train_notredame/checkpoint_notredame_no_aug.pth",
+#"../../pretrained/train_yosemite_with_aug/checkpoint_yosemite_with_aug.pth",
+#"../../pretrained/pretrained_all_datasets/HardNet++.pth"
+"../../pretrained/6Brown/hardnetBr6.pth"
     ]
 
 for model_weights in mws:

pretrained/3rd_party/HardNetPS/README.md

@@ -0,0 +1,58 @@
+This version of HardNet is trained on [PS dataset](https://github.com/rmitra/PS-Dataset) by Mitra et.al. with torch package and then converted to PyTorch. 
+
+The structure of the network has minor changes, use this definition:
+
+class HardNetPS(nn.Module):
+   def __init__(self):
+        super(HardNetPS, self).__init__()
+        self.features = nn.Sequential(
+        nn.Conv2d(1, 32, kernel_size=3, padding=1, bias = True),
+        nn.BatchNorm2d(32, affine=True),
+        nn.ReLU(),
+        nn.Conv2d(32, 32, kernel_size=3, padding=1, bias = True),
+        nn.BatchNorm2d(32, affine=True),
+        nn.ReLU(),
+        nn.Conv2d(32, 64, kernel_size=3, stride=2, padding=1, bias = True),
+        nn.BatchNorm2d(64, affine=True),
+        nn.ReLU(),
+        nn.Conv2d(64, 64, kernel_size=3, padding=1, bias = True),
+        nn.BatchNorm2d(64, affine=True),
+        nn.ReLU(),
+        nn.Conv2d(64, 128, kernel_size=3, stride=2,padding=1, bias = True),
+        nn.BatchNorm2d(128, affine=True),
+        nn.ReLU(),
+        nn.Conv2d(128, 128, kernel_size=3, padding=1, bias = True),
+        nn.BatchNorm2d(128, affine=True),
+        nn.ReLU(),
+        nn.Conv2d(128, 128, kernel_size=8, bias = True)
+    )
+    return
+    def input_norm(self,x):
+        flat = x.view(x.size(0), -1)
+        mp = torch.mean(flat, dim=1)
+        sp = torch.std(flat, dim=1) + 1e-7
+        return (x - mp.unsqueeze(-1).unsqueeze(-1).unsqueeze(-1).expand_as(x)) / sp.unsqueeze(-1).unsqueeze(-1).unsqueeze(1).expand_as(x)
+
+    def forward(self, input):
+        x_features = self.features(self.input_norm(input))
+        x = x_features.view(x_features.size(0), -1)
+        return L2Norm()(x)
+
+
+If you use this weights, please cite:
+
+
+@ARTICLE{2018arXiv180101466M,
+   author = {{Mitra}, R. and {Doiphode}, N. and {Gautam}, U. and {Narayan}, S. and 
+    {Ahmed}, S. and {Chandran}, S. and {Jain}, A.},
+    title = "{A Large Dataset for Improving Patch Matching}",
+  journal = {ArXiv e-prints},
+archivePrefix = "arXiv",
+   eprint = {1801.01466},
+ primaryClass = "cs.CV",
+ keywords = {Computer Science - Computer Vision and Pattern Recognition},
+     year = 2018,
+    month = jan,
+   adsurl = {http://adsabs.harvard.edu/abs/2018arXiv180101466M},
+  adsnote = {Provided by the SAO/NASA Astrophysics Data System}
+}