Inception Score of Test Images

[chmxu]

Hi,
I got the test set according to your split and I used the inception score implementation here https://github.com/sbarratt/inception-score-pytorch.git. The problem is I got 3.50 on 12000 test images and 3.49 on 7284 filterd images that the repeated ones are removed. I have tried all suggestions in original repo to have a better score. Do you have any idea about this problem?

[tengteng95]

Thanks for your attention! Sorry for not making this part clear. Now we have updated the ReadMe in our github with details for calculating all the metrics used in our paper. We use the same evaluation codes as previous work Deform. By the way, what do you mean by "7284 filterd images that the repeated ones are removed" ? It should be noted that tranfering a person from Pose A to Pose B, and from Pose B to Pose A are different pairs.

…

------------------ 原始邮件 ------------------ 发件人: "loadder"<[email protected]>; 发送时间: 2019年4月22日(星期一) 上午10:32 收件人: "tengteng95/Pose-Transfer"<[email protected]>; 抄送: "Subscribed"<[email protected]>; 主题: [tengteng95/Pose-Transfer] Inception Score of Test Images (#1) Hi, I got the test set according to your split and I used the inception score implementation here https://github.com/sbarratt/inception-score-pytorch.git. The problem is I got 3.50 on 12000 test images and 3.49 on 7284 filterd images that the repeated ones are removed. I have tried all suggestions in original repo to have a better score. Do you have any idea about this problem? — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub, or mute the thread.

[chmxu]

Thanks for your attention! Sorry for not making this part clear. Now we have updated the ReadMe in our github with details for calculating all the metrics used in our paper. We use the same evaluation codes as previous work Deform. By the way, what do you mean by "7284 filterd images that the repeated ones are removed" ? It should be noted that tranfering a person from Pose A to Pose B, and from Pose B to Pose A are different pairs.
…
------------------ 原始邮件 ------------------ 发件人: "loadder"[email protected]; 发送时间: 2019年4月22日(星期一) 上午10:32 收件人: "tengteng95/Pose-Transfer"[email protected]; 抄送: "Subscribed"[email protected]; 主题: [tengteng95/Pose-Transfer] Inception Score of Test Images (#1) Hi, I got the test set according to your split and I used the inception score implementation here https://github.com/sbarratt/inception-score-pytorch.git. The problem is I got 3.50 on 12000 test images and 3.49 on 7284 filterd images that the repeated ones are removed. I have tried all suggestions in original repo to have a better score. Do you have any idea about this problem? — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub, or mute the thread.

hi, thanks for your reply! I have checked these two implementations of inception score and find that in the pretrained tensorflow inception model, the dimension of output is 1008 while that in pytorch is 1000 (actual number of class in ILSVRC2012). When I use your code on 12000 'to' images i got a similar score to yours. I dont know if this is reasonable since the repo I used is also a famous implementation and widely-used one and this can obviously result in a huge performance gap.

[tengteng95]

For fair comparisons, we just use the same evaluation codes in previous works Deform and PG2. We also provide quantitative results of previous works under our testing set using these evaluation codes in our paper.

[chmxu]

Thanks!

[budui]

@tengteng95 Hi, which version tensorflow is needed for Inception Score?
I test tensorflow 1.4.1(python2), 1.7.0rc(python3). but I got the same error:

Traceback (most recent call last):
  File "tool/getMetrics_market.py", line 2, in <module>
    from inception_score import get_inception_score
  File "/media/main/Data3/RayHY/Pose-Transfer/tool/inception_score.py", line 96, in <module>
    _init_inception()
  File "/media/main/Data3/RayHY/Pose-Transfer/tool/inception_score.py", line 92, in _init_inception
    logits = tf.matmul(tf.squeeze(pool3), w)
  File "/usr/local/lib/python3.5/dist-packages/tensorflow/python/ops/math_ops.py", line 2108, in matmul
    a, b, transpose_a=transpose_a, transpose_b=transpose_b, name=name)
  File "/usr/local/lib/python3.5/dist-packages/tensorflow/python/ops/gen_math_ops.py", line 4209, in mat_mul
    name=name)
  File "/usr/local/lib/python3.5/dist-packages/tensorflow/python/framework/op_def_library.py", line 787, in _apply_op_helper
    op_def=op_def)
  File "/usr/local/lib/python3.5/dist-packages/tensorflow/python/framework/ops.py", line 3292, in create_op
    compute_device=compute_device)
  File "/usr/local/lib/python3.5/dist-packages/tensorflow/python/framework/ops.py", line 3332, in _create_op_helper
    set_shapes_for_outputs(op)
  File "/usr/local/lib/python3.5/dist-packages/tensorflow/python/framework/ops.py", line 2496, in set_shapes_for_outputs
    return _set_shapes_for_outputs(op)
  File "/usr/local/lib/python3.5/dist-packages/tensorflow/python/framework/ops.py", line 2469, in _set_shapes_for_outputs
    shapes = shape_func(op)
  File "/usr/local/lib/python3.5/dist-packages/tensorflow/python/framework/ops.py", line 2399, in call_with_requiring
    return call_cpp_shape_fn(op, require_shape_fn=True)
  File "/usr/local/lib/python3.5/dist-packages/tensorflow/python/framework/common_shapes.py", line 627, in call_cpp_shape_fn
    require_shape_fn)
  File "/usr/local/lib/python3.5/dist-packages/tensorflow/python/framework/common_shapes.py", line 691, in _call_cpp_shape_fn_impl
    raise ValueError(err.message)
ValueError: Shape must be rank 2 but is rank 1 for 'MatMul' (op: 'MatMul') with input shapes: [2048], [2048,1008].

[budui]

solution for above error:
use logits = tf.matmul(tf.squeeze(pool3, [1, 2]), w) to replace
logits = tf.matmul(tf.squeeze(pool3), w)
ref: biuyq/CT-GAN#5 (comment)

But now, I got a new error:

Loading images...
Compute inception score...
2019-05-05 00:08:54.772060: I tensorflow/core/common_runtime/gpu/gpu_device.cc:977] Creating TensorFlow device (/gpu:0) -> (device: 0, name: TITAN X (Pascal), pci bus id: 0000:02:00.0)
2019-05-05 00:08:54.772760: I tensorflow/core/common_runtime/gpu/gpu_device.cc:977] Creating TensorFlow device (/gpu:1) -> (device: 1, name: TITAN X (Pascal), pci bus id: 0000:03:00.0)
2019-05-05 00:08:54.772997: I tensorflow/core/common_runtime/gpu/gpu_device.cc:977] Creating TensorFlow device (/gpu:2) -> (device: 2, name: TITAN X (Pascal), pci bus id: 0000:82:00.0)
2019-05-05 00:08:54.773213: I tensorflow/core/common_runtime/gpu/gpu_device.cc:977] Creating TensorFlow device (/gpu:3) -> (device: 3, name: TITAN X (Pascal), pci bus id: 0000:83:00.0)
.Traceback (most recent call last):
  File "tool/getMetrics_market.py", line 125, in <module>
    test(generated_images_dir, annotations_file_test)
  File "tool/getMetrics_market.py", line 93, in test
    inception_score = get_inception_score(generated_images)
  File "/media/main/Data3/RayHY/Pose-Transfer/tool/inception_score.py", line 43, in get_inception_score
    pred = sess.run(softmax, {'ExpandDims:0': inp})
  File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/client/session.py", line 778, in run
    run_metadata_ptr)
  File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/client/session.py", line 961, in _run
    % (np_val.shape, subfeed_t.name, str(subfeed_t.get_shape())))
ValueError: Cannot feed value of shape (10, 128, 64, 3) for Tensor u'ExpandDims:0', which has shape '(1, ?, ?, 3)'

[tengteng95]

@budui Hi, I haven't met this problem before and I use tensorflow1.4.1(python3).

[studabyd]

@budui Hello, I met the same error(the new error you reffered) with you, have you solved it. I want to your method If you do it. Thank you!

[budui]

@studabyd docker.
Ref: https://github.com/budui/Human-Pose-Transfer/blob/master/README.md#evaluate

[studabyd]

OK, thank you for your advice.