Check failed: work_element_count > 0

[qiaomu-miao]

Hello,

I tried to run the demo code with torch=2.1.2 and tensorflow=2.12.0 with cuda=11.8. It shows the error below and the program aborted:

2024-05-23 16:32:35.814947: I tensorflow/core/platform/cpu_feature_guard.cc:182] This TensorFlow binary is optimized to use available CPU instructions in performance-critical operations.
To enable the following instructions: AVX2 FMA, in other operations, rebuild TensorFlow with the appropriate compiler flags
2024-05-23 16:32:37.248128: W tensorflow/compiler/tf2tensorrt/utils/py_utils.cc:38] TF-TRT Warning: Could not find TensorRT Loading OmniGlue (and its submodules: SuperPoint & DINOv2)...
2024-05-23 16:32:41.019127: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1635] Created device /job:localhost/replica:0/task:0/device:GPU:0 with 22504 MB memory: -> device: 0, name: NVIDIA RTX A5000, pci bus id: 0000:e1:00.0, compute capability: 8.6
2024-05-23 16:32:52.298761: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1635] Created device /job:localhost/replica:0/task:0/device:GPU:0 with 22504 MB memory: -> device: 0, name: NVIDIA RTX A5000, pci bus id: 0000:e1:00.0, compute capability: 8.6
WARNING:tensorflow:From /home/qiaomu/code/3D/omniglue/src/omniglue/superpoint_extract.py:40: load (from tensorflow.python.saved_model.loader_impl) is deprecated and will be removed in a future version.
Instructions for updating:
Use tf.saved_model.load instead.
2024-05-23 16:32:52.353076: I tensorflow/compiler/mlir/mlir_graph_optimization_pass.cc:353] MLIR V1 optimization pass is not enabled
2024-05-23 16:32:53.883790: I tensorflow/compiler/xla/stream_executor/cuda/cuda_dnn.cc:424] Loaded cuDNN version 8700
2024-05-23 16:32:55.415750: I tensorflow/compiler/xla/stream_executor/cuda/cuda_blas.cc:637] TensorFloat-32 will be used for the matrix multiplication. This will only be logged once.
2024-05-23 16:32:55.428150: F ./tensorflow/core/util/gpu_launch_config.h:129] Check failed: work_element_count > 0 (0 vs. -1222967296)
Aborted (core dumped)

Do you have any idea of this error? What are the versions of pytorch, tensorflow, and cuda in your program?

Thanks.

[Starlit404]

same question

[arjunkarpur]

Hi, thanks for raising the issues. We're taking a look at this and will follow up soon