CC-3DT++: The Velocity-Similarity Enhanced Tracker of CR3DT

[KimJunHan]

CC-3DT++: The Velocity-Similarity Enhanced Tracker of CR3DT

Now I test CC-3DT++ from git source
CC-3DT++: Link: https://github.com/ETH-PBL/cc-3dt-pp
Use Detection & Tracking Results from Google Drive that are provied fro CC-3DT++ git page

But
when I put the command "(base) root@3dd97ec2720a:~/cc-3dt-pp# ./full_eval.sh mini /root/cc-3dt-pp/data/3drdt/detection_results/mini_results_nusc.json"

the problem occur

Computing metrics for class bicycle...

0%| | 0/2 [00:00<?, ?
ValueError: Length of names must match number of levels in MultiIndex.

but the results in json file that involded bicycle Car, Bus other class

Let me Know the solution If you have some

------------------------- there is all the error code in terminal that I am facing ------------------------------

base) root@3dd97ec2720a:~/cc-3dt-pp# ./full_eval.sh mini /root/cc-3dt-pp/data/3drdt/detection_results/mini_results_nusc.json
/opt/conda/envs/cc3dt/lib/python3.9/site-packages/timm/models/helpers.py:7: FutureWarning: Importing from timm.models.helpers is deprecated, please import via timm.models
warnings.warn(f"Importing from {name} is deprecated, please import via timm.models", FutureWarning)
[01/23 22:32:43 Vis4D]: Environment info: PyTorch version: 2.1.2+cu118
Is debug build: False
CUDA used to build PyTorch: 11.8
ROCM used to build PyTorch: N/A

OS: Ubuntu 20.04.6 LTS (x86_64)
GCC version: (Ubuntu 9.4.0-1ubuntu1~20.04.2) 9.4.0
Clang version: Could not collect
CMake version: Could not collect
Libc version: glibc-2.31

Python version: 3.9.23 | packaged by conda-forge | (main, Jun 4 2025, 17:57:12) [GCC 13.3.0] (64-bit runtime)
Python platform: Linux-6.8.0-90-generic-x86_64-with-glibc2.31
Is CUDA available: True
CUDA runtime version: 11.8.89
CUDA_MODULE_LOADING set to: LAZY
GPU models and configuration:
GPU 0: NVIDIA GeForce RTX 3080 Ti
GPU 1: NVIDIA GeForce RTX 3080 Ti

Nvidia driver version: 550.163.01
cuDNN version: Could not collect
HIP runtime version: N/A
MIOpen runtime version: N/A
Is XNNPACK available: True

CPU:
Architecture: x86_64
CPU op-mode(s): 32-bit, 64-bit
Byte Order: Little Endian
Address sizes: 46 bits physical, 48 bits virtual
CPU(s): 20
On-line CPU(s) list: 0-19
Thread(s) per core: 2
Core(s) per socket: 10
Socket(s): 1
NUMA node(s): 1
Vendor ID: GenuineIntel
CPU family: 6
Model: 85
Model name: Intel(R) Core(TM) i9-10900X CPU @ 3.70GHz
Stepping: 7
CPU MHz: 3200.114
CPU max MHz: 4700.0000
CPU min MHz: 1200.0000
BogoMIPS: 7399.70
Virtualization: VT-x
L1d cache: 320 KiB
L1i cache: 320 KiB
L2 cache: 10 MiB
L3 cache: 19.3 MiB
NUMA node0 CPU(s): 0-19
Vulnerability Gather data sampling: Mitigation; Microcode
Vulnerability Itlb multihit: KVM: Mitigation: VMX disabled
Vulnerability L1tf: Not affected
Vulnerability Mds: Not affected
Vulnerability Meltdown: Not affected
Vulnerability Mmio stale data: Mitigation; Clear CPU buffers; SMT vulnerable
Vulnerability Reg file data sampling: Not affected
Vulnerability Retbleed: Mitigation; Enhanced IBRS
Vulnerability Spec rstack overflow: Not affected
Vulnerability Spec store bypass: Mitigation; Speculative Store Bypass disabled via prctl
Vulnerability Spectre v1: Mitigation; usercopy/swapgs barriers and __user pointer sanitization
Vulnerability Spectre v2: Mitigation; Enhanced / Automatic IBRS; IBPB conditional; RSB filling; PBRSB-eIBRS SW sequence; BHI SW loop, KVM SW loop
Vulnerability Srbds: Not affected
Vulnerability Tsx async abort: Mitigation; TSX disabled
Vulnerability Vmscape: Mitigation; IBPB before exit to userspace
Flags: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx pdpe1gb rdtscp lm constant_tsc art arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc cpuid aperfmperf pni pclmulqdq dtes64 monitor ds_cpl vmx est tm2 ssse3 sdbg fma cx16 xtpr pdcm pcid dca sse4_1 sse4_2 x2apic movbe popcnt tsc_deadline_timer aes xsave avx f16c rdrand lahf_lm abm 3dnowprefetch cpuid_fault epb cat_l3 cdp_l3 ssbd mba ibrs ibpb stibp ibrs_enhanced tpr_shadow flexpriority ept vpid ept_ad fsgsbase tsc_adjust bmi1 avx2 smep bmi2 erms invpcid cqm mpx rdt_a avx512f avx512dq rdseed adx smap clflushopt clwb intel_pt avx512cd avx512bw avx512vl xsaveopt xsavec xgetbv1 xsaves cqm_llc cqm_occup_llc cqm_mbm_total cqm_mbm_local dtherm ida arat pln pts hwp hwp_act_window hwp_epp hwp_pkg_req vnmi avx512_vnni md_clear flush_l1d arch_capabilities ibpb_exit_to_user

Versions of relevant libraries:
[pip3] flake8==7.3.0
[pip3] flake8_import_order==0.19.2
[pip3] numpy==1.23.5
[pip3] pytorch-lightning==2.6.0
[pip3] torch==2.1.2+cu118
[pip3] torchaudio==2.1.2+cu118
[pip3] torchmetrics==1.8.2
[pip3] torchvision==0.16.2+cu118
[pip3] triton==2.1.0
[conda] mkl 2025.3.0 h0e700b2_463 conda-forge
[conda] mkl-include 2025.3.0 hf2ce2f3_463 conda-forge
[conda] numpy 1.23.5 pypi_0 pypi
[conda] pytorch-lightning 2.6.0 pypi_0 pypi
[conda] torch 2.1.2+cu118 pypi_0 pypi
[conda] torchaudio 2.1.2+cu118 pypi_0 pypi
[conda] torchmetrics 1.8.2 pypi_0 pypi
[conda] torchvision 0.16.2+cu118 pypi_0 pypi
[conda] triton 2.1.0 pypi_0 pypi
[01/23 22:32:43 Vis4D]: GPU available: True (cuda), used: True
I0123 22:32:43.864930 138545818862784 setup.py:164] GPU available: True (cuda), used: True
[01/23 22:32:43 Vis4D]: TPU available: False, using: 0 TPU cores
I0123 22:32:43.866408 138545818862784 setup.py:167] TPU available: False, using: 0 TPU cores
[01/23 22:32:43 Vis4D]: You are using a CUDA device ('NVIDIA GeForce RTX 3080 Ti') that has Tensor Cores. To properly utilize them, you should set torch.set_float32_matmul_precision('medium' | 'high') which will trade-off precision for performance. For more details, read https://pytorch.org/docs/stable/generated/torch.set_float32_matmul_precision.html#torch.set_float32_matmul_precision
I0123 22:32:43.867619 138545818862784 cuda.py:171] You are using a CUDA device ('NVIDIA GeForce RTX 3080 Ti') that has Tensor Cores. To properly utilize them, you should set torch.set_float32_matmul_precision('medium' | 'high') which will trade-off precision for performance. For more details, read https://pytorch.org/docs/stable/generated/torch.set_float32_matmul_precision.html#torch.set_float32_matmul_precision
[01/23 22:32:44 Vis4D]: Load checkpoint from http path: https://download.pytorch.org/models/resnet101-63fe2227.pth
/root/cc-3dt-pp/vis4d/common/ckpt.py:375: UserWarning: The model and loaded state dict do not match exactly

unexpected key in source state_dict: fc.weight, fc.bias

rank_zero_warn(err_msg)
[01/23 22:32:44 Vis4D]: Load checkpoint from local path: /root/cc-3dt-pp/data/3drdt/cc-3dt/cc-3dt/cc_3dt_frcnn_r101_fpn_24e_nusc_f24f84.pt
/root/cc-3dt-pp/vis4d/common/ckpt.py:375: UserWarning: The model and loaded state dict do not match exactly

unexpected key in source state_dict: faster_rcnn_head.rpn_head.rpn_conv.weight, faster_rcnn_head.rpn_head.rpn_conv.bias, faster_rcnn_head.rpn_head.rpn_cls.weight, faster_rcnn_head.rpn_head.rpn_cls.bias, faster_rcnn_head.rpn_head.rpn_box.weight, faster_rcnn_head.rpn_head.rpn_box.bias, faster_rcnn_head.roi_head.fc_cls.weight, faster_rcnn_head.roi_head.fc_cls.bias, faster_rcnn_head.roi_head.fc_reg.weight, faster_rcnn_head.roi_head.fc_reg.bias, faster_rcnn_head.roi_head.shared_convs.0.weight, faster_rcnn_head.roi_head.shared_convs.0.bias, faster_rcnn_head.roi_head.shared_convs.1.weight, faster_rcnn_head.roi_head.shared_convs.1.bias, faster_rcnn_head.roi_head.shared_convs.2.weight, faster_rcnn_head.roi_head.shared_convs.2.bias, faster_rcnn_head.roi_head.shared_convs.3.weight, faster_rcnn_head.roi_head.shared_convs.3.bias, faster_rcnn_head.roi_head.shared_fcs.0.weight, faster_rcnn_head.roi_head.shared_fcs.0.bias, faster_rcnn_head.roi_head.shared_fcs.1.weight, faster_rcnn_head.roi_head.shared_fcs.1.bias, bbox_3d_head.shared_convs.0.weight, bbox_3d_head.shared_convs.0.bias, bbox_3d_head.shared_convs.1.weight, bbox_3d_head.shared_convs.1.bias, bbox_3d_head.dep_convs.0.weight, bbox_3d_head.dep_convs.0.bias, bbox_3d_head.dep_convs.1.weight, bbox_3d_head.dep_convs.1.bias, bbox_3d_head.dep_convs.2.weight, bbox_3d_head.dep_convs.2.bias, bbox_3d_head.dep_convs.3.weight, bbox_3d_head.dep_convs.3.bias, bbox_3d_head.dim_convs.0.weight, bbox_3d_head.dim_convs.0.bias, bbox_3d_head.dim_convs.1.weight, bbox_3d_head.dim_convs.1.bias, bbox_3d_head.dim_convs.2.weight, bbox_3d_head.dim_convs.2.bias, bbox_3d_head.dim_convs.3.weight, bbox_3d_head.dim_convs.3.bias, bbox_3d_head.rot_convs.0.weight, bbox_3d_head.rot_convs.0.bias, bbox_3d_head.rot_convs.1.weight, bbox_3d_head.rot_convs.1.bias, bbox_3d_head.rot_convs.2.weight, bbox_3d_head.rot_convs.2.bias, bbox_3d_head.rot_convs.3.weight, bbox_3d_head.rot_convs.3.bias, bbox_3d_head.cen_2d_convs.0.weight, bbox_3d_head.cen_2d_convs.0.bias, bbox_3d_head.cen_2d_convs.1.weight, bbox_3d_head.cen_2d_convs.1.bias, bbox_3d_head.cen_2d_convs.2.weight, bbox_3d_head.cen_2d_convs.2.bias, bbox_3d_head.cen_2d_convs.3.weight, bbox_3d_head.cen_2d_convs.3.bias, bbox_3d_head.fc_dep.weight, bbox_3d_head.fc_dep.bias, bbox_3d_head.fc_dep_uncer.weight, bbox_3d_head.fc_dep_uncer.bias, bbox_3d_head.fc_dim.weight, bbox_3d_head.fc_dim.bias, bbox_3d_head.fc_rot.weight, bbox_3d_head.fc_rot.bias, bbox_3d_head.fc_cen_2d.weight, bbox_3d_head.fc_cen_2d.bias

rank_zero_warn(err_msg)
LOCAL_RANK: 0 - CUDA_VISIBLE_DEVICES: [0,1]
[01/23 22:32:45 Vis4D]: Found /root/cc-3dt-pp/data/val.pkl generated at 2026-01-22T12:54:25.023082 and loading it...
[01/23 22:32:45 Vis4D]: Preprocessing 81 frames takes 0.01 seconds.
[01/23 22:32:45 Vis4D]: Distribution of instances among all 10 categories:

category	#instances	category	#instances	category	#instances
bicycle	52	motorcycle	259	pedestrian	1358
bus	41	car	2568	trailer	0
truck	124	constructio..	0	traffic_cone	39
barrier	0
total	4441
[01/23 22:32:45 Vis4D]: Loading NuScenesDetection v1.0-mini mini_val using /root/cc-3dt-pp/data/3drdt/detection_results/mini_results_nusc.json takes 0.05 seconds.
/opt/conda/envs/cc3dt/lib/python3.9/site-packages/lightning/pytorch/trainer/connectors/data_connector.py:434: The 'test_dataloader' does not have many workers which may be a bottleneck. Consider increasing the value of the num_workers argumenttonum_workers=19in theDataLoader` to improve performance.
[01/23 22:33:10 Vis4D]: Testing: 50/81, ETA: 0:00:15, 2.01it/s
[01/23 22:33:23 Vis4D]: Running evaluator NuScenes 3D Detection Evaluator...
Initializing nuScenes detection evaluation
Loaded results from vis4d-workspace/cc_3dt_pp_kf3d_nusc_mini/2026-01-23_22-32-41/detect_3d/detect_3d_predictions.json. Found detections for 81 samples.
Loading annotations for mini_val split from nuScenes version: v1.0-mini
100%	█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████	81/81 [00:00<00:00, 461.67it/s]
Loaded ground truth annotations for 81 samples.
Filtering predictions
=> Original number of boxes: 3420
=> After distance based filtering: 3257
=> After LIDAR and RADAR points based filtering: 3257
=> After bike rack filtering: 3241
Filtering ground truth annotations
=> Original number of boxes: 4441
=> After distance based filtering: 3785
=> After LIDAR and RADAR points based filtering: 3393
=> After bike rack filtering: 3393
Accumulating metric data...
Calculating metrics...
[01/23 22:33:26 Vis4D]: detect_3d/mAP: 0.2914
[01/23 22:33:26 Vis4D]: detect_3d/mATE: 0.6542
[01/23 22:33:26 Vis4D]: detect_3d/mASE: 0.4685
[01/23 22:33:26 Vis4D]: detect_3d/mAOE: 0.6740
[01/23 22:33:26 Vis4D]: detect_3d/mAVE: 0.8577
[01/23 22:33:26 Vis4D]: detect_3d/mAAE: 0.3494
[01/23 22:33:26 Vis4D]: detect_3d/NDS: 0.3453
[01/23 22:33:26 Vis4D]: Showing results for metric: detect_3d
[01/23 22:33:26 Vis4D]:
High-level metrics:
mAP: 0.2914
mATE: 0.6542
mASE: 0.4685
mAOE: 0.6740
mAVE: 0.8577
mAAE: 0.3494
NDS: 0.3453
Eval time: 1.4s

Per-class results:
Object Class AP ATE ASE AOE AVE AAE
car 0.450 0.393 0.166 0.119 0.515 0.181
truck 0.481 0.338 0.253 0.096 0.192 0.000
bus 0.490 0.630 0.081 0.056 1.983 0.009
trailer 0.000 1.000 1.000 1.000 1.000 1.000
construction_vehicle 0.000 1.000 1.000 1.000 1.000 1.000
pedestrian 0.455 0.530 0.260 0.662 0.913 0.401
motorcycle 0.223 0.576 0.267 1.335 0.304 0.022
bicycle 0.197 0.668 0.257 0.798 0.955 0.182
traffic_cone 0.618 0.406 0.401 nan nan nan
barrier 0.000 1.000 1.000 1.000 nan nan
[01/23 22:33:26 Vis4D]: Running evaluator NuScenes 3D Tracking Evaluator...
[01/23 22:33:26 Vis4D]: Showing results for metric: track_3d
[01/23 22:33:26 Vis4D]: Currently only save the json file.

Loading NuScenes tables for version v1.0-mini...
Loading nuScenes-lidarseg...
32 category,
8 attribute,
4 visibility,
911 instance,
12 sensor,
120 calibrated_sensor,
31206 ego_pose,
8 log,
10 scene,
404 sample,
31206 sample_data,
18538 sample_annotation,
4 map,
404 lidarseg,
Done loading in 0.404 seconds.

Reverse indexing ...
Done reverse indexing in 0.1 seconds.

Initializing nuScenes tracking evaluation
Loaded results from /root/cc-3dt-pp/vis4d-workspace/cc_3dt_pp_kf3d_nusc_mini/2026-01-22_13-16-18/track_3d/track_3d_predictions.json. Found detections for 81 samples.
Loading annotations for mini_val split from nuScenes version: v1.0-mini
100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 81/81 [00:00<00:00, 479.21it/s]
Loaded ground truth annotations for 81 samples.
Filtering tracks
=> Original number of boxes: 3329
=> After distance based filtering: 3170
=> After LIDAR and RADAR points based filtering: 3170
=> After bike rack filtering: 3154
Filtering ground truth tracks
=> Original number of boxes: 4402
=> After distance based filtering: 3748
=> After LIDAR and RADAR points based filtering: 3358
=> After bike rack filtering: 3358
Accumulating metric data...
Computing metrics for class bicycle...

0%| | 0/2 [00:00<?, ?it/s]/opt/conda/envs/cc3dt/lib/python3.9/site-packages/motmetrics/mot.py:178: DeprecationWarning: np.bool is a deprecated alias for the builtin bool. To silence this warning, use bool by itself. Doing this will not modify any behavior and is safe. If you specifically wanted the numpy scalar type, use np.bool_ here.
Deprecated in NumPy 1.20; for more details and guidance: https://numpy.org/devdocs/release/1.20.0-notes.html#deprecations
oids_masked = np.zeros_like(oids, dtype=np.bool)
/opt/conda/envs/cc3dt/lib/python3.9/site-packages/motmetrics/mot.py:180: DeprecationWarning: np.bool is a deprecated alias for the builtin bool. To silence this warning, use bool by itself. Doing this will not modify any behavior and is safe. If you specifically wanted the numpy scalar type, use np.bool_ here.
Deprecated in NumPy 1.20; for more details and guidance: https://numpy.org/devdocs/release/1.20.0-notes.html#deprecations
hids_masked = np.zeros_like(hids, dtype=np.bool)
Traceback (most recent call last):
File "/root/cc-3dt-pp/eval_nusc.py", line 110, in
evaluate(
File "/root/cc-3dt-pp/eval_nusc.py", line 96, in evaluate
eval_tracking(version, output_dir, result_path, root, eval_set)
File "/root/cc-3dt-pp/eval_nusc.py", line 76, in eval_tracking
_ = nusc_eval.main()
File "/opt/conda/envs/cc3dt/lib/python3.9/site-packages/nuscenes/eval/tracking/evaluate.py", line 204, in main
metrics, metric_data_list = self.evaluate()
File "/opt/conda/envs/cc3dt/lib/python3.9/site-packages/nuscenes/eval/tracking/evaluate.py", line 135, in evaluate
accumulate_class(class_name)
File "/opt/conda/envs/cc3dt/lib/python3.9/site-packages/nuscenes/eval/tracking/evaluate.py", line 131, in accumulate_class
curr_md = curr_ev.accumulate()
File "/opt/conda/envs/cc3dt/lib/python3.9/site-packages/nuscenes/eval/tracking/algo.py", line 123, in accumulate
thresholds, recalls = self.compute_thresholds(gt_box_count)
File "/opt/conda/envs/cc3dt/lib/python3.9/site-packages/nuscenes/eval/tracking/algo.py", line 303, in compute_thresholds
_, scores = self.accumulate_threshold(threshold=None)
File "/opt/conda/envs/cc3dt/lib/python3.9/site-packages/nuscenes/eval/tracking/algo.py", line 273, in accumulate_threshold
events = acc.events.loc[frame_id]
File "/opt/conda/envs/cc3dt/lib/python3.9/site-packages/nuscenes/eval/tracking/mot.py", line 61, in events
self.cached_events_df = MOTAccumulatorCustom.new_event_dataframe_with_data(self._indices, self._events)
File "/opt/conda/envs/cc3dt/lib/python3.9/site-packages/nuscenes/eval/tracking/mot.py", line 36, in new_event_dataframe_with_data
idx = pd.MultiIndex.from_tuples(indices, names=['FrameId', 'Event'])
File "/opt/conda/envs/cc3dt/lib/python3.9/site-packages/pandas/core/indexes/multi.py", line 202, in new_meth
return meth(self_or_cls, *args, **kwargs)
File "/opt/conda/envs/cc3dt/lib/python3.9/site-packages/pandas/core/indexes/multi.py", line 560, in from_tuples
return cls.from_arrays(arrays, sortorder=sortorder, names=names)
File "/opt/conda/envs/cc3dt/lib/python3.9/site-packages/pandas/core/indexes/multi.py", line 487, in from_arrays
return cls(
File "/opt/conda/envs/cc3dt/lib/python3.9/site-packages/pandas/core/indexes/multi.py", line 331, in new
result._set_names(names)
File "/opt/conda/envs/cc3dt/lib/python3.9/site-packages/pandas/core/indexes/multi.py", line 1430, in _set_names
raise ValueError(
ValueError: Length of names must match number of levels in MultiIndex.

[RoyYang0714]

Hi, I remember it is a potential issue with the Nuscenes evaluation code. I think I resolved it by using pandas==1.1.5.