Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

_pickle.UnpicklingError: invalid load key, '<'. #2

Open
junminren2 opened this issue Dec 27, 2023 · 17 comments
Open

_pickle.UnpicklingError: invalid load key, '<'. #2

junminren2 opened this issue Dec 27, 2023 · 17 comments

Comments

@junminren2
Copy link

hi, I found sth maybe needs to be revised:
the link is changed, below is correct.
!wget -P checkpoints https://huggingface.co/16lemoing/dot/blob/main/cvo_raft_patch_8.pth
!wget -P checkpoints https://huggingface.co/16lemoing/dot/blob/main/movi_f_raft_patch_4_alpha.pth
!wget -P checkpoints https://huggingface.co/16lemoing/dot/blob/main/movi_f_cotracker_patch_4_wind_8.pth
!wget -P datasets https://huggingface.co/16lemoing/dot/blob/main/demo.zip
!unzip datasets/demo.zip -d datasets/
When I installed any package and ran the command below.
!python demo.py --visualization_modes spaghetti_last_static --video_path orange.mp4
!python demo.py --visualization_modes spaghetti_last_static --video_path treadmill.mp4
I found an error here:
Traceback (most recent call last):
File "/content/dot/demo.py", line 310, in
main(args)
File "/content/dot/demo.py", line 269, in main
model = create_model(args).cuda()
File "/content/dot/dot/models/init.py", line 7, in create_model
model = DenseOpticalTracker(
File "/content/dot/dot/models/dense_optical_tracking.py", line 23, in init
self.point_tracker = PointTracker(height, width, tracker_config, tracker_path, estimator_config, estimator_path)
File "/content/dot/dot/models/point_tracking.py", line 19, in init
self.model.load_state_dict(torch.load(tracker_path, map_location=device))
File "/usr/local/lib/python3.10/dist-packages/torch/serialization.py", line 1028, in load
return _legacy_load(opened_file, map_location, pickle_module, **pickle_load_args)
File "/usr/local/lib/python3.10/dist-packages/torch/serialization.py", line 1246, in _legacy_load
magic_number = pickle_module.load(f, **pickle_load_args)
_pickle.UnpicklingError: invalid load key, '<'.
I checked the track_path: movi_f_cotracker_patch_4_wind_8.pth and I guess there is a problem with the saving format of model. I really need your help, Thanks.
@16lemoing
Copy link
Owner

Hi, I have found what is causing your issue. You are not using the correct download links.
Here are the correct links (replace blob by resolve):

wget -P checkpoints https://huggingface.co/16lemoing/dot/resolve/main/cvo_raft_patch_8.pth
wget -P checkpoints https://huggingface.co/16lemoing/dot/resolve/main/movi_f_raft_patch_4_alpha.pth
wget -P checkpoints https://huggingface.co/16lemoing/dot/resolve/main/movi_f_cotracker_patch_4_wind_8.pth
wget -P datasets https://huggingface.co/16lemoing/dot/resolve/main/demo.zip

I have already updated the README. Let me know if it works now.

@junminren2
Copy link
Author

Hi, I have found what is causing your issue. You are not using the correct download links. Here are the correct links (replace blob by resolve):

wget -P checkpoints https://huggingface.co/16lemoing/dot/resolve/main/cvo_raft_patch_8.pth
wget -P checkpoints https://huggingface.co/16lemoing/dot/resolve/main/movi_f_raft_patch_4_alpha.pth
wget -P checkpoints https://huggingface.co/16lemoing/dot/resolve/main/movi_f_cotracker_patch_4_wind_8.pth
wget -P datasets https://huggingface.co/16lemoing/dot/resolve/main/demo.zip

I have already updated the README. Let me know if it works now.

Thanks for your help, it could work now. But I found the speed is very slow like this.
image
There is any method to accelerate it?

@junminren2
Copy link
Author

Hi, I have found what is causing your issue. You are not using the correct download links. Here are the correct links (replace blob by resolve):

wget -P checkpoints https://huggingface.co/16lemoing/dot/resolve/main/cvo_raft_patch_8.pth
wget -P checkpoints https://huggingface.co/16lemoing/dot/resolve/main/movi_f_raft_patch_4_alpha.pth
wget -P checkpoints https://huggingface.co/16lemoing/dot/resolve/main/movi_f_cotracker_patch_4_wind_8.pth
wget -P datasets https://huggingface.co/16lemoing/dot/resolve/main/demo.zip

I have already updated the README. Let me know if it works now.

Thanks for your help, it could work now. But I found the speed is very slow like this. image There is any method to accelerate it?

Refine target step: 0% 0/51 [00:00<?, ?it/s]/content/dot/dot/models/optical_flow.py:73: UserWarning: Torch3D is not available. For optimal speed and memory consumption, consider setting it up.
coarse_flow, coarse_alpha = interpolate(data["src_points"], data["tgt_points"], self.coarse_grid,

@16lemoing
Copy link
Owner

Thanks for your help, it could work now. But I found the speed is very slow like this.
There is any method to accelerate it?

Speed depends on the length of the input video. The time bottleneck is the computation of inital tracks with CoTracker. For instance, it takes ~40 sec for tracking a batch of 2048 points for a video with 120 frames.

You can trade accuracy for speed by using fewer point tracks during inference, e.g.:

python demo.py --visualization_modes spaghetti_last_static --video_path treadmill.mp4 --num_tracks 1024 --sim_tracks 1024

UserWarning: Torch3D is not available.

You can enable Torch3D by running:

cd dot/utils/torch3d/ && python setup.py install && cd ../../..

But this will not change speed much. It is mostly for reducing memory consumption for high resolution inputs.

@junminren2
Copy link
Author

Thanks for your help, it could work now. But I found the speed is very slow like this.
There is any method to accelerate it?

Speed depends on the length of the input video. The time bottleneck is the computation of inital tracks with CoTracker. For instance, it takes ~40 sec for tracking a batch of 2048 points for a video with 120 frames.

You can trade accuracy for speed by using fewer point tracks during inference, e.g.:

python demo.py --visualization_modes spaghetti_last_static --video_path treadmill.mp4 --num_tracks 1024 --sim_tracks 1024

UserWarning: Torch3D is not available.

You can enable Torch3D by running:

cd dot/utils/torch3d/ && python setup.py install && cd ../../..

But this will not change speed much. It is mostly for reducing memory consumption for high resolution inputs.

Thanks for your help, Moing. I tried inputting my own video to get a demo, but I found the program quickly finished with just output '^C'. Do you know what problems happened?

@16lemoing
Copy link
Owner

Hi,don't you have any error message apart from '^C'? Can you share the video so that I can try and see if I can reproduce the issue?

@junminren2
Copy link
Author

junminren2 commented Jan 2, 2024 via email

@junminren2
Copy link
Author

Hi,don't you have any error message apart from '^C'? Can you share the video so that I can try and see if I can reproduce the issue?

Could you please send me your private email? Thanks

@16lemoing
Copy link
Owner

16lemoing commented Jan 2, 2024 •
edited
Loading

Hi, you can send me a message at [email protected]

@junminren2
Copy link
Author

Hi, you can send me a message at [email protected]

Thanks moing, have you already received my email? Thanks

@16lemoing
Copy link
Owner

Not yet. Can you try again? There was a typo in my original reply. You can also try [email protected]

@junminren2
Copy link
Author

Not yet. Can you try again? There was a typo in my original reply. You can also try [email protected]

Thanks Moing, I already sent to your email [email protected]. Did you received well?

@16lemoing
Copy link
Owner

Hi @junminren2 ! I have received your video. It seems that your video is too high resolution ~900 frames at 1080p (reduced to 480p by default with the demo script) to be loaded all at once in memory. I suggest you try with a shorter video first. I can try and implement an online mode that loads frames one by one, to handle long videos with high-resolution frames like yours. Let me know if you are interested.

@junminren2
Copy link
Author

junminren2 commented Jan 24, 2024 via email

@16lemoing
Copy link
Owner

Hi! This feature has also been requested in another issue (#8) so I will try to implement it in the next few weeks.

@junminren2
Copy link
Author

junminren2 commented Jan 24, 2024 via email

@16lemoing
Copy link
Owner

You can specify a mask on the first frame to filter tracks (see varanus example in the README), more advanced filtering is not yet supported.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@16lemoing @junminren2