Do videos placed in videos_dgp need to be labeled?

[vcorbit]

Hi,

I'm trying to run DGP on some DLC-labeled data I have from a long time ago. My DLC version is not up to date, but I already have the project folder, labeled data, and I ran up to iteration-7 on the DLC network, last year.

When I try to run DGP, I get through to "Running DGP with labeled frames only" and this section:

Creating training datasets
Selected additional 95 hidden frames
Skipped 0 high motion energy (me) frames since in visible window or close to higher me hidden frame
Selected additional 95 hidden frames
Skipped 0 high motion energy (me) frames since in visible window or close to higher me hidden frame
Selected additional 95 hidden frames
Skipped 0 high motion energy (me) frames since in visible window or close to higher me hidden frame

Video: 2mt_drtop_concat_cropped has 0 visible frames selected; 95 hidden frames selected.
Video: 2mt_b2279_concat_cropped has 0 visible frames selected; 95 hidden frames selected.
Video: 2mt_c2293_concat_cropped has 0 visible frames selected; 95 hidden frames selected.
n_hidden_frames_total 5985
n_visible_frames_total 0
n_frames_total 5985

But then I get this error:

Traceback (most recent call last):
File "/tigress/vcorbit/DGP/deepgraphpose/demo/run_dgp_demo.py", line 209, in
step=1)
File "/tigress/vcorbit/DGP/deepgraphpose/src/deepgraphpose/models/fitdgp.py", line 390, in fit_dgp_labeledonly
loss, total_loss, total_loss_visible, placeholders = dgp_loss(data_batcher, dgp_cfg)
File "/tigress/vcorbit/DGP/deepgraphpose/src/deepgraphpose/models/fitdgp.py", line 885, in dgp_loss
limb_full = np.reshape(limb_full, [joint_loc_full.shape[0], 2, -1])
File "/home/vcorbit/.conda/envs/dgp/lib/python3.6/site-packages/numpy/core/fromnumeric.py", line 292, in reshape
return _wrapfunc(a, 'reshape', newshape, order=order)
File "/home/vcorbit/.conda/envs/dgp/lib/python3.6/site-packages/numpy/core/fromnumeric.py", line 56, in _wrapfunc
return getattr(obj, method)(*args, **kwds)
ValueError: cannot reshape array of size 0 into shape (0,2,newaxis)

So, I'm a bit confused by the instructions, should the videos I include in the videos_dgp folder for "test phase" be videos that already have DLC labels? Or can they be videos that the network has never seen?

[waq1129]

Hi
videos_dgp should include all videos (labeled or not labeled). But you have to have some labeled frames input to DGP-labeled, as the ones you use for DLC. It seems like you have no labeled/visible frames at all. That's why it throws this error.

Best,
Anqi

[vcorbit]

Hi Anqi,

Thanks for clarifying. Another clarification question - do ALL of my labeled videos need to be in that videos_dgp folder? I'm having trouble getting the DGP to finish running in my max time limit on our University server, so I thought I'd try to reduce the number of videos in that folder. However when I did that, I got this error:
Traceback (most recent call last):
File "/tigress/vcorbit/DGP/deepgraphpose/demo/run_dgp_demo.py", line 209, in
step=1)
File "/tigress/vcorbit/DGP/deepgraphpose/src/deepgraphpose/models/fitdgp.py", line 341, in fit_dgp_labeledonly
S0=S0)
File "/tigress/vcorbit/DGP/deepgraphpose/src/deepgraphpose/dataset.py", line 869, in init
self.datasets.append(Dataset(video_file, self.dlc_config, self.paths))
File "/tigress/vcorbit/DGP/deepgraphpose/src/deepgraphpose/dataset.py", line 312, in init
self.video_clip = VideoFileClip(str(self.video_path))
File "/home/vcorbit/.conda/envs/dgp/lib/python3.6/site-packages/moviepy/video/io/VideoFileClip.py", line 91, in init
fps_source=fps_source)
File "/home/vcorbit/.conda/envs/dgp/lib/python3.6/site-packages/moviepy/video/io/ffmpeg_reader.py", line 33, in init
fps_source)
File "/home/vcorbit/.conda/envs/dgp/lib/python3.6/site-packages/moviepy/video/io/ffmpeg_reader.py", line 276, in ffmpeg_parse_infos
"path.")%filename)
OSError: MoviePy error: the file /tigress/vcorbit/DLC/lineartrack-new-VC-2020-01-29/videos_dgp/3442mouse2_dlc.mp4 could not be found!
Please check that you entered the correct path.

I'm pretty sure this came about because it's looking for a video that I have labels for, but I removed the actual video from videos_dgp. My DLC network has a total of ~1200 labeled frames across 42 videos, and so far I haven't been able to complete a run of DGP before my server job times out. Any advice you have for how to handle this large amount of data would be great! Maybe I just need to copy my DLC project folder and reduce the number of labels/videos for DGP to run in a timely manner?

[waq1129]

Hi

Is this missing video a labeled one or an unlabeled one?

[vcorbit]

The missing video was a labeled video.