Problem about batch size and video length

[yifanjiang19]

Hello!
I have some questions. When I set the image_batch, video_batch, and video_length both equal to 16, it works well. But when I set them to 48, it occurs:

Traceback (most recent call last):
File "train.py", line 133, in
trainer.train(generator, image_discriminator, video_discriminator)
File "mocogan/src/trainers.py", line 273, in train
opt_generator)
File "mocogan/src/trainers.py", line 212, in train_generator
l_generator += self.category_criterion(fake_categorical.squeeze(), generated_categories)
File "/usr/local/lib/python2.7/dist-packages/torch/nn/modules/module.py", line 325, in call
result = self.forward(*input, **kwargs)
File "/usr/local/lib/python2.7/dist-packages/torch/nn/modules/loss.py", line 601, in forward
self.ignore_index, self.reduce)
File "/usr/local/lib/python2.7/dist-packages/torch/nn/functional.py", line 1140, in cross_entropy
return nll_loss(log_softmax(input, 1), target, weight, size_average, ignore_index, reduce)
File "/usr/local/lib/python2.7/dist-packages/torch/nn/functional.py", line 1053, in nll_loss
raise ValueError('Expected 2 or 4 dimensions (got {})'.format(dim))
ValueError: Expected 2 or 4 dimensions (got 3)

Do the batch size number and video length have a relation? Should I set them same?
Thanks~

[sergeytulyakov]

Thanks for reporting, I'll have a look into that

[bragilee]

@yueruchen

Hi,
I have the similar problem before.
I think we need to check arguments passed to data loader (data.py) and also model (model.py) if necessary. Some arguments are pre-set manually. Hope it will help you. :)

[Cpruce]

It seems video length is hardcoded to 16

mocogan/src/train.py

Line 108 in 937a99d

video_dataset = data.VideoDataset(dataset, 16, 2, video_transforms)

Though changing video length still results in an error like:
ValueError: Expected target size (32, 17), got torch.Size([32])

[Cpruce]

When increasing the video length (32 in this case), it seems that the difference+1 between the new video length and 16 is added to the next dimension.

Symptom occurs at

mocogan/src/trainers.py

Line 211 in 937a99d

l_generator += self.category_criterion(fake_categorical.squeeze(), generated_categories)

video_length==16 (works):
('fake_categorical', (32, 102))
('generated_categories', (32,))

video_length==32 (does not work):
('fake_categorical', (32, 102, 17))
('generated_categories', (32,))

Update:

I've narrowed the issue down to the video_discriminator

The generated_categories seem correct, just the output shape of the video discriminator is adding another dimension that should not be there

Update 2:

I've got the architecture to train with video length 32 by increasing the stride in the VideoDiscriminator's last Conv3d layer. However, I'm almost positive this is not the correct approach and I run out of memory.

@sergeytulyakov did you test with any other video lengths? Either way, how do you recommend handling different video lengths?

[HeegerGao]

I've got the same problem. Is there any new progress about this question?