bug: video that is flipped or rotated in Cockpit does not record / save in this adjusted orientation.

[vshie]

Bug description

Video streams can (very nicely) be rotated (even multiple times1) or flipped. However, when the recording is downloaded, the video is in the original orientation.

Steps to reproduce

Rotate video stream in edit interface menu, then make a recording. Rotation/flip of displayed video will not match recording.

Primary pain point(s)

The browser decodes, and then we flip, but João mentions that it may be possible to hint the correct orientation in the recording?

Additional context

No response

Prerequisites

• I have checked to make sure that a similar request has not already been filed or fixed.

[joaoantoniocardoso]

Yeah, the video's metadata contains a rotation field for that purpose. Cockpit should be able to change it after processing the chunks.

[rafaellehmkuhl]

Yeah, the video's metadata contains a rotation field for that purpose. Cockpit should be able to change it after processing the chunks.

Problem is we have to reprocess the chunk, right?

[joaoantoniocardoso]

Yeah, the video's metadata contains a rotation field for that purpose. Cockpit should be able to change it after processing the chunks.

Problem is we have to reprocess the chunk, right?

So the two options are:

1. Change the metadata of the final .mp4 video file and don't mess with the file data, should be quick.
2. Decode, change rotate/flip frame frame, then re-encode every frame. We might want to do it since we are going to have a video processing pipeline anyway (for color lens and color correction, for example)

[ES-Alexander]

Option 2 seems useful, but advanced. In cases where there isn't already processing happening it would be preferable to support option 1, especially for the sake of lower end systems that could struggle with encoding.

[rafaellehmkuhl]

Although option 1 seems ideal, I'm struggling to find any way to do this in the browser @joaoantoniocardoso. If you have any clue let me know.

Would be ffmpeg.wasm to go.

[joaoantoniocardoso]

Would be ffmpeg.wasm to go.

Yep, the ffmpeg can do that, I expect the wasm version to also be capable. Maybe something like https://www.npmjs.com/mp4js could also do that

[ES-Alexander]

Noting that semantically the video recorder mini-widget acts on the incoming video stream, and is unrelated to any video display widget configuration options. Multiple display widgets could show the same stream in different ways, and we generally don't want or expect things like the widget scale or cropping to affect the recorded video.

If we wanted to record a stream as flipped/rotated then we'd need to add controls for that in the video configuration section, and potentially remove them from the video widget (if we don't expect users to need to handle them independently for different outputs).