Codec metadata missing

[snakehill]

It works awesome for online uploads like on YouTube, and it seems to be the absolutely way to go, especially now that Google has stopped making available its VR180 application.

However, unlike Google's Spatial Media Metadata Injector, this doesn't add the usual codec metadata. So most (offline) media players won't and don't recognize and play it properly.

This tool:

Google's Injector:

To make it work in everything, I first use Google's Injector and then your tool, but that seems a little redundant and limits the projection possibilities.

[Vargol]

Hi,

just to check, are you using VR180 or Equirectangular video's and the GUI our the command line ?
If you used the command line what command did you use.

Those extra values are Equirectangular values not VR180 values e.,g.
Projection would be Mesh or Fisheye for VR180 (the actual value is 'mshp' the GI your video would be 'equi').

The last I used Google's Spatial Media Metadata Injector, and a very quick look suggests that's not changed, it didn't support VR180 at all, that why I forked it.

I ran a quick check and the metadata is there for Equirectangular
Loaded file...
Track 0
Stereo Mode: left-right
Spherical Mode: equirectangular
[Yaw: 0.00, Pitch: 0.00, Roll: 0.00]
[Clip Top: 0, Bottom: 0, Left: 1073741823 Right: 1073741823]

and for VR180 mesh

Loaded file...
Track 0
Stereo Mode: left-right
Spherical Mode: mesh
Mesh Projection:
[Mesh count 2]
[Yaw: 0.00, Pitch: 0.00, Roll: 0.00]
[Clip Top: 0, Bottom: 0, Left: 0 Right: 0]

Note the clip values for mesh are zero as they are not actually there, that's something that needs tiding up, the clip values are only valid for Equirectangular videos.

If the values survive a pass through my tools then that suggests they are not the actual spatial metadata as that mp4 atom gets totally re-written by my version of the tool.

[snakehill]

Thanks for the swift reply!

It goes for both 360 and 3D 180. In players (like VLC) it will keep playing stretched out or with the images side by side, opposed to it allowing to move through the space, I assume due to the codec data missing as shown.

After using Google's Spatial Media Metadata Injector, it works as expected (although in VLC, the VR180 will act like 360 footage with the left and right eye next to/opposing each other, but that seems to be the case always). It allows one to move through the space instead of it being a flat (stretched out or side-by-side) video.

Using Google's Injector alone, makes VR180 videos act weirdly on YouTube (like 360 footage), but then adding your injection fixes that and keeps allowing one to move through the space in media players like VLC.

I used the command line as well as the GUI with the same results (tried Python 2.7 and the most recent 3.9):
python spatialmedia -i -s left-right -m equirectangular final.mp4 final_injected.mp4

[Vargol]

using your command line I get the metadata you're expecting

[Vargol]

Okay, I think I figured it, something I'd long forgotten.

Google metadata injection only injects the old spatial metadata, my tool does version 2 which is the type recognised by Youtube. It appears that for H264 videos VLC recognises version 2 (which is why it worked for me) but doesn't for H265 , which I'd argue is a bug for VLC (or maybe upstream in libav) as the metadata is a container thing not a codec thing.

[Vargol]

Seems they've fixed it in VLC 4 if it ever gets released.

[snakehill]

I checked and indeed, on h.264 it works fine. But, Google's injection also works on h.265, so that would lead me to believe there still is something wrong here.

[Vargol]

Like I said, Google injector uses an old obsolete metadata format, it is that metadata VLC is reading for H265, not the newer (but 3 -5 years old) format that my version uses, which VLC does parse for H264 and H265 in VLC 4's development version.

[Vargol]

Okay, I've had a go at adding v1 metadata back

-1, --force_v1_360_equi_metadata
Add v1 metadata as well as v2 metadata to the video.only really valid for 360 equirectangular videos, but some video player only enable VR if
they recognise v1 metadata

e.g.

python spatialmedia -1 -i -s left-right -m equirectangular equi-broke_x265.mp4 equi_test_v1.mp4

Its seems to work okay

[snakehill]

You're the best! Especially if you could add a checkbox for it in the GUI also. 😊🙏

If so, please keep that possibility enabled for all options (180 also), for example naming it something like "add (deprecated) equirectangular v1 metadata" to make clear it's not really intended for the other options).

Personally, I would imagine the GUI like this. Please see it just as a suggestion, do with it as you please, I would be more than happy if there was just a functioning checkbox for the v1 metadata added given I can change the text around myself also. I also found a typo, "foramtted", but this would get rid of that too:

The changes:

• Current 360 video: => Current video:
• Click Open => Click 'Open'
• My video is foramtted as.... => Format:
• equi-rectangular => equirectangular (it's safe to replace all dashed instances within gui.py)
• My video is 180 or 360 degree... => Degrees:
• My video layout is... => Layout:
• My video has spatial audio (ambiX ACN/SN3D format) => Spatial audio (AmbiX ACN/SN3D): (still goes fine with the "with head-locked stereo" appearance)
• Add equirectangular v1 metadata:

Do you have any means of accepting donations?