AV1Packet incompatible with sample builder

[danjenkins]

When trying to use AV1Packet as a depacketiser I get an error...

cannot use &(codecs.AV1Packet literal) (value of type *"github.com/pion/rtp/codecs".AV1Packet) as "github.com/pion/rtp".Depacketizer value in assignment: *"github.com/pion/rtp/codecs".AV1Packet does not implement "github.com/pion/rtp".Depacketizer (missing method IsPartitionHead)

[tmatth]

I was hoping the solution was just as simple as patching https://github.com/pion/rtp/blob/master/codecs/av1_packet.go with:

diff --git a/codecs/av1_packet.go b/codecs/av1_packet.go
index 7aa3a55..d52e920 100644
--- a/codecs/av1_packet.go
+++ b/codecs/av1_packet.go
@@ -104,6 +104,8 @@ type AV1Packet struct {
       // Each AV1 RTP Packet is a collection of OBU Elements. Each OBU Element may be a full OBU, or just a fragment of one.
       // AV1Frame provides the tools to construct a collection of OBUs from a collection of OBU Elements
       OBUElements [][]byte
+
+       videoDepacketizer
}

// Unmarshal parses the passed byte slice and stores the result in the AV1Packet this method is called upon
@@ -156,3 +158,11 @@ func (p *AV1Packet) Unmarshal(payload []byte) ([]byte, error) {

       return payload[1:], nil
}
+
+// IsPartitionHead checks whether if this is a head of the AV1 partition
+func (*AV1Packet) IsPartitionHead(payload []byte) bool {
+       if len(payload) < 1 {
+               return false
+       }
+       return (payload[0] & 0x08) != 0
+}

but I'm a bit skeptical.

[danjenkins]

@tmatth did you try it out? I can give it a go if not?

[tmatth]

@tmatth did you try it out? I can give it a go if not?

I did not, it's basically just the same code from the vp9 depacketizer and I'm guessing it means it's checking the RTP marker bit (which is also at position 0x08 for av1), see:
https://aomediacodec.github.io/av1-rtp-spec/#42-rtp-header-marker-bit-m vs.
https://tools.ietf.org/id/draft-ietf-payload-vp9-09.html#rfc.section.4.1

but again I'd be surprised if this worked.

[mikebevz]

Any news on this fix?

[jech]

@tmatth I'm pretty sure your patch is not correct.

IsPartitionHead should be checking the Z bit in the AV1 aggregation header. That's bit 0x80, see https://aomediacodec.github.io/av1-rtp-spec/#44-av1-aggregation-header.

You'll also need some code to remove the aggregation headers when you generate samples. This will require some changes to the samplebuilder itself.

[tmatth]

@tmatth I'm pretty sure your patch is not correct.

I kind of assumed that'd be the case.

IsPartitionHead should be checking the Z bit in the AV1 aggregation header. That's bit 0x80, see https://aomediacodec.github.io/av1-rtp-spec/#44-av1-aggregation-header.

You'll also need some code to remove the aggregation headers when you generate samples. This will require some changes to the samplebuilder itself.

That's good to know. FWIW I started working an rtp muxer for AV1 in C so that should give me a better handle on the format, but if someone else wants to take a stab at this here it'd be most welcome.

[tmatth]

That's good to know. FWIW I started working an rtp muxer for AV1 in C so that should give me a better handle on the format, but if someone else wants to take a stab at this here it'd be most welcome.

Oh I see you've already started on this, tremendous: #264

[JoeTurki]

Since AV1 RTP packets are different, using the same interface for packetization and depacketization wouldn't be correct. I've implemented AV1Depacketizer, which works with SampleBuilder to reconstruct OBUs with obu_size_fields=true, handle defragmentation, and return the complete units. Here how it can be used to fix the IVF reader pion/webrtc#3041 I'll update some examples with the SampleBuilder soon.