Interceptor Framework for RTP/RTCP Processing or Direct RTP/RTCP Access for Custom Packetization

[kinsleykajiva]

Is your feature request related to a problem? Please describe.

There's currently no way to intercept, modify, or inject custom encoders, transform packets, or interoperate into the RTP/RTCP packet processing pipeline. This prevents implementing:
Enhanced monitoring (inspect packet-level metrics (jitter, packet loss patterns, RTT)) , Custom FEC (forward error correction) schemes , Research and experimentation (Cannot prototype new RTP extensions or RTCP feedback mechanisms)

Describe the solution you'd like

Expose RTP/RTCP send and receive hooks via a MediaStreamTrack or RtpTransceiver API. This should allow access to RtpReceiver and RtpSender transport streams .

Implement a pluggable interceptor framework similar to Pion's interceptor package, allowing developers to insert custom processing logic into the media pipeline:

1. Interceptor Interface:

   interface Interceptor {
       // Called for each outgoing RTP packet
       RTCRtpPacket interceptRTP(RTCRtpPacket packet, RTCRtpSender sender);
       
       // Called for each incoming RTP packet
       RTCRtpPacket interceptIncomingRTP(RTCRtpPacket packet, RTCRtpReceiver receiver);
       
       // Called for outgoing RTCP packets
       RTCRtcpPacket interceptRTCP(RTCRtcpPacket packet);
       
       // Called for incoming RTCP packets
       RTCRtcpPacket interceptIncomingRTCP(RTCRtcpPacket packet);
   }

2. Interceptor Chain Management:

   interface InterceptorRegistry {
       void add(Interceptor interceptor);
       void remove(Interceptor interceptor);
       List<Interceptor> getInterceptors();
   }
   
   // Usage
   peerConnection.getInterceptorRegistry().add(new CustomInterceptor());

3. Built-in Interceptors: Provide commonly-needed interceptors:

   • NACKInterceptor: Handle NACKs as per RFC 4585
   • TWCCInterceptor: Transport-Wide Congestion Control per draft-holmer-rmcat-transport-wide-cc-extensions
   • StatsInterceptor: Collect per-SSRC statistics
   • RecordingInterceptor: Dump packets to file for debugging

4. Packet Access Objects: Expose packet structures:

   class RTCRtpPacket {
       RTCRtpHeader header;
       byte[] payload;
       List<RTCRtpHeaderExtension> extensions;
   }
   
   class RTCRtpHeader {
       int payloadType;
       int sequenceNumber;
       long timestamp;
       long ssrc;
       List<Long> csrcs;
   }

Describe alternatives you've considered

Tried to implement own interface but due to local dev complications with webrtc code base and JNI complexity but ran into a wall trying to do this , hence this request .

Additional context

• Interceptors are essential for building production-grade WebRTC applications that need observability and control

• Pion WebRTC's interceptor framework: github.com/pion/interceptor

• Key specifications:

  • RFC 3550 - RTP: A Transport Protocol for Real-Time Applications
  • RFC 3551 - RTP Profile for Audio and Video Conferences
  • RFC 4585 - Extended RTP Profile for RTCP-Based Feedback
  • RFC 5104 - Codec Control Messages in AVPF
  • RFC 5761 – Multiplexing RTP Data and Control Packets on a Single Port

• Direct packet access enables advanced analytics, media transformation, and interoperability.