Skip to content

Pull Stream from SRS error "[h264 @ 000000007DA17600] no frame!" #224

@nex6328

Description

@nex6328

Describe the bug
When using the webrtc-java library to connect to the SRS streaming media server and pull a WebRTC video stream, the video playback is extremely slow (approximately one frame updates every 2 seconds). Simultaneously, the console frequently reports the following H.264 decoding error:
[h264 @ 000000007DA17600] no frame!
The client fails to receive and render the video frames smoothly and correctly.

To Reproduce
Steps to reproduce the behavior:

  1. Start the SRS Service
  2. DJI Drone push h264 rtmp stream
  3. WHEP pull

Expected behavior
Observe Behavior: The WebRTC Java client connects to SRS but fails to pull the stream correctly. The video frame update is very slow (about 2 seconds per frame), and the console outputs the error: [h264 @ 000000007DA17600] no frame!
but use ffmpeg pull rtmp or flv normal, and use web js library normal

Screenshots
Image

Desktop (please complete the following information):

  • Windows 10 19045.6396

Additional context
create offer

o=- 5814958837898991714 2 IN IP4 127.0.0.1
s=-
t=0 0
a=group:BUNDLE 0
a=extmap-allow-mixed
a=msid-semantic: WMS
m=video 9 UDP/TLS/RTP/SAVPF 96 97 98 99 100 101 35 36 37 38 103 104 107 108 109 114 115 116 117 118 39 40 41 42 43 44 45 46 47 48 119 120 121 49
c=IN IP4 0.0.0.0
a=rtcp:9 IN IP4 0.0.0.0
a=ice-ufrag:y3BZ
a=ice-pwd:aqPibVjEEn/pL4lBuaxxDo1b
a=ice-options:trickle
a=fingerprint:sha-256 D4:F7:AD:B2:B3:54:E2:5D:F5:CE:D0:DB:50:4C:22:BF:65:CB:3D:8C:DA:99:A0:AE:B0:08:C2:E1:B3:43:08:58
a=setup:actpass
a=mid:0
a=extmap:1 urn:ietf:params:rtp-hdrext:toffset
a=extmap:2 http://www.webrtc.org/experiments/rtp-hdrext/abs-send-time
a=extmap:3 urn:3gpp:video-orientation
a=extmap:4 http://www.ietf.org/id/draft-holmer-rmcat-transport-wide-cc-extensions-01
a=extmap:5 http://www.webrtc.org/experiments/rtp-hdrext/playout-delay
a=extmap:6 http://www.webrtc.org/experiments/rtp-hdrext/video-content-type
a=extmap:7 http://www.webrtc.org/experiments/rtp-hdrext/video-timing
a=extmap:8 http://www.webrtc.org/experiments/rtp-hdrext/color-space
a=extmap:9 urn:ietf:params:rtp-hdrext:sdes:mid
a=extmap:10 urn:ietf:params:rtp-hdrext:sdes:rtp-stream-id
a=extmap:11 urn:ietf:params:rtp-hdrext:sdes:repaired-rtp-stream-id
a=recvonly
a=rtcp-mux
a=rtcp-rsize
a=rtpmap:96 VP8/90000
a=rtcp-fb:96 goog-remb
a=rtcp-fb:96 transport-cc
a=rtcp-fb:96 ccm fir
a=rtcp-fb:96 nack
a=rtcp-fb:96 nack pli
a=rtpmap:97 rtx/90000
a=fmtp:97 apt=96
a=rtpmap:98 VP9/90000
a=rtcp-fb:98 goog-remb
a=rtcp-fb:98 transport-cc
a=rtcp-fb:98 ccm fir
a=rtcp-fb:98 nack
a=rtcp-fb:98 nack pli
a=fmtp:98 profile-id=0
a=rtpmap:99 rtx/90000
a=fmtp:99 apt=98
a=rtpmap:100 VP9/90000
a=rtcp-fb:100 goog-remb
a=rtcp-fb:100 transport-cc
a=rtcp-fb:100 ccm fir
a=rtcp-fb:100 nack
a=rtcp-fb:100 nack pli
a=fmtp:100 profile-id=2
a=rtpmap:101 rtx/90000
a=fmtp:101 apt=100
a=rtpmap:35 VP9/90000
a=rtcp-fb:35 goog-remb
a=rtcp-fb:35 transport-cc
a=rtcp-fb:35 ccm fir
a=rtcp-fb:35 nack
a=rtcp-fb:35 nack pli
a=fmtp:35 profile-id=1
a=rtpmap:36 rtx/90000
a=fmtp:36 apt=35
a=rtpmap:37 VP9/90000
a=rtcp-fb:37 goog-remb
a=rtcp-fb:37 transport-cc
a=rtcp-fb:37 ccm fir
a=rtcp-fb:37 nack
a=rtcp-fb:37 nack pli
a=fmtp:37 profile-id=3
a=rtpmap:38 rtx/90000
a=fmtp:38 apt=37
a=rtpmap:103 H264/90000
a=rtcp-fb:103 goog-remb
a=rtcp-fb:103 transport-cc
a=rtcp-fb:103 ccm fir
a=rtcp-fb:103 nack
a=rtcp-fb:103 nack pli
a=fmtp:103 level-asymmetry-allowed=1;packetization-mode=1;profile-level-id=42001f
a=rtpmap:104 rtx/90000
a=fmtp:104 apt=103
a=rtpmap:107 H264/90000
a=rtcp-fb:107 goog-remb
a=rtcp-fb:107 transport-cc
a=rtcp-fb:107 ccm fir
a=rtcp-fb:107 nack
a=rtcp-fb:107 nack pli
a=fmtp:107 level-asymmetry-allowed=1;packetization-mode=0;profile-level-id=42001f
a=rtpmap:108 rtx/90000
a=fmtp:108 apt=107
a=rtpmap:109 H264/90000
a=rtcp-fb:109 goog-remb
a=rtcp-fb:109 transport-cc
a=rtcp-fb:109 ccm fir
a=rtcp-fb:109 nack
a=rtcp-fb:109 nack pli
a=fmtp:109 level-asymmetry-allowed=1;packetization-mode=1;profile-level-id=42e01f
a=rtpmap:114 rtx/90000
a=fmtp:114 apt=109
a=rtpmap:115 H264/90000
a=rtcp-fb:115 goog-remb
a=rtcp-fb:115 transport-cc
a=rtcp-fb:115 ccm fir
a=rtcp-fb:115 nack
a=rtcp-fb:115 nack pli
a=fmtp:115 level-asymmetry-allowed=1;packetization-mode=0;profile-level-id=42e01f
a=rtpmap:116 rtx/90000
a=fmtp:116 apt=115
a=rtpmap:117 H264/90000
a=rtcp-fb:117 goog-remb
a=rtcp-fb:117 transport-cc
a=rtcp-fb:117 ccm fir
a=rtcp-fb:117 nack
a=rtcp-fb:117 nack pli
a=fmtp:117 level-asymmetry-allowed=1;packetization-mode=1;profile-level-id=4d001f
a=rtpmap:118 rtx/90000
a=fmtp:118 apt=117
a=rtpmap:39 H264/90000
a=rtcp-fb:39 goog-remb
a=rtcp-fb:39 transport-cc
a=rtcp-fb:39 ccm fir
a=rtcp-fb:39 nack
a=rtcp-fb:39 nack pli
a=fmtp:39 level-asymmetry-allowed=1;packetization-mode=0;profile-level-id=4d001f
a=rtpmap:40 rtx/90000
a=fmtp:40 apt=39
a=rtpmap:41 H264/90000
a=rtcp-fb:41 goog-remb
a=rtcp-fb:41 transport-cc
a=rtcp-fb:41 ccm fir
a=rtcp-fb:41 nack
a=rtcp-fb:41 nack pli
a=fmtp:41 level-asymmetry-allowed=1;packetization-mode=1;profile-level-id=f4001f
a=rtpmap:42 rtx/90000
a=fmtp:42 apt=41
a=rtpmap:43 H264/90000
a=rtcp-fb:43 goog-remb
a=rtcp-fb:43 transport-cc
a=rtcp-fb:43 ccm fir
a=rtcp-fb:43 nack
a=rtcp-fb:43 nack pli
a=fmtp:43 level-asymmetry-allowed=1;packetization-mode=0;profile-level-id=f4001f
a=rtpmap:44 rtx/90000
a=fmtp:44 apt=43
a=rtpmap:45 AV1/90000
a=rtcp-fb:45 goog-remb
a=rtcp-fb:45 transport-cc
a=rtcp-fb:45 ccm fir
a=rtcp-fb:45 nack
a=rtcp-fb:45 nack pli
a=fmtp:45 level-idx=5;profile=0;tier=0
a=rtpmap:46 rtx/90000
a=fmtp:46 apt=45
a=rtpmap:47 AV1/90000
a=rtcp-fb:47 goog-remb
a=rtcp-fb:47 transport-cc
a=rtcp-fb:47 ccm fir
a=rtcp-fb:47 nack
a=rtcp-fb:47 nack pli
a=fmtp:47 level-idx=5;profile=1;tier=0
a=rtpmap:48 rtx/90000
a=fmtp:48 apt=47
a=rtpmap:119 red/90000
a=rtpmap:120 rtx/90000
a=fmtp:120 apt=119
a=rtpmap:121 ulpfec/90000
a=rtpmap:49 flexfec-03/90000
a=rtcp-fb:49 goog-remb
a=rtcp-fb:49 transport-cc
a=fmtp:49 repair-window=10000000

handle answer:

v=0
o=SRS/5.0.213(Bee) 140323946873104 2 IN IP4 0.0.0.0
s=SRSPlaySession
t=0 0
a=ice-lite
a=group:BUNDLE 0
a=msid-semantic: WMS MATRICE_4T/1581F7K3C253S00DRCQC
m=video 9 UDP/TLS/RTP/SAVPF 109
c=IN IP4 0.0.0.0
a=ice-ufrag:0w206674
a=ice-pwd:uec7048782g08mfgth1863qen44ek69v
a=fingerprint:sha-256 E2:55:13:4F:44:D2:09:C5:2A:25:03:F8:19:D7:6B:A8:38:5A:1C:FA:96:F2:48:A1:88:6E:9E:E2:E8:34:C6:3A
a=setup:passive
a=mid:0
a=extmap:4 http://www.ietf.org/id/draft-holmer-rmcat-transport-wide-cc-extensions-01
a=sendonly
a=rtcp-mux
a=rtcp-rsize
a=rtpmap:109 H264/90000
a=rtcp-fb:109 transport-cc
a=rtcp-fb:109 nack
a=rtcp-fb:109 nack pli
a=fmtp:109 level-asymmetry-allowed=1;packetization-mode=1;profile-level-id=42e01f
a=ssrc:10559 cname:272198w22446b011
a=ssrc:10559 label:video-984g6m94
a=candidate:0 1 udp 2130706431 119.96.25.250 8000 typ host generation 0

code

@Override
    public void onTrack(RTCRtpTransceiver transceiver) {
        final MediaStreamTrack track = transceiver.getReceiver().getTrack();

        if (track.getKind().equals(MediaStreamTrack.VIDEO_TRACK_KIND)) {
            final VideoTrack videoTrack = (VideoTrack) track;
            videoTrack.addSink(videoFrame -> {
                synchronized (lock) {
                    try {
                        if (active.get()) {
                            VideoFrameBuffer buffer = videoFrame.buffer;
                            ByteBuffer byteBuffer = ByteBuffer.allocate(buffer.getWidth() * buffer.getHeight() * 4);
                            VideoBufferConverter.convertFromI420(buffer, byteBuffer, FourCC.ARGB);
                            gathers.forEach(gather -> gather.apply(byteBuffer));

                            PixelBuffer<ByteBuffer> pixelBuffer = new PixelBuffer<>(buffer.getWidth(), buffer.getHeight(), byteBuffer, PixelFormat.getByteBgraPreInstance());
                            this.frame.set(new WritableImage(pixelBuffer));
                            byteBuffer.clear();
                        }
                    } catch (Exception e) {
                        e.printStackTrace();
                    } finally {
                        videoFrame.release();
                    }
                }
            });
        }
        PeerConnectionObserver.super.onTrack(transceiver);
    }

Metadata

Metadata

Assignees

Labels

No labels
No labels

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions