Consider integration with Janus

[stblassitude]

In a number of scenarios, it might be beneficial to have the speaker's camera, screenshare and audio sent to a relay server, and potentially combined there into a single stream. This would allow multiple speakers (and other participants like herald, signal angel, translators) to join the same session.

Janus appears to allow a single stream from one participant to be multiplied to all participants. This should help with problems that we have seen where sessions with many participants have low quality, because the video stream has to be sent by the speaker's system to all participants, maxing out their upload capacity.

[stblassitude]

I think this sounds like it could be useful: https://janus.conf.meetecho.com/docs/videoroom.html

[derchrisuk]

This is already on my re-design list

[iSchluff]

Use cases I could think of that might be interesting to look at:

• Let the Speaker/s join into a video-room and forward each participants RTP stream to an external video mixer without the need for a webview/chromedriver
• Send an RTP-Stream from a video mixer into a video-room for low-latency preview from browsers
  • Combine with audio-only return-channel from the participants for ultimate translator integration

I suppose both of these would need some kind of sidecar app which coordinates the creation of rooms and forwarding of streams as we probably don't want to allow the clients to have full control over the stream api. Also this can be used to provide an interface to the video mixer.

It also might make sense to relay the forwarded streams over an rtsp server or something like that to be able to interact with the RTP streams more easily. (ffmpeg and friends need an SDP file to figure out the stream-mapping inside an RTP stream)

Maybe the sidecar app could also speak RTSP directly to coordinate between Janus and the RTP-client (e.g. with https://github.com/aler9/gortsplib)