"Unable to connect to the voice call: timed out waiting for peer connection" on mobile app but working with the web version on the same device

[ngthwi]

Summary

Cannot start/join to call

On an iPad, Calls doesn't work with the app despite working with Safari, which confirms that ports 443,8443 TCP/UDP are correctly opened.

Environment Information

• Device Name: iPhone and iPad
• OS Version: 18.3.1
• Mattermost App Version: 2.25.1 (Build 607)
• Mattermost Server Version: 10.5 (self-hosted)
• Calls (com.mattermost.calls - 1.5.0)

Steps to reproduce

1. Open Mattermost app
2. Start call

Expected behavior

Calls started/joined successfully.

Observed behavior (that appears unintentional)

Cannot start/join the voice call. The following error appears: "Unable to connect to the voice call: timed out waiting for peer connection".

Possible fixes

No fix found.

[ngthwi]

Here's the log from the app:

2025/02/26 00:54:17:997 INFO Launch app in Home screen
2025/02/26 00:54:18:009 INFO channel switch to admin jb****************4tw 9 ms
2025/02/26 00:54:18:009 INFO websocket connecting to wss://mymattermostserver/api/v4/websocket?connection_id=&sequence_number=0
2025/02/26 00:54:18:014 INFO Running "Home"
2025/02/26 00:54:18:058 DEBUG Time to Interaction 61.50283299386501
2025/02/26 00:54:18:113 DEBUG Notification token registered apple_rn-v2:70 ****************7d5
2025/02/26 00:54:18:203 DEBUG Performance metric: {
    metric = "mobile_load";
    timestamp = [1740527658200](tel:1740527658200);
    value = "624.0396250039339";
}
2025/02/26 00:54:18:332 INFO websocket connecting to wss://mymattermostserver/api/v4/websocket?connection_id=&sequence_number=0
2025/02/26 00:54:19:780 INFO websocket connected to wss://mymattermostserver/api/v4/websocket?connection_id=&sequence_number=0
2025/02/26 00:54:19:781 INFO wss://mymattermostserver/api/v4/websocket?connection_id=&sequence_number=0 got connection id  sh****************oe8c
2025/02/26 00:54:19:781 INFO wss://mymattermostserver/api/v4/websocket?connection_id=&sequence_number=0 got the expected new connection id
2025/02/26 00:54:19:812 INFO websocket connected to wss://mymattermostserver/api/v4/websocket?connection_id=&sequence_number=0
2025/02/26 00:54:19:812 INFO wss://mymattermostserver/api/v4/websocket?connection_id=&sequence_number=0 got connection id  kb ****************9g4o
2025/02/26 00:54:19:812 INFO wss://mymattermostserver/api/v4/websocket?connection_id=&sequence_number=0 got a new connection due to long timeout, or server restart, or sequence number is not found
2025/02/26 00:54:19:819 WARNING Handling Javascript error error on fetchChannelBookmarks 0
2025/02/26 00:54:19:819 ERROR error on fetchChannelBookmarks Your license does not support channel bookmarks.
2025/02/26 00:54:19:995 DEBUG Process models on entry Cold Start 43 6ms
2025/02/26 00:54:20:001 INFO WEBSOCKET RECONNECT MODELS BATCHING TOOK 61ms
2025/02/26 00:54:20:013 DEBUG Process models on entry WebSocket Reconnect 42 4ms
2025/02/26 00:54:20:016 INFO WEBSOCKET RECONNECT MODELS BATCHING TOOK 7ms
2025/02/26 00:54:25:158 INFO Running "ChannelInfo"
2025/02/26 00:54:27:128 DEBUG calls: ws error {
    "AT_TARGET" = 2;
    "BUBBLING_PHASE" = 3;
    "CAPTURING_PHASE" = 1;
    NONE = 0;
    bubbles = 0;
    cancelBubble = 0;
    cancelable = 0;
    composed = 0;
    composedPath = "<null>";
    currentTarget = "<null>";
    defaultPrevented = 0;
    eventPhase = 0;
    initEvent = "<null>";
    isTrusted = 0;
    message = "The operation couldn\U2019t be completed. (OSStatus error -9836.)";
    preventDefault = "<null>";
    returnValue = 1;
    srcElement =     {
        CLOSED = 3;
        CLOSING = 2;
        CONNECTING = 0;
        OPEN = 1;
        "_eventEmitter" =         {
            "_nativeModule" =             {
                addListener = "<null>";
                close = "<null>";
                connect = "<null>";
                getConstants = "<null>";
                ping = "<null>";
                removeListeners = "<null>";
                send = "<null>";
                sendBinary = "<null>";
            };
        };
        "_socketId" = 0;
        "_subscriptions" =         (
        );
        addEventListener = "<null>";
        dispatchEvent = "<null>";
        onclose = "<null>";
        onerror = "<null>";
        onmessage = "<null>";
        onopen = "<null>";
        readyState = 3;
        removeEventListener = "<null>";
        url = "https://mymattermostserver/api/v4/websocket?connection_id=&sequence_number=0";
    };
    stopImmediatePropagation = "<null>";
    stopPropagation = "<null>";
    target =     {
        CLOSED = 3;
        CLOSING = 2;
        CONNECTING = 0;
        OPEN = 1;
        "_eventEmitter" =         {
            "_nativeModule" =             {
                addListener = "<null>";
                close = "<null>";
                connect = "<null>";
                getConstants = "<null>";
                ping = "<null>";
                removeListeners = "<null>";
                send = "<null>";
                sendBinary = "<null>";
            };
        };
        "_socketId" = 0;
        "_subscriptions" =         (
        );
        addEventListener = "<null>";
        dispatchEvent = "<null>";
        onclose = "<null>";
        onerror = "<null>";
        onmessage = "<null>";
        onopen = "<null>";
        readyState = 3;
        removeEventListener = "<null>";
        url = "https://mymattermostserver/api/v4/websocket?connection_id=&sequence_number=0";
    };
    timeStamp = [1740527667125](tel:1740527667125);
    type = error;
}
2025/02/26 00:54:27:128 DEBUG calls: ws close, code: 1006 reason: The operation couldn’t be completed. (OSStatus error -9836.) message: <null>
2025/02/26 00:54:28:134 DEBUG calls: attempting ws reconnection to https://mymattermostserver/api/v4/websocket
2025/02/26 00:54:28:176 DEBUG calls: ws error {
[.....]

[streamer45]

From the logs provided, it looks like it's failing to establish the initial websocket connection, which comes before any connectivity check through the UDP port.

The error reported (The operation couldn\U2019t be completed. (OSStatus error -9836.)) doesn't really ring a bell. It seems like the system isn't happy with the request for some reason. Looking further, there could be some issues with the TLS certificates.

Are you running MM on a self-signed certificate perhaps?

[ngthwi]

MM is running with a Let's Encrypt certificate.
This is from TLS Inspector:

The certificate is on a reverse proxy, does MM need the certificate on its settings?

[streamer45]

The certificate is on a reverse proxy, does MM need the certificate on its settings?

If TLS termination is happening at the proxy, I wouldn't expect that to be needed. But for some reasons, the mobile device you are using is failing with a errSSLPeerProtocolVersion error. May be worth checking your reverse proxy config, especially any specific setting dealing with TLS version and crypto algorithm used (e.g. is TLS v1.3 enforced perhaps?)

[ngthwi]

Here's the log prior to the one in the first post (I've just edited it):

2025/02/26 00:54:17:997 INFO Launch app in Home screen
2025/02/26 00:54:18:009 INFO channel switch to admin jb****************4tw 9 ms
2025/02/26 00:54:18:009 INFO websocket connecting to wss://mymattermostserver/api/v4/websocket?connection_id=&sequence_number=0
2025/02/26 00:54:18:014 INFO Running "Home"
2025/02/26 00:54:18:058 DEBUG Time to Interaction 61.50283299386501
2025/02/26 00:54:18:113 DEBUG Notification token registered apple_rn-v2:70 ****************7d5
2025/02/26 00:54:18:203 DEBUG Performance metric: {
    metric = "mobile_load";
    timestamp = [1740527658200](tel:1740527658200);
    value = "624.0396250039339";
}
2025/02/26 00:54:18:332 INFO websocket connecting to wss://mymattermostserver/api/v4/websocket?connection_id=&sequence_number=0
2025/02/26 00:54:19:780 INFO websocket connected to wss://mymattermostserver/api/v4/websocket?connection_id=&sequence_number=0
2025/02/26 00:54:19:781 INFO wss://mymattermostserver/api/v4/websocket?connection_id=&sequence_number=0 got connection id  sh****************oe8c
2025/02/26 00:54:19:781 INFO wss://mymattermostserver/api/v4/websocket?connection_id=&sequence_number=0 got the expected new connection id
2025/02/26 00:54:19:812 INFO websocket connected to wss://mymattermostserver/api/v4/websocket?connection_id=&sequence_number=0
2025/02/26 00:54:19:812 INFO wss://mymattermostserver/api/v4/websocket?connection_id=&sequence_number=0 got connection id  kb ****************9g4o
2025/02/26 00:54:19:812 INFO wss://mymattermostserver/api/v4/websocket?connection_id=&sequence_number=0 got a new connection due to long timeout, or server restart, or sequence number is not found
2025/02/26 00:54:19:819 WARNING Handling Javascript error error on fetchChannelBookmarks 0
2025/02/26 00:54:19:819 ERROR error on fetchChannelBookmarks Your license does not support channel bookmarks.
2025/02/26 00:54:19:995 DEBUG Process models on entry Cold Start 43 6ms
2025/02/26 00:54:20:001 INFO WEBSOCKET RECONNECT MODELS BATCHING TOOK 61ms
2025/02/26 00:54:20:013 DEBUG Process models on entry WebSocket Reconnect 42 4ms

We can see:

2025/02/26 00:54:18:332 INFO websocket connecting to wss://mymattermostserver/api/v4/websocket?connection_id=&sequence_number=0
2025/02/26 00:54:19:780 INFO websocket connected to wss://mymattermostserver/api/v4/websocket?connection_id=&sequence_number=0

Doesn't it imply that there's no issue with TLS?

[streamer45]

Looks like we are missing the DEBUG level logs coming from the calls component. Example:

2025/02/18 09:31:12:586 DEBUG calls: ws new conn id from server
2025/02/18 09:31:12:586 DEBUG calls: ws setting original conn id
2025/02/18 09:31:12:586 DEBUG calls: ws open, sending join msg
2025/02/18 09:31:12:712 DEBUG calls: join ack received, initializing connection
2025/02/18 09:31:12:870 DEBUG RTCPeer: created new client {"iceServers":[{"urls":["stun:stun.global.calls.mattermost.com:3478"]},{"urls":["stun:stun.global.calls.mattermost.com:3478"]}],"logger":{},"dcSignaling":false}

[ngthwi]

Here's the log from the server side:

debug [2025-02-26 00:54:19.765 +01:00] Received HTTP request                         caller="web/handlers.go:185" method=GET url=/api/v4/license/client request_id=4s8******16r user_id=7j********mmahy status_code=200
debug [2025-02-26 00:54:19.765 +01:00] Received HTTP request                         caller="web/handlers.go:185" method=GET url=/api/v4/channels/jb8y********ym5q4tw/posts request_id=6u******foo user_id=7j********mmahy status_code=200
debug [2025-02-26 00:54:19.768 +01:00] Received HTTP request                         caller="web/handlers.go:185" method=GET url=/api/v4/license/client request_id=a5u*********kgjpsray user_id=7j********mmahy status_code=200
debug [2025-02-26 00:54:19.770 +01:00] CSRF Header check failed for request - Please upgrade your web application or custom app to set a CSRF Header caller="web/handlers.go:531" path=/api/v4/users/ids ip="__docker_container_ip__:random_port" session_id=t94j********7a user_id=7j********mmahy
debug [2025-02-26 00:54:19.771 +01:00] CSRF Header check failed for request - Please upgrade your web application or custom app to set a CSRF Header caller="web/handlers.go:531" path=/api/v4/users/sessions/device ip="__docker_container_ip__:random_port" session_id=t94j********7a user_id=7j********mmahy
debug [2025-02-26 00:54:19.772 +01:00] CSRF Header check failed for request - Please upgrade your web application or custom app to set a CSRF Header caller="web/handlers.go:531" path=/api/v4/channels/members/me/view ip="__docker_container_ip__:random_port" session_id=t94j********7a user_id=7j********mmahy
debug [2025-02-26 00:54:19.774 +01:00] CSRF Header check failed for request - Please upgrade your web application or custom app to set a CSRF Header caller="web/handlers.go:531" path=/api/v4/users/sessions/device ip="__docker_container_ip__:random_port" session_id=t94j********7a user_id=7j********mmahy
debug [2025-02-26 00:54:19.774 +01:00] CSRF Header check failed for request - Please upgrade your web application or custom app to set a CSRF Header caller="web/handlers.go:531" path=/api/v4/channels/members/me/view ip="__docker_container_ip__:random_port" session_id=t94j********7a user_id=7j********mmahy

And here’s the corresponding log from the app again:

2025/02/26 00:54:18:332 INFO websocket connecting to wss://mymattermostserver/api/v4/websocket?connection_id=&sequence_number=0
2025/02/26 00:54:19:780 INFO websocket connected to wss://mymattermostserver/api/v4/websocket?connection_id=&sequence_number=0
2025/02/26 00:54:19:781 INFO wss://mymattermostserver/api/v4/websocket?connection_id=&sequence_number=0 got connection id  sh****************oe8c
2025/02/26 00:54:19:781 INFO wss://mymattermostserver/api/v4/websocket?connection_id=&sequence_number=0 got the expected new connection id
2025/02/26 00:54:19:812 INFO websocket connected to wss://mymattermostserver/api/v4/websocket?connection_id=&sequence_number=0
2025/02/26 00:54:19:812 INFO wss://mymattermostserver/api/v4/websocket?connection_id=&sequence_number=0 got connection id  kb ****************9g4o
2025/02/26 00:54:19:812 INFO wss://mymattermostserver/api/v4/websocket?connection_id=&sequence_number=0 got a new connection due to long timeout, or server restart, or sequence number is not found
2025/02/26 00:54:19:819 WARNING Handling Javascript error error on fetchChannelBookmarks 0
2025/02/26 00:54:19:819 ERROR error on fetchChannelBookmarks Your license does not support channel bookmarks.
2025/02/26 00:54:19:995 DEBUG Process models on entry Cold Start 43 6ms
2025/02/26 00:54:20:001 INFO WEBSOCKET RECONNECT MODELS BATCHING TOOK 61ms
2025/02/26 00:54:20:013 DEBUG Process models on entry WebSocket Reconnect 42 4ms

[streamer45]

@ngthwi To rule out a server-side misconfiguration, would you be able to connect to our public MM instance (https://community.mattermost.com/) and see if you can successfully start a call over there (from the mobile app, that is)?

[ngthwi]

Calls is working with the app on the public MM instance.
I could start a call with a bot.

[streamer45]

Alright, then there could be some sort of networking issue (firewall block) on the server side. You are running MM in docker it seems? Could you provide the output of docker ps command and the MM config.json? You can share in a direct message on our public server if you prefer. You can find me as @claudio.costa over there.