Pool server works with ipv6 but not ipv4

[gombadi]

Hi
I have just added a server to the NTS pool - syd-o1.gombadi.com.
It has both ipv4 & ipv6 addresses

The ipv6 side seems to be fine with the score climbing but the ipv4 side is dropping. Looking at the monitor logs from the pool I see ipv6 has

% cat nts-pool-v6.log| jq "." -M
{
  "keyexchange": {
    "description": "",
    "exchange_duration": 1.05528247,
    "exchange_start": 1763856670,
    "num_cookies": 8,
    "status": "Success"
  },
  "ntp_with_ke_cookie": {
    "leap_indicates_synchronized": true,
    "offset": 0.002444131533253037,
    "received_cookies": 2,
    "remote_residence_time": 0.00008312892170695796,
    "request_sent": 1763856671,
    "requested_cookies": 2,
    "roundtrip_duration": 0.2544472939927241,
    "status": "Success",
    "stratum": 3
  },
  "ntp_with_ntp_cookie": {
    "leap_indicates_synchronized": true,
    "offset": 0.002443098929348192,
    "received_cookies": 1,
    "remote_residence_time": 0.0000978580210585748,
    "request_sent": 1763856672,
    "requested_cookies": 1,
    "roundtrip_duration": 0.25493588397627137,
    "status": "Success",
    "stratum": 3
  }
}

while ipv4 has

% cat nts-pool-v4.log| jq "." -M
{
  "keyexchange": {
    "description": "",
    "exchange_duration": 2.040772649,
    "exchange_start": 1763856791,
    "num_cookies": 8,
    "status": "Success"
  },
  "ntp_with_ke_cookie": {
    "leap_indicates_synchronized": true,
    "offset": 0.0023880933882640893,
    "received_cookies": 3,
    "remote_residence_time": 0.00009656487966342011,
    "request_sent": 1763856793,
    "requested_cookies": 3,
    "roundtrip_duration": 0.2547722720202925,
    "status": "Success",
    "stratum": 3
  },
  "ntp_with_ntp_cookie": {
    "leap_indicates_synchronized": false,
    "offset": null,
    "received_cookies": 0,
    "remote_residence_time": null,
    "request_sent": 1763856793,
    "requested_cookies": 1,
    "roundtrip_duration": null,
    "status": "Timeout",
    "stratum": null
  }
}

It seems that the ipv4 section for ntp_with_ntp_cookie fails for some reason.

Both requests go to the same ntpd-rs server with the firewall open on udp 123 and tcp 4460 on both v4 & v6.

Does anyone have any thoughts on what could be going wrong?
Thanks.

[davidv1992]

I have taken a more detailed look at your servers logs, also at a few more entries. It in general just intermittently reacts to the udp packets. The weird thing is that there is a pattern to the intermittent behavior, namely it responds succesfully if the monitor is asking for additional cookies, but fails to respond when it does not. As to why that would happen, I have absolutely no idea, but it is a really interesting way of misbehaving for an NTS capable NTP server.

[gombadi]

Interesting and it only seems to be an issue on ipv4 requests which is strange.

[davidv1992]

Yep, it is very odd. My best guess is that somewhere along the line something is filtering out small packets. The surprising part is that this is exactly the other way round from what is usually observed.

Your best bet for figuring out what is probably trying to connect to it with NTS from various places and see how it behaves, possibly while capturing incoming traffic to see what is and isn't arriving. Unfortunately, this is getting out of scope for our development plans on the pool so I am afraid we really don't have the bandwidth to assist with this.

[davidv1992]

Ok, we now have a second server with the same behavior from the perspective of our monitor. All indications are that this is an ISP filter somewhere along the path. If you are able, can you provide a packet capture from your server showing about an hour's worth of UDP traffic from 51.15.56.203 (the monitor)? I might be able to learn something more from that.

[gombadi]

@davidv1992

I will leave it up for a day or so.
https://www.gombadi.com/joog7om7iQu2ieth/20251206-nts-51.15.56.203.pcap.gz

[davidv1992]

Thx, I have the data I need. Apparantly the problem is in the return journey. I'll go have a talk with our hoster.