Skip to content

Commit 18a53d1

Browse files
cmonroecmonroe
authored
Add HBO Max/Max streaming detection (#3193)
Extend NDPI_PROTOCOL_HBO with the domains Warner Bros. Discovery's Max service now uses, so HBO Max video sessions are classified instead of falling back to generic TLS/QUIC: * h264.io WBD video CDN - QUIC video streams * max.com Max app - API and media hosts * discomax.com shared WBD streaming API and image hosts These domains are WBD-wide.. CNN, Discovery+ and TNT Sports streaming traffic using the same infrastructure is also reported as HBO. Direct cnn.com/cnn.net website traffic still maps to NDPI_PROTOCOL_CNN, since SNI matching takes precedence over certificate-SAN matching. Add a small sample capture containing the QUIC video flow plus TLS/QUIC control flows, and its expected result. Signed-off-by: Chad Monroe <chad@monroe.io>
1 parent 1ac558b commit 18a53d1

3 files changed

Lines changed: 29 additions & 20 deletions

File tree

‎src/lib/ndpi_content_match.c.inc‎

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1297,6 +1297,8 @@ static ndpi_protocol_match host_match[] =
12971297
{ "dtvbb.tv", "DirecTV", NDPI_PROTOCOL_DIRECTV, NDPI_PROTOCOL_CATEGORY_STREAMING, NDPI_PROTOCOL_FUN, NDPI_PROTOCOL_DEFAULT_LEVEL },
12981298
{ "dtvce.com", "DirecTV", NDPI_PROTOCOL_DIRECTV, NDPI_PROTOCOL_CATEGORY_STREAMING, NDPI_PROTOCOL_FUN, NDPI_PROTOCOL_DEFAULT_LEVEL },
12991299

1300+
{ "discomax.com", "HBO", NDPI_PROTOCOL_HBO, NDPI_PROTOCOL_CATEGORY_STREAMING, NDPI_PROTOCOL_FUN, NDPI_PROTOCOL_DEFAULT_LEVEL },
1301+
{ "h264.io", "HBO", NDPI_PROTOCOL_HBO, NDPI_PROTOCOL_CATEGORY_STREAMING, NDPI_PROTOCOL_FUN, NDPI_PROTOCOL_DEFAULT_LEVEL },
13001302
{ "hbo.com", "HBO", NDPI_PROTOCOL_HBO, NDPI_PROTOCOL_CATEGORY_STREAMING, NDPI_PROTOCOL_FUN, NDPI_PROTOCOL_DEFAULT_LEVEL },
13011303
{ "hbogo.co.th", "HBO", NDPI_PROTOCOL_HBO, NDPI_PROTOCOL_CATEGORY_STREAMING, NDPI_PROTOCOL_FUN, NDPI_PROTOCOL_DEFAULT_LEVEL },
13021304
{ "hbogo.com", "HBO", NDPI_PROTOCOL_HBO, NDPI_PROTOCOL_CATEGORY_STREAMING, NDPI_PROTOCOL_FUN, NDPI_PROTOCOL_DEFAULT_LEVEL },
@@ -1307,6 +1309,7 @@ static ndpi_protocol_match host_match[] =
13071309
{ "hbomax.com", "HBO", NDPI_PROTOCOL_HBO, NDPI_PROTOCOL_CATEGORY_STREAMING, NDPI_PROTOCOL_FUN, NDPI_PROTOCOL_DEFAULT_LEVEL },
13081310
{ "hbomaxcdn.com", "HBO", NDPI_PROTOCOL_HBO, NDPI_PROTOCOL_CATEGORY_STREAMING, NDPI_PROTOCOL_FUN, NDPI_PROTOCOL_DEFAULT_LEVEL },
13091311
{ "hbonow.com", "HBO", NDPI_PROTOCOL_HBO, NDPI_PROTOCOL_CATEGORY_STREAMING, NDPI_PROTOCOL_FUN, NDPI_PROTOCOL_DEFAULT_LEVEL },
1312+
{ "max.com", "HBO", NDPI_PROTOCOL_HBO, NDPI_PROTOCOL_CATEGORY_STREAMING, NDPI_PROTOCOL_FUN, NDPI_PROTOCOL_DEFAULT_LEVEL },
13101313
{ "maxgo.com", "HBO", NDPI_PROTOCOL_HBO, NDPI_PROTOCOL_CATEGORY_STREAMING, NDPI_PROTOCOL_FUN, NDPI_PROTOCOL_DEFAULT_LEVEL },
13111314

13121315
{ "vudu.com", "Vudu", NDPI_PROTOCOL_VUDU, NDPI_PROTOCOL_CATEGORY_STREAMING, NDPI_PROTOCOL_FUN, NDPI_PROTOCOL_DEFAULT_LEVEL },

‎tests/cfgs/default/pcap/sites2.pcapng‎

15 KB
Binary file not shown.

‎tests/cfgs/default/result/sites2.pcapng.out‎

Lines changed: 26 additions & 20 deletions
Original file line numberDiff line numberDiff line change
@@ -1,52 +1,58 @@
1-
DPI Packets (TCP): 31 (6.20 pkts/flow)
2-
Confidence DPI : 5 (flows)
3-
Num dissector calls: 15 (3.00 diss/flow)
1+
DPI Packets (TCP): 37 (6.17 pkts/flow)
2+
DPI Packets (UDP): 4 (2.00 pkts/flow)
3+
Confidence DPI : 8 (flows)
4+
Num dissector calls: 179 (22.38 diss/flow)
45
LRU cache ookla: 0/0/0 (insert/search/found)
56
LRU cache bittorrent: 0/0/0 (insert/search/found)
67
LRU cache stun: 0/0/0 (insert/search/found)
78
LRU cache tls_cert: 0/0/0 (insert/search/found)
89
LRU cache mining: 0/0/0 (insert/search/found)
910
LRU cache msteams: 0/0/0 (insert/search/found)
10-
LRU cache fpc_dns: 0/5/0 (insert/search/found)
11-
Automa host: 5/5 (search/found)
12-
Automa domain: 5/0 (search/found)
11+
LRU cache fpc_dns: 0/6/0 (insert/search/found)
12+
Automa host: 8/8 (search/found)
13+
Automa domain: 8/0 (search/found)
1314
Automa tls cert: 0/0 (search/found)
14-
Automa risk mask: 0/0 (search/found)
15-
Automa common alpns: 6/6 (search/found)
16-
Patricia risk mask: 0/0 (search/found)
15+
Automa risk mask: 1/0 (search/found)
16+
Automa common alpns: 10/10 (search/found)
17+
Patricia risk mask: 2/0 (search/found)
1718
Patricia risk mask IPv6: 0/0 (search/found)
1819
Patricia risk: 0/0 (search/found)
1920
Patricia risk IPv6: 0/0 (search/found)
20-
Patricia protocols: 6/4 (search/found)
21+
Patricia protocols: 9/7 (search/found)
2122
Patricia protocols IPv6: 0/0 (search/found)
2223
Hash malicious ja4: 0/0 (search/found)
2324
Hash malicious sha1: 1/0 (search/found)
24-
Hash TCP fingerprints: 5/5 (search/found)
25-
Hash public domain suffix: 45/25 (search/found)
26-
Hash ja4 custom protos: 7/0 (search/found)
27-
Hash fp custom protos: 4/0 (search/found)
25+
Hash TCP fingerprints: 6/6 (search/found)
26+
Hash public domain suffix: 88/48 (search/found)
27+
Hash ja4 custom protos: 10/0 (search/found)
28+
Hash fp custom protos: 7/0 (search/found)
2829
Hash url custom protos: 1/0 (search/found)
2930

31+
HBO 18 14542 3
3032
Shein 13 5080 1
3133
Temu 20 7323 1
3234
Taobao 15 7085 1
3335
ParamountPlus 4 797 1
3436
YandexAlice 10 882 1
3537

3638
Acceptable 58 20370 4
37-
Fun 4 797 1
39+
Fun 22 15339 4
3840

39-
Streaming 4 797 1
41+
Streaming 22 15339 4
4042
Shopping 48 19488 3
4143
ConnCheck 10 882 1
4244

4345
JA Host Stats:
4446
IP Address # JA4C
45-
1 192.168.12.67 4
47+
1 192.168.80.237 2
48+
2 192.168.12.67 4
4649

4750

4851
1 TCP 192.168.12.67:47694 <-> 20.15.0.9:443 [proto: 91.435/TLS.Temu][Stack: TLS.Temu][IP: 276/Azure][Encrypted][Confidence: DPI][FPC: 276/Azure, Confidence: IP address][DPI packets: 7][cat: Shopping/27][Breed: Acceptable][10 pkts/1963 bytes <-> 10 pkts/5360 bytes][Goodput ratio: 71/90][0.54 sec][Hostname/SNI: gtm.temu.com][(Advertised) ALPNs: h2;http/1.1][TLS Supported Versions: TLSv1.3;TLSv1.2;TLSv1.1;TLSv1][bytes ratio: -0.464 (Download)][IAT c2s/s2c min/avg/max/stddev: 0/0 61/49 282/342 86/112][Pkt Len c2s/s2c min/avg/max/stddev: 54/54 196/536 571/1514 206/532][nDPI Cli Fingerprint: cf1eb1ae235bac301da32c9e902f6ec5][nDPI Srv Fingerprint: 81dba73ae426][TCP Fingerprint: 2_64_65535_685ad951a756/Android][TLSv1.3][JA4: t13d1712h2_5b57614c22b0_3f5d972527c0][JA3S: 15af977ce25de452b96affa2addb1036][Safari][Cipher: TLS_AES_256_GCM_SHA384][Plen Bins: 8,8,8,0,0,0,0,0,0,25,0,0,0,0,0,0,16,8,8,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,16,0,0]
4952
2 TCP 192.168.12.67:43446 <-> 59.82.122.224:443 [proto: 91.436/TLS.Taobao][Stack: TLS.Taobao][IP: 274/Alibaba][Encrypted][Confidence: DPI][FPC: 274/Alibaba, Confidence: IP address][DPI packets: 8][cat: Shopping/27][Breed: Acceptable][9 pkts/2792 bytes <-> 6 pkts/4293 bytes][Goodput ratio: 82/92][0.78 sec][Hostname/SNI: umdc.taobao.com][(Advertised) ALPNs: http/1.1][(Negotiated) ALPN: http/1.1][TLS Supported Versions: TLSv1.3;TLSv1.2;TLSv1.1;TLSv1][bytes ratio: -0.212 (Download)][IAT c2s/s2c min/avg/max/stddev: 0/0 111/64 269/253 125/109][Pkt Len c2s/s2c min/avg/max/stddev: 54/54 310/716 1078/1514 359/618][nDPI Cli Fingerprint: cf1eb1ae235bac301da32c9e902f6ec5][nDPI Srv Fingerprint: 15a5a115c683][TCP Fingerprint: 2_64_65535_685ad951a756/Android][TLSv1.2][JA4: t13d1713h1_5b57614c22b0_eca864cca44a][ServerNames: *.alibabachengdun.com,*.alibabachengdun.net,umdc.aliapp.org,*.ynuf.aliapp.org,sgynuf.alibaba.com,pum.m.alibaba.com,ynuf.aliapp.org,mum.hzchengdun.com,mum.m.alibaba.com,umdc.alibaba-inc.com,umidiot.aliapp.org,us-mum.alibabachengdun.com,sg-pum.alibabachengdun.com,sg-pum.alibabachengdun.net,umdc.taobao.com,umdc.tmall.com,alibabachengdun.com][JA3S: 00447ab319e9d94ba2b4c1248e155917][Issuer: C=BE, O=GlobalSign nv-sa, CN=GlobalSign Organization Validation CA - SHA256 - G3][Subject: C=CN, ST=ZheJiang, L=HangZhou, O=Alibaba (China) Technology Co., Ltd., CN=*.alibabachengdun.com][Certificate SHA-1: A4:84:85:BF:7A:3D:54:C0:EE:F2:8B:39:E7:ED:56:FB:74:6B:5E:61][Safari][Validity: 2024-09-11 08:46:01 - 2025-09-04 00:00:00][Cipher: TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256][Plen Bins: 0,0,12,0,0,0,0,0,12,0,0,0,0,0,0,0,12,0,0,0,12,0,0,12,0,0,0,0,0,0,0,0,12,0,0,0,0,0,0,0,0,0,0,0,0,25,0,0]
50-
3 TCP 192.168.12.67:46892 <-> 2.23.155.106:443 [proto: 91.434/TLS.Shein][Stack: TLS.Shein][IP: 467/Akamai][Encrypted][Confidence: DPI][FPC: 467/Akamai, Confidence: IP address][DPI packets: 6][cat: Shopping/27][Breed: Acceptable][7 pkts/1067 bytes <-> 6 pkts/4013 bytes][Goodput ratio: 56/90][0.09 sec][Hostname/SNI: img.shein.com][(Advertised) ALPNs: h2;http/1.1][TLS Supported Versions: TLSv1.3;TLSv1.2][bytes ratio: -0.580 (Download)][IAT c2s/s2c min/avg/max/stddev: 0/0 17/5 58/19 21/8][Pkt Len c2s/s2c min/avg/max/stddev: 66/66 152/669 583/1514 178/648][nDPI Cli Fingerprint: 03c638a568483a8c7de6f2dd89b99818][nDPI Srv Fingerprint: 81dba73ae426][TCP Fingerprint: 2_64_65535_685ad951a756/Android][TLSv1.3][JA4: t13d1513h2_8daaf6152771_eca864cca44a][JA3S: 15af977ce25de452b96affa2addb1036][Safari][Cipher: TLS_AES_256_GCM_SHA384][Plen Bins: 0,0,20,0,0,0,0,0,0,0,0,0,0,0,0,0,20,0,0,0,0,0,20,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,40,0,0]
51-
4 TCP 192.168.0.100:52124 <-> 213.180.193.9:80 [proto: 7.440/HTTP.YandexAlice][Stack: HTTP.YandexAlice][IP: 25/Yandex][ClearText][Confidence: DPI][FPC: 25/Yandex, Confidence: IP address][DPI packets: 6][cat: ConnCheck/30][Breed: Acceptable][6 pkts/532 bytes <-> 4 pkts/350 bytes][Goodput ratio: 24/22][0.02 sec][Hostname/SNI: scbh.yandex.net][bytes ratio: 0.206 (Upload)][IAT c2s/s2c min/avg/max/stddev: 0/0 4/4 7/8 3/4][Pkt Len c2s/s2c min/avg/max/stddev: 66/66 89/88 194/144 47/33][URL: scbh.yandex.net/generate_204][StatusCode: 204][User-Agent: yandexmini_2/0.270.1.48.2683694502.20241115.199][TCP Fingerprint: 2_64_29200_d853e95bd80f/Linux][PLAIN TEXT (HEAD /generate)][Plen Bins: 0,0,50,0,50,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0]
52-
5 TCP 192.168.12.67:39974 <-> 151.101.1.233:443 [proto: 91.439/TLS.ParamountPlus][Stack: TLS.ParamountPlus][IP: 0/Unknown][Encrypted][Confidence: DPI][FPC: 0/Unknown, Confidence: Unknown][DPI packets: 4][cat: Streaming/17][Breed: Fun][3 pkts/723 bytes <-> 1 pkts/74 bytes][Goodput ratio: 71/0][0.01 sec][Hostname/SNI: vod-gcs-cedexis.cbsaavideo.com][(Advertised) ALPNs: http/1.1][TLS Supported Versions: TLSv1.3;TLSv1.2][nDPI Cli Fingerprint: 03c638a568483a8c7de6f2dd89b99818][TCP Fingerprint: 2_64_65535_685ad951a756/Android][TLSv1.2][JA4: t13d1513h1_8daaf6152771_eca864cca44a][Safari][PLAIN TEXT (cedexis.c)][Plen Bins: 0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,100,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0]
53+
3 UDP 192.168.80.237:55131 <-> 34.126.228.79:443 [proto: 188.319/QUIC.HBO][Stack: QUIC.HBO][IP: 126/Google][Encrypted][Confidence: DPI][FPC: 188/QUIC, Confidence: DPI][DPI packets: 2][cat: Streaming/17][Breed: Fun][2 pkts/2484 bytes <-> 4 pkts/3808 bytes][Goodput ratio: 97/96][0.02 sec][Hostname/SNI: gcp.amer-free.prd.media.max.com][(Advertised) ALPNs: h3][TLS Supported Versions: GREASE;TLSv1.3][bytes ratio: -0.210 (Download)][IAT c2s/s2c min/avg/max/stddev: 0/0 0/0 0/1 0/0][Pkt Len c2s/s2c min/avg/max/stddev: 1242/82 1242/952 1242/1242 0/502][nDPI Cli Fingerprint: 74b25186621c904c4ef1c296b44002a8][TLSv1.3][QUIC ver: V-1][JA4: q13d0311h3_55b375c5d22e_f2a83c8e78ae][Safari][Plen Bins: 0,16,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,83,0,0,0,0,0,0,0,0,0,0]
54+
4 UDP 192.168.80.237:64732 <-> 34.104.36.211:443 [proto: 188.319/QUIC.HBO][Stack: QUIC.HBO][IP: 126/Google][Encrypted][Confidence: DPI][FPC: 188/QUIC, Confidence: DPI][DPI packets: 2][cat: Streaming/17][Breed: Fun][2 pkts/2484 bytes <-> 4 pkts/3808 bytes][Goodput ratio: 97/96][0.01 sec][Hostname/SNI: gcp.prd.media.h264.io][(Advertised) ALPNs: h3][TLS Supported Versions: GREASE;TLSv1.3][bytes ratio: -0.210 (Download)][IAT c2s/s2c min/avg/max/stddev: 0/0 0/0 0/1 0/0][Pkt Len c2s/s2c min/avg/max/stddev: 1242/82 1242/952 1242/1242 0/502][nDPI Cli Fingerprint: 74b25186621c904c4ef1c296b44002a8][TLSv1.3][QUIC ver: V-1][JA4: q13d0311h3_55b375c5d22e_f2a83c8e78ae][Safari][Plen Bins: 0,16,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,83,0,0,0,0,0,0,0,0,0,0]
55+
5 TCP 192.168.12.67:46892 <-> 2.23.155.106:443 [proto: 91.434/TLS.Shein][Stack: TLS.Shein][IP: 467/Akamai][Encrypted][Confidence: DPI][FPC: 467/Akamai, Confidence: IP address][DPI packets: 6][cat: Shopping/27][Breed: Acceptable][7 pkts/1067 bytes <-> 6 pkts/4013 bytes][Goodput ratio: 56/90][0.09 sec][Hostname/SNI: img.shein.com][(Advertised) ALPNs: h2;http/1.1][TLS Supported Versions: TLSv1.3;TLSv1.2][bytes ratio: -0.580 (Download)][IAT c2s/s2c min/avg/max/stddev: 0/0 17/5 58/19 21/8][Pkt Len c2s/s2c min/avg/max/stddev: 66/66 152/669 583/1514 178/648][nDPI Cli Fingerprint: 03c638a568483a8c7de6f2dd89b99818][nDPI Srv Fingerprint: 81dba73ae426][TCP Fingerprint: 2_64_65535_685ad951a756/Android][TLSv1.3][JA4: t13d1513h2_8daaf6152771_eca864cca44a][JA3S: 15af977ce25de452b96affa2addb1036][Safari][Cipher: TLS_AES_256_GCM_SHA384][Plen Bins: 0,0,20,0,0,0,0,0,0,0,0,0,0,0,0,0,20,0,0,0,0,0,20,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,40,0,0]
56+
6 TCP 192.168.80.237:61384 <-> 76.223.31.142:443 [proto: 91.319/TLS.HBO][Stack: TLS.HBO][IP: 265/AmazonAWS][Encrypted][Confidence: DPI][FPC: 265/AmazonAWS, Confidence: IP address][DPI packets: 6][cat: Streaming/17][Breed: Fun][4 pkts/1818 bytes <-> 2 pkts/140 bytes][Goodput ratio: 85/0][0.03 sec][Hostname/SNI: busy.any-any.prd.api.discomax.com][(Advertised) ALPNs: h2;http/1.1][TLS Supported Versions: GREASE;TLSv1.3;TLSv1.2][bytes ratio: 0.857 (Upload)][IAT c2s/s2c min/avg/max/stddev: 0/16 5/16 15/16 7/0][Pkt Len c2s/s2c min/avg/max/stddev: 66/66 454/70 1514/74 613/4][nDPI Cli Fingerprint: 2f7493a76d6f6bb46601ce74d4097be7][TCP Fingerprint: 194_64_65535_d29295416479/macOS][TLSv1.2][JA4: t13d1313h2_f57a46bbacb6_7f0f34a4126d][Safari][Plen Bins: 0,0,50,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,50,0,0]
57+
7 TCP 192.168.0.100:52124 <-> 213.180.193.9:80 [proto: 7.440/HTTP.YandexAlice][Stack: HTTP.YandexAlice][IP: 25/Yandex][ClearText][Confidence: DPI][FPC: 25/Yandex, Confidence: IP address][DPI packets: 6][cat: ConnCheck/30][Breed: Acceptable][6 pkts/532 bytes <-> 4 pkts/350 bytes][Goodput ratio: 24/22][0.02 sec][Hostname/SNI: scbh.yandex.net][bytes ratio: 0.206 (Upload)][IAT c2s/s2c min/avg/max/stddev: 0/0 4/4 7/8 3/4][Pkt Len c2s/s2c min/avg/max/stddev: 66/66 89/88 194/144 47/33][URL: scbh.yandex.net/generate_204][StatusCode: 204][User-Agent: yandexmini_2/0.270.1.48.2683694502.20241115.199][TCP Fingerprint: 2_64_29200_d853e95bd80f/Linux][PLAIN TEXT (HEAD /generate)][Plen Bins: 0,0,50,0,50,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0]
58+
8 TCP 192.168.12.67:39974 <-> 151.101.1.233:443 [proto: 91.439/TLS.ParamountPlus][Stack: TLS.ParamountPlus][IP: 0/Unknown][Encrypted][Confidence: DPI][FPC: 0/Unknown, Confidence: Unknown][DPI packets: 4][cat: Streaming/17][Breed: Fun][3 pkts/723 bytes <-> 1 pkts/74 bytes][Goodput ratio: 71/0][0.01 sec][Hostname/SNI: vod-gcs-cedexis.cbsaavideo.com][(Advertised) ALPNs: http/1.1][TLS Supported Versions: TLSv1.3;TLSv1.2][nDPI Cli Fingerprint: 03c638a568483a8c7de6f2dd89b99818][TCP Fingerprint: 2_64_65535_685ad951a756/Android][TLSv1.2][JA4: t13d1513h1_8daaf6152771_eca864cca44a][Safari][PLAIN TEXT (cedexis.c)][Plen Bins: 0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,100,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0]

0 commit comments

Comments
 (0)