|
| 1 | +# Optimizing TLS over TCP to reduce latency for NGINX |
| 2 | + |
| 3 | +* [`nginx__dynamic_tls_records`](https://github.com/cloudflare/sslconfig/blob/3e45b99/patches/) |
| 4 | + |
| 5 | +### What we do now |
| 6 | + |
| 7 | +We use a static record size of 4K. |
| 8 | +This gives a good balance of latency and throughput. |
| 9 | + |
| 10 | +#### Configuration |
| 11 | + |
| 12 | +*Example* |
| 13 | + |
| 14 | +```nginx |
| 15 | +http { |
| 16 | + ssl_dyn_rec_enable on; |
| 17 | +} |
| 18 | +``` |
| 19 | + |
| 20 | +#### Optimize latency |
| 21 | + |
| 22 | +By initialy sending small (1 TCP segment) sized records, |
| 23 | +we are able to avoid HoL blocking of the first byte. |
| 24 | +This means TTFB is sometime lower by a whole RTT. |
| 25 | + |
| 26 | +#### Optimizing throughput |
| 27 | + |
| 28 | +By sending increasingly larger records later in the connection, |
| 29 | +when HoL is not a problem, we reduce the overhead of TLS record |
| 30 | +(29 bytes per record with GCM/CHACHA-POLY). |
| 31 | + |
| 32 | +#### Logic |
| 33 | + |
| 34 | +Start each connection with small records |
| 35 | +(1369 byte default, change with `ssl_dyn_rec_size_lo`). |
| 36 | + |
| 37 | +After a given number of records (40, change with `ssl_dyn_rec_threshold`) |
| 38 | +start sending larger records (4229, `ssl_dyn_rec_size_hi`). |
| 39 | + |
| 40 | +Eventually after the same number of records, |
| 41 | +start sending the largest records (`ssl_buffer_size`). |
| 42 | + |
| 43 | +In case the connection idles for a given amount of time |
| 44 | +(1s, `ssl_dyn_rec_timeout`), the process repeats itself |
| 45 | +(i.e. begin sending small records again). |
| 46 | + |
| 47 | +### Configuration directives |
| 48 | + |
| 49 | +#### dyn_rec_enable |
| 50 | +* **syntax**: `dyn_rec_enable bool` |
| 51 | +* **default**: `off` |
| 52 | +* **context**: `http` |
| 53 | + |
| 54 | +#### dyn_rec_timeout |
| 55 | +* **syntax**: `dyn_rec_timeout number` |
| 56 | +* **default**: `1000` |
| 57 | +* **context**: `http` |
| 58 | + |
| 59 | +We want the initial records to fit into one TCP segment |
| 60 | +so we don't get TCP HoL blocking due to TCP Slow Start. |
| 61 | + |
| 62 | +A connection always starts with small records, but after |
| 63 | +a given amount of records sent, we make the records larger |
| 64 | +to reduce header overhead. |
| 65 | + |
| 66 | +After a connection has idled for a given timeout, begin |
| 67 | +the process from the start. The actual parameters are |
| 68 | +configurable. If dyn_rec_timeout is 0, we assume dyn_rec is off. |
| 69 | + |
| 70 | +#### dyn_rec_size_lo |
| 71 | +* **syntax**: `dyn_rec_size_lo number` |
| 72 | +* **default**: `1369` |
| 73 | +* **context**: `http` |
| 74 | + |
| 75 | +Default sizes for the dynamic record sizes are defined to fit maximal |
| 76 | +TLS + IPv6 overhead in a single TCP segment for lo and 3 segments for hi: |
| 77 | +1369 = 1500 - 40 (IP) - 20 (TCP) - 10 (Time) - 61 (Max TLS overhead) |
| 78 | + |
| 79 | +#### dyn_rec_size_hi |
| 80 | +* **syntax**: `dyn_rec_size_hi number` |
| 81 | +* **default**: `4229` |
| 82 | +* **context**: `http` |
| 83 | + |
| 84 | +4229 = (1500 - 40 - 20 - 10) * 3 - 61 |
| 85 | + |
| 86 | +#### dyn_rec_threshold |
| 87 | +* **syntax**: `dyn_rec_threshold number` |
| 88 | +* **default**: `40` |
| 89 | +* **context**: `http` |
| 90 | + |
| 91 | +### License |
| 92 | + |
| 93 | +* [Cloudflare](https://github.com/cloudflare), [Vlad Krasnov](https://github.com/vkrasnov) |
0 commit comments