`InternetAddress.lookup` needs a cache

[gmpassos]

The method InternetAddress.lookup needs a resolution cache:

sdk/sdk/lib/io/socket.dart

Line 152 in c6daa6a

external static Future<List<InternetAddress>> lookup(

Why?

For each connection requiring hostname resolution, a new lookup is performed. This triggers requests to the OS DNS resolver (e.g., bind, dnsmasq, mDNSResponder, etc.). Depending on the application being developed, this behavior can overwhelm the OS.

For instance, while benchmarking shelf with intensive requests, this behavior "crashed" the macOS DNS responder. Additionally, the hostname lookup accounted for a significant portion of the time in each benchmark interaction, resulting in non-performant behavior.

Suggestion

I suggest introducing an InternetAddressOverrides, similar to HttpOverrides, and akin to what Java does with InetAddress.

With this feature, developers could customize how their application handles DNS resolution without needing to modify the code that creates a Socket (which may not always be accessible for changes). Whether the default behavior includes a cached lookup would depend on the Dart team's decision, but an InternetAddressOverrides would empower developers to choose the best behavior for their specific applications.

Example

To work around this issue, I have created tools that allow cached addresses for HTTP requests:
https://github.com/gmpassos/multi_domain_secure_server/blob/master/lib/src/connector_tools.dart#L7

[dart-github-bot]

Summary: InternetAddress.lookup repeatedly calls the OS DNS resolver, impacting performance and potentially crashing it under load. A caching mechanism, perhaps via InternetAddressOverrides, is proposed for improved efficiency.

[lrhn]

One could argue that the OS should cache DNS lookups, so you don't have to.

I think having a separate DNS lookup cache that you can choose to use might be better than having one that everybody must use, since caches cost memory too.

[a-siva]

I agree with @lrhn's comment above that having a separate DNS lookup cache would be better than adding it generically to InternetAddress.lookup

[gmpassos]

@irhn @a-siva

Most operating systems have their own DNS cache. However, interprocess communication may not be performant enough to handle a high volume of requests, potentially leading to crashes or a denial of service (DoS) for the local DNS resolver/cache.

The main idea behind my proposal is to provide an optional DNS cache within a Dart program. This implementation would not require changes to the existing Socket code or modifications to third-party packages. Additionally, the default behavior of hostname resolution would remain unchanged unless the developer explicitly defines a hostname lookup cache, similar to how HttpOverrides works.

[gmpassos]

Many applications, such as Node.js, JVM, Google Chrome, Mozilla Firefox, Microsoft Edge, Safari, Slack, Spotify, and Docker, implement their own DNS cache. Do not underestimate the importance of this feature, as it is crucial for improving performance and reliability in real-world applications.

In Dart, however, adding a cache layer to InternetAddress.lookup is currently impossible unless you manually create the Socket connection. This makes it difficult to achieve the same functionality when relying on third-party packages that handle the connection.