Try-ports is detrimental to good engineering

[gcb]

Change Intent

try-ports logic on dart devtools, and its defaulting to 10, is detrimental and cause more issues on build systems than convenience (not sure who is the target. developers who run devtools one after another and forget them in the background? do we want to foster this? I honestly tried to understand when/why it was added but there was two migrations on git where this code was moved that made history hard to follow)

Justification

This logic goes against every single other daemon behaviour. (i.e. user specify --port=8000 but daemon listen on port 8001, or 43993 or whatever other port. See bellow).

Also, probably a security issue somewhere. (e.g. deploy to stage where port 8000 is only accessible via office vpn and 8001 is connected to the your http cache egress, and devtools decide to start on port 8001).

If the caller have logic to handle that (i.e. read the command output and attach to a different port) then it should explicitly use that --try-ports param and be happy. (or --port=0 see bellow)

Most daemons respect port parameters. It is counter-intuitive to start a webserver at a random port and search for it. If you use the --port command, and the port is already used, the common behaviour of most servers/daemons is to fail. I cannot think of other servers that does this by default. Specially undocumented like this.

Extra points:

• it's confusing that try-ports needs 1 to respect the value of --port. More confusing that the sensible value zero results in dart using a random port, ignoring --port=8000 entirely. crazy.
• This eccentric and non-standard behaviour is not documented on most help text/online guide/https://docs.flutter.dev/tools/devtools/cli/etc. Only verbose help (dart help -v <topic>) for devtools will mention it exists! I will be honest, I didn't even know there was verbose help until I looked into a patch for this and saw the hidden param docs in the source. I expected help would have helped me.
• still on help text, the hidden help for this param is missing all the edge cases (e.g. 0 means use random port; or if --port failed because of permission like using port 80 as regular user, use random port; etc)
• The entire functionality of this (using unknown ports and figuring out by parsing output) is already handled by --port=0, which is already mentioned on the regular help for devtools just fine.
• • having to set --try-ports=1 to have --ports= behave like it is described in its own help text is not mentioned though.

Impact

More secure deployments. Saner server startup. Predictable behavior. Less silent failures on CI because server decided on a port on its own instead of failing.

Some program which already reads the output of dart devtools and parse it to get the port would fail if also using --port. If not using port it would not notice any change.

Mitigation

set the default port to 0 (use any port).

Users who do not care about port and would be happy with random port will not even notice the change.

Better yet, --port could accept ranges (the implementation would be much simpler than what is there today for try-ports) and default to --port=9100-9110

Change Timeline

No response

Associated CLs

No response

[bkonyi]

It looks like this behavior was added in flutter/devtools@87a9883 and the default value of --try-ports was set to 10 in flutter/devtools#2458.

I have to agree that this behavior is strange. I understand it's an annoyance to run dart devtools only to fail to bind to the default port, so this seems like it was a way to make things less annoying for developers on the CLI. I'm not sure this should be the default behavior, especially if --port is specified. If a user is specifying a specific port and DevTools can't be served on that port, the command should fail. cc @kenzieschmoll for their input.

I don't think this port selection behavior is much more of a security concern than providing --port=0 and binding to a random port that may already be forwarded for another reason.

try-ports logic on dart devtools, and its defaulting to 10, is detrimental and cause more issues on build systems than convenience.

Out of curiosity, how is this causing issues with build systems? I can't think of a scenario where the Dart VM service or DevTools should be enabled in that context.

[gcb]

flutter/devtools#1037

thanks for the history! Still don't see a valid use case though.

flutter/devtools#2305

Ironically asks for random ports, like you would get with ports=0.

I have to agree that this behavior is strange. I understand it's an annoyance to run dart devtools only to fail to bind to the default port, so this seems like it was a way to make things less annoying for developers on the CLI. I'm not sure this should be the default behavior, especially if --port is specified. If a user is specifying a specific port and DevTools can't be served on that port, the command should fail. cc @kenzieschmoll for their input.

That's how I see it also. If --port is specified it should be honored. Everything else is better served by --port=0 which can be the default and the --try-ports is never needed for any case. And I still fail to see who benefits from having tons of devtools lying around using the ports.

I don't think this port selection behavior is much more of a security concern than providing --port=0 [...]

Correct, but then you are opt-in (if we don't make that the default. I personally think --port=9100 being the default is better, and --port=0 if you want any port. But --port=0 might be a good middle ground so you guys can avoid a flood of tickets until a major version?).
Also, having a random port is better than sequential from a security POV, because sequential ports are more likely to be open by firewalls, being guessed, etc.
I am assuming the use case that don't care about port just use --launch-browser and never pay any attention to the port.

Out of curiosity, how is this causing issues with build systems?

enabling debug ports is very common to measure performance and code coverage in most languages. and opening a port range on the firewall is also common. so those are usually close together and with this 'feature' are a recipe for disaster.

A late functional stage build had a port used by something transient, devtools picked another random port. Much later on, things failed. The original port was not even used at that point, so we spent a lot of time hunting firewall and network issues and bad configs, when we were listening on the wrong port. Which then also caused some wasted time trying to figure out why the log said port X and configs said port Y and it worked everywhere but on that build step with the same config files. Again, remember I had to look at sources with the intent to suggest a patch before i even found out about help -v which is the only place with some documentation on this.