Show download/upload speed in CLI

[pavelzw]

would be cool to see the current download/upload speed in the cli. maybe also a counter how much data was already successfully transferred

[mitanshu-23]

Can I try this one?

[pavelzw]

sure!

i think something like this would be ideal:

$ ./conda-mirror -c config.yaml
🪞 Mirroring conda-forge to s3://rattler-testing/conda-forge-mini...
########################################     390/390     Finished adding packages in win-32 (1.5 GB)
#---------------------------------------      54/5040    Mirroring linux-ppc64le (10.5 GB) python-suitesparse-graphblas-7.4.3.2-py38h48a2adc_0.conda
<spinner> 10 min 20 s / 1.5 Gb/s ↕ / 202.5 GB

for formatting the data we could use https://docs.rs/indicatif/latest/indicatif/struct.HumanBytes.html and https://docs.rs/indicatif/latest/indicatif/struct.HumanDuration.html
for the spinner we should use https://docs.rs/indicatif/latest/indicatif/struct.ProgressBar.html

i think it's fair to try to estimate upload/download speed by having some sort of counter over the last ~20 seconds and just add the package sizes together to estimate how much was mirrored in the last couple of seconds.
not sure it makes sense to estimate the upload and download speed separately as they are probably tightly coupled: if your upload speed is limited to 10Mb/s then your download speed probably is also around that on average.

[mitanshu-23]

Thanks!!, I’m working on this and looking forward to completing it!

[mitanshu-23]

Hello @pavelzw,

Currently, we set the progress bar message before downloading each file:
pb.set_message(format!( "Mirroring {} {}", subdir.as_str(), console::style(&filename).dim() ));

This works fine for just showing the current file being mirrored.

However, now that we want to display download speed, there’s a limitation: because we download each file entirely at once, we can only calculate speed after the file has finished downloading. This means the set_message to show speed would need to be called after the download completes, which would overwrite the previous message about the currently downloading file.

• With this approach, we may not know which file is currently in progress.
• An alternative is to use pb.println method to log each file’s speed after it finishes, but for many files, this could produce excessive output.

Or to keep overwriting the message with the most recently mirrored file and the file currently being mirrored—but in my opinion, this is not a suitable approach.

[pavelzw]

i think adding a second progress bar that's exclusively for time+speed+total downloaded should work, no? we can then pass this progress bar into dispatch_tasks_add