Download/Upload only missing frameworks

[Robuske]

Question

Maybe I am not understanding correctly, but if I run rome download two times it will download everything the two times? And the same for upload? It doesn't check the versions to know if it actually needs to do that?

Rome version:

0.20.0.56

[tmspzz]

For upload you can run rome list first as explained in the CI setup here https://github.com/blender/Rome/blob/master/README.md#--cache-builds-workflow-recommended

This avoids unnecessary uploads.

For download, where would it get the version information to check? From the .version files? That doesn’t guarantee you still have the same binary since there is no checksum.

[Robuske]

It doesn't guarantee, but isn't what Carthage itself uses for the --cache-builds option? And don't you use ir for the list --missing? What would be the difference?
Seems good enough for at least having the option of doing that with a --check-version or something like that. And if you don't check the version for the list --missing command, it could be added there and to the upload command too

I just started using Rome and I like how much time it can save, but I couldn't find a way of only running it when really needed on the CI (Bitrise already has its own cache), part of the problems is that Carthage doesn't have a way (that I know of) of telling if I need to build something before actually doing it. But I really think that Rome should have that option by itself

[tmspzz]

@Robuske I'll try to explain how things work

.version files are irrelevant. Rome doens't read .version files. This is because on a fresh checkout you wouldn't have .version files to begin with. Your Carthage folder is empty so there is nothing to read there.

When you run rome download also the .version files are downloaded. It's a chicken and egg problem. So, you can't really limit what is downloaded. The information that would help in scoping the oeration is itself being downloaded. If you have alternative ideas, I'm ears 👍

The list command doesn't looks at .version files either. It looks at the Cartfile.resolved at queries whatever cache you pointed rome to for the existence of a bunch of files.

You can however limit uploads by running:

rome list --missing --platform iOS | awk '{print $1}' | xargs rome upload --platform iOS

If you want to use all the bandwidth at your disposal you can even add --concurrently to upload and download commands. However the --verbose output very hard to follow.

[tmspzz]

Wait, I see what you're saying. YourCarthage/ folder is not empty to begin with.

So you're suggesting to read the .version files to limit download operations if they are available.
That could be useful but still .dYSM and .bcsymbolmap are not checksummed in the .version file so they would need to be downloaded again.

[Robuske]

Exactly, I may already have everything, or most of the cache, I just want to download what is missing. The main use case is any CI service that already has its own cache, avoiding downloading everything again could save time and money, S3 is cheap, but not free
Yeah, dSYM and bcsymbolmap may be a problem, but like I said, I would trust the version file, if it was changed, download all files, ignore them otherwise, because of that it shouldn't be the default behaviour, but having as a option is enough

Something I am not sure I understood from what you said
If I have Alamofire 4 and upload it
Them update to Alamofire 5, the cache is reported as missing or not?

[tmspzz]

Something I am not sure I understood from what you said
If I have Alamofire 4 and upload it
Them update to Alamofire 5, the cache is reported as missing or not?

I'm not sure what you mean. Can you be more clear?

[Robuske]

What the rome list --missing tests for? Having any version of the framework, or the version specified on Carthage.resolved?

[tmspzz]

@Robuske rome list tests for the version listed in Carthage.resolved to be in the cache.

I have made the improvement you have suggested, but only to download. I don't see how it would make sense for upload or list.

[Robuske]

If that's how it works it should be enough, thank you

You made already? I don't see any PR/branch/merge

[tmspzz]

@Robuske I need to finalize a CLI switch for this feature and clean up a few things first. I'll write here when there is a pre-release version you can test :)

[Robuske]

No problem, didn't expect it to be implemented that fast, thank you
I thought about contributing, but I've never touched Haskel before, so I was having a hard time understanding it