Feature: HTTP store cache

[nunojpg]

For very slow networks where we might have to retry a update over several days, have you considered to cache the chunks coming from HTTP into a cache partition?

I would be happy to contribute such feature but would like to know your thoughts on it before.

[florolf]

This is basically what #4 is about: If you skip the parts of the target partition that have already been written to, resuming an interrupted update will always make forward progress.

I agree that being able to deal with interrupted updates/flaky network connections is a very desirable feature. I felt that the approach in that PR is a little too expensive computationally, but there's a way to solve that. I've shamefully neglected that PR because I didn't have access to the kind of hardware I wanted to run some benchmarks on back then and then other things got in the way. This has now largely been rectified and I definitely want to get in a resumption mechanism soon.

Having a separate cache somewhere doesn't really sound appealing to me. This adds quite a bit of complexity and needlessly so: casync-nano is able to use the already-written-to parts of the target as a store as well (https://github.com/florolf/casync-nano/blob/master/target.h#L21-L25). While using this feature currently has some performance impacts, this is not a fundamental problem, but rather just the implementation being kinda lazy. Especially when combined with some kind of resumption mechanism, this feature is basically equivalent to what you propose.

[nunojpg]

Great.
To be honest I forgot that I can use the other partition as a dirty cache while there are downloads in progress.
Thanks for explaining!