Add getzim.sh script

[mkg20001]

The script allows the user to pick a source and download it from cli without having to visit http://wiki.kiwix.org/wiki/Content_in_all_languages
It also automatically resumes the download and verifies the md5sum

[momack2]

Seems related to #58?

[lidel]

Thank you @mkg20001!

Quick feedback:

• Add or switch to non-interactive mode. We need it for Automate snapshot updates #58, something like:
  • getzim --list should return a list of available wikis
  • getzim --wiki <id> --latest would return the name of latest snapshot of a specified wiki
  • getzim --wiki <id> --get <snapshot> would download specified snapshot
• Do you know why I can't see wikipedia on the list?
• Instead of grepping Wiki HTML, try grepping directory listings from https://download.kiwix.org/zim/ (and https://download.kiwix.org/zim/wikipedia/ etc)
  • Should be enough for now, in the future we wil lconsider switching to https://wiki.kiwix.org/wiki/OPDS (as suggested in Automate snapshot updates #58 (comment)), btu for now API is not stable enough

[lidel]

It did not work for me unless I enabled following redirect to HTTPS:

Suggested change

	curl -s http://wiki.kiwix.org/wiki/Content_in_all_languages > .content
	curl -sL http://wiki.kiwix.org/wiki/Content_in_all_languages > .content

[kelson42]

Maybe just better use the https version https://wiki.kiwix.org/wiki/Content_in_all_languages

[mkg20001]

I've changed the script to work with the download.kiwix.org site directly

There's a command bash getzim.sh choose that allows to create a download command

One of these for example is bash getzim.sh download other archlinux en all notag any which downloads the english archlinux zim

Note that currently SHA256 validation is broken and needs to be fixed

[lidel]

@mkg20001 Apologies for missing notification on this! I did a test run with bash getzim.sh download wikipedia wikipedia tr all maxi any and download worked as expected 👍

Let me know if you have time to make two changes before we merge this:

1. Rename any to latest (I believe you already sort to ensure it returns latest entry, so it is a better name anyway)
2. Add url command to print the URL behind latest without downloading it, something like:

   $ bash getzim.sh url wikipedia wikipedia tr all maxi latest
   https://download.kiwix.org/zim/wikipedia/wikipedia_tr_all_maxi_2019-10.zim

   The idea is to use this in automation (Automate snapshot updates #58) to tell if latest newer than the last successful snapshot build.

[mkg20001]

The url command did already exist (download_url), but now I've aliased it to url as well and documented it

Any is now latest

[mkg20001]

Now the url command also returns json and prints the log to stderr

[lidel]

Thank you!
Works as expected, can be refined later if needed, let's merge this.

@mkg20001 are you able to apply below fix? (i can't commit to your repo)

[mkg20001]

The cache being filled without cache_update was sorta a unintended feature, but I think we can just make this a proper feature

[lidel]

@mkg20001 is there anything you want to add, or ok to merge? (can be a separate PR)

[mkg20001]

Not really. Don't fix it, if it ain't broken :P

[lidel]

Thanks again!