CI tests fail if only upgrade tests are required in `tests.toml`

[oleole39]

I tried to exclude all default tests but upgrade, but upgrade tests were skipped because: Error: All installs failed, therefore the following tests cannot be performed...

test_format = 1.0

[default]

    exclude = ["install.root", "install.subdir", "install.nourl", "install.multi", "backup_restore", "change_url"]

    # -------------------------------
    # Commits to test upgrade from
    # -------------------------------

    test_upgrade_from.d9beaa2fa546a3c2acce2457d8d817bcbbbdd520.name = "Upgrade from 1.2.35~ynh3"

including install.root again is enough to have upgrade tests working.

Cf. the first 3 CI run in this PR: YunoHost-Apps/kanboard_ynh#234

[orhtej2]

What is the issue and how would you like to address it? This is not a very smart tool, it needs to be able to install first (on root iirc) in order to upgrade later 🤷

[oleole39]

it needs to be able to install first (on root iirc) in order to upgrade later 🤷

EDIT: How comes it requires root install step (=installing latest version) to complete to then perform the upgrade step, given that in my current case, I ask for an upgrade from an earlier commit ?

But I have added this issue also as a reminder as I have not dived into the script so far.

[orhtej2]

EDIT: How comes it requires root install step (=installing latest version) to complete to then perform the upgrade step, given that in my current case, I ask for an upgrade from an earlier commit ?

This is not a very smart tool

It asserts that the app is installable as this is a requirement for the app getting level 1. It was not intended for testing individual edge cases.

[oleole39]

Also I remember that originally even if install failed on root, CI would continue all the tests (which would obviously fail most often).
So I guess adding this error message and skipping tests systematically was already a first improvement to the original script - saving CI time - even if not fully accurate as in this issue (at error message is misleading here).

I'll see later if I can improve the logic for this case. I believe it wouldn't hurt at least?