Testing

[EhudKirsh]

This is in response to #322 (comment). I prefer to keep #272 strictly about Wrappers/Overlays, as its name suggests.
Sometimes a thread brings up an idea, but that doesn't necessarily mean it should be discussed and designed in depth where it originated, because it's not necessarily in scope there, and this can make posts like #272 significantly longer than they already are.

I have questions regarding testing:

1. What tests are difficult to conduct and why? You alluded to OAuth. I'm also thinking of premium plans. I can test pCloud WebDAV while I still have a premium subscription to it. But how will it work? It seems complicated to ask users to help, due to the nature of communicating about it after each release and also the privacy aspect, which surely means it won't be automated.
   This can be a feature that each user can run locally, and AeroFTP notifies the user if a test failed and lets them know that they can submit a bug report. There can be a variable that starts with a FALSE value after each update/install, and it turns TRUE after running whatever tests the user chooses ahead of time.

2. What else do you think is worth testing? Maybe certain or all past reported bugs?

[axpnet]

Agreed, and thank you for opening a dedicated thread. Let's keep testing here and leave #272 to Wrappers/Overlays as you suggest.

We also published the test matrix this discussion prompted: docs/TESTING.md. It is honest about what is automated versus manually verified, and it is a living document that we update as new tests land.

1. What is hard to test, and why. Three things make a provider hard to cover automatically:

• Interactive OAuth: Google Drive, OneDrive, Dropbox, Box, pCloud, Zoho and similar need a real consented token. We cannot ship those tokens into public CI, and the consent step is interactive, so they cannot run unattended for everyone.
• Paid or plan-gated behavior: some paths only exist on premium tiers (pCloud WebDAV is a good example, as you note; also larger multipart thresholds and higher quotas). We do not hold every paid plan, so coverage there depends on whoever does.
• Live nondeterminism: real calls are rate-limited and occasionally WAF-gated (we had to work around a User-Agent block on one provider), so they are not reproducible enough for an unattended pass/fail gate. Our own live runs use dedicated dev and test accounts rather than anyone's real data, but those credentials still cannot be shipped into public CI, so we keep them as manual per-release checks.

Because of that the layers split cleanly: deterministic tests (offline parsing, error-mapping, crypto) run in CI for everyone; the lab-backed protocols (SFTP, FTP, FTPS, S3, WebDAV) plus Backblaze B2 run as automated integration tests; everything else is documented manual verification per release against our dev and test accounts, recorded with byte-identity proof.

Your local self-test idea is exactly the right shape for the rest, and it answers the privacy point you raised: a user runs it with their own accounts and their credentials and data never leave their machine. The user runs the providers they actually use, the app reports pass or fail, and on failure it offers a pre-filled bug report. The flag that starts FALSE after each update and flips TRUE after a run is a nice nudge so a fresh install reminds the user to re-run. We have opened two wishlist items in this direction: a single pre-release smoke entrypoint (P2) and systematic deterministic parsing and error-mapping tests for the providers that have unit coverage only (P1). A user-facing self-test would build naturally on top of those.

And yes please: if you can run pCloud WebDAV while you still have premium, that is exactly the kind of help that turns a pending cell in the matrix into a dated pass. Even a manual round-trip (upload, download, byte compare, a listing) reported here is valuable.

2. What else is worth testing. A few priorities:

• Regression tests for past reported bugs: yes. Every fixed bug should leave behind a deterministic test that pins it so it cannot silently return. We already do this on the freshest fixes (the WebDAV 404/409 upload mapping and the OpenDrive size-limit error both shipped with their own regression tests); the plan is to make it systematic and to backfill the historically reported ones where we can reproduce them deterministically.
• Provider response-shape and error-mapping from captured real responses, so a change on a provider's side or in a dependency fails a test rather than a user (this is P1).
• Cross-implementation interop where an independent reference exists, as we did for the Cryptomator format against the official cryptomator-cli.
• A standard edge-case sweep: empty files, unicode and spaced names, deep nesting, empty folders, over-long names, symlink handling. We run this for the Cryptomator format already and it is worth generalizing.

Thanks again for pushing on this. It is shaping the testing phase directly.

[EhudKirsh]

Looking at https://github.com/axpdev-lab/aeroftp/issues and https://github.com/axpdev-lab/aeroftp/issues?q=sort%3Aupdated-desc%20is%3Aissue%20label%3Abug%20state%3Aclosed%20author%3AEhudKirsh, I see some notable past and present bugs that are worth mentioning here:

1. [BUG] Koofr API can't connect in v3.8.1, getting 404 #213 - I happened to catch an API connection issue that returned 404 just for Koofr in one version, as soon as it was introduced. This should be among the most basic problems that testing should swiftly detect.
2. [BUG] OpenDrive API contextMenu.setAsPrivate/Public returns a code 400 error #252 and [BUG] Non-WebDAV MEGAcmd fails to upload files and delete folders. #263 show that it's not just enough to upload and delete some item.
   The problem can affect only files or only folders, so both should be tested.
3. [BUG] Zooming in on a previewed image hides its edges #239, [BUG] WebDAV MEGAcmd fails to preview images #264, [FEATURE] Add a '2FA Code' input field when connecting MEGA #128 (comment) and others are about previewing images,
   so perhaps you'd like to comment on whether such tests can be conducted unattended.