Tests are not robust to parallel running

[Altai-man]

Right now, test files assume running in a certain order specified by file names.

I am not sure whether we want to teach Comma about something like "Execute those tests in order in a single thread", but right now parallel execution, even with a single worker, will lead to failed tests, which is not always stable.

[jnthn]

@Altai-man So they only pass because prove, say, will sort the test files by filename?

[Altai-man]

@jnthn not sure about prove, but zef test ., at the very least, passes. while Comma breaks with "no cache prepared".

[JJ]

I see the extent of the problem now. As indicated by @antoniogamiz, in this answer in Stackoverflow, the problem is that we are never updating a cache. Caches can't simply be updated. What we were doing before was creating a cache in a program, and then using the cache from another program. As ugexe says, the first time we do "load" is when we cache the results, so by creating the Pod::To::Cached object we "freeze" the cache. Subsequent calls to load will simply not work.
I'm not sure how to get out of this, since all that part of Rakudo is under-documented. Don't know if we can somehow unlock the cache and update it; or somehow not call load so that we don't freeze it. I would be very grateful if you suggested any solution.

[antoniogamiz]

@JJ did you want to comment that in this issue or in #16?

[JJ]

@antoniogamiz probably in the other issue... Let's leave this for the parallel update, which is a different problem. It would require not only the tests being independent, but also use random temporary directory to avoid race conditions.