-
Notifications
You must be signed in to change notification settings - Fork 109
Description
Currently, we only test all tested-versions for a certain library once—when the scheduled job discovers new versions. After the list is updated, we only test the version defined in metadata-version field. (Example of the index file)
Idea: When metadata or tests for a certain library change, we should run tests with all tested-versions as well (not just the version the tests are currently assigned to) to see if the change broke support for some tested-versions.
For discussion: Once contributed to the repository, the metadata can only be extended. Extending metadata should not break support for already tested versions, since those library versions are already released and never change. The metadata that was once appropriate for that version should always remain valid for that version. We should therefore consider testing with all tested-versions only if the tests change.