Adding a new test attribute "NoIterativeLeakTesting" that signals TestRunner to omit tests from our iterative leak tests #3349
Conversation
…tRunner to omit tests from our iterative leak tests. Applied to several tests with long execution times. Those test still run in other test phases, just not the leak test phase where they crowd out other tests due to time constraints.
|
Good to have this attribute! I'm not sure whether we want to disable leak testing on the paste and library import tests. I know those are woefully long running for functional tests, but do we have other shorter tests that are going to exercise most of the code that the paste and library import ones do? EncyclopeDia is a good one to exclude for sure. I think we can also exclude all the DiaSearch and DdaSearch tests: most of the execution time is there running the search and percolator, and the rest of it should already be leak-covered by other tests. |
| /// (e.g. memory hungry or excessively time consuming given that it has to be run many times for leak detection) | ||
| /// </summary> | ||
| [AttributeUsage(AttributeTargets.Method, AllowMultiple = false)] | ||
| public sealed class NoIterativeLeakTestingAttribute : Attribute |
There was a problem hiding this comment.
Can we give this attribute a shorter name. Maybe "NoLeakTestingAttribute"?
Also, can we give it a default zero-argument constructor which sets the Reason to "TestExclusionReason.EXCESSIVE_TIME"?
There was a problem hiding this comment.
I'm open to a rename, but I do think there's value in having to declare why it's being set.
I suppose we can run it through code coverage to be certain. And possibly the paste test, at least, could be broken down into multiple tests some of which might be suitable candidates. |
…" per Nick suggestion
…k/20250127_NoIterativeLeakTesting
…going into master
Applied to several tests with long execution times. Those test still run in other test phases, just not the leak test phase where they crowd out other tests due to time constraints.