Skip to content

Commit aacacd7

Browse files
authored
Skip evaluating parameterized test arguments for disabled tests (#507)
Similar to how parameterized test arguments are left unevaluated for tests that aren't included in a `Plan`, also leave them unevaluated for tests in a `Plan` that won't run, e.g. because of having the `.disabled()` trait attached. ### Motivation: Since the changes in #366, parameterized test arguments are evaluated lazily, only for tests which are included in the `Runner.Plan` that is being prepared for execution. A code comment included in that change further suggested that arguments are only evaluated for tests which will actually run as part of the plan, as opposed to being skipped. However the actual code didn't reflect that behavior, by mistake. That means wasted work is being performed when setting up a `Runner.Plan`, evaluating the arguments of tests which won't end up actually running. ### Modifications: Only perform the step of evaluating test arguments/cases for tests which are marked with a `.run` action in the `Runner.Plan`. ### Result: Less throwaway work is performed when setting up a `Runner.Plan` for greater efficiency. ### Checklist: - [x] Code and documentation should follow the style of the [Style Guide](https://github.com/apple/swift-testing/blob/main/Documentation/StyleGuide.md). - [x] If public symbols are renamed or modified, DocC references should be updated.
1 parent 2cdfccf commit aacacd7

File tree

3 files changed

+25
-7
lines changed

3 files changed

+25
-7
lines changed

Sources/Testing/Running/Runner.Plan.swift

Lines changed: 8 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -253,12 +253,14 @@ extension Runner.Plan {
253253
// can be evaluated lazily only once it is determined that the test will
254254
// run, to avoid unnecessary work. But now is the appropriate time to
255255
// evaluate them.
256-
do {
257-
try await test.evaluateTestCases()
258-
} catch {
259-
let sourceContext = SourceContext(backtrace: Backtrace(forFirstThrowOf: error))
260-
let issue = Issue(kind: .errorCaught(error), comments: [], sourceContext: sourceContext)
261-
action = .recordIssue(issue)
256+
if case .run = action {
257+
do {
258+
try await test.evaluateTestCases()
259+
} catch {
260+
let sourceContext = SourceContext(backtrace: Backtrace(forFirstThrowOf: error))
261+
let issue = Issue(kind: .errorCaught(error), comments: [], sourceContext: sourceContext)
262+
action = .recordIssue(issue)
263+
}
262264
}
263265

264266
actionGraph.updateValue(action, at: keyPath)

Tests/TestingTests/PlanTests.swift

Lines changed: 14 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -428,6 +428,20 @@ struct PlanTests {
428428
#expect(step.test.id.nameComponents.count == 1, "Test is not top-level: \(step.test)")
429429
}
430430
}
431+
432+
@Test("Test cases of a disabled test are not evaluated")
433+
func disabledTestCases() async throws {
434+
var configuration = Configuration()
435+
configuration.setEventHandler { event, context in
436+
guard case .testSkipped = event.kind else {
437+
return
438+
}
439+
let testSnapshot = try #require(context.test.map({ Test.Snapshot(snapshotting: $0 )}))
440+
#expect(testSnapshot.testCases?.isEmpty ?? false)
441+
}
442+
443+
await runTestFunction(named: "disabled(x:)", in: ParameterizedTests.self, configuration: configuration)
444+
}
431445
}
432446

433447
// MARK: - Fixtures

Tests/TestingTests/Test.Case.ArgumentTests.swift

Lines changed: 3 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -152,7 +152,7 @@ struct Test_Case_ArgumentTests {
152152
// MARK: - Fixture tests
153153

154154
@Suite(.hidden)
155-
private struct ParameterizedTests {
155+
struct ParameterizedTests {
156156
@Test(.hidden, arguments: ["value"])
157157
func oneParameter(x: String) {}
158158

@@ -170,4 +170,6 @@ private struct ParameterizedTests {
170170

171171
@Test(.hidden, arguments: ["value": 123])
172172
func oneDictionaryElementTupleParameter(x: (key: String, value: Int)) {}
173+
174+
@Test(.disabled(), arguments: [1, 2, 3]) func disabled(x: Int) {}
173175
}

0 commit comments

Comments
 (0)