Skip to content

Reporting failing tests despite all tests running green #311

@acn-sbuad

Description

@acn-sbuad

We are using this action in our workflows to better present the test reports after running a workflow. However, lately we have experienced the trx-parser action failing and reporting on failed tests when it is logged that all test cases have passed.

The worfklow

log from the task running the tests

Test run for /home/runner/work/altinn-notifications/altinn-notifications/test/Altinn.Notifications.Tests/bin/Release/net8.0/Altinn.Notifications.Tests.dll (.NETCoreApp,Version=v8.0)
VSTest version 17.11.0 (x64)

Starting test execution, please wait...
A total of 1 test files matched the specified pattern.
Results File: /home/runner/work/altinn-notifications/altinn-notifications/TestResults/_fv-az1052-128_2024-08-19_12_52_21.trx

Passed!  - Failed:     0, Passed:   269, Skipped:     0, Total:   269, Duration: 644 ms - Altinn.Notifications.Tests.dll (net8.0)
Test run for /home/runner/work/altinn-notifications/altinn-notifications/test/Altinn.Notifications.IntegrationTests/bin/Release/net8.0/Altinn.Notifications.IntegrationTests.dll (.NETCoreApp,Version=v8.0)
VSTest version 17.11.0 (x64)

Starting test execution, please wait...
A total of 1 test files matched the specified pattern.
%4|1724071950.202|GETPID|rdkafka#producer-1| [thrd:main]: Failed to acquire idempotence PID from broker localhost:9092/1: Broker: Coordinator load in progress: retrying
Results File: /home/runner/work/altinn-notifications/altinn-notifications/TestResults/_fv-az[105](https://github.com/Altinn/altinn-notifications/actions/runs/10453675204/job/28944613679#step:11:106)2-128_2024-08-19_12_52_32.trx

Passed!  - Failed:     0, Passed:   112, Skipped:     0, Total:   112, Duration: 2 m 47 s - Altinn.Notifications.IntegrationTests.dll (net8.0)

Attachments:
  /home/runner/work/altinn-notifications/altinn-notifications/TestResults/07f1e20a-abd9-4e89-b7e3-c4e6f22c186e/coverage.opencover.xml
  /home/runner/work/altinn-notifications/altinn-notifications/TestResults/f5447407-a6c1-4ffd-b1ca-3c0cf28c63f0/coverage.opencover.xml

log from the trx-parser task:

Run NasAmin/trx-parser@v0.6.0
Finding Trx files in: /home/runner/work/altinn-notifications/altinn-notifications/TestResults
Files count: 6
Processing 2 trx files
Transforming file /home/runner/work/altinn-notifications/altinn-notifications/TestResults/_fv-az[10](https://github.com/Altinn/altinn-notifications/actions/runs/10453675204/job/28944613679#step:13:11)52-128_2024-08-19_12_52_21.trx
Transforming file /home/runner/work/altinn-notifications/altinn-notifications/TestResults/_fv-az1052-[12](https://github.com/Altinn/altinn-notifications/actions/runs/10453675204/job/28944613679#step:13:13)8_2024-08-19_12_52_32.trx
Checking for failing tests
Creating PR check for ALTINN NOTIFICATIONS.TESTS
Check time is: Mon, 19 Aug 2024 12:55:57 GMT
Created check: altinn.notifications.tests-check with response status 201
Creating PR check for ALTINN NOTIFICATIONS.INTEGRATIONTESTS
Check time is: Mon, 19 Aug 2024 12:55:58 GMT
Created check: altinn.notifications.integrationtests-check with response status 201
Error: At least one failing test was found

Metadata

Metadata

Assignees

Labels

No labels
No labels

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions