After hook and Before hooks are not running if a failure occrurs

[dmbartle]

Current behavior

The Before and After hooks will only run when tests pass.

Desired behavior

We want to be able to use the cucumber style After hook to run some test cleanup, even when tests fail. But currently the After hook will only run when the test passes.

Using the cypress afterEach hook is an option but it would be nice to use the cucumber After hook as it supports @tags

Test code to reproduce

Here is a link to an older test feature file from the cypress repo which illustrates the issue. The scenarios that illustrate this problem are commented out as they cause steps to fail, (Scenario: With Untagged After having test errors in steps part 1 & 2). The steps file only needed a minor tweak to pull { Given, Then, After, Before } from the right place.

https://github.com/badeball/cypress-cucumber-preprocessor/blob/77dbacb65741d559ae0eedd461f02970249f75d8/cypress/integration/BeforeAndAfterSteps.feature

Versions

• Cypress version: cypress@10.7.0
• Preprocessor version: @badeball/cypress-cucumber-preprocessor@12.1.0
• ** esbuild version**: @bahmutov/cypress-esbuild-preprocessor@2.1.3
• Node version: v16.14.0

Checklist

• I've read the FAQ.
• I've read Instructions for logging issues.
• I'm not using cypress-cucumber-preprocessor@4.3.1 (package name has changed and it is no longer the most recent version, see #689).

[dmbartle]

Related issue:

#538

[badeball]

The Before and After hooks will only run when tests pass.

This is by design and matches behavior found in Cucumber.

If you want afterEach behavior then use afterEach. You can use doesFeatureMatch(..) for conditions, as seen here.

[dmbartle]

Okay, thanks for the quick reply.

[hammzj]

Hello, I believe we need to reopen this. The behavior of an After hook is to run post-scenario, regardless of whether the scenario passed or failed.

See here:

After hooks run after the last step of each scenario, even when the step result is failed, undefined, pending, or skipped.

While that above is for Cucumber-Java, the same behavior works for Cucumber-JS where a tagged hook will still run post-scenario. I can reproduce this behavior if needed.

[badeball]

Re-opening, as this does in fact seem to the behavior exhibited by Cucumber, despite what I was thinking earlier.

This is likely a pretty substantial change and I don't have the ability to look into it right now. Anyone feeling particularly compelled can take a shot at it meanwhile.

[badeball]

As per now, the plugin will register a single afterEach hook. If a step fails, this hook will ensure that appropriate messages are sent to the backend (example).

In order to ensure that After hooks are run despite step failure, they would have to be part of an afterEach block. This is just due to how Cypress works - there isn't any try-catch here.

However, errors ocurring in afterEach blocks are "uncatchable", as in they will cause the spec to finish early. In other words, you only have one opportunity to handle failures in Cypress and this is used by the plugin to create appropriate messages / reports.

Thus, if After hooks were to run as part of the internal afterEach, then failures in such hooks could not be handled properly and failures here would finish the spec early.

For this reason, I believe it is unlikely that this issue will be fixed.