Skip to content

Add Support for Cucumber using @split[feature:treatment] tags #247

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 4 commits into from
Jul 26, 2021
Merged

Add Support for Cucumber using @split[feature:treatment] tags #247

Show file tree
Hide file tree
Changes from 2 commits
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
fd19e7e
Add CucumberSplit
aslakhellesoy Jul 13, 2021
87d6d4f
Add URL in JavaDoc
aslakhellesoy Jul 13, 2021
5024560
Improve the Cucumber example to be more realistic
aslakhellesoy Jul 18, 2021
77b4753
disambiguation
aslakhellesoy Jul 18, 2021
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 10 additions & 0 deletions testing/pom.xml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -23,5 +23,15 @@
<groupId>junit</groupId>
<artifactId>junit</artifactId>
</dependency>
<dependency>
<groupId>io.cucumber</groupId>
<artifactId>cucumber-java</artifactId>
<version>6.10.4</version>
Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This should probably be test scope, no?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

probably

Copy link
Author

@aslakhellesoy aslakhellesoy Jul 17, 2021 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've thought a bit more about this. I think there are two options here - compile scope or provided scope. (See maven docs for details).

The junit dependency is already compile scope, so using the same scope for cucumber-java and cucumber-junit would be consistent with that.

Alternatively, making it provided scope would indicate that dependent modules would have to explicitly add the dependencies themselves. I think that's ok too, if you don't want consumers of the library transitively download cucumber unless they opt into it with an explicit dependency in their own pom/gradle file.

I'll defer to you to decide what's most appropriate.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If someone is already using this framework (our Java-testing) without cucumber and I merge this PR. Regardless of the provided or not, I believe the build will fail for not finding the dependency in the classpath, right?

I need time to sync internally but if that's the case we may need to split this into a separate submodule.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I believe the build will fail for not finding the dependency in the classpath, right?

No it won't.

In Java, classes are only loaded when they are referenced by another class. Classes are not loaded by mere presence in a jar file.

No other classes in this library reference io.split.client.testing.cucumber.CucumberSplit, so it won't be loaded.

</dependency>
<dependency>
<groupId>io.cucumber</groupId>
<artifactId>cucumber-junit</artifactId>
<version>6.10.4</version>
</dependency>
</dependencies>
</project>
51 changes: 51 additions & 0 deletions testing/src/main/java/io/split/client/testing/cucumber/CucumberSplit.java
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,51 @@
package io.split.client.testing.cucumber;

import io.cucumber.java.Scenario;
import io.split.client.testing.SplitClientForTest;

import java.util.Collection;
import java.util.regex.Matcher;
import java.util.regex.Pattern;

/**
* <p>
* Simple Cucumber plugin for Split.
* </p>
* <p>
* Cucumber scenarios annotated with {@code @split[feature:treatment]} tags can be used to
* configure a {@link SplitClientForTest} instance.
* </p>
* <p>
* To use it, define a <a href="https://cucumber.io/docs/cucumber/api/#hooks">Before Hook</a> that invokes the {@link CucumberSplit#configureSplit(SplitClientForTest, Scenario)}
* method. Example:
* </p>
*
* <pre>
* import io.cucumber.java.Before;
* import io.split.client.testing.SplitClientForTest;
*
* public class StepDefinitions {
* private final SplitClientForTest splitClient = new SplitClientForTest();
*
* &#64;Before
* public void configureSplit(Scenario scenario) {
* CucumberSplit.configureSplit(splitClient, scenario);
* }
* }
* </pre>
*/
public class CucumberSplit {
private static final Pattern SPLIT_TAG_PATTERN = Pattern.compile("^@split\\[(.*):(.*)]");

public static void configureSplit(SplitClientForTest splitClient, Scenario scenario) {
Collection<String> tags = scenario.getSourceTagNames();
for (String tag : tags) {
Matcher matcher = SPLIT_TAG_PATTERN.matcher(tag);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Question here:

  • if more than 1 tag matches this, do you end up overwriting the registered (feature, treatment) pair?

Copy link
Author

@aslakhellesoy aslakhellesoy Jul 17, 2021 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes. This is intentional.

Tags that are placed at the top level Feature level are inherited by all the Scenarios. The getSourceTagNames() method returns a Collection of the tags from both the Feature and the current Scenario. The inherited Feature tags are always before the Scenario tags in the collection (it would have been more appropriate to declare the return type as List rather than Collection since it's ordered).

We're taking advantage of this here to allow users to define "default" treatments at the feature level, and (optionally) override them at the scenario level. I have illustrated this in the last scenario where the default dollars=off treatment from the feature level is overridden to dollars=on for that particular scenario.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've pushed some more changes to (hopefully) illustrate better how this works.

if (matcher.matches()) {
String feature = matcher.group(1);
String treatment = matcher.group(2);
splitClient.registerTreatment(feature, treatment);
}
}
}
}
8 changes: 8 additions & 0 deletions testing/src/test/java/io/split/client/testing/cucumber/RunCucumberTest.java
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
package io.split.client.testing.cucumber;

import io.cucumber.junit.Cucumber;
import org.junit.runner.RunWith;

@RunWith(Cucumber.class)
public class RunCucumberTest {
}
22 changes: 22 additions & 0 deletions testing/src/test/java/io/split/client/testing/cucumber/StepDefinitions.java
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,22 @@
package io.split.client.testing.cucumber;

import io.cucumber.java.Before;
import io.cucumber.java.Scenario;
import io.cucumber.java.en.Then;
import io.split.client.testing.SplitClientForTest;

import static org.junit.Assert.assertEquals;

public class StepDefinitions {
private final SplitClientForTest splitClient = new SplitClientForTest();

@Then("split {string} should be {string}")
public void split_should_be(String split, String expectedValue) {
assertEquals(expectedValue, splitClient.getTreatment("arbitraryKey", split));
}

@Before
public void configureSplit(Scenario scenario) {
CucumberSplit.configureSplit(splitClient, scenario);
}
}
1 change: 1 addition & 0 deletions testing/src/test/resources/cucumber.properties
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
cucumber.publish.quiet=true
19 changes: 19 additions & 0 deletions testing/src/test/resources/io/split/client/testing/cucumber/Split.feature
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,19 @@
@split[cappuccino:off]
@split[dollars:on]
Feature: Split
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Would you mind educating me how the unit test binds to the split listed in this file?

also, is the idea to have a single descritptor file or one per test class or test case?

Copy link
Author

@aslakhellesoy aslakhellesoy Jul 17, 2021 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not sure what you mean by unit test or descriptor file. I'll clarify some of the terminology used by Cucumber (which is closer to an acceptance testing tool than a unit testing tool).

A .feature file (sometimes called gherkin document) contains multiple scenarios (sometimes called examples).
Each Scenario is an acceptance test and consists of the following parts:

  • One or more Steps (Given, When, Then).
  • Step Definitions, which are Java methods annotated with @Given, When or @Then.

Cucumber creates a new instance of every class with step definitions before the scenario runs. This is similar to how JUnit creates a new instance of a test class for every @Test method it executes. However, the step definition is not a test - think of it as a method that is "invoked" by one of the scenario's steps.

For each step in the scenario, Cucumber finds a matching step definition (based on the expression on the annotation) and invokes the method.

Step definition classes aren't scoped to a particular .feature file. The steps in a .feature file can invoke step definitions from any step definition class.

Scenarios can have tags. The code in this library looks for tags of the format @split[feature:treatment] and uses that to register the appropriate treatments. This is done on a new instance of SplitClientForTest for every scenario.

This is an example of how to use Split with Cucumber

@split[cappuccino:on]
@split[dollars:off]
Scenario: with cappuccino, but without dollars
Then split "cappuccino" should be "on"
And split "dollars" should be "off"

Scenario: without cappuccino, but with dollars
Then split "cappuccino" should be "off"
And split "dollars" should be "on"

@split[dollars:off]
Scenario: without cappuccino, nor dollars
Then split "cappuccino" should be "off"
And split "dollars" should be "off"