Merge pull request #85 from jroper/tck-docs

Added documentation for the TCK to the specification

streams/spec/src/main/asciidoc/microprofile-reactive-streams-operators-spec.asciidoc

@@ -41,3 +41,5 @@ include::examples.asciidoc[]
 include::java-streams.asciidoc[]
 
 include::spi.asciidoc[]
+
+include::tck.asciidoc[]

streams/spec/src/main/asciidoc/tck.asciidoc

@@ -0,0 +1,101 @@
+//
+// Copyright (c) 2018 Contributors to the Eclipse Foundation
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+//     http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+//
+
+[[reactivestreamstck]]
+== TCK
+
+The MicroProfile Reactive Streams Operators TCK is provided to test compliance of implementations to the specification.
+It provides a mechanism both for testing implementations independent from a MicroProfile container, as well as implementations provided by a container.
+
+=== Structure
+
+There are two parts to the TCK, an API TCK, and an SPI TCK.
+In addition, the whole TCK can either be run directly against an SPI implementation, or against a container with an Arquillian adapter.
+
+The TCK uses TestNG for testing, and contains many test classes.
+These test classes are brought together using TestNG `@Factory` annotated methods, so that running the TCK can be done by simply running a single test class that is a factory for the rest of the test classes.
+
+Additionally, this TCK also uses the https://github.com/reactive-streams/reactive-streams-jvm/tree/master/tck[Reactive Streams TCK].
+The Reactive Streams TCK is used to test Reactive Streams `Publishers`, `Subscribers` and `Processors` to ensure that they conform to the Reactive Streams spec.
+Wherever possible, this TCK will use it to verify the `Publishers`, `Subscribers` and `Processors` built by the API.
+For example, the returned `Processor` built by building a simple `map` stage is run through the Reactive Streams `Processor` TCK verification.
+
+It should be noted that the Reactive Streams TCK is structured in such a way that each numbered requirement in the Reactive Streams specification has a test, even if that requirement is untestable by the TCK, or if its optional. In the case where requirements are optional or untested, the Reactive Streams TCK skips the test. Consequently, when running this TCK, there are a significant number of skipped tests.
+
+==== API TCK
+
+This tests the API, to ensure that every API call results in the a graph with the right stages being built.
+It also tests that the necessary validation, for example for nulls, is done on each API call.
+The API TCK is particularly useful for clean room implementations that don't depend on the Eclipse MicroProfile Reactive Streams Operators API artifact.
+Passing the API TCK ensures that different implementations of the SPI will be compatible with different implementations of the API.
+
+The API TCK doesn't need an implementation of the SPI to run, and so can be run against any implementation of the API.
+It is run as part of the Eclipse MicroProfile Reactive Streams Operators API's continuous integration.
+
+===== Running the API TCK
+
+The API TCK can be run by running the `org.eclipse.microprofile.reactive.streams.tck.api.ReactiveStreamsApiVerification` class.
+For convenience, implementations might decide to subclass this in their `src/test/java` folder, so that it gets automatically picked up and run by build tools like Maven.
+
+==== SPI TCK
+
+This tests implementations of the SPI, to ensure that each different stage defined by the SPI behaves correctly, and to ensure that the implementation of that stage conforms to the Reactive Streams specification.
+In general, each stage has a verification test, which tests the behavior of that stage, error, completion and cancellation propagation, and anything else that is necessary for each stage.
+Additionally, each stage defines one or more Reactive Streams TCK verification classes, which tests that the `Publisher`, `Processor` or `Subscriber` built by a graph that contains just that stage conforms to the Reactive Streams specification.
+
+===== Running the SPI TCK
+
+The SPI TCK can be run by running the whole TCK (this includes running the API TCK, since the SPI TCK uses the API to create the stages, it doesn't make sense to verify an SPI without verifying that the API on top of it is doing the right thing too).
+
+The whole TCK can be run by creating a subclass of `org.eclipse.microprofile.reactive.streams.tck.ReactiveStreamsTck`.
+This requires passing an instance of `org.reactivestreams.tck.TestEnvironment` to the super constructor, which contains configuration like timeouts.
+Typically, the timeouts set by using the default constructor for `TestEnvironment` should be satisfactory.
+The `createEngine` method also needs to be implemented, this must create and return a `ReactiveStreamsEngine` for the TCK to test against.
+Optionally, a `shutdownEngine` method can be overridden to shutdown the engine if it holds any resources like thread pools.
+
+.An example of using the TCK
+====
+[source, java]
+----
+public class MyReactiveStreamsTckTest extends ReactiveStreamsTck<MyEngine> {
+
+  public MyReactiveStreamsTckTest() {
+    super(new TestEnvironment());
+  }
+
+  @Override
+  protected MyEngine createEngine() {
+    return new MyEngine();
+  }
+
+  @Override
+  protected void shutdownEngine(MyEngine engine) {
+    engine.shutdown();
+  }
+}
+----
+====
+
+==== Running the TCK in a container
+
+A test class for running the TCK in a MicroProfile container is provided so that containers can verify compliance with the spec.
+This container verification comes in the `org.eclipse.microprofile.reactive.streams:microprofile-reactive-streams-operators-tck-arquillian` artifact, which can be added as a dependency to a project that wishes to use it.
+
+To run this TCK, the container must provide a `ReactiveStreamsEngine` to be tested as an injectable `ApplicationScoped` bean, and the MicroProfile Reactive Streams Operators API must be on the classpath.
+Having ensured this, the TCK can then be run by executing `org.eclipse.microprofile.reactive.streams.tkc.arquillian.ReactiveStreamsArquillianTck`.
+This class deploys the TCK to an Arquillian compatible container, and then runs all the tests in the container in its own configured TestNG suite on the container.
+
+For convenience, implementations may want to subclass this class in their own `src/test/java` class, so that it can automatically be run by build tools like Maven.