📝 Update testing

* Rearrange testing section
* pytest vs unittest
* pytest monkeypatching
* Add behave
* Mocking with Typer
* Expand glossary

Author: veit

docs/appendix/glossary.rst

@@ -4,6 +4,12 @@ Glossary
 .. glossary::
    :sorted:
 
+   Acceptance test
+   User Acceptance Test
+       Verification that software functions as intended from the user’s
+       perspective and that users accept the software. Acceptance tests are
+       primarily used in :term:` Extreme Programming`.
+
    Argument
        A value that is passed to a function. There are two types of arguments:
 
@@ -80,6 +86,14 @@ Glossary
           * :ref:`pytest_fail`
           * :class:`python3:Exception`
 
+   Extreme Programming
+   XP
+       Software development methodology that aims to improve software quality
+       and responsiveness to changing customer requirements. As a form of agile
+       software development, it advocates frequent releases in short development
+       cycles to increase productivity and introduce control points where new
+       requirements can be taken into account.
+
    F-string
        :doc:`String </types/strings/built-in-modules/string>` literal preceded
        by an ``f`` or ``F``.
@@ -712,8 +726,27 @@ Glossary
 
    Test-driven development
    TDD
-       A software development strategy in which the tests are written before the
-       code.
+       A technique for creating software that guides software development by
+       writing tests. It was developed in the late 1990s by Kent Beck as part of
+       Extreme Programming. Essentially, it involves repeating three simple
+       steps:
+
+       * Write a test for the next feature to be added.
+       * Write the function code until the test passes.
+       * Refactor both the new and old code to make it well structured.
+
+       Although these three steps, often summarised as *‘red – green –
+       refactor’*, form the core of the process, there is also an important
+       first step, in which a list of test cases is created. One of these tests
+       is then selected, *‘Red – Green – Refactor’* is applied to it, and the
+       next test is selected. During the process, further tests are added to
+       this list.
+
+       .. seealso::
+          * `Canon TDD <https://tidyfirst.substack.com/p/canon-tdd>`_ by Kent
+            Beck
+          * `Test-driven development by example
+            <https://archive.org/details/est-driven-development-by-example/test-driven-development-by-example/>`_ by Kent Beck
 
    ``try``
        A keyword that protects a part of the code that can throw an

docs/test/behave/example.rst

@@ -0,0 +1,61 @@
+Example
+=======
+
+#. After installing behave, we can create a file called :file:`install.feature`
+   in the :file:`features` directory with the following content:
+
+   .. code-block:: gherkin
+
+      Feature: showing off behave
+
+        Scenario: run a simple test
+           Given we have behave installed
+            When we implement a test
+            Then behave will test it for us!
+
+   .. seealso::
+      `Features <https://behave.readthedocs.io/en/latest/tutorial/#features>`_
+
+#. Next, we create the file :file:`install.py` in the directory
+   :file:`features/steps`:
+
+   .. code-block:: python
+
+      from behave import *
+
+
+      @given("we have behave installed")
+      def step_impl(context):
+          pass
+
+
+      @when("we implement a test")
+      def step_impl(context):
+          assert True is not False
+
+
+      @then("behave will test it for us")
+      def step_impl(context):
+          assert context.failed is False
+
+   .. seealso::
+      `Python Step Implementations
+      <https://behave.readthedocs.io/en/latest/tutorial/#python-step-implementations>`_
+
+#. Call ``behave``
+
+   .. code-block:: console
+
+      $ behave
+      USING RUNNER: behave.runner:Runner
+      Feature: showing off behave # features/install.feature:1
+
+        Scenario: run a simple test        # features/install.feature:3
+          Given we have behave installed   # features/steps/install.py:3 0.000s
+          When we implement a test         # features/steps/install.py:7 0.000s
+          Then behave will test it for us  # features/steps/install.py:11 0.000s
+
+      1 feature passed, 0 failed, 0 skipped
+      1 scenario passed, 0 failed, 0 skipped
+      3 steps passed, 0 failed, 0 skipped
+      Took 0min 0.000s

docs/test/behave/index.rst

@@ -0,0 +1,59 @@
+behave
+======
+
+`behave <https://behave.readthedocs.io/en/latest/>`_ implements behaviour-driven
+development (:abbr:`BDD (Behavior-Driven Development)`). BDD is an agile
+software development technique that promotes collaboration between development,
+quality assurance and non-technical or business staff on a software project. The
+term was originally coined in 2003 by `Daniel Terhorst-North
+<https://dannorth.net/introducing-bdd>`_ in response to :term:`test-driven
+development <Test-driven development>` and encompasses :term:`acceptance testing
+<Acceptance test>` or customer-test-driven development practices as found in
+:term:`extreme programming <Extreme programming>`. In `Selling BDD to the
+Business <https://speakerdeck.com/tastapod/selling-bdd-to-the-business>`_, he
+gave the following definition:
+
+    *‚BDD is a second-generation, outside–in, pull-based, multiple-stakeholder,
+    multiple-scale, high-automation, agile methodology. It describes a cycle of
+    interactions with well-defined outputs, resulting in the delivery of
+    working, tested software that matters.‘*
+
+Gherkin
+-------
+
+Description language based on natural written language for the textual
+specification of software requirements. Only certain keywords are predefined.
+
+.. code-block:: gherkin
+
+   Scenario: Add an item to the database
+     Given an empty database
+      When an item with a summary is added
+      Then the number of items should be 1
+       and the queried item from the db should correspond to the added object.
+
+Each scenario is an example intended to illustrate a specific aspect of the
+application’s behaviour.
+
+Installation
+------------
+
+You can install behave in your :ref:`virtual environments <venv>` with:
+
+.. tab:: Linux/macOS
+
+   .. code-block:: console
+
+      $ python -m pip install behave
+
+.. tab:: Windows
+
+   .. code-block:: ps1con
+
+      C:> python -m pip install behave
+
+.. toctree::
+   :titlesonly:
+   :hidden:
+
+   example

docs/test/index.rst

@@ -19,8 +19,9 @@ Basically, a distinction is made between static and dynamic test procedures.
    :titlesonly:
    :hidden:
 
-   pytest/index
    unittest
+   pytest/index
+   behave/index
    mock
    hypothesis
    tox

docs/test/mock.rst

@@ -123,6 +123,9 @@ We can then simply use this fixture to test the version in
    def test_version(items_cli):
        assert items_cli("version") == items.__version__
 
+.. seealso::
+   `Typer Learn Testing <https://typer.tiangolo.com/tutorial/testing/>`_
+
 Mocking of attributes
 ---------------------
 

docs/test/pytest/builtin-fixtures.rst

@@ -270,38 +270,53 @@ code is restored and everything that was changed by the patch is undone.
 
 The ``monkeypatch`` fixture offers the following functions:
 
-+-------------------------------------------------------+-----------------------+
-| Function                                              | Description           |
-+=======================================================+=======================+
-| :samp:`setattr(TARGET, NAME, VALUE, raising=True)`    | sets an attribute     |
-| [1]_                                                  |                       |
-+-------------------------------------------------------+-----------------------+
-| :samp:`delattr(TARGET, NAME, raising=True)` [1]_      | deletes an attribute  |
-+-------------------------------------------------------+-----------------------+
-| :samp:`setitem(DICT, NAME, VALUE)`                    | sets a dict entry     |
-|                                                       |                       |
-+-------------------------------------------------------+-----------------------+
-| :samp:`delitem(DICT, NAME, raising=True)` [1]_        | deletes a dict entry  |
-+-------------------------------------------------------+-----------------------+
-| :samp:`setenv(NAME, VALUE, prepend=None)` [2]_        | sets an environment   |
-|                                                       | variable              |
-+-------------------------------------------------------+-----------------------+
-| :samp:`delenv(NAME, raising=True)` [1]_               | deletes an environment|
-|                                                       | variable              |
-+-------------------------------------------------------+-----------------------+
-| :samp:`syspath_prepend(PATH)`                         | expands the path      |
-|                                                       | ``sys.path``          |
-+-------------------------------------------------------+-----------------------+
-| :samp:`chdir(PATH)`                                   | changes the current   |
-|                                                       | working directory     |
-+-------------------------------------------------------+-----------------------+
++-----------------------------------------------+-----------------------+
+| Function                                      | Description           |
++===============================================+=======================+
+| :meth:`monkeypatch.setattr(obj, name, value,  | sets an attribute     |
+| raising=True)                                 |                       |
+| <pytest.MonkeyPatch.setattr>`                 |                       |
+| [1]_                                          |                       |
++-----------------------------------------------+-----------------------+
+| :meth:`monkeypatch.delattr(obj, name,         | deletes an attribute  |
+| raising=True)                                 |                       |
+| <pytest.MonkeyPatch.delattr>`                 |                       |
+| [1]_                                          |                       |
++-----------------------------------------------+-----------------------+
+| :meth:`monkeypatch.setitem(mapping, name,     | sets a dict entry     |
+| value) <pytest.MonkeyPatch.setitem>`          |                       |
++-----------------------------------------------+-----------------------+
+| :meth:`monkeypatch.delitem(obj, name,         | deletes a dict entry  |
+| raising=True) <pytest.MonkeyPatch.delitem>`   |                       |
+| [1]_                                          |                       |
++-----------------------------------------------+-----------------------+
+| :meth:`monkeypatch.setenv(name, value,        | sets an environment   |
+| prepend=None) <pytest.MonkeyPatch.setenv>`    | variable              |
+| [2]_                                          |                       |
++-----------------------------------------------+-----------------------+
+| :meth:`monkeypatch.delenv(name, raising=True) | deletes an environment|
+| <pytest.MonkeyPatch.delenv>`                  | variable              |
+| [1]_                                          |                       |
++-----------------------------------------------+-----------------------+
+| :meth:`monkeypatch.syspath_prepend(path)      | expands the path      |
+| <pytest.MonkeyPatch.syspath_prepend>`         | :py:data:`sys.path`   |
++-----------------------------------------------+-----------------------+
+| :meth:`monkeypatch.chdir(path)                | changes the current   |
+| <pytest.MonkeyPatch.chdir>`                   | working directory     |
++-----------------------------------------------+-----------------------+
+| :meth:`monkeypatch.context()                  | changes the current   |
+| <pytest.MonkeyPatch.context>`                 | context               |
++-----------------------------------------------+-----------------------+
 
 .. [1] The ``raising`` :term:`parameter` tells pytest whether an exception
        should be thrown if the element is not (yet) present.
 .. [2] The ``prepend`` :term:`parameter` of ``setenv()`` can be a character. If
        it is set, the value of the environment variable is changed to
        :samp:`{VALUE} + prepend + {OLD_VALUE}`
 
+RMonkey patching of environment variables
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
 We can use ``monkeypatch`` to redirect the :abbr:`CLI (Command Line Interface)`
 to a temporary directory for the database in two ways. Both methods require
 knowledge of the application code. Let’s take a look at the method
@@ -392,6 +407,44 @@ environment variable :envvar:`ITEMS_DB_DIR` that can be easily patched:
        monkeypatch.setenv("ITEMS_DB_DIR", str(tmp_path))
        assert run_items_cli("config") == str(tmp_path)
 
+Monkey patching dictionaries
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+The path could also have been specified in a dictionary, for example:
+
+.. code-block:: python
+   :caption: conf.py
+
+   DEFAULT_CONFIG = {"database": "items_db"}
+
+
+   def create_connection(config=None):
+       """Create a connection string from input or defaults."""
+       config = config or DEFAULT_CONFIG
+       return f"Location={config['database']};"
+
+For testing purposes, we can change the values in the ``DEFAULT_CONFIG``
+dictionary:
+
+.. code-block:: python
+   :caption: tests/test_conf.py
+
+   from items import conf
+
+
+   def test_connection(monkeypatch):
+       monkeypatch.setitem(conf.DEFAULT_CONFIG, "database", "test_db")
+
+Alternatively, you could have defined a fixture with:
+
+.. code-block:: python
+   :caption: tests/conftest.py
+
+   @pytest.fixture
+   def mock_test_database(monkeypatch):
+       """Set the DEFAULT_CONFIG database to test_db."""
+       monkeypatch.setitem(app.DEFAULT_CONFIG, "database", "test_db")
+
 Remaining built-in fixtures
 ---------------------------
 

docs/test/pytest/fixtures.rst

@@ -20,6 +20,13 @@ But before you familiarise yourself with fixtures and use them to test Items,
 let’s take a look at a small example fixture and learn how fixtures and test
 functions are connected.
 
+.. seealso::
+   * `pytest fixtures <https://docs.pytest.org/en/latest/explanation/fixtures.html>`_
+   * `pytest fixtures reference
+     <https://docs.pytest.org/en/latest/reference/fixtures.html>`_
+   * `How to use fixtures
+     <https://docs.pytest.org/en/latest/how-to/fixtures.html#how-to-fixtures>`_
+
 First steps with fixtures
 -------------------------
 

docs/test/pytest/index.rst

@@ -4,15 +4,25 @@ pytest
 :doc:`pytest <pytest:index>` is an alternative to Python’s :doc:`../unittest`
 module that simplifies testing even further.
 
-Features
---------
-
-* More detailed information about failed ``assert`` statements
-* Automatic detection of test modules and functions
-* Modular fixtures for the management of small or :term:`parameterised
-  <Parameter>`, long-lived test resources
-* Can also execute unit tests without presets
-* Extensive plug-in architecture, with over 800 external plug-ins
+* pytest automatically recognises tests based on filenames and functions that
+  start with ``test_``, while unittest derives test classes and methods from
+  :class:`unittest.TestCase`. This results in simpler, more readable syntax with
+  less boilerplate code.
+* unittest provides a set of assertion methods (for example,
+  :func:`assertEqual`, :func:`assertTrue`, :func:`assertRaises`). With pytest,
+  the same assertions can be defined, but using Python’s standard :func:`assert`
+  statement. This often results in more meaningful error messages and better
+  introspection.
+* unittest only provides :func:`setUp` and :func:`tearDown` methods for
+  fixtures. In pytest, on the other hand, :doc:`fixtures <fixtures>` are defined
+  as functions, which promotes reusability and simplifies the management of
+  test dependencies.
+* Parametrised tests are possible in unittest, but require additional effort.
+  pytest, however, includes the :doc:`decorator  <../../functions/decorators>`
+  ``@pytest.mark.parametrize``, which makes it easy to run test functions with
+  different inputs and expected outputs.
+* pytest has an extensive ecosystem with over 800 :doc:`plugins` for advanced
+  testing requirements; unittest is more limited in its extensibility.
 
 Installation
 ------------

docs/test/unittest.rst

@@ -15,16 +15,6 @@ It provides the following test concepts:
    Test Fixture
        is a consistent test environment.
 
-       .. seealso::
-          * `pytest fixtures
-            <https://docs.pytest.org/en/latest/explanation/fixtures.html>`_
-          * `About fixtures
-            <https://docs.pytest.org/en/latest/explanation/fixtures.html#about-fixtures>`_
-          * `Fixtures reference
-            <https://docs.pytest.org/en/latest/reference/fixtures.html>`_
-          * `How to use fixtures
-            <https://docs.pytest.org/en/latest/how-to/fixtures.html#how-to-fixtures>`_
-
    Test Suite
        is a collection of several :term:`test cases <Test Case>`.