Add guide on creating package docs (#4196)

* Add guide on creating package API docs

Signed-off-by: Jonas Otto <[email protected]>
Co-authored-by: Chris Lalancette <[email protected]>
(cherry picked from commit 84dd416)

source/How-To-Guides.rst

@@ -22,6 +22,7 @@ If you are new and looking to learn the ropes, start with the :doc:`Tutorials <T
 
    How-To-Guides/Installation-Troubleshooting
    How-To-Guides/Developing-a-ROS-2-Package
+   How-To-Guides/Documenting-a-ROS-2-Package
    How-To-Guides/Ament-CMake-Documentation
    How-To-Guides/Ament-CMake-Python-Documentation
    How-To-Guides/Migrating-from-ROS1

source/How-To-Guides/Documenting-a-ROS-2-Package.rst

@@ -0,0 +1,245 @@
+Documenting a ROS 2 package
+###########################
+
+.. contents:: Table of Contents
+   :depth: 2
+   :local:
+
+
+This guide introduces the standard way to create documentation for ROS 2 packages.
+For packages with binary releases this also results in the docs being hosted at ``docs.ros.org/en/<distro>/p/<package>/``.
+For information on how to contribute to this documentation on docs.ros.org, see :doc:`Contributing to ROS 2 Documentation <../The-ROS2-Project/Contributing/Contributing-To-ROS-2-Documentation>`.
+
+Prerequisites
+-------------
+
+- :doc:`Install ROS <../../Installation>`
+
+- `Install rosdoc2 <https://github.com/ros-infrastructure/rosdoc2#installation>`__
+
+Package Documentation Overview
+------------------------------
+
+The type of documentation discussed in this guide is referred to as "package docs" or "API docs".
+For ROS packages that have been released on ROS Index, their documentation will be built on the ROS buildfarm, included on docs.ros.org, and visible via the ``API Docs`` button on index.ros.org.
+
+The tool responsible for generating ROS 2 package docs is `rosdoc2 <https://github.com/ros-infrastructure/rosdoc2>`__.
+
+``rosdoc2`` is a convenient wrapper around the commonly used `Sphinx <https://www.sphinx-doc.org/>`__ documentation framework.
+Sphinx allows freeform written documentation as well as API documentation for python code generated from comments in the code.
+The `breathe <https://breathe.readthedocs.io/en/latest/>`__ + `exhale <https://exhale.readthedocs.io/en/latest/>`__ packages allow integration with Doxygen, to include autogenerated C++ API documentation as well.
+
+``rosdoc2`` creates a default configuration for packages with no documentation or configuration at all, and applies options for a uniform theme and integration with other packages.
+
+Building Package Docs
+---------------------
+
+To generate the documentation for a package in HTML format with ``rosdoc2``, run:
+
+.. code-block:: console
+
+   rosdoc2 build --package-path <package-path>
+
+The documentation is written to ``docs_output/<package-name>/index.html`` and can be viewed in a browser.
+
+Configuration
+-------------
+
+There are three configuration locations for ROS package docs: ``rosdoc2.yaml`` for general settings,
+``conf.py`` for sphinx settings and the ``Doxyfile`` for doxygen settings.
+For all of those, a default is assumed or generated if not present, so none of them is strictly required.
+However, it might be necessary to create and modify those once you want to use features such as custom textual documentation pages.
+
+rosdoc2.yaml
+^^^^^^^^^^^^
+
+This is the main entrypoint for rosdoc2.
+It specifies generic settings, and can be used to control the execution of specific builders (Doxygen and Sphinx) and decides which builders to run.
+
+``rosdoc2`` provides a multitude of configuration options, which can be adjusted in a config file ``rosdoc2.yaml``.
+To generate a default ``rosdoc2.yaml`` which you can then further customize, run:
+
+.. code-block:: console
+
+   rosdoc2 default_config --package-path <package-path>
+
+And add ``<rosdoc2>rosdoc2.yaml</rosdoc2>`` to the export section in your ``package.xml``:
+
+.. code-block:: xml
+
+    <package>
+        <!-- [...] -->
+        <export>
+            <!-- [...] -->
+            <rosdoc2>rosdoc2.yaml</rosdoc2>
+        </export>
+    </package>
+
+However, for most packages,  the default settings in ``rosdoc2`` will suffice, and no custom config is necessary.
+More information about ``rosdoc2.yaml`` can be found in the `rosdoc2 readme <https://github.com/ros-infrastructure/rosdoc2#using-a-rosdoc2yaml-file-to-control-how-your-package-is-documented>`__.
+
+conf.py, rosdoc2_settings
+^^^^^^^^^^^^^^^^^^^^^^^^^
+
+The final output of the package docs is (almost) always built by Sphinx.
+Each Sphinx project is configured by a ``conf.py`` file in the ``doc`` directory.
+If no configuration is present, a default Sphinx project is created and used when building the documentation.
+If however a ``conf.py`` Sphinx config is found in the ``doc`` subdirectory of the package, this is used instead.
+A custom Sphinx project is required if you want to include a standalone reStructuredText documentation page.
+A standalone documentation page can be used to list multiple tutorials and guides; if that's something you want for your package you'll need to create a custom Sphinx project.
+
+``rosdoc2`` provides additional settings to ``conf.py`` and overrides some.
+Information about changes done to the Sphinx settings are logged to the console with a ``[rosdoc2]`` prefix.
+
+Doxyfile
+^^^^^^^^
+
+Doxygen is a tool for automatically generating C++ API docs from code comments.
+While Doxygen can also generate HTML output directly, in the usual workflow for ROS packages, Doxygen produces machine readable output in XML format which is then consumed by Sphinx and integrated with the rest of the documentation.
+Doxygen-only docs are possible by only enabling the Doxygen builder in ``rosdoc2.yaml``, but this is uncommon.
+
+Customizing Sphinx Documentation
+--------------------------------
+
+Creating a Sphinx Project
+^^^^^^^^^^^^^^^^^^^^^^^^^
+
+In order to add standalone documentation pages in addition to the automatically generated API docs, a custom Sphinx project is necessary.
+This should be created in a subdirectory called ``doc`` in the package directory.
+A new Sphinx project can be created by running ``sphinx-quickstart`` in the ``doc`` directory, answering ``no`` to "Separate source and build directories".
+The wizard requires entering the project name, author and version, but this can later be removed and will be provided to Sphinx by ``rosdoc2`` from your packages ``package.xml``.
+More information about creating a sphinx project can be found on the `Sphinx quickstart page <https://www.sphinx-doc.org/en/master/usage/quickstart.html>`__,
+
+Customizing ``index.rst``
+^^^^^^^^^^^^^^^^^^^^^^^^^
+
+The ``sphinx-quickstart`` wizard creates an ``index.rst`` file, which is the custom landing page for your package, similar to a Github ``README`` file.
+
+Adding Python API-Docs
+^^^^^^^^^^^^^^^^^^^^^^
+
+By default ``rosdoc2``  uses the `sphinx-apidoc tool <https://www.sphinx-doc.org/en/master/man/sphinx-apidoc.html>`__ and the `autodoc Sphinx extension <https://www.sphinx-doc.org/en/master/usage/extensions/autodoc.html>`__ to automatically generate documentation for python code.
+In order for autodoc to find the Python modules in your package, it must be added to the python search path in ``conf.py``:
+
+.. code-block:: python
+
+    sys.path.insert(0, os.path.abspath('.'))
+
+This is because ``rosdoc2`` wraps the custom ``conf.py`` with more configuration from a script which will be placed in the package.
+In this case the ``.``  path in ``os.path.abspath`` refers to the package's directory root, not the package's ``doc`` directory due to the interaction between rosdoc2 and ``conf.py``.
+
+By default, package API docs are already reachable through the "Module Index" link that is present on the landing page.
+For the API docs to also appear in the table of contents, simply add a link to the ``modules`` page to your ``index.rst``:
+
+.. code-block:: rst
+
+    .. toctree::
+       :maxdepth: 2
+       :caption: Contents:
+
+       Python Modules <modules>
+
+Adding C++ API-Docs
+^^^^^^^^^^^^^^^^^^^
+
+If you would like to add your automatically generated API docs back to your custom landing page, add the line ``generated/index``  to your documentation page where you would like the API docs to appear:
+
+.. code-block:: rst
+
+   .. toctree::
+      :maxdepth: 2
+
+      C++ API Docs <generated/index>
+
+This adds the elements "Class Hierarchy", "File Hierarchy" and "Reference" to the table of contents in the sidebar.
+To make those appear under one "C++ API Docs" heading for a less cluttered sidebar, a separate file such as ``cpp_api_docs.rst`` can be added, which links to the generated docs:
+
+.. code-block:: rst
+    :caption: cpp_api_docs.rst
+
+    C++ API Docs
+    ============
+
+    These are the autogenerated docs for the internal implementation.
+
+    .. toctree::
+       :maxdepth: 3
+       :caption: Contents:
+
+       generated/index
+
+Which then also needs to be added in  ``index.rst`` to appear in the sidebar:
+
+.. code-block:: rst
+    :caption: index.rst
+
+    .. toctree::
+       :maxdepth: 2
+       :caption: Contents:
+
+       cpp_api_docs
+
+
+Including an existing README.md
+-------------------------------
+
+If your git repository already has an existing ``README.md``, it is possible to reuse this as the landing page for the documentation, without duplicating the contents.
+To correctly include a Markdown file in Sphinx while preserving relative links and images, some additional effort is required.
+
+First, create a proxy-file ``readme_include.md`` next to ``index.rst``.
+This is a markdown file which just includes the original README.md, but preserves the relative image paths, which would otherwise break in the next step:
+
+.. code-block:: markdown
+    :caption: readme_include.md
+
+    ```{include} ../README.md
+    :relative-images:
+    ```
+
+
+Then, include the contents of this file from ``index.rst`` using ``myst`` to include markdown from rst:
+
+.. code-block:: rst
+    :caption: index.rst
+
+    .. include:: readme_include.md
+       :parser: myst_parser.sphinx_
+
+
+This also requires adding ``myst_parser`` to the extensions in ``conf.py``:
+
+.. code-block:: python
+    :caption: conf.py
+
+    extensions = ["myst_parser"]
+
+
+CI, docs.ros.org
+----------------
+
+The ROS build farm uses ``rosdoc2`` to build the package documentation hosted at ``docs.ros.org/en/<distro>/p/<package>/``.
+To enable this, the repository containing the documentation must be configured in `rosdistro/{DISTRO}/distribution.yaml <https://github.com/ros/rosdistro/blob/master/{DISTRO}/distribution.yaml>`__.
+This would usually be the package source repository:
+
+.. code-block:: yaml
+
+      <package_name>:
+        doc:
+          type: git
+          url: https://github.com/<github_username>/<package_name>.git
+          version: main
+        release:
+        [...]
+
+The buildfarm hosts the documentation for every distribution separately, and periodically rebuilds it from the latest commit on the specified branch.
+It is not required to tag a new release to update the hosted documentation.
+To view the status of your package's documentation build, search for ``doc__<package_name>`` on `<https://build.ros2.org>`__.
+One job is created for every distribution for which the package is released.
+On each job page, you can see when a build was last triggered, as well as the status and logs of each build.
+
+Further Reading
+---------------
+
+* ``rosdoc2`` readme: https://github.com/ros-infrastructure/rosdoc2/blob/main/README.md
+* ROS 2 design document on package documentation: https://design.ros2.org/articles/per_package_documentation.html
+* The ROS 2 cookbook: https://github.com/mikeferguson/ros2_cookbook/blob/main/pages/rosdoc2.md