Add "Plugin marketplace" infrastructure to hermes docs and codebase

.github/ISSUE_TEMPLATE/add-plugin-to-marketplace.yml

@@ -0,0 +1,76 @@
+# SPDX-FileCopyrightText: 2024 Helmholtz-Zentrum Dresden-Rossendorf
+# SPDX-License-Identifier: CC-BY-SA-4.0
+# SPDX-FileContributor: David Pape
+
+name: "Add Plugin to Marketplace"
+description: "I want to add a plugin to the Hermes plugin marketplace."
+title: "[New Plugin]: "
+labels: ["documentation"]
+
+body:
+  - type: markdown
+    attributes:
+      value: |
+        Thank you for building a plugin for Hermes and sharing it with the community!
+
+        The [Hermes plugin marketplace](https://hermes.software-metadata.pub#plugins) is a curated list of plugins on our website that allows you to share plugins that you developed, and others to find them.
+
+        Via this issue template, you can send us the required information to add your plugin to the marketplace. Alternatively, you may file a pull request, adding the plugin to [`plugins.json`](https://github.com/softwarepub/hermes/tree/develop/docs/source/plugins.json) yourself.
+
+  - id: "name"
+    type: "input"
+    attributes:
+      label: "Name"
+      description: "The name of the plugin"
+      placeholder: "Foobar Harvesting and Quux Deposit Plugin"
+    validations:
+      required: true
+
+  - id: "author"
+    type: "input"
+    attributes:
+      label: "Author"
+      description: "The author of the plugin, usually a team or organization"
+      placeholder: "Team Quux at Fizzbuzz Institute"
+
+  - id: "description"
+    type: "textarea"
+    attributes:
+      label: "Description"
+      description: "A short description of your plugin"
+      placeholder: "Plugin for harvesting foobar files and uploading deposits to quux repo."
+
+  - id: "steps"
+    type: "dropdown"
+    attributes:
+      label: "Steps"
+      description: "Steps of the Hermes workflow targeted by your plugin"
+      multiple: true
+      options: ["harvest", "process", "curate", "deposit", "postprocess"]
+
+  - id: "harvested-files"
+    type: "input"
+    attributes:
+      label: "Harvested files"
+      description: "The types of files your plugin harvests (if it is a harvest plugin)"
+      placeholder: "foobar, foobar.yml, foobar.json"
+
+  - id: "repository-url"
+    type: "input"
+    attributes:
+      label: "Repository"
+      description: "The link to the repository where users can find and inspect the source code of your plugin"
+      placeholder: "https://git.example.com/quux/hermes-plugin-quux"
+
+  - id: "pypi-url"
+    type: "input"
+    attributes:
+      label: "PyPI URL"
+      description: "The link to your project on PyPI"
+      placeholder: "https://pypi.org/project/hermes-plugin-quux/"
+
+  - id: "comments"
+    type: "textarea"
+    attributes:
+      label: "Comments"
+      description: "Any additional comments you would like to add"

docs/source/_ext/plugin_markup.py

@@ -0,0 +1,128 @@
+# SPDX-FileCopyrightText: 2024 Helmholtz-Zentrum Dresden-Rossendorf
+# SPDX-License-Identifier: Apache-2.0
+# SPDX-FileContributor: David Pape
+
+import json
+import re
+from pathlib import Path
+from typing import Any, Dict
+
+from docutils import nodes
+from jsonschema import validate
+from sphinx.application import Sphinx
+from sphinx.util import logging
+from sphinx.util.console import colorize
+from sphinx.util.docutils import SphinxDirective
+
+from hermes.commands.marketplace import (
+    SchemaOrgOrganization,
+    SchemaOrgSoftwareApplication,
+    schema_org_hermes,
+)
+
+
+logger = logging.getLogger(__name__)
+
+
+def log_message(text: str, detail: str = None) -> None:
+    message = colorize("bold", "[Plugin Markup]") + " " + text
+    if detail is not None:
+        message += " " + colorize("darkgreen", detail)
+    logger.info(message)
+
+
+def keywordify(text: str) -> str:
+    """Make keyword-friendly text.
+
+    The result will only contain lowercase a through z and hyphens, e.g.:
+
+    * CITATION.cff → citation-cff
+    * codemeta.json → codemeta-json
+    * LICENSE → license
+    """
+    text = text.casefold()
+    return re.sub(r"[^a-z]", "-", text)
+
+
+def plugin_to_schema_org(plugin: Dict[str, Any]) -> SchemaOrgSoftwareApplication:
+    """Convert plugin metadata from the used JSON format to Schema.org.
+
+    The ``plugin`` is transformed into a ``schema:SoftwareApplication``. For most
+    attributes of the plugin, a mapping into Schema.org terms is performed. The author
+    is expressed as a ``schema:Organization`` using the given author field as the
+    ``name``. The steps targeted by the plugin are expressed using the ``keyword`` field
+    by transforming them to the keywords ``hermes-step-<STEP>`` where ``<STEP>`` is the
+    name of the workflow step. The ``harvested_files`` are also transformed into
+    keywords by making the text "keyword-friendly" and prepending ``hermes-harvest-``.
+    If the plugin is marked as a Hermes ``builtin``, this is expressed using
+    ``schema:isPartOf``.
+    """
+    steps = plugin.get("steps", [])
+    keywords = [f"hermes-step-{step}" for step in steps]
+    if "harvest" in steps:
+        harvested_files = plugin.get("harvested_files", [])
+        keywords += [f"hermes-harvest-{keywordify(file)}" for file in harvested_files]
+
+    return SchemaOrgSoftwareApplication(
+        name=plugin.get("name"),
+        url=plugin.get("repository_url"),
+        install_url=plugin.get("pypi_url"),
+        abstract=plugin.get("description"),
+        author=SchemaOrgOrganization(name=au) if (au := plugin.get("author")) else None,
+        is_part_of=schema_org_hermes if plugin.get("builtin", False) else None,
+        keywords=keywords or None,
+    )
+
+
+class PluginMarkupDirective(SphinxDirective):
+    """A Sphinx directive to render the ``plugins.json`` file to Schema.org markup.
+
+    The plugins file and its jsonschema are passed to the directive as parameters, i.e.,
+    in Markdown:
+
+    .. code-block:: markdown
+
+       ```{plugin-markup} path/to/plugins.json path/to/plugins-schema.json
+       ```
+
+    For each plugin listed in the file, a ``<script type="application/ld+json">`` tag
+    is generated.
+    """
+
+    required_arguments = 2
+
+    def run(self) -> list[nodes.Node]:
+        filename = Path(self.get_source_info()[0])  # currently processing this file
+        directory = filename.parent
+
+        plugins_file = directory / self.arguments[0]
+        log_message("reading plugins file", detail=str(plugins_file))
+        with open(plugins_file) as file:
+            plugin_data = json.load(file)
+
+        plugins_schema_file = directory / self.arguments[1]
+        log_message("reading plugins schema file", detail=str(plugins_schema_file))
+        with open(plugins_schema_file) as file:
+            plugin_schema = json.load(file)
+
+        log_message("validating plugins")
+        validate(plugin_data, plugin_schema)
+
+        log_message("converting plugins to markup")
+        tags = []
+        for plugin in plugin_data:
+            markup = plugin_to_schema_org(plugin).model_dump_jsonld()
+            tag = f'<script type="application/ld+json">{markup}</script>'
+            tags.append(nodes.raw(text=tag, format="html"))
+
+        return tags
+
+
+def setup(app: Sphinx):
+    """Wire up the directive so that it can be used as ``plugin-markup``."""
+    app.add_directive("plugin-markup", PluginMarkupDirective)
+    return {
+        "version": "0.1",
+        "parallel_read_safe": True,
+        "parallel_write_safe": True,
+    }

docs/source/_templates/plugins.md

@@ -0,0 +1,33 @@
+{#
+SPDX-FileCopyrightText: 2024 Helmholtz-Zentrum Dresden-Rossendorf
+SPDX-License-Identifier: CC-BY-SA-4.0
+SPDX-FileContributor: David Pape
+#}
+
+{% for step in ("harvest", "process", "curate", "deposit", "postprocess") %}
+
+### {{ step|title }}
+
+<ul>
+{%- for plugin in data -%}
+  {%- if step in plugin.steps -%}
+  <li style="margin-top: 0.5rem;">
+    {%- if plugin.repository_url -%}
+    <a href="{{ plugin.repository_url }}" rel="nofollow">{{ plugin.name }}</a>
+    {%- else -%}
+    {{ plugin.name }}
+    {%- endif -%}
+    <span style="color: gray;"> by <em>{{ plugin.author }}</em><br></span>
+    {%- if plugin.description -%}
+    {{ plugin.description }}<br>
+    {%- endif -%}
+    {%- if plugin.builtin -%}
+    <span style="color: gray;">This plugin is built into Hermes.</span><br>
+    {%- elif plugin.pypi_url -%}
+    <span style="color: gray;">Install via <a href="{{ plugin.pypi_url }}">PyPI</a>.</span><br>
+    {%- endif -%}
+  </li>
+  {%- endif -%}
+{%- endfor -%}
+</ul>
+{% endfor %}

docs/source/conf.py

@@ -5,6 +5,7 @@
 # SPDX-FileContributor: Stephan Druskat
 # SPDX-FileContributor: Oliver Bertuch
 # SPDX-FileContributor: Michael Meinel
+# SPDX-FileContributor: David Pape
 
 
 # Configuration file for the Sphinx documentation builder.
@@ -22,14 +23,16 @@
 import os
 import sys
 sys.path.insert(0, os.path.abspath('../../src'))
+sys.path.append(os.path.abspath('_ext'))
 
 
 # -- Project information -----------------------------------------------------
 
 project = 'HERMES Workflow'
 copyright = '2024, HERMES project'
 author = 'Oliver Bertuch, Stephan Druskat, Guido Juckeland, Jeffrey Kelling, ' + \
-         'Oliver Knodel, Michael Meinel, Tobias Schlauch, Sophie Kernchen'
+         'Oliver Knodel, Michael Meinel, Tobias Schlauch, Sophie Kernchen, ' + \
+         'David Pape'
 
 
 # The full version, including alpha/beta/rc tags
@@ -60,6 +63,9 @@
     'autoapi.extension',
     'sphinxcontrib.mermaid',
     'sphinx_togglebutton',
+    'sphinxcontrib.datatemplates',
+    # Custom extensions, see `_ext` directory.
+    'plugin_markup',
 ]
 
 language = 'en'

docs/source/index.md

@@ -8,6 +8,7 @@ SPDX-License-Identifier: CC-BY-SA-4.0
 SPDX-FileContributor: Oliver Bertuch
 SPDX-FileContributor: Stephan Druskat
 SPDX-FileContributor: Michael Meinel
+SPDX-FileContributor: David Pape
 -->
 
 ![](_static/img/header.png)
@@ -19,22 +20,34 @@ This is work in progress.
 ```
 
 Research software must be formally published to satisfy FAIR principles and unlock academic credit. Publication
-repositories enable this and provide PIDs for software versions, but only through tedious, mostly manual process. 
+repositories enable this and provide PIDs for software versions, but only through tedious, mostly manual process.
 
 The HERMES workflow enables automated publication of rich research software metadata and artifacts to publication
-repositories using open source tooling. 
+repositories using open source tooling.
 
-We follow a *push based* model and run in continuous integration (CI) infrastructures integrated in common code platforms 
-such as GitHub or GitLab to avoid going out of service and overcome limitations of pull-based web services. 
+We follow a *push based* model and run in continuous integration (CI) infrastructures integrated in common code platforms
+such as GitHub or GitLab to avoid going out of service and overcome limitations of pull-based web services.
 
 Rich descriptive metadata is the key element to useful software publications. We harvest existing metadata from source
 code repos and connected platforms, then process, collate and present them for curation, thus preparing software for
-automatic submission to publication repositories. 
+automatic submission to publication repositories.
 
 ![](_static/img/workflow-overview.svg)
 
+## Plugins
+
+```{plugin-markup} plugins.json plugins-schema.json
+```
+
+Hermes is built to be extensible for your needs.
+This is a list of available plugins for the different steps in the Hermes workflow:
+
+```{datatemplate:json} plugins.json
+:template: plugins.md
+```
+
 ## Documentation
- 
+
 <!--
 ```{toctree}
  cli
@@ -69,13 +82,13 @@ Concept Paper <https://arxiv.org/abs/2201.09015>
 
 This is an open repository to collect feedback on the HERMES workflow.
 
-We see our project as part of a global and inter-disciplinary effort to improve the state of the art in 
+We see our project as part of a global and inter-disciplinary effort to improve the state of the art in
 research software engineering, maintenance and scholarly communications around research software. We therefore
 appreciate any feedback you may have on the HERMES project itself and any of its outputs.
 
 **How to give feedback**
 
-Either [create an issue](https://github.com/hermes-hmc/workflow/issues/new/choose) in our project repository or 
+Either [create an issue](https://github.com/hermes-hmc/workflow/issues/new/choose) in our project repository or
 [send us an email](mailto:[email protected]?subject=HERMES%20WOrkflow%20Reachout).
 
 ## Acknowledgements
@@ -92,4 +105,4 @@ in the framework of the [Helmholtz Metadata Collaboration](https://helmholtz-met
 
 * [](genindex)
 * [](modindex)
-* [](search)
+* [](search)