RHDH Dynamic Plugin Factory

A comprehensive tool for building and exporting dynamic plugins for Red Hat Developer Hub (RHDH) from Backstage plugin source code.

Overview

The RHDH Plugin Factory automates the process of converting Backstage plugins into RHDH dynamic plugins. It provides:

Source Repository Management: Clone and checkout plugin source repositories
Patch & Overlay System: Apply custom modifications to plugin source code before exporting
Dependency Management: Automated yarn installation with TypeScript compilation
Dynamic Plugin Packaging: Build, export and package plugins using the RHDH CLI
Container Image Publishing: Optionally push to container registries (Quay, OpenShift, etc.)

Prerequisites

Local Development

Python: 3.8 or higher
Node.js: 22 or higher (specified in default.env)
Yarn: Latest version via Corepack
Git: For cloning and checking out remote git repositories
Buildah: For building and pushing container images (if using --push-images)

Installation

You can use the RHDH Plugin Factory either locally or via a container image.

Local Setup

Clone the repository:

git clone https://github.com/redhat-developer/rhdh-dynamic-plugin-factory
cd rhdh-dynamic-plugin-factory

Install Python dependencies:

pip install -r requirements.txt -r requirements.dev.txt

Set up custom environment variables:

# The default.env file contains required version settings
# Copy and customize if needed
cp default.env .env

Container Image Setup

Using Pre-built Community Images

Pre-built container images are published to quay.io/rhdh-community/dynamic-plugins-factory with tags corresponding to the version of RHDH they were designed for:

# Pull the latest version
podman pull quay.io/rhdh-community/dynamic-plugins-factory:latest

# Or pull a specific RHDH version
podman pull quay.io/rhdh-community/dynamic-plugins-factory:1.8

Building the Image

Alternatively, you can build the container image locally using Podman or Docker:

podman build -t rhdh-dynamic-plugin-factory:latest .

Or with Docker:

docker build -t rhdh-dynamic-plugin-factory:latest .

Container Requirements

The container requires specific capabilities and device access for building dynamic plugins:

Volume Mounts: Mount your configuration, workspace, and output directories to the /config, /workspace and /outputs directories respectively
Device Access: Mount /dev/fuse for filesystem operations
SELinux Context: Use :z flag for volume mounts on SELinux-enabled systems when using podman

Basic Container Usage

podman run --rm -it \
  --device /dev/fuse \
  -v ./config:/config:z \
  -v ./workspace:/workspace:z \
  -v ./outputs:/outputs:z \
  quay.io/rhdh-community/dynamic-plugins-factory:latest \

Key Differences from Local Usage:

Paths for --config-dir, --repo-path, and --output-dir don't need to be defined since they use the default values of /config, /workspace and /outputs respectively.
Usage of volume mounts to map your local directories to these container paths
The --device /dev/fuse flag is required for buildah operations inside the container
Use :z or :Z SELinux labels when running on RHEL/Fedora/CentOS systems

Configuration

Directory Structure

The factory expects the following directory structure:

./
├── config/                                   # Configuration directory (set with --config-dir)
│   ├── .env                                  # Optional: Override environment variables
│   ├── source.json                           # Source repository configuration
│   ├── plugins-list.yaml                     # List of plugins to build
│   ├── patches/                              # Optional: Patch files to apply
│   └── <path-to-plugin-in-workspace>/overlays/   # Optional: Files to overlay on plugin source
├── workspace/                                # Source code location (set with --repo-path)
└── outputs/                                  # Build output directory (set with --output-dir)

Configuration Files

1. `default.env` (Provided)

This file contains required version settings and defaults for RHDH CLI:

# Tooling versions
RHDH_CLI_VERSION="1.7.2"

2. `config/source.json` (Required for remote repositories)

Defines the source repository to clone:

{
  "repo": "https://github.com/backstage/community-plugins",
  "repo-ref": "main",
}

Fields:

repo: Repository URL (HTTPS or SSH)
repo-ref: Git reference (branch, tag, or commit SHA)

3. `config/plugins-list.yaml` (Required)

Lists plugins to build with optional build arguments:

# Simple plugins (no additional arguments)
plugins/todo:
plugins/todo-backend:

# Plugins with embed packages
plugins/scaffolder-backend: --embed-package @backstage/plugin-scaffolder-backend-module-github

# Multiple embed packages
plugins/search-backend: |
  --embed-package @backstage/plugin-search-backend-module-catalog
  --embed-package @backstage/plugin-search-backend-module-techdocs

4. `config/.env` (Optional)

Override default settings to publish to a remote image registry:

# Registry configuration (required only with --push-images)
REGISTRY_URL=quay.io
REGISTRY_USERNAME=your_username
REGISTRY_PASSWORD=your_password
REGISTRY_NAMESPACE=your_namespace
REGISTRY_INSECURE=false

# Logging
LOG_LEVEL=DEBUG
WORKSPACE_PATH=<path_to_workspace_with_respect_to_plugin_repo_root>

LOG_LEVEL can be set to one of DEBUG, INFO (default), WARN, ERROR, or CRITICAL

WORKSPACE_PATH can be set in lieu of the --workspace-path argument

Patches and Overlays

WARNING: This is a destructive operation

Patches and overlays modify files directly in the --repo-path directory. These operations are destructive and will permanently change the repository contents.

When using --use-local with a local repository, patches and overlays WILL modify your local files

Consider using version control OR cloning a fresh copy of your repository if you need to preserve the original state

Patches Directory (`config/patches/`)

Place .patch files to apply modifications to the source code:

config/
└── patches/
    └── 001-fix-dependency.patch

Patches are applied using the override-sources.sh script before building.

Overlays Directory (`config/<path-to-plugin-root-with-respect-to-workspace>/overlays/`)

Place files that should be copied over the source code:

config/
  └── plugins/
      └── my-plugin/
          └── overlay/
              └── custom-config.ts

Usage

Basic Command Structure

python -m rhdh_dynamic_plugin_factory.cli [OPTIONS]

Command-Line Options

Option	Default	Description
`--config-dir`	`/config`	Configuration directory containing `source.json`, `plugins-list.yaml`, patches, and overlays
`--repo-path`	`/workspace`	Path where plugin source code will be cloned/stored
`--workspace-path`	(required)	Path to the workspace from repository root (e.g., `workspaces/todo`)
`--output-dir`	`/outputs`	Directory for build artifacts (`.tgz` files and container images)
`--push-images` / `--no-push-images`	`true`	Whether to push container images to registry. Defaults to not pushing if no argument is provided
`--use-local`	`false`	Use local repository instead of cloning from source.json
`--log-level`	`INFO`	Logging level: `DEBUG`, `INFO`, `WARNING`, `ERROR`, `CRITICAL`
`--verbose`	`false`	Show verbose output with file and line numbers

Example Usage

Build plugins from backstage/community-plugins (todo workspace)

Local Execution:

python -m rhdh_dynamic_plugin_factory.cli \
  --config-dir ./config \
  --repo-path ./workspace \
  --workspace-path workspaces/todo \
  --output-dir ./outputs \
  --no-push-images \
  --log-level DEBUG

Container Execution:

podman run --rm -it \
  --device /dev/fuse \
  -v ./config:/config:z \
  -v ./workspace:/workspace:z \
  -v ./outputs:/outputs:z \
  -e LOG_LEVEL=DEBUG \
  quay.io/rhdh-community/dynamic-plugins-factory:latest \
  --workspace-path workspaces/todo \
  --no-push-images

Both methods will:

Load configurations from the local ./config directory
Clone the repository specified in ./config/source.json to ./workspace
Apply patches and overlays from ./config/patches/ and ./config/<plugin-path>/overlay/ to ./workspace
Install dependencies in ./workspace/workspaces/todo
Export and package plugins listed in ./config/plugins-list.yaml
Output artifacts to the local ./outputs/ directory
Skip pushing to registry (--no-push-images)

Build and push to registry

Local Execution:

# Set registry credentials in config/.env or environment
export REGISTRY_URL=quay.io
export REGISTRY_USERNAME=myuser
export REGISTRY_PASSWORD=mytoken
export REGISTRY_NAMESPACE=mynamespace

python -m rhdh_dynamic_plugin_factory.cli \
  --config-dir ./config \
  --repo-path ./workspace \
  --workspace-path workspaces/announcements \
  --output-dir ./outputs \
  --push-images

Container Execution:

# Pass registry credentials as environment variables
podman run --rm -it \
  --device /dev/fuse \
  -v ./config:/config:z \
  -v ./workspace:/workspace:z \
  -v ./outputs:/outputs:z \
  -e REGISTRY_URL=quay.io \
  -e REGISTRY_USERNAME=myuser \
  -e REGISTRY_PASSWORD=mytoken \
  -e REGISTRY_NAMESPACE=mynamespace \
  quay.io/rhdh-community/dynamic-plugins-factory:latest \
  --workspace-path workspaces/announcements \
  --push-images

Note: For security, consider providing registry configurations through the config/.env file

# Using environment file
podman run --rm -it \
  --device /dev/fuse \
  -v ./config:/config:z \
  -v ./workspace:/workspace:z \
  -v ./outputs:/outputs:z \
  --env-file ./config/.env \
  quay.io/rhdh-community/dynamic-plugins-factory:latest \
  --repo-path /workspace \
  --workspace-path workspaces/announcements \
  --output-dir /outputs \
  --push-images

Using a local repository (skip cloning)

If you already have the source code locally, use the --use-local flag to skip cloning from source.json AND/OR not include source.json in the config folder:

Local Execution:

# Ensure workspace already exists at --repo-path
# The --use-local flag will skip cloning even if source.json exists

python -m rhdh_dynamic_plugin_factory.cli \
  --config-dir ./config \
  --repo-path ./existing-workspace \
  --workspace-path . \
  --output-dir ./outputs \
  --use-local \
  --no-push-images

Container Execution:

# Mount your existing workspace directory
podman run --rm -it \
  --device /dev/fuse \
  -v ./config:/config:z \
  -v /path/to/existing-workspace:/workspace:z \
  -v ./outputs:/outputs:z \
  quay.io/rhdh-community/dynamic-plugins-factory:latest \
  --config-dir /config \
  --workspace-path . \
  --use-local \
  --no-push-images

Note: When using --use-local, patches and overlays will still be applied to your local repository. Make sure you have backups or are using version control before running the tool with a local repository.

Output

Build Artifacts

The factory also produces the following outputs in the directory specified by --output-dir:

outputs/
├── plugin-name-dynamic-1.0.0.tgz           # Plugin tarball
├── plugin-name-dynamic-1.0.0.tgz.integrity # Integrity checksum
└── ...

Container Images

When --push-images is enabled, images are tagged as:

${REGISTRY_URL}/${REGISTRY_NAMESPACE}/plugin-name-dynamic:1.0.0

NOTE: If the repository name (ex: plugin-name-dynamic) in the namespace specified by REGISTRY_NAMESPACE does not exist, the dynamic plugin factory will create a new registry. Depending on the registry specified by REGISTRY_URL, the newly created repository may be private. This will be the case for quay.io.

Examples

See the examples directory for complete configuration examples:

TODO Workspace Example

Located at examples/example-config-todo

This example contains a custom scalprum-config.json file and uses the standard backstage community plugins (BCP) workspace format. The process is very similar if you also want to include a custom backstage.json or app-config.dynamic.yaml file.

Local Execution:

python -m rhdh_dynamic_plugin_factory.cli \
  --config-dir ./examples/example-config-todo \
  --repo-path ./workspace \
  --workspace-path workspaces/todo \
  --output-dir ./outputs \
  --no-push-images

Container Execution:

podman run --rm -it \
  --device /dev/fuse \
  -v ./examples/example-config-todo:/config:z \
  -v ./workspace:/workspace:z \
  -v ./outputs:/outputs:z \
  quay.io/rhdh-community/dynamic-plugins-factory:latest \
  --workspace-path workspaces/todo \
  --no-push-images

Package Verification

Afterwards, you can verify that the plugin functions correctly by dynamically installing them into your RHDH instance by adding the newly published plugins into the dynamic-plugins.yaml file of your RHDH instance. You may need to add additional pluginConfig fields if your plugin requires additional configurations.

includes:
  - dynamic-plugins.default.yaml
plugins:
  - package: oci://quay.io/{REGISTRY_NAMESPACE}/backstage-community-plugin-todo:0.12.0!backstage-community-plugin-todo
    disabled: false
    pluginConfig: # `pluginConfig` may be required for additional configurations such as mounting the components
      dynamicPlugins:
        frontend:
          backstage-community.plugin-todo:
            mountPoints:
              - mountPoint: entity.page.todo/cards
                importName: EntityTodoContent
            entityTabs:
              - path: /todo
                title: Todo
                mountPoint: entity.page.todo
  - package: oci://quay.io/{REGISTRY_NAMESPACE}/backstage-community-plugin-todo-backend:0.13.0!backstage-community-plugin-todo-backend 
    disabled: false

We will show an example of how to verify with RHDH Local.

If the OCI images were pushed to a previously non-existent repository, please ensure it's now a public repository (or you can setup podman credentials for subsequent steps)
Create a dynamic-plugins.override.yaml file with the plugin config above. Replace the {REGISTRY_NAMESPACE} with the registry namespace used to publish the plugins.

Import a catalog entity the todo plugin with. We will be importing the backstage repository via config/app-config/app-config.local.yaml:

catalog:
  locations:
    - type: url
      target: https://github.com/backstage/backstage/blob/master/catalog-info.yaml
      rules: 
        - allow: [Component]

Start the RHDH instance with podman compose up -d
Navigate to http://localhost:7007/catalog/default/component/backstage/todo and verify that the backstage catalog entity appears, and a todo tab appears containing a table of all TODO comments in the source code of the catalog entity.
Clean up with podman compose down --volumes

Gitlab Workspace Example

Located at examples/example-config-gitlab

This example contains overlays used to override entire files contained in the gitlab workspace at https://github.com/immobiliare/backstage-plugin-gitlab which does not use the standard BCP workspace format. The --workspace-path is set to . (root of repository).

Local Execution:

python -m rhdh_dynamic_plugin_factory.cli \
  --config-dir ./examples/example-config-gitlab \
  --repo-path ./workspace \
  --workspace-path . \
  --output-dir ./outputs \

Container Execution:

podman run --rm -it \
  --device /dev/fuse \
  -v ./examples/example-config-gitlab:/config:z \
  -v ./workspace:/workspace:z \
  -v ./outputs:/outputs:z \
  quay.io/rhdh-community/dynamic-plugins-factory:latest \
  --workspace-path . \

AWS ECS Workspace Example

Located at examples/example-config-aws-ecs

This example contains a patches folder used for small patches as well as custom export arguments for the ecs backend plugin in the plugins-list.yaml to embed additional packages during the dynamic plugin export. This workspace also does not use the standard BCP workspace format.

Local Execution:

python -m rhdh_dynamic_plugin_factory.cli \
  --config-dir ./examples/example-config-aws-ecs \
  --repo-path ./workspace \
  --workspace-path . \
  --output-dir ./outputs \

Container Execution:

podman run --rm -it \
  --device /dev/fuse \
  -v ./examples/example-config-aws-ecs:/config:z \
  -v ./workspace:/workspace:z \
  -v ./outputs:/outputs:z \
  quay.io/rhdh-community/dynamic-plugins-factory:latest \
  --repo-path /workspace \

Development

Project Structure

rhdh-dynamic-plugin-factory/
├── src/rhdh_dynamic_plugin_factory/
|   ├── __init__.py         
│   ├── __main__.py              # Package entry point
│   ├── cli.py                   # CLI implementation
│   ├── config.py                # Configuration classes
│   ├── logger.py                # Logging setup
|   └── utils.py                 # Utility functions
├── scripts/
│   ├── export-workspace.sh      # Plugin export script
│   └── override-sources.sh      # Patch/overlay script
├── tests/                       # Unit Tests
├── examples/                    # Example configuration sets
├── default.env                  # Default environment settings
├── requirements.txt             # Python dependencies
└── requirements.dev.txt         # Development dependencies

Running Tests

Run all tests

pytest tests/

Run tests with verbose output

pytest tests/ -v

Run tests for a specific module

pytest tests/test_config.py -v

Run a specific test class or method

pytest tests/test_config.py::TestPluginFactoryConfigLoadFromEnv -v
pytest tests/test_config.py::TestPluginFactoryConfigLoadFromEnv::test_load_from_env_valid_configuration -v

Run with coverage reporting

pytest tests/ --cov=src/rhdh_dynamic_plugin_factory --cov-report=term-missing

Resources

To learn more about how dynamic plugins work refer to the dynamic plugins documentation in the RHDH Repository

Name		Name	Last commit message	Last commit date
Latest commit History 7 Commits
.cursor/rules		.cursor/rules
.github		.github
examples		examples
scripts		scripts
src/rhdh_dynamic_plugin_factory		src/rhdh_dynamic_plugin_factory
tests		tests
.dockerignore		.dockerignore
.gitignore		.gitignore
Dockerfile		Dockerfile
LICENSE		LICENSE
README.md		README.md
default.env		default.env
pytest.ini		pytest.ini
requirements.dev.txt		requirements.dev.txt
requirements.txt		requirements.txt

License

redhat-developer/rhdh-dynamic-plugin-factory

Folders and files

Latest commit

History

Repository files navigation

RHDH Dynamic Plugin Factory

Overview

Prerequisites

Local Development

Installation

Local Setup

Container Image Setup

Using Pre-built Community Images

Building the Image

Container Requirements

Basic Container Usage

Configuration

Directory Structure

Configuration Files

1. default.env (Provided)

2. config/source.json (Required for remote repositories)

3. config/plugins-list.yaml (Required)

4. config/.env (Optional)

Patches and Overlays

Patches Directory (config/patches/)

Overlays Directory (config/<path-to-plugin-root-with-respect-to-workspace>/overlays/)

Usage

Basic Command Structure

Command-Line Options

Example Usage

Build plugins from backstage/community-plugins (todo workspace)

Build and push to registry

Using a local repository (skip cloning)

Output

Build Artifacts

Container Images

Examples

TODO Workspace Example

Package Verification

Gitlab Workspace Example

AWS ECS Workspace Example

Development

Project Structure

Running Tests

Run all tests

Run tests with verbose output

Run tests for a specific module

Run a specific test class or method

Run with coverage reporting

Resources

About

Resources

License

Uh oh!

Stars

Watchers

Forks

Releases

Packages 0

Uh oh!

Languages

1. `default.env` (Provided)

2. `config/source.json` (Required for remote repositories)

3. `config/plugins-list.yaml` (Required)

4. `config/.env` (Optional)

Patches Directory (`config/patches/`)

Overlays Directory (`config/<path-to-plugin-root-with-respect-to-workspace>/overlays/`)

Packages