Skip to content

docs(py): clarify genkit.plugins namespace discovery and add test#5543

Open
cabljac wants to merge 1 commit into
mainfrom
port/genkit-plugins-namespace-discovery
Open

docs(py): clarify genkit.plugins namespace discovery and add test#5543
cabljac wants to merge 1 commit into
mainfrom
port/genkit-plugins-namespace-discovery

Conversation

@cabljac

@cabljac cabljac commented Jun 15, 2026

Copy link
Copy Markdown
Contributor

What

Reword the genkit.plugins package docstring so it correctly explains our
plugin-discovery setup, and add a regression test for the runtime discovery that
nothing currently covers. Doc + test only, no behaviour change.

Why

In #5521 I'd added pkgutil.extend_path to
genkit/plugins/__init__.py, then removed it again once it turned out to be a
no-op, leaving a comment so the next person doesn't re-add it. While reviewing, we
realised the comment's stated reason was wrong: it claimed extend_path only
picks up portions that ship their own genkit/plugins/__init__.py. Reading the
CPython source, that isn't how it works (modern extend_path handles PEP 420
portions with no __init__.py).

The real reason extend_path is useless here:

# pkgutil.extend_path, paraphrased
parent_package, _, final_name = name.rpartition('.')   # "genkit", "plugins"
search_path = sys.modules[parent_package].__path__     # genkit.__path__, NOT sys.path

It only searches the parent package's __path__. genkit is a regular
single-directory package, so its __path__ doesn't span the separately installed
plugin distributions, and there's nothing for extend_path to find. Our runtime
extend_plugin_namespace (in genkit/_core/_plugins.py, called from
genkit/__init__.py) scans sys.path instead, which does reach them.

I've reworded the docstring to say that, and to clarify that genkit.plugins is a
regular package (it ships __init__.py) whose __path__ is extended at runtime,
rather than a strict PEP 420 namespace package.

Test

Adds tests/genkit/core/plugins_discovery_test.py covering extend_plugin_namespace:

  • a fake plugin laid out on a temporary sys.path entry (as a bare PEP 420
    portion, matching how real plugins ship) is discovered and importable
  • discovery is idempotent (no duplicate __path__ entries on repeat calls)
  • a sys.path entry without genkit/plugins is ignored

There was no test for this mechanism before. Run:

cd py/packages/genkit && uv run pytest tests/genkit/core/plugins_discovery_test.py -v

All three pass locally; ruff check/format clean.

Worth a careful look at

  • My reading of extend_path vs extend_plugin_namespace above, in case I've
    misjudged the behaviour.
  • Whether the test isolation (snapshot/restore of genkit.plugins.__path__ and
    sys.path) is solid enough not to leak into other tests.

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request updates the documentation in genkit/plugins/__init__.py to clarify how plugin namespace discovery works, and adds unit tests in plugins_discovery_test.py to verify the runtime discovery mechanism. Feedback is provided to improve test isolation in the test fixture by deleting the dynamically imported submodule attribute from the parent module, preventing potential test pollution.

Important

The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.

Comment on lines +50 to +51
sys.modules.pop(f'genkit.plugins.{FAKE_PLUGIN}', None)
importlib.invalidate_caches()

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

When a submodule is imported in Python, it is set as an attribute on its parent module (e.g., genkit.plugins.fake_discovery_plugin). Simply popping the submodule from sys.modules does not remove this attribute from the parent module. To ensure complete test isolation and prevent potential test pollution, we should also delete the attribute from genkit.plugins if it exists.

Suggested change
sys.modules.pop(f'genkit.plugins.{FAKE_PLUGIN}', None)
importlib.invalidate_caches()
sys.modules.pop(f'genkit.plugins.{FAKE_PLUGIN}', None)
if hasattr(genkit.plugins, FAKE_PLUGIN):
delattr(genkit.plugins, FAKE_PLUGIN)
importlib.invalidate_caches()

@cabljac cabljac force-pushed the port/genkit-plugins-namespace-discovery branch 3 times, most recently from 2ab0d82 to 2985ddd Compare June 15, 2026 11:16
Reword the genkit.plugins package docstring to correctly explain why
pkgutil.extend_path is not used here. extend_path searches the parent
package's __path__ (genkit.__path__) rather than sys.path, and because
genkit is a regular single-directory package its __path__ does not span
the separately installed plugin distributions, so extend_path would find
nothing. Runtime discovery via extend_plugin_namespace scans sys.path
instead. Also clarify that genkit.plugins is a regular package (it ships
__init__.py) whose __path__ is extended at runtime, not a strict PEP 420
namespace package.

Add a regression test for extend_plugin_namespace: a fake plugin laid out
on a temporary sys.path entry is discovered and importable, discovery is
idempotent, and a sys.path entry without genkit/plugins is ignored.
@cabljac cabljac force-pushed the port/genkit-plugins-namespace-discovery branch from 2985ddd to 9a664cc Compare June 15, 2026 12:53
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

python Python

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant