Throw informative error when KolibriPluginBase cannot import from properties

[nucleogenesis]

KolibriPluginBase takes string properties pointing to module names which it then imports. However, when there are issues with this import, the Django error messages are basically worthless - pointing to an entirely different issue.

So - there should be a try/catch earlier on in the instantiation of a class inheriting KolibriPluginBase that tries to import the module and complains directly if that module does not exist or if it has an error itself.

For example, if a class assigns "api_urls" to the property untrainslated_view_urls and that api_urls.py file itself has a broken import, the dev will see an error completely unrelated to that file and that import. Ideally, this will be caught far earlier and the user can be told in this case a more contextually helpful error message.

Current State of Error Handling

The KolibriPluginBase class (in kolibri/plugins/init.py) uses the _return_module() method (lines 296-309) to import modules specified by plugin properties. The current implementation:

def _return_module(self, module_name):
    if module_has_submodule(sys.modules[self.module_path], module_name):
        models_module_name = "%s.%s" % (self.module_path, module_name)
        try:
            return import_module(models_module_name)
        except Exception as e:
            logging.warning(
                "Tried to import module {module_name} from {plugin} but an error was raised".format(
                    plugin=self.module_path, module_name=module_name
                )
            )
            logging.exception(e)
    return None

The problem: While this logs a generic warning, it:

1. Swallows the actual exception details (only logs to console, not shown to developer in most cases)
2. Returns None, which causes errors to surface much later in unrelated parts of the codebase
3. Doesn't distinguish between different failure scenarios

Specific Misconfiguration Scenarios

Here are common mistakes that produce confusing error messages:

1. Typo in the module name:

   class MyPlugin(KolibriPluginBase):
       untranslated_view_urls = "api_url"  # Should be "api_urls"

2. Module exists but has a broken import:

   # In api_urls.py
   from nonexistent_module import something  # ImportError

3. Module file doesn't exist:

   class MyPlugin(KolibriPluginBase):
       untranslated_view_urls = "urls"  # File doesn't exist

4. Syntax errors in the module:

   # In api_urls.py
   urlpatterns = [  # Missing closing bracket

Where Errors Surface

When _return_module() returns None, the error manifests in different places depending on which property was misconfigured:

1. URL registration (kolibri/plugins/utils/urls.py:44-45):

   url_module = plugin_instance.url_module  # Returns None
   api_url_module = plugin_instance.api_url_module  # Returns None

   Later when trying to access url_module.urlpatterns (line 67), you get:

   AttributeError: 'NoneType' object has no attribute 'urlpatterns'
   

2. Settings application (kolibri/plugins/utils/settings.py:134):

   plugin_settings_module = plugin_instance.settings_module  # Returns None

   If settings are expected, subsequent code may fail with obscure errors.

3. Options loading (kolibri/plugins/utils/options.py:111-117):

   plugin_options = plugin_instance.options_module  # Returns None
   if plugin_options and hasattr(plugin_options, "option_spec"):  # Safely handles None

   This one is better handled, but still doesn't tell the developer what went wrong.

Desired Behavior

The improved error handling should:

1. Provide clear, actionable error messages that include:

   • Which plugin caused the error (module path)
   • Which property was misconfigured (e.g., untranslated_view_urls)
   • What went wrong (module not found vs. import error)

2. Include full exception information when a module exists but has import errors:

   Error in plugin 'kolibri.plugins.learn':
   Failed to import module 'api_urls' specified in 'untranslated_view_urls' property.
   
   ImportError: cannot import name 'something' from 'nonexistent_module'
   [full traceback]
   
   Please check:
   - The module file exists at: kolibri/plugins/learn/api_urls.py
   - All imports in that file are correct
   - The module name is spelled correctly in the plugin definition
   

3. Distinguish between error types:

   • Module doesn't exist: "Module 'api_urls' not found. Expected file: kolibri/plugins/learn/api_urls.py"
   • Import failed: Show the actual ImportError with traceback
   • Syntax error: Show the SyntaxError with traceback

4. Fail early and explicitly rather than returning None and causing confusion downstream

Understanding the Plugin Architecture

For students new to Kolibri, here's how the plugin system works:

Plugin Properties: When you create a plugin class, you can specify module names as string properties:

• untranslated_view_urls: Module containing URL patterns for API endpoints (no language prefix)
• translated_view_urls: Module containing URL patterns for views with translated content
• root_view_urls: Module containing URL patterns attached to the domain root
• django_settings: Module containing Django settings to augment
• kolibri_options: Module containing configuration options
• kolibri_option_defaults: Module containing default value overrides

Example:

class Learn(KolibriPluginBase):
    untranslated_view_urls = "api_urls"  # Points to kolibri/plugins/learn/api_urls.py
    translated_view_urls = "urls"         # Points to kolibri/plugins/learn/urls.py
    kolibri_options = "options"           # Points to kolibri/plugins/learn/options.py

Module Loading Process:

1. Django startup triggers plugin registration
2. For each enabled plugin, the system accesses properties like .url_module (lines 311-336)
3. These properties call _return_module() to import the specified module
4. The imported modules are used by various parts of the system (URL routing, settings, etc.)

Implementation Hints

When implementing the fix, consider:

1. The _return_module() method is the central place to add better error handling
2. You might want to create custom exception classes for different error scenarios
3. Check Django's error messages for import failures as inspiration for helpful messages
4. Consider using the existing development mode flag to provide even more detailed debugging info (see here for example: https://github.com/learningequality/kolibri/blob/develop/kolibri/core/webpack/hooks.py#L87 )
5. Look at how the properties (url_module, settings_module, etc.) use _return_module() - they currently have their own warning messages that could be improved too

Testing Your Changes

To test your improvements, you can:

1. Create a test plugin with intentional errors
2. Try each misconfiguration scenario listed above
3. Verify that error messages clearly identify the problem
4. Ensure the error appears immediately at startup (fail fast)
5. Check that correct configurations still work normally

---

Additional Resources:

• Plugin Architecture Documentation
• Django's import_module function: Python docs
• Django's module_has_submodule: Used to check if a module exists before importing

[AllanOXDi]

Hey @nucleogenesis ! I have tried to recreate this issue you raised. I imported something that does not exist on the api_url and when I run the Kolibri, it seems to be pointing me to the right thing. in other words, we are trying to catch the error before and the Django error message is useful.

WARNING 2022-10-06 17:58:52,817 Tried to import module api_urls from kolibri.plugins.learn but an error was raised ERROR 2022-10-06 17:58:52,818 cannot import name 'something' from 'rest_framework' (/Users/Otodiallan/.pyenv/versions/3.9.9/envs/kolibri-py3.9/lib/python3.9/site-packages/rest_framework/__init__.py) Traceback (most recent call last): File "/Users/Otodiallan/Desktop/kolibri/kolibri/plugins/__init__.py", line 245, in _return_module return import_module(models_module_name) File "/Users/Otodiallan/.pyenv/versions/3.9.9/lib/python3.9/importlib/__init__.py", line 127, in import_module return _bootstrap._gcd_import(name[level:], package, level) File "<frozen importlib._bootstrap>", line 1030, in _gcd_import File "<frozen importlib._bootstrap>", line 1007, in _find_and_load File "<frozen importlib._bootstrap>", line 986, in _find_and_load_unlocked File "<frozen importlib._bootstrap>", line 680, in _load_unlocked File "<frozen importlib._bootstrap_external>", line 850, in exec_module File "<frozen importlib._bootstrap>", line 228, in _call_with_frames_removed File "/Users/Otodiallan/Desktop/kolibri/kolibri/plugins/learn/api_urls.py", line 4, in <module> from rest_framework import something ImportError: cannot import name 'something' from 'rest_framework' (/Users/Otodiallan/.pyenv/versions/3.9.9/envs/kolibri-py3.9/lib/python3.9/site-packages/rest_framework/__init__.py) WARNING 2022-10-06 17:58:52,873 kolibri.plugins.learn defined api_urls untranslated view urls but the module was not found ERROR 2022-10-06 17:58:52,873 'NoneType' object has no attribute 'startswith' Traceback (most recent call last): File "/Users/Otodiallan/Desktop/kolibri/kolibri/plugins/__init__.py", line 308, in api_url_module return import_module(module) File "/Users/Otodiallan/.pyenv/versions/3.9.9/lib/python3.9/importlib/__init__.py", line 118, in import_module if name.startswith('.'): AttributeError: 'NoneType' object has no attribute 'startswith'

[AllanOXDi]

I am wondering if you could give me more details