New semantic analyzer: Namespace package with __init__.py breaks analysis

[fireundubh]

I'm getting a lot of errors reported that are not errors.

• Classes are reported as invalid types
• Calls to methods are reported as modules not having those attributes
• Instances of classes are reported as modules not callable
• Types passed to, e.g., isinstance are reported as incompatible type Module (suggests that all custom types are treated as type Module)
• Base classes are reported as invalid base classes
• Calls to instances of classes are reported as not having the specified attributes
• Package imports in __init__.py are reported as already defined

I am not sure whether these are issues are caused by the new semantic analyzer or by the Mypy plugin for PyCharm which has not been updated yet.

In comparison, with the new semantic analyzer disabled, there are no errors. The application works correctly.

[ilevkivskyi]

Can you give some examples of the code that triggers this?

[fireundubh]

Invalid type "Presenters.BasePresenter" (35:20)
Module has no attribute "get_presenter" (35:36)

presenter: BasePresenter = PresenterHelper.get_presenter(options=presenter_args)

BasePresenter is imported from a Presenters package. get_presenter is a static method.

Argument 2 to "isinstance" has incompatible type "Tuple[Module, Module, Module]"; expected "Union[type, Tuple[Union[type, Tuple[Any, ...]], ...]]" (96:12)
BasePresenter? has no attribute "save_to_file" (97:13)

if isinstance(presenter, (ConsolePresenter, JsonPresenter, MarkdownPresenter)):
    presenter.save_to_file(records, args.documents, args.output)

ConsolePresenter, JsonPresenter, and MarkdownPresenter are imported from a package.

Invalid type "Presenters.ThemeManager" (36:24)
Module not callable (36:39)

theme_manager: ThemeManager = ThemeManager(args.output)

ThemeManager is also imported from a package.

Argument "formatter_class" to "ArgumentParser" has incompatible type Module; expected "Type[HelpFormatter]" (8:15)

from Extensions import HelpFormatterEx

if __name__ == '__main__':
    _parser = argparse.ArgumentParser(formatter_class=HelpFormatterEx)

HelpFormatterEx inherits from argparse.HelpFormatter and is imported from a package.

The overarching pattern seems to be related to package imports.

[ilevkivskyi]

Can you show the place in Extensions where HelpFormatterEx is defined? Also do you have a module with the same name?

[fireundubh]

Tree:

Extensions
  __init__.py
  HelpFormatterEx.py

__init__.py

from Extensions.HelpFormatterEx import HelpFormatterEx

HelpFormatterEx.py

import argparse

class HelpFormatterEx(argparse.HelpFormatter):
    def _format_action_invocation(self, action: argparse.Action) -> str:
# snip

[ilevkivskyi]

There must be something else, this test passes cleanly:

[case testNewAnalyzerArgParseSubmoduleAsClass_newsemanal]
import argparse
from Extensions import HelpFormatterEx

if __name__ == '__main__':
    _parser = argparse.ArgumentParser(formatter_class=HelpFormatterEx)

[file Extensions/__init__.py]
from Extensions.HelpFormatterEx import HelpFormatterEx

[file Extensions/HelpFormatterEx.py]
import argparse

class HelpFormatterEx(argparse.HelpFormatter):
    def _format_action_invocation(self, action: argparse.Action) -> str:
        ...

Do you have any tricky import cycles and/or forward references there?

[fireundubh]

Extensions is a package within a package, so it's really more like MyApplication.Extensions.HelpFormatterEx.HelpFormatterEx.

I just tried to repro this specifically in a new project, but I got the same result as you. It's possible the error is somewhere else and tripping up the other checks.

I'll try to narrow it down.

[ilevkivskyi]

I'll try to narrow it down.

OK, thanks!

[fireundubh]

Okay, I found the error. The issue was I had an empty and errant __init__.py in my namespace package.

Using the docs example tree as a reference:

mynamespace/
    # No __init__.py here. <---- had an __init__.py there
    subpackage_a/
        # Sub-packages have __init__.py.
        __init__.py
        module.py

If that file exists, mypy reports a ton of false-positive errors. Deleting that file clears up the report.

[ilevkivskyi]

Hm, interesting. Could you please post a self contained repro that you have?