Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add type annotations to _pytest.compat and _pytest._code.code #6205

Merged
merged 9 commits into from
Nov 17, 2019

Conversation

bluetech
Copy link
Member

This PR starts with some small prerequisite commits, and then adds type annotations to _pytest.compat and _pytest._code.code.

Let me know if this is too big for one PR, and I will split it. Note that some part of the diff is just adding -> None to some test functions, so that mypy will check them without check_untyped_defs = True (which needs more work).

Add some Python 3.8 type: ignores; all are already fixed in the next
mypy release, so can be removed once we upgrade.

Also move some flake8 ignores which seem to have changed places.
pytest doesn't support these PyPy versions anymore, so no need to have
checks for them.
It doesn't help much IMO, just adds indirection and makes it harder to
type.
The previous test was better in that it used fakes to test all of the
real code paths. The problem with that is that it makes it impossible to
simplify the code with `isinstance` checks. So let's just simulate the
issue directly with a monkeypatch.
Source was previously iterable because it implements `__getitem__()`,
which is apparently a thing from before `__iter__()` was introduced.
To reduce mypy's and my own confusion, implement `__iter__()` directly.
@@ -60,7 +60,7 @@ def __getitem__(self, key: int) -> str:
raise NotImplementedError()

@overload # noqa: F811
def __getitem__(self, key: slice) -> "Source":
def __getitem__(self, key: slice) -> "Source": # noqa: F811
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There seems to be a discrepancy between where CI flake8 and my flake8 report this - on the decorator or the def. Maybe because of Python 3.8? So for now I add it to both.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe because of Python 3.8?

Very likely, yes.
Isn't there a flake8 plugin to handle this automatically maybe?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There seems to be some discussion regarding this here, I'm too lazy to read through it though...

Copy link
Contributor

@blueyed blueyed left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Awesome!

@@ -38,13 +46,12 @@ class Code:
def __init__(self, rawcode) -> None:
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we type the rawcode arg here?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not really. I spent some time trying to type getrawcode but left it aside for now.

@@ -134,7 +141,7 @@ def eval(self, code, **vars):
f_locals.update(vars)
return eval(code, self.f_globals, f_locals)

def exec_(self, code, **vars):
def exec_(self, code, **vars) -> None:
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can the args be typed here also?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

They can be anything really AFAIK, but I'm not entirely sure, so I left it out.


def __init__(self, tb, excinfo=None):
def __init__(
self, tb: Union[TracebackType, Iterable[TracebackEntry]], excinfo=None
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can excinfo be typed?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, will add.

path=None,
lineno: Optional[int] = None,
firstlineno: Optional[int] = None,
excludepath=None,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can excludepath be typed?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's can be a py.path AFAIU, so I'm leaving it unannotated.

# Type ignored because mypy thinks the slice is a List[TracebackEntry]
# instead of the Traceback subclass, even though we override __getitem__().
# Should figure out why it does that.
traceback = traceback[:max_frames] + traceback[-max_frames:] # type: ignore
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The slice is a Traceback, but only the expression traceback[:max_frames] + traceback[-max_frames:] is a List[…] then.
No idea how to fix it, but it might help you.. :)

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Right, I misunderstood the situation. Looks like a deficiency in mypy/typeshed for now. I'll update the comment.

reprfuncargs: Optional["ReprFuncArgs"],
reprlocals: Optional["ReprLocals"],
filelocrepr: Optional["ReprFileLocation"],
style: str,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Literal for style?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, will add.

@@ -976,6 +1035,7 @@ def __init__(self, lines, reprfuncargs, reprlocals, filelocrepr, style):

def toterminal(self, tw) -> None:
if self.style == "short":
assert self.reprfileloc is not None
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could (additionally) be checked/validated in __init__? But it's fine like that also.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Wouldn't happen unless the code is buggy; seems fine to only do it here.

@@ -1074,13 +1074,14 @@ def try_makedirs(cache_dir) -> bool:

def get_cache_dir(file_path: Path) -> Path:
"""Returns the cache directory to write .pyc files for the given .py file path"""
if sys.version_info >= (3, 8) and sys.pycache_prefix:
# Type ignored until added in next mypy release.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Would be good to link an issue in general, but ok for now.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

mypy can detect outdated type: ignores so it's easy to get rid of when updating mypy.

warnings.warn(FUNCARGNAMES, stacklevel=2)
return self.fixturenames


Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm ok with inlining, but wondered how it made typing harder (from the commit message)?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's a mixin class, it uses fixturenames which is not defined in the class so mypy gets confused. There are ways to type this but it's not worth the bother in this case IMO.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Agreed.

assert ex.value.lineno == 1
assert ex.value.offset in {5, 7} # cpython: 7, pypy3.6 7.1.1: 5
assert ex.value.text is not None
assert ex.value.text.strip(), "x x"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could check the full expected text instead?
assert ex.value.text == "xyz xyz\n"

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, that's better.

@blueyed
Copy link
Contributor

blueyed commented Nov 16, 2019

Note that some part of the diff is just adding -> None to some test functions, so that mypy will check them without check_untyped_defs = True (which needs more work).

IIRC this (check_untyped_defs) could be enabled based on modules also?

@bluetech
Copy link
Member Author

IIRC this (check_untyped_defs) could be enabled based on modules also?

Yes, that's an option to. The -> None is not too bad, though.

@bluetech
Copy link
Member Author

Thanks for the review @blueyed, updated.

Copy link
Contributor

@blueyed blueyed left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍

@bluetech bluetech merged commit fa578d7 into pytest-dev:features Nov 17, 2019
@bluetech bluetech deleted the type-annotations-8 branch November 17, 2019 07:45
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants