Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Failing tests: sympy.parsing.sympy_tokenize not found (and others) #40

Closed
sanjayankur31 opened this issue Nov 14, 2018 · 4 comments
Closed

Failing tests: sympy.parsing.sympy_tokenize not found (and others) #40

sanjayankur31 opened this issue Nov 14, 2018 · 4 comments

Comments

@sanjayankur31
Copy link
Contributor

sanjayankur31 commented Nov 14, 2018 •
edited
Loading

Running python setup.py test in a fresh python3 virtual environment shows this failure:

nineml (unittest.loader._FailedTest) ... ERROR

======================================================================
ERROR: nineml (unittest.loader._FailedTest)
----------------------------------------------------------------------
ImportError: Failed to import test module: nineml
Traceback (most recent call last):
  File "/usr/lib64/python3.7/unittest/loader.py", line 468, in _find_test_path
    package = self._get_module_from_name(name)
  File "/usr/lib64/python3.7/unittest/loader.py", line 375, in _get_module_from_name
    __import__(name)
  File "/home/asinha/rpmbuild/SOURCES/nineml-1.0/nineml/__init__.py", line 12, in <module>
    from . import abstraction  # @IgnorePep8
  File "/home/asinha/rpmbuild/SOURCES/nineml-1.0/nineml/abstraction/__init__.py", line 10, in <module>
    from . import dynamics
  File "/home/asinha/rpmbuild/SOURCES/nineml-1.0/nineml/abstraction/dynamics/__init__.py", line 8, in <module>
    from .base import Dynamics
  File "/home/asinha/rpmbuild/SOURCES/nineml-1.0/nineml/abstraction/dynamics/base.py", line 15, in <module>
    from nineml.abstraction.componentclass import (
  File "/home/asinha/rpmbuild/SOURCES/nineml-1.0/nineml/abstraction/componentclass/__init__.py", line 2, in <module>
    from .base import ComponentClass, Parameter
  File "/home/asinha/rpmbuild/SOURCES/nineml-1.0/nineml/abstraction/componentclass/base.py", line 19, in <module>
    from ..expressions import Alias, Constant
  File "/home/asinha/rpmbuild/SOURCES/nineml-1.0/nineml/abstraction/expressions/__init__.py", line 1, in <module>
    from .base import (ODE, Expression, ExpressionWithSimpleLHS, ExpressionSymbol,
  File "/home/asinha/rpmbuild/SOURCES/nineml-1.0/nineml/abstraction/expressions/base.py", line 31, in <module>
    from .parser import Parser  # @IgnorePep8
  File "/home/asinha/rpmbuild/SOURCES/nineml-1.0/nineml/abstraction/expressions/parser.py", line 11, in <module>
    from sympy.parsing.sympy_tokenize import NAME, OP
ModuleNotFoundError: No module named 'sympy.parsing.sympy_tokenize'

sympy.parsing.simpy_tokenize is not part of sympy any more. It seems it was dropped in sympy 1.2 in favour of the standard library's tokenize: https://github.com/sympy/sympy/wiki/Release-Notes-for-1.2

it is used in nineml/abstraction/expressions/parser.py:11:from sympy.parsing.sympy_tokenize import NAME, OP
@sanjayankur31
Copy link
Contributor Author

sed -i 's/sympy.parsing.sympy_tokenize/tokenize/' nineml/abstraction/expressions/parser.py should fix it as per https://docs.python.org/3/library/tokenize.html

@sanjayankur31 sanjayankur31 changed the title Failing tests: sympy.parsing.sympy_tokenize not found Failing tests: sympy.parsing.sympy_tokenize not found (and others) Nov 14, 2018
@sanjayankur31
Copy link
Contributor Author

After that bit, a few tests still fail but with other issues:

(ins)(nineml-virt) [asinha@ankur  nineml-python-1.0]$ nosetests test/
..................E..............E.....E..E.......................................................................................................................................................................................
======================================================================
ERROR: test_trigger (unittests.abstraction_test.dynamics_test.OnCondition_test)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/asinha/dump/nineml-virt/lib64/python3.7/site-packages/sympy/core/cache.py", line 93, in wrapper
    retval = cfunc(*args, **kwargs)
  File "/home/asinha/dump/nineml-virt/lib64/python3.7/site-packages/sympy/core/function.py", line 439, in __new__
    'given': n})
TypeError: factorial takes exactly 1 argument (0 given)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/asinha/rpmbuild/SOURCES/nineml-python-1.0/nineml/abstraction/expressions/parser.py", line 92, in _parse_expr
    local_dict=self.inline_randoms_dict)
  File "/home/asinha/dump/nineml-virt/lib64/python3.7/site-packages/sympy/parsing/sympy_parser.py", line 965, in parse_expr
    return eval_expr(code, local_dict, global_dict)
  File "/home/asinha/dump/nineml-virt/lib64/python3.7/site-packages/sympy/parsing/sympy_parser.py", line 878, in eval_expr
    code, global_dict, local_dict)  # take local objects in preference
  File "<string>", line 1, in <module>
  File "/home/asinha/dump/nineml-virt/lib64/python3.7/site-packages/sympy/core/cache.py", line 95, in wrapper
    retval = func(*args, **kwargs)
  File "/home/asinha/dump/nineml-virt/lib64/python3.7/site-packages/sympy/core/function.py", line 439, in __new__
    'given': n})
TypeError: factorial takes exactly 1 argument (0 given)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/asinha/rpmbuild/SOURCES/nineml-python-1.0/test/unittests/abstraction_test/dynamics_test.py", line 706, in test_trigger
    c = OnCondition(trigger=expr)
  File "/home/asinha/rpmbuild/SOURCES/nineml-python-1.0/nineml/abstraction/dynamics/transitions.py", line 500, in __init__
    self._trigger = Trigger(rhs=trigger)
  File "/home/asinha/rpmbuild/SOURCES/nineml-python-1.0/nineml/abstraction/dynamics/transitions.py", line 395, in __init__
    Expression.__init__(self, rhs)
  File "/home/asinha/rpmbuild/SOURCES/nineml-python-1.0/nineml/abstraction/expressions/base.py", line 72, in __init__
    self.rhs = rhs
  File "/home/asinha/rpmbuild/SOURCES/nineml-python-1.0/nineml/abstraction/expressions/base.py", line 83, in rhs
    self._rhs = Parser().parse(rhs)
  File "/home/asinha/rpmbuild/SOURCES/nineml-python-1.0/nineml/abstraction/expressions/parser.py", line 75, in parse
    return self._parse_expr(expr)
  File "/home/asinha/rpmbuild/SOURCES/nineml-python-1.0/nineml/abstraction/expressions/parser.py", line 96, in _parse_expr
    "{}\n\n{}".format(expr, e))
nineml.exceptions.NineMLMathParseError: Could not parse math-inline expression: And__(Gt__(V, 1.0), !(Lt__(V, 10.0)))

factorial takes exactly 1 argument (0 given)

======================================================================
ERROR: test_double_negation (unittests.abstraction_test.expressions_test.C89ToSympy_test)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/asinha/rpmbuild/SOURCES/nineml-python-1.0/nineml/abstraction/expressions/parser.py", line 92, in _parse_expr
    local_dict=self.inline_randoms_dict)
  File "/home/asinha/dump/nineml-virt/lib64/python3.7/site-packages/sympy/parsing/sympy_parser.py", line 960, in parse_expr
    code = stringify_expr(s, local_dict, global_dict, transformations)
  File "/home/asinha/dump/nineml-virt/lib64/python3.7/site-packages/sympy/parsing/sympy_parser.py", line 866, in stringify_expr
    tokens = transform(tokens, local_dict, global_dict)
  File "/home/asinha/dump/nineml-virt/lib64/python3.7/site-packages/sympy/parsing/sympy_parser.py", line 616, in factorial_notation
    result = _add_factorial_tokens('factorial2', result)
  File "/home/asinha/dump/nineml-virt/lib64/python3.7/site-packages/sympy/parsing/sympy_parser.py", line 52, in _add_factorial_tokens
    raise TokenError()
tokenize.TokenError

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/asinha/rpmbuild/SOURCES/nineml-python-1.0/test/unittests/abstraction_test/expressions_test.py", line 177, in test_double_negation
    expr = Expression('!!a')
  File "/home/asinha/rpmbuild/SOURCES/nineml-python-1.0/nineml/abstraction/expressions/base.py", line 72, in __init__
    self.rhs = rhs
  File "/home/asinha/rpmbuild/SOURCES/nineml-python-1.0/nineml/abstraction/expressions/base.py", line 83, in rhs
    self._rhs = Parser().parse(rhs)
  File "/home/asinha/rpmbuild/SOURCES/nineml-python-1.0/nineml/abstraction/expressions/parser.py", line 75, in parse
    return self._parse_expr(expr)
  File "/home/asinha/rpmbuild/SOURCES/nineml-python-1.0/nineml/abstraction/expressions/parser.py", line 96, in _parse_expr
    "{}\n\n{}".format(expr, e))
nineml.exceptions.NineMLMathParseError: Could not parse math-inline expression: !!a



======================================================================
ERROR: test_negation (unittests.abstraction_test.expressions_test.C89ToSympy_test)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/asinha/rpmbuild/SOURCES/nineml-python-1.0/nineml/abstraction/expressions/parser.py", line 92, in _parse_expr
    local_dict=self.inline_randoms_dict)
  File "/home/asinha/dump/nineml-virt/lib64/python3.7/site-packages/sympy/parsing/sympy_parser.py", line 960, in parse_expr
    code = stringify_expr(s, local_dict, global_dict, transformations)
  File "/home/asinha/dump/nineml-virt/lib64/python3.7/site-packages/sympy/parsing/sympy_parser.py", line 866, in stringify_expr
    tokens = transform(tokens, local_dict, global_dict)
  File "/home/asinha/dump/nineml-virt/lib64/python3.7/site-packages/sympy/parsing/sympy_parser.py", line 614, in factorial_notation
    result = _add_factorial_tokens('factorial', result)
  File "/home/asinha/dump/nineml-virt/lib64/python3.7/site-packages/sympy/parsing/sympy_parser.py", line 52, in _add_factorial_tokens
    raise TokenError()
tokenize.TokenError

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/asinha/rpmbuild/SOURCES/nineml-python-1.0/test/unittests/abstraction_test/expressions_test.py", line 173, in test_negation
    expr = Expression('!a')
  File "/home/asinha/rpmbuild/SOURCES/nineml-python-1.0/nineml/abstraction/expressions/base.py", line 72, in __init__
    self.rhs = rhs
  File "/home/asinha/rpmbuild/SOURCES/nineml-python-1.0/nineml/abstraction/expressions/base.py", line 83, in rhs
    self._rhs = Parser().parse(rhs)
  File "/home/asinha/rpmbuild/SOURCES/nineml-python-1.0/nineml/abstraction/expressions/parser.py", line 75, in parse
    return self._parse_expr(expr)
  File "/home/asinha/rpmbuild/SOURCES/nineml-python-1.0/nineml/abstraction/expressions/parser.py", line 96, in _parse_expr
    "{}\n\n{}".format(expr, e))
nineml.exceptions.NineMLMathParseError: Could not parse math-inline expression: !a



======================================================================
ERROR: test_triple_negation (unittests.abstraction_test.expressions_test.C89ToSympy_test)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/asinha/rpmbuild/SOURCES/nineml-python-1.0/nineml/abstraction/expressions/parser.py", line 92, in _parse_expr
    local_dict=self.inline_randoms_dict)
  File "/home/asinha/dump/nineml-virt/lib64/python3.7/site-packages/sympy/parsing/sympy_parser.py", line 960, in parse_expr
    code = stringify_expr(s, local_dict, global_dict, transformations)
  File "/home/asinha/dump/nineml-virt/lib64/python3.7/site-packages/sympy/parsing/sympy_parser.py", line 866, in stringify_expr
    tokens = transform(tokens, local_dict, global_dict)
  File "/home/asinha/dump/nineml-virt/lib64/python3.7/site-packages/sympy/parsing/sympy_parser.py", line 618, in factorial_notation
    raise TokenError
tokenize.TokenError

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/asinha/rpmbuild/SOURCES/nineml-python-1.0/test/unittests/abstraction_test/expressions_test.py", line 181, in test_triple_negation
    expr = Expression('!!!a')
  File "/home/asinha/rpmbuild/SOURCES/nineml-python-1.0/nineml/abstraction/expressions/base.py", line 72, in __init__
    self.rhs = rhs
  File "/home/asinha/rpmbuild/SOURCES/nineml-python-1.0/nineml/abstraction/expressions/base.py", line 83, in rhs
    self._rhs = Parser().parse(rhs)
  File "/home/asinha/rpmbuild/SOURCES/nineml-python-1.0/nineml/abstraction/expressions/parser.py", line 75, in parse
    return self._parse_expr(expr)
  File "/home/asinha/rpmbuild/SOURCES/nineml-python-1.0/nineml/abstraction/expressions/parser.py", line 96, in _parse_expr
    "{}\n\n{}".format(expr, e))
nineml.exceptions.NineMLMathParseError: Could not parse math-inline expression: !!!a



----------------------------------------------------------------------
Ran 226 tests in 90.965s

FAILED (errors=4)

These are the packages I've got here in the venv:

$ pip freeze
future==0.17.1
h5py==2.8.0
lxml==4.2.5
mpmath==1.0.0
nineml==1.0
nose==1.3.7
numpy==1.15.4
PyNEST==2.14.0
PyYAML==3.13
six==1.11.0
sympy==1.3

$ python --version
Python 3.7.1

@tclose
Copy link
Contributor

tclose commented Nov 15, 2018

Those new errors look to be caused by ! being treated as a factorial symbol in the tokenize module, where it should be treated as negation. We used to be able to add a special cases for this I think, although that functionality may have been lost in the tokenize migration. I will investigate tomorrow.

@sanjayankur31
Copy link
Contributor Author

Thanks very much! I reckon travis didn't catch it because it pulled in sympy 1.1 for some reason.

@tclose tclose mentioned this issue Nov 15, 2018
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@sanjayankur31 @tclose