Support running tests with mxmake/pip-installed packages #1020
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
As proof of concept, it only runs the unit tests.
This was referenced May 28, 2025
Closed
Use GIT_CLONE_DEPTH=1 env variable for mxdev.
With 'make test' on 3.13 I get two errors in tests, so I wonder if that is specific for Python 3.13.
```
Error in test testNormalizeLatin1 (Products.CMFPlone.tests.testUnicodeSplitter.TestCaseNormalizer.testNormalizeLatin1)
Traceback (most recent call last):
File "/opt/hostedtoolcache/Python/3.13.3/x64/lib/python3.13/unittest/case.py", line 58, in testPartExecutor
yield
File "/opt/hostedtoolcache/Python/3.13.3/x64/lib/python3.13/unittest/case.py", line 651, in run
self._callTestMethod(testMethod)
File "/opt/hostedtoolcache/Python/3.13.3/x64/lib/python3.13/unittest/case.py", line 606, in _callTestMethod
if method() is not None:
File "/home/runner/work/buildout.coredev/buildout.coredev/src/Products.CMFPlone/Products/CMFPlone/tests/testUnicodeSplitter.py", line 153, in testNormalizeLatin1
saved = _setlocale(*LATIN1)
File "/home/runner/work/buildout.coredev/buildout.coredev/src/Products.CMFPlone/Products/CMFPlone/tests/testUnicodeSplitter.py", line 41, in _setlocale
raise ValueError(
ValueError: Unsupported locale. These tests need at least one of the following locales available on your system: ('en_US.ISO-8859-1', 'en_US.ISO8859-15', 'en_GB.ISO8859-15', 'de_DE@euro', 'fr_FR@euro', 'nl_NL@euro')
```
Taken over from bare.yml.
Deprecation and other warnings are drowning out the useful output.
Windows does not recognize the 'export' command.
It would be good if this all works.
It does not exist on Mac, and likely not on Windows either.
Running `make` works, so that is good to know. But there are several [test failures](https://github.com/plone/buildout.coredev/actions/runs/15449045805/job/43486236797?pr=1020): ``` Ran 1961 tests with 17 failures, 4 errors and 1 skipped in 1 minutes 15.319 seconds. ``` Let me paste the complete test output (long): ``` Run tests using ./run-tests.sh Running zope-testrunner --auto-color --auto-progress --auto-path -u -s borg.localrole -s collective.monkeypatcher -s diazo -s five.customerize -s five.intid -s plone.alterego -s plone.api -s plone.app.caching -s plone.app.content -s plone.app.contentlisting -s plone.app.contentmenu -s plone.app.contentrules -s plone.app.contenttypes -s plone.app.customerize -s plone.app.dexterity -s plone.app.discussion -s plone.app.event -s plone.app.i18n -s plone.app.intid -s plone.app.iterate -s plone.app.layout -s plone.app.linkintegrity -s plone.app.locales -s plone.app.lockingbehavior -s plone.app.multilingual -s plone.app.portlets -s plone.app.querystring -s plone.app.redirector -s plone.app.registry -s plone.app.relationfield -s plone.app.testing -s plone.app.textfield -s plone.app.theming -s plone.app.upgrade -s plone.app.users -s plone.app.uuid -s plone.app.versioningbehavior -s plone.app.viewletmanager -s plone.app.vocabularies -s plone.app.widgets -s plone.app.workflow -s plone.app.z3cform -s plone.autoform -s plone.batching -s plone.behavior -s plone.base -s plone.browserlayer -s plone.cachepurging -s plone.caching -s plone.classicui -s plone.contentrules -s plone.dexterity -s plone.distribution -s plone.event -s plone.exportimport -s plone.folder -s plone.formwidget.namedfile -s plone.i18n -s plone.indexer -s plone.intelligenttext -s plone.keyring -s plone.locking -s plone.memoize -s plone.namedfile -s plone.outputfilters -s plone.portlet.collection -s plone.portlet.static -s plone.portlets -s plone.protect -s plone.registry -s plone.resource -s plone.resourceeditor -s plone.rest -s plone.restapi -s plone.rfc822 -s plone.scale -s plone.schema -s plone.schemaeditor -s plone.session -s plone.staticresources -s plone.stringinterp -s plone.subrequest -s plone.supermodel -s plone.testing -s plone.theme -s plone.transformchain -s plone.uuid -s plone.volto -s plone.z3cform -s plonetheme.barceloneta -s Products.CMFDiffTool -s Products.CMFDynamicViewFTI -s Products.CMFEditions -s Products.CMFPlacefulWorkflow -s Products.CMFPlone -s Products.CMFUid -s Products.DateRecurringIndex -s Products.DCWorkflow -s Products.ExtendedPathIndex -s Products.GenericSetup -s Products.isurlinportal -s Products.MimetypesRegistry -s Products.PlonePAS -s Products.PluggableAuthService -s Products.PluginRegistry -s Products.PortalTransforms -s Products.statusmessages -s Products.ZopeVersionControl -s repoze.xmliter Running zope.testrunner.layer.UnitTests tests: Set up zope.testrunner.layer.UnitTests in 0.000 seconds. Error in test test_transform_filename (diazo.tests.test_wsgi.TestXSLTMiddleware.test_transform_filename) Traceback (most recent call last): File "C:\hostedtoolcache\windows\Python\3.11.9\x64\Lib\unittest\case.py", line 57, in testPartExecutor yield File "C:\hostedtoolcache\windows\Python\3.11.9\x64\Lib\unittest\case.py", line 623, in run self._callTestMethod(testMethod) File "C:\hostedtoolcache\windows\Python\3.11.9\x64\Lib\unittest\case.py", line 579, in _callTestMethod if method() is not None: File "D:\a\buildout.coredev\buildout.coredev\.venv\Lib\site-packages\diazo\tests\test_wsgi.py", line 180, in test_transform_filename os.unlink(filename) PermissionError: [WinError 32] The process cannot access the file because it is being used by another process: 'C:\\Users\\RUNNER~1\\AppData\\Local\\Temp\\tmp7wemmb49' Failure in test test_rebuild (plone.app.redirector.tests.test_storage.TestStorage.test_rebuild) Traceback (most recent call last): File "C:\hostedtoolcache\windows\Python\3.11.9\x64\Lib\unittest\case.py", line 57, in testPartExecutor yield File "C:\hostedtoolcache\windows\Python\3.11.9\x64\Lib\unittest\case.py", line 623, in run self._callTestMethod(testMethod) File "C:\hostedtoolcache\windows\Python\3.11.9\x64\Lib\unittest\case.py", line 579, in _callTestMethod if method() is not None: File "D:\a\buildout.coredev\buildout.coredev\.venv\Lib\site-packages\plone\app\redirector\tests\test_storage.py", line 517, in test_rebuild self.assertTrue(time1 < new_time < time2) File "C:\hostedtoolcache\windows\Python\3.11.9\x64\Lib\unittest\case.py", line 715, in assertTrue raise self.failureException(msg) AssertionError: False is not true Failure in test test_storage_date (plone.app.redirector.tests.test_storage.TestStorage.test_storage_date) Traceback (most recent call last): File "C:\hostedtoolcache\windows\Python\3.11.9\x64\Lib\unittest\case.py", line 57, in testPartExecutor yield File "C:\hostedtoolcache\windows\Python\3.11.9\x64\Lib\unittest\case.py", line 623, in run self._callTestMethod(testMethod) File "C:\hostedtoolcache\windows\Python\3.11.9\x64\Lib\unittest\case.py", line 579, in _callTestMethod if method() is not None: File "D:\a\buildout.coredev\buildout.coredev\.venv\Lib\site-packages\plone\app\redirector\tests\test_storage.py", line 123, in test_storage_date self.assertTrue(time1 < info[1] < time2) File "C:\hostedtoolcache\windows\Python\3.11.9\x64\Lib\unittest\case.py", line 715, in assertTrue raise self.failureException(msg) AssertionError: False is not true Failure in test test_storage_update_mixed (plone.app.redirector.tests.test_storage.TestStorage.test_storage_update_mixed) Traceback (most recent call last): File "C:\hostedtoolcache\windows\Python\3.11.9\x64\Lib\unittest\case.py", line 57, in testPartExecutor yield File "C:\hostedtoolcache\windows\Python\3.11.9\x64\Lib\unittest\case.py", line 623, in run Failure in test testDetectPersistentSubObjectChange (Products.ZopeVersionControl.tests.testVersionControl.VersionControlTestsWithCommits.testDetectPersistentSubObjectChange) Traceback (most recent call last): File "C:\hostedtoolcache\windows\Python\3.11.9\x64\Lib\unittest\case.py", line 57, in testPartExecutor yield File "C:\hostedtoolcache\windows\Python\3.11.9\x64\Lib\unittest\case.py", line 623, in run self._callTestMethod(testMethod) File "C:\hostedtoolcache\windows\Python\3.11.9\x64\Lib\unittest\case.py", line 579, in _callTestMethod if method() is not None: File "D:\a\buildout.coredev\buildout.coredev\.venv\Lib\site-packages\Products\ZopeVersionControl\tests\testVersionControl.py", line 643, in testDetectPersistentSubObjectChange self.assertTrue(repository.isResourceChanged(folder2)) File "C:\hostedtoolcache\windows\Python\3.11.9\x64\Lib\unittest\case.py", line 715, in assertTrue raise self.failureException(msg) AssertionError: False is not true Failure in test testIsResourceChanged (Products.ZopeVersionControl.tests.testVersionControl.VersionControlTestsWithCommits.testIsResourceChanged) Traceback (most recent call last): File "C:\hostedtoolcache\windows\Python\3.11.9\x64\Lib\unittest\case.py", line 57, in testPartExecutor yield File "C:\hostedtoolcache\windows\Python\3.11.9\x64\Lib\unittest\case.py", line 623, in run self._callTestMethod(testMethod) File "C:\hostedtoolcache\windows\Python\3.11.9\x64\Lib\unittest\case.py", line 579, in _callTestMethod if method() is not None: File "D:\a\buildout.coredev\buildout.coredev\.venv\Lib\site-packages\Products\ZopeVersionControl\tests\testVersionControl.py", line 89, in testIsResourceChanged self.assertTrue(repository.isResourceChanged(document)) File "C:\hostedtoolcache\windows\Python\3.11.9\x64\Lib\unittest\case.py", line 715, in assertTrue raise self.failureException(msg) AssertionError: False is not true Ran 1961 tests with 17 failures, 4 errors and 1 skipped in 1 minutes 15.319 seconds. Tearing down left over layers: Tear down zope.testrunner.layer.UnitTests in 0.000 seconds. ```
|
This is now also tested on GitHub Actions, by running Windows did not quite work, giving several test failures. This would likely happen when using So I think this is ready for merge. Let's do Jenkins for the minor version updates: @jenkins-plone-org please run jobs |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Just try it:
make test. For the moment as proof of concept this only runs the unit tests. But locally it works for the normal tests as well. I did not try the robot tests, they may need some more setup withrfbrowser.Main changes:
qa.testdomain in theMakefile, and runmxmake updateto update some relates files.run-tests.shscript in the repo root, instead of the automatically generated.mxmake/files/run-tests.sh.zope.testrunner7.4 so we can use--auto-path. This is needed, because otherwise we would need to add amxmake-test-pathoption in each section inmxsources.ini, like was done as example in PR Run tests from makefile #985.This PR also includes a minor sync of constraints and checkouts.
I wanted to add a GitHub Action job to run
make test. But this could use a change inmxdevso we do not make a full clone of each checkout, but use a git depth of 1. I have opened an issue. [Update: that PR is merged and released, and we use it now.]