Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

WMAgent: Creating a script for setting up additional environment variables to allow running unit tests under WMAgent.venv #12266

Open
todor-ivanov opened this issue Feb 19, 2025 · 0 comments
Labels
deployment Issue related to deployment of the services New Feature Unittest

Comments

@todor-ivanov
Copy link
Contributor

Impact of the new feature
Unit tests

Is your feature request related to a problem? Please describe.
Whenever one is working with a WMAgent virtual environment it is natural to develop and test/run unit tests directly in it. So far I've been running random unit test just by activating the environment and running the test directly, with the following sequence of commands:

[cmst1@vocms0260 WMAgent.venv3]$ . ./bin/activate
Setting up WMCore related environment variables:

(WMAgent.venv3) cmst1@vocms0260:WMAgent.venv3 $ ipython -i $WMA_DEPLOY_DIR/srv/WMCore/test/python/Utils_t/Utilities_t.py 
Python 3.9.21 (main, Dec  5 2024, 00:00:00) 
Type 'copyright', 'credits' or 'license' for more information
IPython 8.18.1 -- An enhanced Interactive Python. Type '?' for help.
%reload_ext autoreload
%autoreload 2
.........
----------------------------------------------------------------------
Ran 10 tests in 0.038s

OK

It is also natural to do the above, since at the already activated environment we have access to all the relevant backend databases and services etc. But...
While working on few issues lately together with @mapellidario and creating new unit tests that rely on database connections (both MariaDB and CouchDB), we figured out for few of the unit tests in our repository we need some more environment to be added which are mandatory for the unittests to be able run.

For that purpose we've created a short script which allows us to run those tests upon sourcing it inside the already active environment. Example: [1]

The current issue is about adopting this script into the WMcore codebase, such that it is available from the main repository and installed WMCore packages. And to also test the full set of unit tests we already have.

Describe the solution you'd like
Adopt the above mentioned script in the WMCore repository

Additional context
[1]

cmst1@vocms0260:WMAgent.venv3 $ . ./bin/activate
Setting up WMCore related environment variables:

(WMAgent.venv3) cmst1@vocms0260:WMAgent.venv3 $ . $WMA_ROOT_DIR/srv/WMCoreAux/deploy/deploy-unittest-venv.sh 
Activating: . /data/WMAgent.venv3/bin/activate
Setting up WMCore related environment variables:
Requirement already satisfied: nose in ./lib/python3.9/site-packages (1.3.7)

[notice] A new release of pip is available: 24.3.1 -> 25.0.1
[notice] To update, run: pip install --upgrade pip
Setting up WMCore related environment variables:
--------------
CREATE DATABASE wmagent_unittest
--------------

(WMAgent.venv3) cmst1@vocms0260:WMAgent.venv3 $ ipython -i $WMA_DEPLOY_DIR/srv/WMCore/test/python/WMCore_t/WorkQueue_t/WorkQueue_t.py 
Python 3.9.21 (main, Dec  5 2024, 00:00:00) 
Type 'copyright', 'credits' or 'license' for more information
IPython 8.18.1 -- An enhanced Interactive Python. Type '?' for help.
%reload_ext autoreload
%autoreload 2
Reading mocked CRIC data from the file /data/WMAgent.venv3/srv/WMCore/test/python/../data/Mock/CRICMockData.json
Running setUp with emulated services.
/data/WMAgent.venv3/srv/WMCore/src/python/WMCore/Database/DBFactory.py:83: SADeprecationWarning: The create_engine.convert_unicode parameter and corresponding dialect-level parameters are deprecated, and will be removed in a future release.  Modern DBAPIs support Python Unicode natively and this parameter is unnecessary.
  create_engine(self.dburl,
Installing WorkQueue app into database: http://127.0.0.1:5984/workqueue_t
...
Using MockDBSApi
...
.Running setUp with emulated services.
...
Using MockCRICApi
Using MockRucioApi: acct=wma_test, url=https://cms-rucio-auth-int.cern.ch, auth

@todor-ivanov todor-ivanov added deployment Issue related to deployment of the services New Feature Unittest labels Feb 19, 2025
@todor-ivanov todor-ivanov changed the title Creating a script for setting up addition environment variables to allow running unit tests under WMAgent.venv WMAgent: Creating a script for setting up additional environment variables to allow running unit tests under WMAgent.venv Feb 19, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
deployment Issue related to deployment of the services New Feature Unittest
Projects
None yet
Development

No branches or pull requests

1 participant