many enhancements to the contributing guide, some cleanups for the Makefile

including a new target: `make cleanup`

Co-authored-by: Michael R. Crusoe <[email protected]>

Author: Jasper

CONTRIBUTING.md

@@ -1,32 +1,68 @@
 Style guide:
-- PEP-8
-- Python 3.7+ compatible code
+- PEP-8 (as implemented by the `black` code formatting tool)
+- Python 3.6+ compatible code
 - PEP-484 type hints
-- Vertically align the type hints in function definitions
 
-The development is done using ``git``, we encourage you to get familiar with it.
+The development is done using `git`, we encourage you to get familiar with it.
 Here's a rough guide (improvements are welcome!) 
 To get the code and start working on the changes you can start a console and:
-- Clone the cwltool: ``git clone https://github.com/common-workflow-language/cwltool.git``
-- Switch to cwltool directory: ``cd cwltool``
+- Clone the cwltool: `git clone https://github.com/common-workflow-language/cwltool.git`
+- Switch to cwltool directory: `cd cwltool`
 
-In order to contribute to the development of ``cwltool``, the source code needs to pass the test before your changes are accepted.
-There are a couple ways to test the code with your changes: let `tox` manage installation and test running in virtual environments, or do it manually (preferably in a virtual environment):
-- Install ``tox`` preferably using the OS' package manager, otherwise it can be installed with ``pip install --user -U tox``
+In order to contribute to the development of `cwltool`, the source code needs to
+pass the test before your changes are accepted. There are a couple ways to test
+the code with your changes: let `tox` manage installation and test running in
+virtual environments, or do it manually (preferably in a virtual environment):
+- Install `tox` preferably using the OS' package manager, otherwise it can be
+  installed with `pip install --user -U tox`
 - Make your changes to the code and add tests for new cool things you're adding!
-- Run the tests with the command ``tox``, it's recommended to use some parameters as tox will try to run all the checks in all available python interpreters.
-- The important tests to run are ``unit tests`` and  ``type tests``.
-  To run these two in Python 3.7, we can tell tox to run only those tests by running: ``tox -e py37-unit,py37-mypy2,py37-mypy3``.
-- Run ``tox -l`` to see all available tests and runtimes
+- Run the tests with the command `tox`, it's recommended to use some parameters
+  as tox will try to run all the checks in all available python interpreters.
+- The important tests to run are "unit tests" and "type tests". 
+  To run these two in Python 3.7 (for example), we can tell tox to run only those
+  tests by running: `tox -e py37-mypy`. (the `mypy` tox target also runs the
+  unit tests; to just run the type checker use `make mypy` as shown below)
+- Run `tox -l` to see all available tests and runtimes
 
 For the more traditional workflow:
-- Create a virtual environment: ``python3 -m venv cwltool``
-- To begin using the virtual environment, it needs to be activated: ``source bin/activate``
-- To check if you have the virtual environment set up: ``which python`` and it should point to python executable in your virtualenv
-- Install cwltool: ``pip install -e .``
-- Check the version which might be different from the version installed in general on any system: ``cwltool --version``
+- Create a virtual environment: `python3 -m venv cwltool`
+  (if you have multiple version of Python installed, then you can replace `python3`
+  in this command with the version you want to use, `python3.10` for example)
+- To begin using the virtual environment, it needs to be activated: `source venv/bin/activate`
+- To check if you have the virtual environment set up: `which python`
+  and it should point to python executable in your virtualenv
+- Install cwltool in development mode, along with development tools:
+  `make install-dep dev`
+- Check the `cwltool` version which might be different from the version installed in
+  general on your system: `cwltool --version`
 - Make your changes to the code and add tests for new cool things you're adding!
-- Run the unit-tests to see : ``python setup.py test``
-- After you're done working on ``cwltool``, you can deactivate the virtual environment: ``deactivate``
+- Run the unit-tests to see : `make test`
+- After you're done working on `cwltool`, you can deactivate the virtual
+  environment: `deactivate`
 
-When tests are passing, you can simply commit and create a PR on ``cwltool`` repo
+Before you commit your code (or at least before you push to GitHub) it is
+recommended to run `make cleanup`
+(which is a shortcut for `make sort_imports format flake8 diff_pydocstyle_report`)
+to fix common issues and point out any remaining code formatting issues
+that cannot be fixed automatically.
+
+When tests are passing, you can create git commits in a new branch and push to
+GitHub to make a pull request on `cwltool` repo
+
+Useful `make` commands:
+The `make` commands below can help you to install the dependencies, format and test your code
+
+- `make help` to show the list of commands that can be used with the make function
+- `make install-dep` will install the dependencies needed by `cwltool`
+- `make format` will clean up your code according to the accepted python standard
+- `make test` will perform the tests of `cwltool` (can take a while to run all tests)
+- `make sort_imports` will clean up and sort your import statements
+- `make remove_unused_imports` will remove any unneeded `import` statements
+- `make flake8` will catch format issues that `make format` cannot fix automatically
+- `make diff_pydocstyle_report` check Python docstring style for changed files only
+- `make diff-cover` to run the tests and point out which lines of code that have been
+   changed as compared to the `main` branch are missing test coverage. This is also calculated
+   automatically after all the automatic tests run when you open a Pull Request on GitHub;
+   running it locally can help you confirm that any changes or additions
+   to the tests are sufficient before pushing to GitHub.
+- `make cleanup` is a shortcut for `make sort_imports format flake8 diff_pydocstyle_report`

Makefile

@@ -36,14 +36,17 @@ VERSION=3.1.$(shell TZ=UTC git log --first-parent --max-count=1 \
 mkfile_dir := $(dir $(abspath $(lastword $(MAKEFILE_LIST))))
 UNAME_S=$(shell uname -s)
 
-## all         : default task
+## all                    : default task (install cwltool in dev mode)
 all: dev
 
-## help        : print this help message and exit
+## help                   : print this help message and exit
 help: Makefile
 	@sed -n 's/^##//p' $<
 
-## install-dep : install most of the development dependencies via pip
+## cleanup                : shortcut for "make sort_imports format flake8 diff_pydocstyle_report"
+cleanup: sort_imports format flake8 diff_pydocstyle_report
+
+## install-dep            : install most of the development dependencies via pip
 install-dep: install-dependencies
 
 install-dependencies: FORCE
@@ -53,20 +56,20 @@ install-dependencies: FORCE
 install-doc-dep:
 	pip install -r docs/requirements.txt
 
-## install-deb-dep: install most of the dev dependencies via apt-get
+## install-deb-dep        : install many of the dev dependencies via apt-get
 install-deb-dep:
 	sudo apt-get install $(DEBDEVPKGS)
 
-## install     : install the ${MODULE} module and schema-salad-tool
+## install                : install the cwltool module with the "deps" dependency (galaxy-tool-util)
 install: FORCE
 	pip install .[deps]
 
-## dev     : install the ${MODULE} module in dev mode
+## dev                    : install the cwltool module in dev mode
 dev: install-dep
 	pip install -e .[deps]
 
 
-## dist        : create a module package for distribution
+## dist                   : create a module package for distribution
 dist: dist/${MODULE}-$(VERSION).tar.gz
 
 check-python3:
@@ -76,11 +79,11 @@ check-python3:
 dist/${MODULE}-$(VERSION).tar.gz: check-python3 $(SOURCES)
 	python setup.py sdist bdist_wheel
 
-## docs	       : make the docs
+## docs                   : make the docs
 docs: FORCE
 	cd docs && $(MAKE) html
 
-## clean       : clean up all temporary / machine-generated files
+## clean                  : clean up all temporary / machine-generated files
 clean: check-python3 FORCE
 	rm -f ${MODULE}/*.pyc tests/*.pyc *.so ${MODULE}/*.so
 	rm -Rf ${MODULE}/__pycache__/
@@ -89,37 +92,37 @@ clean: check-python3 FORCE
 	rm -f diff-cover.html
 
 # Linting and code style related targets
-## sorting imports using isort: https://github.com/timothycrosley/isort
+## sort_import            : sorting imports using isort: https://github.com/timothycrosley/isort
 sort_imports: $(PYSOURCES) mypy-stubs
 	isort $^
 
 remove_unused_imports: $(PYSOURCES)
 	autoflake --in-place --remove-all-unused-imports $^
 
 pep257: pydocstyle
-## pydocstyle      : check Python docstring style
+## pydocstyle             : check Python docstring style
 pydocstyle: $(PYSOURCES)
 	pydocstyle --add-ignore=D100,D101,D102,D103 $^ || true
 
 pydocstyle_report.txt: $(PYSOURCES)
 	pydocstyle setup.py $^ > $@ 2>&1 || true
 
-## diff_pydocstyle_report	: check Python docstring style for changed files only
+## diff_pydocstyle_report : check Python docstring style for changed files only
 diff_pydocstyle_report: pydocstyle_report.txt
 	diff-quality --compare-branch=main --violations=pydocstyle --fail-under=100 $^
 
-## codespell   : check for common mispellings
+## codespell              : check for common mispellings
 codespell:
 	codespell -w $(shell git ls-files | grep -v cwltool/schemas | grep -v cwltool/jshint/ | grep -v mypy-stubs)
 
-## format      : check/fix all code indentation and formatting (runs black)
+## format                 : check/fix all code indentation and formatting (runs black)
 format:
 	black --exclude cwltool/schemas setup.py cwltool.py cwltool tests mypy-stubs
 
 format-check:
 	black --diff --check --exclude cwltool/schemas setup.py cwltool.py cwltool tests mypy-stubs
 
-## pylint      : run static code analysis on Python code
+## pylint                 : run static code analysis on Python code
 pylint: $(PYSOURCES)
 	pylint --msg-template="{path}:{line}: [{msg_id}({symbol}), {obj}] {msg}" \
                 $^ -j0|| true
@@ -154,18 +157,18 @@ diff-cover: coverage.xml
 diff-cover.html: coverage.xml
 	diff-cover --compare-branch=main $^ --html-report $@
 
-## test        : run the ${MODULE} test suite
+## test                   : run the ${MODULE} test suite
 test: check-python3 $(PYSOURCES)
 	python -m pytest -rs ${PYTEST_EXTRA}
 
-## testcov     : run the ${MODULE} test suite and collect coverage
+## testcov                : run the ${MODULE} test suite and collect coverage
 testcov: check-python3 $(PYSOURCES)
 	python -m pytest -rs --cov --cov-config=.coveragerc --cov-report= ${PYTEST_EXTRA}
 
 sloccount.sc: $(PYSOURCES) Makefile
 	sloccount --duplicates --wide --details $^ > $@
 
-## sloccount   : count lines of code
+## sloccount              : count lines of code
 sloccount: $(PYSOURCES) Makefile
 	sloccount $^