-
Notifications
You must be signed in to change notification settings - Fork 32
Developer Introduction
The general concept of the plugin is that the user starts VisTrails and either builds or uses and existing workflow, made up of connected modules. These modules are GUI wrappers around an object called a ProcessUnit, which represents some process that changes a set of input files into a set of output files. These processes are command-line executable tools (such as python or BASH scripts or cdo tools) that take arguments of the form ./script_name POSITIONAL_ARGUMENT input_file(s) output_file --KEYWORD ARGUMENT --KEYWORD ARGUMENT, with some flexibility as to how the arguments are arranged.
The way these sets of files are passed from one module to another is by being collected in an object called a DataSet. A DataSet is an object that has a pattern to generate file names (looking something like "/%colour%/%animal%"), and a set of constraints (something like colours = ['blue', green'], animals = ['kangaroo', 'moose']).
You can then ask the DataSet to give you the correct file for a blue kangaroo, or a green moose and it will give you /blue/kangaroo or /green/moose.
The DataSet is an abstract class, there are two implementations - The PatternDataSet is constructed by globbing a pattern on the file system and extracting all the constraints from the matching file names. This represents real files on the file system and is used for initial input. A FileCreator doesn't look at the file system - it holds an output pattern, a set of constraints and a hash table of which combinations of constraints make real files. This is used for communication between VisTrails modules.
The ProcessUnit object handles the creation of the shell commands you want the plugin to run. The object is constructed with three arguments - a list of input DataSets to get input files, a pattern to write out the results and a shell command to link inputs with outputs.
As an example, if we use the animal/colour DataSet from before (called AnimalColour), we could create a ProcessUnit as:
ProcessUnit([AnimalColour], "/this/is/the/output/%animal%/%colour%.thing.filetype", "echo")
When we call the ProcessUnit.execute() method, the following commands will be run:
echo /blue/kangaroo /this/is/the/output/kangaroo/blue.thing.filetype
echo /green/kangaroo /this/is/the/output/kangaroo/green.thing.filetype
echo /blue/moose /this/is/the/output/moose/blue.thing.filetype
echo /green/mooose /this/is/the/output/moose/green.thing.filetype
If instead we construct the ProcessUnit with:
ProcessUnit([AnimalColour], "/different/output/%animal%.filetype", "echo")
calling execute will give:
echo /blue/kangaroo /green/kangaroo /different/output/kangaroo.filetype
echo /blue/moose /green/moose /different/output/moose.filetype
Development on the plugin follows quite a simple workflow. If new features are introduced then they are first developed on a shared devel branch. Any emergency hotfixes can be committed straight on to the master branch after being tested, then will be merged across to devel.
Periodic releases are tagged off the master branch. This allows users who want a more stable experience or are not looking to be involved with development of the plugin to pull down a stable version.
The plugin tags are based on semantic versioning, with MAJOR.MINOR.PATCH version names.