-
Notifications
You must be signed in to change notification settings - Fork 32
Developer Introduction
The general concept of the plugin is that the user starts VisTrails and either builds or uses and existing workflow, made up of connected modules. These modules are GUI wrappers around an object called a ProcessUnit
, which represents some process that changes a set of input files into a set of output files. These processes are command-line executable tools (such as python
or BASH
scripts or cdo
tools) that take arguments of the form ./script_name POSITIONAL_ARGUMENT input_file(s) output_file --KEYWORD ARGUMENT --KEYWORD ARGUMENT
, with some flexibility as to how the arguments are arranged.
The way these sets of files are passed from one module to another is by being collected in an object called a DataSet
. A DataSet
is an object that has a pattern to generate file names (looking something like "/%colour%/%animal%"
), and a set of constraints (something like colours = ['blue', green'], animals = ['kangaroo', 'moose']
).
You can then ask the DataSet
to give you the correct file for a blue kangaroo, or a green moose and it will give you /blue/kangaroo
or /green/moose
.
The DataSet
is an abstract class, there are two implementations - The PatternDataSet
is constructed by globbing a pattern on the file system and extracting all the constraints from the matching file names. This represents real files on the file system and is used for initial input. A FileCreator
doesn't look at the file system - it holds an output pattern, a set of constraints and a hash table of which combinations of constraints make real files. This is used for communication between VisTrails modules.
The ProcessUnit
object handles the creation of the shell commands you want the plugin to run. The object is constructed with three arguments - a list of input DataSet
s to get input files, a pattern to write out the results and a shell command to link inputs with outputs.
As an example, if we use the animal/colour DataSet from before (called AnimalColour
), we could create a ProcessUnit as:
ProcessUnit([AnimalColour], "/this/is/the/output/%animal%/%colour%.thing.filetype", "echo")
When we call the ProcessUnit.execute()
method, the following commands will be run:
echo /blue/kangaroo /this/is/the/output/kangaroo/blue.thing.filetype
echo /green/kangaroo /this/is/the/output/kangaroo/green.thing.filetype
echo /blue/moose /this/is/the/output/moose/blue.thing.filetype
echo /green/mooose /this/is/the/output/moose/green.thing.filetype
If instead we construct the ProcessUnit
with:
ProcessUnit([AnimalColour], "/different/output/%animal%.filetype", "echo")
calling execute
will give:
echo /blue/kangaroo /green/kangaroo /different/output/kangaroo.filetype
echo /blue/moose /green/moose /different/output/moose.filetype
Development on the plugin follows quite a simple workflow. If new features are introduced then they are first developed on a shared devel
branch. Any emergency hotfixes can be committed straight on to the master
branch after being tested, then will be merged across to devel
.
Periodic releases are tagged off the master
branch. This allows users who want a more stable experience or are not looking to be involved with development of the plugin to pull down a stable version.
The plugin tags are based on semantic versioning, with MAJOR.MINOR.PATCH
version names.