Skip to content

Commit

Permalink
Updated documentation
Browse files Browse the repository at this point in the history 
- Written tutorials.
- Updated existing documentation.
  • Loading branch information
@devos50
devos50 committed Jul 19, 2021
1 parent 534a1c7 commit 0d4394c
Show file tree
Hide file tree
Showing 27 changed files with 601 additions and 492 deletions.
3 changes: 3 additions & 0 deletions .gitignore
View file
Original file line number Diff line number Diff line change
Expand Up @@ -53,3 +53,6 @@ output/
.idea

.DS_Store

# docs
docs/_build/**
120 changes: 4 additions & 116 deletions README.md
View file
Original file line number Diff line number Diff line change
@@ -1,8 +1,8 @@
Gumby
=====

An experiment runner framework for IPv8 and Tribler.
Gumby allows developers and scientists to devise complex experiments and run them on the DAS5 supercomputer.
An experiment runner framework to run local and distributed experiments.
Gumby allows developers and scientists to design complex experiments and run them on the DAS5 supercomputer.

Notable features:
- Run IPv8/Tribler experiments with thousands of instances in a local or remote (DAS5) environment.
Expand All @@ -26,117 +26,5 @@ Next, clone this repository from GitHub by running the following command:
git clone https://github.com/tribler/gumby
```

## Running your first experiment

The configuration of Gumby experiments are defined by configuration files.
To show how a Gumby experiment is structured and executed, we provide a basic experiment that simply generates some system I/O and plots some metrics.
After cloning the Gumby repository (see above), simply call `gumby/run.py` passing your experiment's config file path as argument.

Example:

```
gumby/run.py gumby/experiments/dummy/local_processguard.conf
```

This will run the experiment, which should take around 15 seconds.
When the experiment terminates, it will plot various system metrics like CPU usage, I/O read and written.
These metrics can be found in the `output` directory, together with various log files.

_When running the experiment for the first time with `ggplot2` installed, it will compile the `ggplot2` R package and its dependencies.
This can take a while to complete._

## Anatomy of an experiment

We now briefly explain how the experiment that was executed in the previous section, is composed.

#### Configuration file

Each experiment should be defined by a configuration file that contains information about the environment and parameters of the experiment.
The content of the `local_processguard` experiment configuration file is shown below:

```
experiment_name = LocalProcessGuard
local_instance_cmd = process_guard.py -c "(yes CPU > /dev/null & find /etc /usr > /dev/null ; wait)" -t 10 -m $OUTPUT_DIR -o $OUTPUT_DIR --network
post_process_cmd = graph_process_guard_data.sh
```

The configuration file defines some keys and values.
Each experiment requires the `experiment_name` variable to be set.
The `local_instance_cmd` describes the command that Gumby should run.
This usually points to an executable file, i.e. a Python file or a script written in bash.
In the example above, it invokes the `process_guard.py` script with a specific command.
The `process_guard.py` script is (another) wrapper around a command and is responsible for monitoring and writing away various statistics of the process, like CPU, memory and I/O usage.
The command that should be executed by the process guard is provided by the `-c` argument, and one can spawn multiple, parallel processes by passing the desired number instanced with `-n`.
The `post_process_cmd` variable defines an optional script that should be executed *after* the experiment is finished.
The `graph_process_guard_data.sh` scripts reads the statistics as written away by the `process_guard` file, and plots them with R.

This should provide you with a basic understanding of how to run a simple experiment on your local computer.
The remainder of this document will explain more advanced concepts of Gumby.

## Running experiments on the DAS5.

Gumby supports running experiments on the [DAS5 compute cluster](https://www.cs.vu.nl/das5).
An example configuration file to do so is given below:

```
experiment_name = "simple_das5"
local_setup_cmd = 'das4_setup.sh'
local_instance_cmd = 'das4_reserve_and_run.sh'
# How many nodes do we want?
node_amount = 4
# Kill the processes if they don't die after this many seconds
node_timeout = 350
# How many processes do we want to spawn?
instances_to_run = 100
# What command do we want to run on each instance?
das4_node_command = "basic_experiment.py"
```

This configuration file will run a basic experiment on the DAS5 compute cluster.
The `das4_response_and_run.sh` script will automatically find a cluster with available nodes and reserve them for the time indicated by `node_timeout`.
The `local_setup_cmd` will prepare the environment on the remote HEAD node.
Note that you can specify the total number of nodes you want to reserve with the `node_amount` variable, and the total instances you want to run with `instances_to_run`.
Each instance executes the `das4_node_command`, (which is `basic_experiment.py` in this experiment).

## Integrating IPv8/Tribler

For specific information on how to use Gumby to run an experiment with IPv8 or Tribler, please have a look at the TrustChain experiment.
The required files for this experiment can be found in the `gumby/experiments/trustchain` directory.

## Scenario files

Often, you want to execute a specific action on one or more running instances, at specified time intervals.
Gumby allows you to specify a scenario file, which describes one or more actions to be executed during the experiment.
An example of the scenario file used during the basic TrustChain experiment is given below:

```
&module gumby.modules.tribler_module.TriblerModule
&module experiments.trustchain.trustchain_module.TrustchainModule
@0:1 isolate_ipv8_overlay TrustChainCommunity
@0:2 start_session
@0:5 init_trustchain
@0:10 introduce_peers max_peers=10
@0:30 annotate start-creating-blocks
@0:30 start_requesting_signatures
@0:90 stop_requesting_signatures
@0:95 write_overlay_statistics
@0:95 write_trustchain_statistics
@0:95 commit_blocks_to_db
@0:100 stop_session
@0:105 stop
```

The scenario file to be used can specified with the `scenario_file` variable in the experiment configuration file.
Note that scenario files are only correctly loaded if you use `launch_scenario.py` as `das4_node_command`.

## Tutorial
A tutorial for creating your first Gumby experiment is availble [here](docs/hello_world.rst).
## Tutorials
A tutorial for creating your first Gumby experiment is available [here](docs/tutorials/experiment_tutorial_1.rst).
20 changes: 20 additions & 0 deletions docs/Makefile
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,20 @@
# Minimal makefile for Sphinx documentation
#

# You can set these variables from the command line, and also
# from the environment for the first two.
SPHINXOPTS ?=
SPHINXBUILD ?= sphinx-build
SOURCEDIR = .
BUILDDIR = _build

# Put it first so that "make" without argument is like "make help".
help:
@$(SPHINXBUILD) -M help "$(SOURCEDIR)" "$(BUILDDIR)" $(SPHINXOPTS) $(O)

.PHONY: help Makefile

# Catch-all target: route all unknown targets to Sphinx using the new
# "make mode" option. $(O) is meant as a shortcut for $(SPHINXOPTS).
%: Makefile
@$(SPHINXBUILD) -M $@ "$(SOURCEDIR)" "$(BUILDDIR)" $(SPHINXOPTS) $(O)
87 changes: 15 additions & 72 deletions docs/community_isolation.rst
View file
Original file line number Diff line number Diff line change
@@ -1,83 +1,26 @@
***********************
Isolating IPv8 overlays
***********************

The purpose of this document is to show a means of isolating IPv8 overlays from outside interference in Gumby experiments.
This document assumes the reader has a basic understanding of running Gumby experiments, creating IPv8 overlays and running them through the ``TriblerExperimentScriptClient`` class.
This document assumes the reader has a basic understanding of running Gumby experiments, creating IPv8 overlays and running them.

*************************************
Isolating and replacing IPv8 overlays
*************************************
As you may have noticed, the overlays loaded by the ``TriblerExperimentScriptClient`` are the live overlays as loaded by Tribler (which you can toggle by setting the correct flags in the ``SessionConfig``).
As you may have noticed, some of the overlays loaded through Gumby are the actual overlays that are also re-used in production.
In some cases this may be desirable functionality, in other cases one would like to isolate these overlays as such that they do not communicate with third parties.

How have we solved this in the past?
As you may know, part of the unique identification of a IPv8 overlay is its master peer definition.
Previously, one was required to create a subclass of the overlay under test in Gumby which had a different master peer definition.
Even though this is still possible, a system has been implemented in Gumby which allows you to easily isolate and/or replace these existing Tribler overlays or add your own.
As you may know, part of the unique identification of a IPv8 overlay is its community ID.
Previously, one was required to create a subclass of the overlay under test in Gumby which had a different overlay ID.
Even though this is still possible, a system has been implemented in Gumby which allows you to easily isolate and/or replace these existing overlays or add your own.

Isolation
---------
To demonstrate the use of overlay isolation we will use the following subclass of ``TriblerExperimentScriptClient``:

.. code-block:: python
class MyTriblerExperimentScriptClientSubclass(TriblerExperimentScriptClient):
def create_overlay_loader(self):
loader = super(MyTriblerExperimentScriptClientSubclass, self).create_community_loader()
loader.isolate("HiddenTunnelCommunity")
return loader
What we are doing here is overwriting the ``TriblerExperimentScriptClient.create_community_loader`` method and modifying the default ``IsolatedCommunityLoader`` it returns.
Specifically, we are asking the ``IsolatedCommunityLoader`` to isolate a community with the name ``"HiddenTunnelCommunity"``, which happens to be one of the communities loaded by default (if we don't modify the ``SessionConfig``).

You can also perform isolation with your own communities.
To do this, you will have to write your own launcher.
We will go over the basics of this in the following section.

My First CommunityLauncher
--------------------------
Now that you know how to isolate existing communities, let's go over adding your own communities to ``TriblerExperimentScriptClient``.
In this example we will create a custom ``CommunityLauncher``, in the next section we will discuss more advanced functionality which the ``CommunityLauncher`` offers.
Consider the following minimal example, which sets up a launcher for some community class ``MyFirstCommunity``:

.. code-block:: python
class MyFirstCommunityLauncher(CommunityLauncher):
def get_name(self):
return "MyFirstCommunity"
def get_community_class(self):
return MyFirstCommunity
class MyTriblerExperimentScriptClientSubclass(TriblerExperimentScriptClient):
def create_community_loader(self):
loader = super(MyTriblerExperimentScriptClientSubclass, self).create_community_loader()
# Register our custom community
loader.set_launcher(MyFirstCommunityLauncher())
# Which we can isolate as well
loader.isolate("MyFirstCommunity")
return loader
Isolating a particular community is as simple as adding the following line to a scenario file (before the IPv8 session starts):

As you can see, a minimal ``CommunityLauncher`` implementation requires a name and a community class to be defined.
A launcher is uniquely identified by its name.
**If you use ``set_launcher`` with an existing name, the current launcher will be overwritten.**
In some cases overwriting a default community entirely may be desired though.
.. code-block:: none
CommunityLauncher Interface
---------------------------
The reference for the methods of the ``CommunityLauncher`` is as follows:
@0:0 isolate_ipv8_overlay PingPongCommunity
========================================== =========== ===========
Method Type Description
========================================== =========== ===========
``get_name()`` *str* The unique name of this launcher.
``not_before()`` *list(str)* The names of launchers which should be loaded before this launcher is launched. Use in combination with ``prepare()`` to retrieve runtime information from other communities.
``should_launch(session)`` *bool* Checks the Session parameters to see if this community should be loaded.
``prepare(ipv8, session)`` *None* Prepare this launcher with information from the current ``Session``.
``finalize(ipv8, session, community)`` *None* Called after the overlay has been loaded.
``get_overlay_class()`` *Overlay* The class of the overlay to be loaded by IPv8.
``get_my_peer(ipv8, session)`` *Member* The IPv8 member to use for this community.
``should_load_now(session)`` *bool* Load this overlay right now, should be ``True`` in most cases. The ``IsolatedCommunityWrapper`` uses this mechanism to provide a custom master member.
``get_args(session)`` *tuple* The arguments to supply to the ``__init__`` method of the loaded overlay class.
``get_kwargs(session)`` *dict* The named arguments to supply to the ``__init__`` method of the loaded overlay class.
========================================== =========== ===========
In this case, Gumby will automatically search for the launcher associated with the ``PingPongCommunity`` and replace the existing launcher with an instance of ``IsolatedIPv8LauncherWrapper``.
All information in the existing launcher will be copied to the instance of ``IsolatedIPv8LauncherWrapper``.
The community ID will be randomized such that it becomes more difficult for the Gumby experiment to interfere with deployed communities.
51 changes: 51 additions & 0 deletions docs/conf.py
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,51 @@
# Configuration file for the Sphinx documentation builder.
#
# This file only contains a selection of the most common options. For a full
# list see the documentation:
# https://www.sphinx-doc.org/en/master/usage/configuration.html

# -- Path setup --------------------------------------------------------------

# If extensions (or modules to document with autodoc) are in another directory,
# add these directories to sys.path here. If the directory is relative to the
# documentation root, use os.path.abspath to make it absolute, like shown here.
#
# import os
# import sys
# sys.path.insert(0, os.path.abspath('.'))


# -- Project information -----------------------------------------------------

project = 'Gumby'
copyright = '2021, Tribler Team' # pylint: disable=redefined-builtin
author = 'Tribler Team'

# -- General configuration ---------------------------------------------------

# Add any Sphinx extension module names here, as strings. They can be
# extensions coming with Sphinx (named 'sphinx.ext.*') or your custom
# ones.
extensions = [
]

# Add any paths that contain templates here, relative to this directory.
templates_path = ['_templates']

# List of patterns, relative to source directory, that match files and
# directories to ignore when looking for source files.
# This pattern also affects html_static_path and html_extra_path.
exclude_patterns = ['_build', 'Thumbs.db', '.DS_Store']


# -- Options for HTML output -------------------------------------------------

# The theme to use for HTML and HTML Help pages. See the documentation for
# a list of builtin themes.
#
html_theme = 'alabaster'

# Add any paths that contain custom static files (such as style sheets) here,
# relative to this directory. They are copied after the builtin static files,
# so a file named "default.css" will overwrite the builtin "default.css".
html_static_path = ['_static']
Loading

0 comments on commit 0d4394c

Please sign in to comment.