-
Notifications
You must be signed in to change notification settings - Fork 9
HowTo
- Download a stable Eclipse distribution (e.g. Indigo)
- install the SoPeCo-Libs feature (update-site)
- install the SoPeCo-Core feature (update-site)
- (Re)Start Eclipse
... r-installation...
The Software Performance Cockpit is a tool which allows for capturing the performance behaviour of software systems and component, infering performance curves and provides an Eclipse-based tool for the visualisation of measurement and analysis results. Using SoPeCo f***** or own measurements comprises some implementation and configuration steps, which are described in the following. These include modelling of the measurement experiment environment and experiment configurations, implementing a software adapter which conducts the single measurements and implementing a Cockpit-Starter which is basicly a configuration of the required Cockpit-Adapters. In the following we will illustrate all these steps by means of a concrete example.
In order to illustrate the idea of how to use the SoPeCo, we introduce a simple example. For the sake of simplicity, in this example we use a Fibonacci-Calculator as the 'Target System'. Thus, our goal is to analyse the response time of two Fibonacci-Calculator algorithms depending on the input parameter. The Fibonacci function is defined as follows:
F(n) = F(n-1)+F(n-2), F(0) = 0, F(1) = 1
We want to compare the response time behaviour of two Fibonacci implementations (recursive and iterative) depending on the input parameter n:
public static int fibonacciRecursive(int n) {
if ((n==1) || (n==2)){
return 1;
}else{
return fibonacciRecursive(n-1)+fibonacciRecursive(n-2);
}
}
public static int fibonacciIterative(int n) {
int f1 = 0;
int f2 = 1;
for(int i = 1; i < n; i++){
int temp = f2;
f2 = f1+f2;
f1 = temp;
}
return f2;
}
Generally, this Fibonacci functions represent any software system (e.g. database system, messaging middleware, etc) with any input parameters. Furthermore, we do not have to know the implementation of the system to be measured, thus, it can be a black box.
For our Fibonacci example, in the SoPeCo-Eclipse we create a new Plugin-Project ('org.example.fibonacci.implementations'), which is the target system to be measured. The project 'FibonacciImplementations' contains two classes ('FibonacciIterative' and 'FibonacciRecursive'), which stand for the two different implementations of the Fibonacci function (iterative and recursive). In order that other projects can access the FibonacciImplementations it is necessary to export the corresponding pacakge in the META_INF/MANIFEST.MF file.
++++ FIGURE 1 ++++
As the first step we have to specify our experiment environment. We will place this Configuration model into the same project which later will start the SoPeCo instance. Thus, we have to create a Plugin-Project called org.example.start.fibonacci. We will describe this project later, at this point we need it just to place our models in a resource directory of this project. For this, we create a Source Folder 'rsc' inside the 'org.example.start.fibonacci' project and than a AdapterRepository model (as depicted in Figure 2) named 'Fibonacci.adapterrepository' inside 'rsc'.
++++ Figure 2 ++++ (creating project)
++++ FIGURE create model file +++
In the newly created model we create a new Adapter Descriptor and call it 'Fibonacci Example Adapter' with the id 'FibonacciAdapter'. An adapter descriptor is used to describe the configuration parameters, dependent and independent parameters of the system to be measured. In our example we have only one independent parameter. Thus, we create a ConfigurationParameterGroup with one Parameter called 'InputParameter' (with arbitrary unique id). We specify the type of the parameter as INTEGER and set the default value to '1'. Now we need to specify the dependent parameter, too. For this, we create the parameter 'ResponseTime' of type DOUBLE. The third parameter is neither a input parameter for the performance curve nor an observed parameter, it is a parameter specifying the implementation type (recursive or iterative) of the Fibonacci function. We call this parameter 'ImplementationType' and set the parameter type to STRING. With this, the adapter descriptor model is complete.
++++ FIGURE 3 ++++ (Adapter Repository)
As the next step, we need a Configuration model, which specifies the measurement behaviour. Thus, we create a Configuration model named 'FibonacciAnalysis.configuration', analogous to the creation of the AdapterRepository model. As the first _Configuration _child we create the Resource Environment describing the satellites used to measure the target systems. In our example we have only one satellite with Satellite ID 'FibonacciSatellite'. A satellite is deployed on the same platform/system as the target system to be measured and conducts single measurements. Entire experiments are controlled by the SoPeCo instance, which is (normally) deployed on another system in order to reduce the influence of SoPeCo to the target system. The SoPeCo instance interacts with the satellite over Remote Method Invocations (RMI). Thus, we have to set the Hostname (IP-Address) and Port specifying the RMI accesspoint of the satellite. For the sake of simplicity, in our example, we will deploy all components (the satellite, the SoPeCo instance and the target system) on the same machine. Thus we set the Hostname to 'localhost' and the port to the standard RMI port '1099'; Finally, the satellite has to reference an adapter descriptor, in our case it is the 'Fibonacci Example Adapter'.
++++ FIGURE 4 +++++ (Resource Environment)
The Measurement Specification is the second mandatory specification (, all other specifications are optional and will be described later). A Measurement Specification consists of a set of Experiment Series. As we want to compare the two Fibonacci implementations, we define two series ('FibonacciRecursive' and 'FibonacciIterative'), for each implementation one. We will describe an Experiment Series specification by means of the 'FibonacciRecursive'. For a Experiment Series an exploration strategy has to be specified describing how the parameter space is traversed. There are many different exploration strategies: Full Exploration, Adaptive Breakdown, Plackett Burmann, .... In our example we use the FullExploration Strategy which systematically explores the entire input parameter space. Then, create an Experiment Run Configuration, which describes one Run of an experiment series. First, we have to point to the used adapter descriptor and the resource environment by creating an Experiment Run Start Info. Secondly, we have to specify a termination criteria for the experiment runs. There are two options: either the run should be terminated after a certain number of measurements (Number Of Measurements Termination Criteria) or the run should be terminated after a specified periof of time (Measurement Time Termination Criteria). We use the first option with a number of measurements of 2. Finally, we must declare how the results of all measurements within a run are aggregated to single observation values for each observation parameter. For this, we define a Data Processing Step with the method Average. Having specified the exploration strategy and the run configuration, the input parameter space has to be defined. For each parameter which should be varied, one can define a Variation (Linear Integer Variation, Linear Double Variation, ...) describing the value boundaries and variation step. for a referenced input parameter. Canstants can be used to set fix configuration parameters. In our example, we vary the 'InputNumber' parameter from 1 to 45 with a step width of 2. In the 'FibonacciRecursive' experiment series we set the 'ImplementationType' Parameter to the constant value 'recursive'. The experiment series configurations for 'FibonacciIterative' are identical to the configurations of 'FibonacciRecursive', only the 'ImplementationType' constant is set to 'iterative'.
Now, we are ready with basic configurations of the measurement. There are some additional options which will be described later. For the first measurement run, these configuration are satisfactory.
++++ FIGURE 5 ++++ (Measurement Specification)
The main implementation part when using SoPeCo for performance analysis is the creation of the Software Adapter. The software adapter is the part, which is deployed with the satellite to the target platform in order to conduct measurements on the target system. Thus, the software adapter must know how to access the target systems, how to perform single measurments and how to capture the target metrics. We illustrate the idea of a software adapter by describing implementation of the software adapter for our Fibonacci example.
First, we need a new Plugin-Project which we call org.example.swadapter.fibonacci in our example. In this project we create an additional Source Folder 'rsc' in which we copy the 'Fibonacci.adapterrepository' file created in the previous section. We will use this file later. A software adapter has dependencies to the following plug-ins:
- org.sopeco.core
- org.sopeco.satellite.swadapter.interfaces
- org.sopeco.persistence
Furthermore, our software adapter must access the two Fibonacci implementations, thus it has an additional dependency to the project org.example.fibonacci.implementations. In order to resolve these dependencies we have to add them into the dependencies list of the META-INF/MANIFEST.MF in the 'org.example.swadapter.fibonacci' project, as depicted in Figure 6.
++++ FIGURE 6 +++++ (MANIFEST Dependencies 'org.example.swadapter.fibonacci')
Now create a package org.example.swadapter.fibonacci with a class 'FibonacciAdapter' inside the 'src' folder. The class 'FibonaciAdapter' should extend the class 'AbstractExperimentRunStarter'. We, first, define some String constants representing the parameter names and valid values. Furthermore, we need a AdapterHandler instance containing the Adapter Repository model.
public class FibonacciAdapter extends AbstractExperimentRunStarter {
/**
* Parameter specifying the response time which ist the observed parameter
* in this example.
*/
public static final String RESPONSE_TIME = "ResponseTime";
/**
* The configuration parameter of the adapter is the input number for the
* fibonacci calculator.
*/
public static final String INPUT_NUMBER = "InputNumber";
/**
* The ImplementationType indicates whether a recursive or an interative
* implementation for the fibonacci algorithm should be used.
*/
public static final String IMPL_TYPE = "ImplementationType";
/**
* variable to store the value of the ImplementationType variable.
*/
private String implType = RECURSIVE;
/**
* Valid values for the ImplementationType parameter
*/
public static final String RECURSIVE = "recursive";
public static final String ITERATIVE = "iterative";
/**
* Singleton for the adapter descriptor.
*/
private static AdapterHandler adapterHandler;
/**
* Create a new adapter.
*/
public FibonacciAdapter() {
super(getAdapterDescriptorInstance());
}
/**
* @return Returns the descriptor (wrapped by a handler) of the adapter.
*/
public static AdapterHandler getAdapterDescriptorInstance() {
if (adapterHandler == null) {
adapterHandler = AdapterHandler.loadDescriptor(
"Fibonacci.adapterrepository", "FibonacciAdapter");
}
return adapterHandler;
}
@Override
public AdapterDescriptor getAdapterDescriptor() {
return adapterHandler.getDescriptor();
}
**...**
This adapterHandler is a singleton instance and is crated (or retrieved) in the function 'getAdapterDescriptorInstance()', then the instance is passed to the constructor of the super class. Each sub-class of the AbstractExperimentRunStarter has to implement the following methods:
-
prepareExperimentSeries(List<ParameterValue<?>> parameterValueList): This method is called by the experiment controller, before the experiment is started. Thus, this method can be used to configure experiment settings which are valid during the entire experiment. As an attribute the experiment controller passes a list of all default (or constant) parameter values. In our example we created for each implementation type of the Fibonacci function an own experiment series, hence the value of the variable implType (defined above) is constant for the duration of an experiment. Thus, we use the function 'prepareExperimentSeries()' in order to set this variable. We traverse the parameterValueList in order to find the corresponding parameter and then save the value of this parameter to the variable implType.
@Override public void prepareExperimentSeries( List<ParameterValue<?>> parameterValueList) { Parameter implTypeParameter = ParameterRegistry .getParameterByName(IMPL_TYPE); for (ParameterValue pv : parameterValueList) { if (pv.getParameter().equals(implTypeParameter)) { implType = pv.getValueString(); break; } } } -
prepareExperimentRun(List<ParameterValue<?>> parameterValueList): This method has a similar function as the method before, however this method is called prior to each experiment run. In our example we use this method to initialise our target DataSet for each run. For this, we call the 'builder()' method (super-class method) in order to initialise the first row. Then, for all parameters the parameter values are set:
@Override public void prepareExperimentRun(List<ParameterValue<?>> parameterValueList) { builder().startInitialRow(); for (ParameterValue pv : parameterValueList) { builder().setValue(pv.getParameter(), pv.getValue()); } } -
startExperimentRun(): This is the main method conducting the actual measurements for the current run. As input parameters do not change during an experiment run, we initially retrieve the value of the input parameter and save it as an Integer variable (inputValue). As we want to repeat the measurements several times for one run, we use a for-loop. The termination condition is determined by the NumberOfExperimentRuns() which was specified in the Measurement Specification as Number of Measurements Termination Creteria. We construct the dataset for each run by copying the previous row of each measurement. For the first measurement no previous row exists, thus skip this step, if i has the value 0. Then we call the Fibonacci calculation function measuring the time this function requires for calculation. Depending on the initially (at the experiment series initialization) captured Fibonacci implementation type (implType) the method calculateFibonacci() calls either the recursive or the iterative algorithm. Having calculated the reponse time we set the it in the target dataset for the run and finish the current dataset row.
/** * Starts the experiment run by calling the FibonacciCalculator with the the * current value of the input parameter. */ @Override public void startExperimentRun() { // get the current value of the configuration parameter // "InputNumber" Parameter inputNumberParameter = ParameterRegistry .getParameterByName(INPUT_NUMBER); ParameterValue<?> inputNumberParameterValue = builder() .getCurrentValue(inputNumberParameter); Integer inputValue = (int) inputNumberParameterValue.getValueAsDouble(); // measure the current configuration n times // in order to get a confident response time value for (int i = 0; i < getNumberOfExperimentRuns(); i++) { // get last values of configuration and observation parameters if (i != 0) builder().startRowCopy(); // measure response time of fibonacci calculation of current input // number long startTime = System.nanoTime(); calculateFibonacci(inputValue.intValue()); double responseTime = (System.nanoTime() - startTime) / 1000; // save the response time in the row of the current run builder().setValue(RESPONSE_TIME, responseTime); builder().finishRow(); } } private void calculateFibonacci(Integer n) { if (implType.equalsIgnoreCase(RECURSIVE)) { FibonacciRecursive.fibonacciRecursive(n); } else if (implType.equalsIgnoreCase(ITERATIVE)) { FibonacciIterative.fibonacciIterative(n); } else { throw new IllegalArgumentException( "Unknown Fibonacci implementation type!"); } } -
abortExperimentRun(): This method is called by the experiment series controller if an experiment run is aborted. This method for example can be used in order to free allocated resources. In our example we do not need this method, thus we leave it unimplemented:
@Override public void abortExperimentRun() { }
With this, the implementation of the software adapter and the project 'org.example.swadapter.fibonacci' is complete. Again, in order that other projects can use our software adapter we have to export the corresponding package in the MANIFEST file.
In the previous section we explained how to implement a software adapter, which performs the measurements. As already mentioned, (in general), a software adapter is deployed on the same platform as the target system, while the SoPeCo instance itself is separated by deploying it on another platform. The SoPeCo then communicates with the software adapter using RMI. For this, we need a satellite starter which starts the rmi-server and publishes the corresponding satellite controller in order to access the software adapter through rmi.
Create a new Plugin-project with the name org.example.satellite.start.fibonacci. In this project create a second source folder rcs where you create a file with the name fibonacciSatellite.properties. In this file we specify the id and location of our satellite. In our example we will run both, the SoPeCo instance and the satellite on the same platform, thus we set the host to 'localhost' and port to '1099'. The id of our satellite is 'FibonacciSatellite'. We will use these properties for the start of the satellite controller.
++++ FIGURE 7 ++++ (fibonacciSatellite.properties)
A satellite starter project has the following dependencies:
- org.sopeco.satellite
- org.sopeco.satellite.swadapter.interfaces
- org.sopeco.satellite.interfaces
Additionally, the starter has to know the corresponding software adapter. Thus, in our example we have also a dependency to the project 'org.example.swadapter.fibonacci'. Modify the dependencies in the META-INF/MANIFEST.MF file of the org.example.satellite.start.fibonacci project accordingly:
++++ FIGURE 8 ++++ (MANIFEST satellite starter)
Now, in the src folder, create a new package org.example.satellite.start.fibonacci with a new class FibonacciSatelliteStarter:
public class FibonacciSatelliteStarter {
SatelliteController satelliteController;
public static void main(String argV[]) throws Exception
{
FibonacciSatelliteStarter satelliteStarter = new FibonacciSatelliteStarter();
satelliteStarter.buildSatellite();
satelliteStarter.startSatelliteServer();
}
private void startSatelliteServer(){
satelliteController.startRmiServer();
}
public synchronized void buildSatellite(){
// create SatelliteController instance
satelliteController = new SatelliteController("fibonacciSatellite.properties");
// create and bind SoftwareAdapter instances
FibonacciAdapter swAdapter = new FibonacciAdapter();
satelliteController.addSoftwareAdapter(swAdapter);
}
}
The FibonacciSatelliteStarter class has a main-method creating a FibonacciSatelliteStarter instance, building the satellite and starting it. The satellite controller is saved in an instance variable (satelliteController). For the creation of the satellite constroller the name of the properties file described before is passed to the constructor. Then an instance of our FibonacciAdapter is created and registered at the satelliteController. With the method startSatelliteServer the satellite will be started and will wait for instructions from the SoPeCo instance.
In section 'Creating Measurement Specification Models' we created the project org.example.start.fibonacci in order to put our configuration models into this project. In the same project we will create the CockpitStarter which creates an instance of the SoPeCo and initiates the measurements. First, we have to add some dependencies to the META-INFMANIFEST.MF file of the org.example.start.fibonacci project. Every CockpitStarter project has at minimum the following dependencies:
- org.sopeco.core
- org.sopeco.experiment
- org.sopeco.experiment.interfaces
- org.sopeco.experimentseries
- org.sopeco.experimentseries.exploration.interfaces
- org.sopeco.maincontroller
- org.sopeco.satellite
- org.sopeco.satellite.interfaces
- org.sopeco.persistence
Furthermore, depending on the configurations of the measurement specification (*.configuration file) additional dependencies are needed. For our example we additionally need only the dependency org.sopeco.experimentseries.exploration.fullexploration, as we use the Full Exploration Strategy.
Next, we create a package org.example.start.fibonacci in the source directory src, where we create the class FibonacciCockpitStarter with the following content:
public class FibonacciCockpitStarter {
MainController mainController;
public static void main(String[] args) throws Exception {
if (args.length == 1) {
FibonacciCockpitStarter cockpit = new FibonacciCockpitStarter();
cockpit.buildCockpit();
cockpit.runCockpit(args[0]);
} else {
throw new RuntimeException("This program requires"
+ " exactly one argument (the path to the"
+ " configuration file).");
}
}
public void buildCockpit() {
// create component instances
this.mainController = new MainController();
TreeRepository dataRepository = openRepository();
ExperimentController experimentController;
try {
experimentController = new ExperimentController();
} catch (RemoteException e) {
throw new IllegalStateException(
"Cannot create experiment controller instance!", e);
}
SFExperimentSeriesInitializer experimentSeriesInitializer = new SFExperimentSeriesInitializer();
FullExplorationStrategyImplementation fullExploration = new FullExplorationStrategyImplementation();
experimentController.setDataRepository(dataRepository);
fullExploration.setExperimentController(experimentController);
experimentSeriesInitializer.setExperimentController(experimentController);
experimentSeriesInitializer.addExplorationController(fullExploration);
mainController.setDataRepository(dataRepository);
mainController.setExperimentSeriesInitializer(experimentSeriesInitializer);
}
private TreeRepository openRepository() {
TreeRepository dataRepository = new TreeRepository();
File repositoryDir = new File("data");
if (!repositoryDir.isDirectory()) {
if (!repositoryDir.mkdir())
throw new IllegalStateException("Could not create directory "
+ repositoryDir.getAbsolutePath()
+ " for dataRepository!");
}
dataRepository.open(repositoryDir.getAbsolutePath());
return dataRepository;
}
public void runCockpit(String pathToConfiguration)
throws InvalidConfigurationException {
mainController.start(pathToConfiguration);
RAdapter.shutDown();
}
}
The main() method of the FibonacciCockpitStarter expects the absolute path of the created *.configuration file as an argument. This argument will be passed, when the FibonacciCockpitStarter will be started (, we describe this later).
If the number of arguments is correct, then the main() method creates a FibonacciCockpitStarter instance, builds it and starts the cockpit. In the buildCockpit() method we have to create all components used by the cockpit and register them at the MainConstroller. The MainController instance is the actual SoPeCo instance coordinating the measurements.
Thus, in the buildCockpit() method we, first, create a new instace of the MainConstroller. Then we open a dataset repository by calling the method openRepository(). This requires a specification of the directory for the repository. In our case, we use the relative path 'data', where all datasets should be stored during measurements. Next, we create instance for the ExperimentController, ExperimentSeriesInitialiser and the FullExplorationStrategyImplementation. For both, the experimentController and the main controller, the dataRepository has to be set. The exploration strategy and the experimentSeriesInitializer must know the experimentController. Furthermore, the experiemntSeriesInitializer must know the used exploration strategy instance. Finally, we have to pass the experimentSeriesInitializer instance to the mainController.
For advanced model configurations, the buildCockpit method is more complex as additional components like analysisController, exportController, etc have to be created. Advanced features, are described later!
Now, we are ready to start the measurements. For this, first, start the satellite: Select the FibonacciSatelliteStarter class in the org.example.satellite.start.fibonacci project and then select Run As -> Java Application in the context menu. In the console the output should look like it is depicted in the next figure:
+++++ FIGURE +++++ (console output)
Next, select the FibonacciCockpitStarter class in the org.sopeco.start.fibonacci project and go to Run As -> Run Configurations in the context menu. There, create a new Java Application configuration by double clicking on the corresponding item. In the Main tab enter org.example.start.fibonacci for the Project field and org.example.start.fibonacci.FibonacciCockpitStarter for the Main class field. Finally, in the Arguments tabs enter the absolute path to the FibonacciAnalysis.configuration file. Now, the cockpit can be started by clicking on the Run button. The measurements may take some time and will provide a console notification as soon as all experiment series are finished.
As we specified the DataRepository path as the local path 'data', there should be a directory named 'data' in the org.example.start.fibonacci project after the measurements are finished (a refresh of the project might be required).
FIGURE data
We can analyse the measured data using the SoPeCo visualization tool. Therfore, go to Window -> Open Perspective -> Other and then select Persistence Visualization:
FIGURE Visualization Perspective