Skip to content

Commit

Permalink
README.md is added for Forecast Client
Browse files Browse the repository at this point in the history
  • Loading branch information
@devosonder
devosonder committed Jul 19, 2022
1 parent e15817f commit 87abbb6
Show file tree
Hide file tree
Showing 2 changed files with 145 additions and 23 deletions.
123 changes: 123 additions & 0 deletions Bitcoin-Factory/Forecast-Client/README.md
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,123 @@
# Forecast Client

The Forecast Client app feeds itself from the Test Server's forecast cases. A forecast case is the best known set of parameters for a certain Asset / Timeframe. The job of this App is to recreate the model discovered by a Tester using the Test Client App. Once this model is recreated, it is used to start forecasting the next candle for that Asset / Timeframe. The forecasts produced by this App are then sent to the Test Server and distributed to the Test Client Apps every time they solve a new test case.

Each tested model (created based on a set of parameters and a custom dataset) has a certain implied Error: the root-mean-square error (RMSE).

https://en.wikipedia.org/wiki/Root-mean-square_deviation

The whole point of crowd-testing is to find the model with the lowest % of error for a certain Asset / Time-Frame.

**Note:** All forecasts are done at the Asset/USDT markets on Binance for now.

When you are running a Test Client App, you are testing certain combinations of parameters for a certain Asset / Time-Frame including a custom dataset for your specific test, which might include a certain combination of indicators data.

The crowd-sourced forecasts you receive after each test, are the ones belonging to ML models with the lowest % error for a certain Asset / Time-Frame.

## How to setup Forecast Client?
This section describes how to install the Farecast Client. It is assumed that docker and Superalgos are already installed and running on the server where Forecast Client will be run.

### Running Docker Container

1. Navigate to the `cd Superalgos/Bitcoin-Factory/DockerBuild` folder.
2. `docker build -t bitcoin-factory-machine-learning .` command to build the docker image.
**Note:** If you have previously run the Bitcoin Factory Test Client on the server where you will run the Forecast Client, you have probably built the doker image. If you have done this process before, you do not need to do it again.
3. `docker run -it --rm --shm-size=4.37gb --name Bitcoin-Factory-ML-Forecasting -v /Superalgos/Bitcoin-Factory/Forecast-Client/notebooks:/tf/notebooks -p 8888:8888 bitcoin-factory-machine-learning` Let's run the docker container using the command. `/Superalgos/Bitcoin-Factory/Forecast-Client/notebooks` Remember to set the specified folder to the Superalgos folder on your own server.
Once the docker container is running correctly you will see at the first terminal an output similar to this:

```text
[I 12:58:36.546 NotebookApp] Writing notebook server cookie secret to /home/ubuntu/.local/share/jupyter/runtime/notebook_cookie_secret
[I 12:58:37.532 NotebookApp] Serving notebooks from local directory: /tf/notebooks
[I 12:58:37.532 NotebookApp] Jupyter Notebook 6.4.10 is running at:
[I 12:58:37.533 NotebookApp] http://aa1b305587bd:8888/?token=49c135d693e0b4d07d8c0164410ee6fc4593ac5e0578a34a
[I 12:58:37.533 NotebookApp] or http://127.0.0.1:8888/?token=49c135d693e0b4d07d8c0164410ee6fc4593ac5e0578a34a
[I 12:58:37.533 NotebookApp] Use Control-C to stop this server and shut down all kernels (twice to skip confirmation).
[C 12:58:37.544 NotebookApp]
To access the notebook, open this file in a browser:
file:///home/ubuntu/.local/share/jupyter/runtime/nbserver-1-open.html
Or copy and paste one of these URLs:
http://aa1b305587bd:8888/?token=49c135d693e0b4d07d8c0164410ee6fc4593ac5e0578a34a
or http://127.0.0.1:8888/?token=49c135d693e0b4d07d8c0164410ee6fc4593ac5e0578a34a
```
**Note:** At that terminal there is no further action required.

### Forecast Client'inin çalıştırılması.
For your Forecast Client App to work and be able to connect to the Test Server you need to:

1. Update your User Profile with several nodes that today you might not have.
2. Create the Signing Account node to allow your Forecast Client app to run with an identity that the Superalgos Network can recognize.
3. Reference from the Task -> Task Server App Reference one of the nodes you added to your profile.
4. Change a config to specify the name of your Forecast Client, so that you can recognize it among other forecast clients on the execution reports.

Continue reading this section for detailed step by step instructions of how to do the above.

***Update your User Profile***

Before you can participate within the Superalgos P2P network, You need to add a few nodes to your User Profile. Once these nodes are added and configured properly you will need to contribute your updated profile to the Governance repo and make sure that it is merged by the PR merging bot.

Here is the complete list of nodes you need to add to your profile and how to configure them.

**Note:** All paths start from the User Profile node.

**Task Server App Node**
1. User Profile -> User Apps
2. User Profile -> User Apps -> Server Apps
3. User Profile -> User Apps -> Server Apps -> Task Server App

Once you have added the Task Server App node, hover over it and rename it using the following name: "Task Server App #1"

Then add the following configuration within the Task Server App node's config:
```json
{
"codeName": "Task-Server-App-1"
}
```

**Bitcoin Factory Forecasts**
4. User Profile -> Forecast Providers
5. User Profile -> Forecast Providers -> Bitcoin Factory Forecasts

Hover over the Bitcoin Factory Forcasts node and rename it using the following name: "Testnet"

### Signing Accounts

Finally, you need to generate/re-generate the signing accounts of your User Profile, so that a new node of type Signing Accounts is created under the "Task-Server-App-1" node. The procedure to do this is the following:

1. At the Governance Project node create a Profile Constructor node.
2. Reference the Profile Constructor to your User Profile.
3. At the Profile Constructor menu, click on Install Signing Accounts. This will generate a new node under "Task-Server-App-1" and save a file to your My-Secrets folder with the Signing Accounts of your User Profile.

Congratulations! Now you are done with your profile.

Remember to save your User Profile plugin, contribute it and check that it was merged at the Governance repository.

**IMPORTANT:** It takes a few minutes for your profile to be auto-merged into the Governance repository and another 5 minutes to be picked up by the running Network Node. After changes to your profile, wait for around 10 minutes before expecting it to be able to connect to the Superalgos Network node.

***Reference the Task Server App***

Go to Bitcoin-Factory-Demo Workspace, change it's name and save it (so to have your own instance of that workspace). Go to Plugins Node and then import your User Profile into the Workspace

Locate the node Task Server App Reference, under your Forecast Client Task, and replace the current reference with a reference to the "Task-Server-App-1" node you created at your User Profile.

By setting up this reference you define the identity under which the forecast client will run on the P2P network. In other words, the signing account held under your "Task-Server-App-1" node acts like a finger print so that other entities running on the network can identify and work with your forecast client.

***Change the Config***

After that, open the config of the Forecast-Client Sensor Bot Instance. It looks like this:

```json
{
"startDate": "2021-01-01",
"networkCodeName": "Testnet",
"clientInstanceForecaster": "devosonder-01",
"clientInstanceBuilder": "devosonder-01"
}
```

* clientInstanceForecaster: **IMPORTANT:** Change this to match the name you gave to your Test Client Instance node you created at your user profile.
* clientInstanceBuilder: **IMPORTANT:** Change this to match the name you gave to your Test Client Instance node you created at your user profile.

**IMPORTANT:** If you are going to be using 2 or more computers, you need to take care of the Signing Accounts file that needs to be present at both / all computers (This is the one that lives in your My-Secrets file). In other words, you cannot generate the signing account at one computer and then generate it again at the second one. If you generate it at one computer and contributed your profile, then you need to copy the file inside the My-Secrets folder to the second computer/s.

**IMPORTANT:** Currently Test-Client and Forecast-Client do not work together on the same server.
45 changes: 22 additions & 23 deletions Bitcoin-Factory/Test-Client/README.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@ The ML Test Client runs from within the Superalgos Platform and connects to a re
It is important to understand that this Test Client APP does not prepare the dataset to be tested. This is done by the Test Server App. That means that this app does not need Superalgos or any other data provider for the purpose of extracting data from it. It only depends on the Test Server which handles the management of the Test Cases and the generation of the datasets to be used at each one of the tests.

### Example of Parameters [Fraction of the actual list]

```text
┌─────────────────────────────────────────────────────────────────────────┬─────────┬────────┐
│ (index) │ 0 │ Values │
├─────────────────────────────────────────────────────────────────────────┼─────────┼────────┤
Expand Down Expand Up @@ -56,7 +56,7 @@ It is important to understand that this Test Client APP does not prepare the dat
* 1503705600000 4367 4212.41 4280.68 4337.44 228.10806799999992
* 1503792000000 4400 4285.54 4332.51 4310.01 350.6925850000002
* 1503878400000 4399.82 4124.54 4310.01 4386.69 603.8416160000002

```
### How does this Test Client App work?

This app is used to autonomously test different set's of parameters to see which Machine Learning models can produce better forecasts.
Expand Down Expand Up @@ -136,7 +136,7 @@ Here is the complete list of nodes you need to add to your profile and how to co
Once you have added the Task Server App node, hover over it and rename it using the following name: "Task Server App #1"

Then add the following configuration within the Task Server App node's config:
```sh
```json
{
"codeName": "Task-Server-App-1"
}
Expand All @@ -156,7 +156,7 @@ For this node, you need to assign a unique name of your choice. The name you cho
Node Title: "Assign-A-Name"

Node Config:
```sh
```json
{
"codeName": "Assign-A-Name"
}
Expand Down Expand Up @@ -188,7 +188,7 @@ By setting up this reference you define the identity under which the test client

After that, open the config of the Test-Client Sensor Bot Instance. It looks like this:

```sh
```json
{
"networkCodeName": "Testnet",
"targetSuperalgosHost": "localhost",
Expand All @@ -214,15 +214,15 @@ Build the Docker Image. Open a console at the Bitcoin-Factory folder inside Supe

### On x86 Processors

```sh
```shell
cd DockerBuild
docker build -t bitcoin-factory-machine-learning .
cd ..
```

### On xArm Processors

```sh
```shell
cd ArmDockerBuild
docker build -t bitcoin-factory-machine-learning .
cd ..
Expand All @@ -245,7 +245,7 @@ Now you are ready to get things rolling! You will need to open two terminals. On
### Docker Container Terminal Output
Once the docker container is running correctly you will see at the first terminal an output similar to this:

```sh
```text
[I 12:58:36.546 NotebookApp] Writing notebook server cookie secret to /home/ubuntu/.local/share/jupyter/runtime/notebook_cookie_secret
[I 12:58:37.532 NotebookApp] Serving notebooks from local directory: /tf/notebooks
[I 12:58:37.532 NotebookApp] Jupyter Notebook 6.4.10 is running at:
Expand All @@ -259,15 +259,14 @@ Once the docker container is running correctly you will see at the first termina
Or copy and paste one of these URLs:
http://aa1b305587bd:8888/?token=49c135d693e0b4d07d8c0164410ee6fc4593ac5e0578a34a
or http://127.0.0.1:8888/?token=49c135d693e0b4d07d8c0164410ee6fc4593ac5e0578a34a

```

**Note:** At that terminal there is no further action required.

### Superalgos Platform Terminal Output
At the Superalgos terminal, once you run the Test Client Task, you will see, after 10 seconds an output similar to this one:

```sh
```text
-------------------------------------------------------- Test Case # 1 / 3192 --------------------------------------------------------
Starting at this GMT Datetime: 2022-03-24T10:00:55.115Z
Expand All @@ -294,7 +293,7 @@ Parameters Received for this Test:

After between 15 and 30 minutes, depending on the Test Case that was assigned to you, you will see an output like this:

```sh
```text
Docker Python Script exited with code 0
Prediction RMSE Error: 368.83
Predictions [candle.max, candle.min, candle.close]: 43278.008,42785.055,43028.305
Expand Down Expand Up @@ -330,11 +329,11 @@ For specific information on how to run the Docker Container in different OS, ple
**Very important**, if you choose to run docker under a sudo user on Linux distros, make sure you run Superalgos also under sudo, otherwise it might not work.

To run docker without sudo on Ubuntu, add the current user to the docker group with:
```sh
```shell
sudo gpasswd -a $USER docker
```
Then either log out and back in, or run the following command to refresh permissions:
```
```shell
sh
newgrp docker
```
Expand All @@ -343,15 +342,15 @@ newgrp docker

Run the container with this command. Change the path if you did not install this App at the commands location.

```sh
```shell
docker run --gpus all -it --rm --shm-size=4.37gb --name Bitcoin-Factory-ML -v C:/Superalgos/Bitcoin-Factory/Test-Client/notebooks:/tf/notebooks -p 8888:8888 bitcoin-factory-machine-learning
```

### on Ubuntu Server / Linux

Run the Docker container with this command. Change the path if you did not install this App at the commands location.

```sh
```shell
docker run --gpus all -it --rm --shm-size=4.37gb --name Bitcoin-Factory-ML -v /Users/Your-User-Name/Superalgos/Bitcoin-Factory/Test-Client/notebooks:/tf/notebooks -p 8888:8888 bitcoin-factory-machine-learning
```

Expand All @@ -374,7 +373,7 @@ Apply & Restart makes the directory available to containers using Docker’s bin

The command to run the container on Mac should be like this (mind Your-User-Name).

```sh
```shell
docker run --gpus all -it --rm --name Bitcoin-Factory-ML --shm-size=4.37gb -v /Users/Your-User-Name/Superalgos/Bitcoin-Factory/Test-Client/notebooks:/tf/notebooks -p 8888:8888 bitcoin-factory-machine-learning
```
You will need to remove ```--gpus all``` for M1 based macs unless the docker image is specifically built to use the metal API.
Expand All @@ -384,10 +383,10 @@ You will need to remove ```--gpus all``` for M1 based macs unless the docker ima

Confirmed working on Raspberry Pi OS (64 bit)
- Node.js install
```
```shell
curl -fsSL https://deb.nodesource.com/setup_17.x | sudo -E bash -
```
```
```shell
sudo apt-get install -y nodejs
```
- Docker install following either [Repository](https://docs.docker.com/engine/install/debian/#install-using-the-repository) or [Convenience Script](https://docs.docker.com/engine/install/debian/#install-using-the-convenience-script) install steps on [docs.docker.com](https://docs.docker.com/engine/install/debian/#installation-methods)
Expand All @@ -399,32 +398,32 @@ Confirmed working on Raspberry Pi OS (64 bit)
### Response from daemon conflict
If you get the error:

```sh
```text
docker: Error response from daemon: Conflict. The container name "/Bitcoin-Factory-ML" is already in use by container ...
```

Use the command

```sh
```shell
docker container prune
```

to fix it.

### Network Client Identity
```sh
```text
"Fatal Error. Can not run this task. The Network Client Identity does not match any node at User Profiles Plugins."
```
This error occurs when the signing account does not match the Governance plugin repository's account. To ensure they are the same, import your user profile on the workspace using the "Add specified User Profile" command under Plugins -> Plugin Project -> Plugin User Profiles.
Add the correct nodes, references and signing account to the plugin as detailed in [App Setup](#app-setup). Save the plugin and push the changes to the Governance repository and wait 10 minutes for it to merge and be picked up by the Forecast Server.

### Unexpected Error
```sh
```text
unexpected error trying to execute a python script inside the docker container"
```
This error relates to an incorrect path when launching the docker container. Ensure the path to the notebooks directory is correct in the docker run command.

After launching the docker container, the path can be verified by using the following command to run a test model:
```sh
```shell
docker exec -it Bitcoin-Factory-ML python /tf/notebooks/Bitcoin_Factory_LSTM.py
```

0 comments on commit 87abbb6

Please sign in to comment.