Skip to content

Commit 26a2ea3

Browse files
committed
Migrate sync to envgen.py
Actualize ENVGEN.md and docker-compose.yml
1 parent 7bdd949 commit 26a2ea3

5 files changed

Lines changed: 331 additions & 529 deletions

File tree

tools/deploy/ENVGEN.md

Lines changed: 116 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -52,6 +52,87 @@ python3 envgen.py --list-services
5252
python3 envgen.py --list-profiles
5353
```
5454

55+
### Binary Sync and Volume Setup
56+
57+
The environment generator also performs binary synchronization for binary services (oracle, validator, daemon-client, api-client). This sets up the volume directory, ensures required folders and SQLite DB exist, and downloads binaries for the specified tag. If the target binary already exists for that tag, it will be reused and download will be skipped.
58+
59+
Key points:
60+
- `NODE_VERSION` is **required** for binary services and triggers automatic sync
61+
- `REPO_URL` and `SQLITE_DB` can be read from `--input` and written via `--output`
62+
- `NODE_VERSION` is written to service envs and consolidated env for Docker usage
63+
- Binaries are placed under `<volume>/<service>/bin/<tag>/<binary>`
64+
- Validator SQLite DB is created under `<volume>/validator/sqlite/<name>.db` (existing `<volume>/sqlite` is migrated)
65+
- Only prompts for binary configs (REPO_URL, NODE_VERSION) when binary services are involved
66+
- Only prompts for validator config (SQLITE_DB) when validator service is involved
67+
68+
Examples:
69+
70+
```bash
71+
# Generate binary service envs (NODE_VERSION required, triggers automatic sync)
72+
python3 envgen.py \
73+
--service validator \
74+
--profile bsctest \
75+
--config-dir deploy/config \
76+
--ntag v1.2.3 \
77+
--volume-dir /opt/openstore/volume
78+
79+
# Generate all services with binary sync
80+
python3 envgen.py \
81+
--profile bsctest \
82+
--config-dir deploy/config \
83+
--output deploy/config/.env.all \
84+
--ntag v1.2.3 \
85+
--volume-dir /opt/openstore/volume
86+
87+
# Interactive mode for binary services (will prompt for NODE_VERSION)
88+
python3 envgen.py \
89+
--service oracle \
90+
--profile bsctest \
91+
--config-dir deploy/config \
92+
--volume-dir /opt/openstore/volume
93+
```
94+
95+
Volume directory layout after sync (existing binaries are reused, not re-downloaded):
96+
97+
```
98+
<volume>/
99+
├── daemon-client/
100+
│ ├── bin/
101+
│ │ └── v1.2.3/
102+
│ │ └── daemon-client
103+
│ └── log/
104+
├── api-client/
105+
│ ├── bin/
106+
│ │ └── v1.2.3/
107+
│ │ └── api-client
108+
│ └── log/
109+
├── validator/
110+
│ ├── bin/
111+
│ │ └── v1.2.3/
112+
│ │ └── validator
113+
│ ├── log/
114+
│ └── sqlite/
115+
│ └── bsctest.db
116+
├── oracle/
117+
│ ├── bin/
118+
│ │ └── v1.2.3/
119+
│ │ └── oracle
120+
│ └── log/
121+
├── redis/
122+
├── postgres/
123+
├── certbot/
124+
│ ├── conf/
125+
│ └── www/
126+
└── nginx/
127+
└── log/
128+
```
129+
130+
Flags for binary services:
131+
- `--ntag`: Node release tag (required for binary services, triggers sync)
132+
- `--volume-dir`: Volume root directory (or set `VOLUME_DIR`)
133+
- `--repo-url`: Custom repository URL (read/write via consolidated env as `REPO_URL`)
134+
- `--sqlite-db`: SQLite DB name (read/write via consolidated env as `SQLITE_DB`)
135+
55136
### Generate for Specific Service
56137

57138
```bash
@@ -82,6 +163,7 @@ python3 envgen.py --profile bsctest
82163
- **POSTGRES_DB**: PostgreSQL database name
83164
- **POSTGRES_USER**: PostgreSQL username
84165
- **POSTGRES_PASSWORD**: PostgreSQL password
166+
- **SQLITE_DB**: SQLite database name (for validator)
85167

86168
### Redis
87169
- **REDIS_HOST**: Redis host (default: redis)
@@ -102,9 +184,17 @@ python3 envgen.py --profile bsctest
102184
- **NGINX_VARIANT**: Configuration type (http/https/none)
103185
- **CERTBOT_EMAIL**: Email for SSL certificates (Let's Encrypt)
104186

105-
### File Storage
187+
### Binary Services
188+
- **NODE_VERSION**: Node release tag (required for binary services)
189+
- **REPO_URL**: Repository URL for downloading binaries (default: https://github.com/Open-Store-Foundation/node)
190+
191+
### Validator
106192
- **FILE_STORAGE_PATH**: Validator file storage path (default: ./tmp/)
107193

194+
### Logging
195+
- **RUST_LOG**: Rust log level (default: info)
196+
- **LOG_PATH**: Log file path
197+
108198
## Deployment Profiles
109199

110200
### BSC Testnet Profile (`bsctest`)
@@ -218,28 +308,46 @@ HISTORICAL_SYNC_BLOCK=...
218308
CONFIRM_COUNT=...
219309
TX_POLL_TIMEOUT_MS=...
220310
311+
# BUILD (binary services only)
312+
NODE_VERSION=...
313+
221314
# DATABASE
222315
DATABASE_URL=...
223316
224-
# SERVICES
317+
# VALIDATOR (validator service only)
318+
SQLITE_DB=...
319+
FILE_STORAGE_PATH=...
320+
321+
# LOG (binary services only)
322+
RUST_LOG=...
323+
LOG_PATH=...
324+
325+
# API (api-client service only)
225326
REDIS_URL=...
226327
CLIENT_HOST_URL=...
227-
FILE_STORAGE_PATH=...
228328
229329
# POSTGRES (postgres service only)
230330
POSTGRES_DB=...
231331
POSTGRES_USER=...
232332
POSTGRES_PASSWORD=...
333+
DATA_SOURCE_NAME=...
334+
335+
# GRAFANA (grafana service only)
336+
GRAFANA_VARIANT=...
337+
GRAFANA_REMOTE_WRITE_URL=...
338+
GRAFANA_REMOTE_WRITE_USER=...
339+
GRAFANA_REMOTE_WRITE_PASSWORD=...
233340
```
234341

235342
### Service-Specific Variables
236343

237-
**Oracle Service**: Telegram, Blockchain, Wallet, Contracts, Sync
238-
**Validator Service**: Telegram, Blockchain, Wallet, Contracts, Sync, Database (SQLite), Services
239-
**Daemon Client**: Telegram, Blockchain, Wallet, Contracts, Sync, Database (PostgreSQL)
240-
**API Client**: Telegram, Wallet, Database (PostgreSQL), Services
344+
**Oracle Service**: Telegram, Blockchain, Wallet, Contracts, Sync, Build, Log
345+
**Validator Service**: Telegram, Blockchain, Wallet, Contracts, Sync, Build, Database, Validator, Log
346+
**Daemon Client**: Telegram, Blockchain, Wallet, Contracts, Sync, Build, Database, Log
347+
**API Client**: Telegram, Wallet, Build, Database, Log, API
241348
**PostgreSQL Database**: Postgres block only
242-
**Nginx Service**: Nginx block only (DOMAIN_NAME, CERTBOT_EMAIL)
349+
**Nginx Service**: Nginx block only (DOMAIN_NAME, CERTBOT_EMAIL)
350+
**Grafana Service**: Grafana block only
243351

244352
## Requirements
245353

0 commit comments

Comments
 (0)