Skip to content

Commit b3f5b64

Browse files
drozdziak1drozdziak1
authored
Prometheus metrics (#54)
* metrics.rs: Refactor dashboard rendering to its own module * Add local and global store metrics * global.rs: Fix global store metrics naming Prometheus does not like several of the characters we use in our symbol naming. * metrics: Use prometheus-client, the official Prometheus Rust crate * Release cutoff v1.2.0: bump version in Cargo.toml * local.rs: Remove old comment block * metrics.rs: Move symbol name fallback from global.rs * agent.rs: Add include keypair hotloading and metrics to diagram * metrics.rs: fix comment
1 parent ec36079 commit b3f5b64

File tree

7 files changed

+790
-337
lines changed

7 files changed

+790
-337
lines changed

‎Cargo.lock

Lines changed: 32 additions & 1 deletion
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.

‎Cargo.toml

Lines changed: 3 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,6 @@
11
[package]
22
name = "pyth-agent"
3-
version = "1.1.0"
3+
version = "1.2.0"
44
edition = "2021"
55

66
[[bin]]
@@ -44,6 +44,8 @@ serde-this-or-that = "0.4.0"
4444
# Rationale, 2023-03-21: https://stackoverflow.com/questions/74462753
4545
typed-html = { git = "https://github.com/bodil/typed-html", rev = "4c13ecca" }
4646
humantime = "2.1.0"
47+
prometheus-client = "0.19.0"
48+
lazy_static = "1.4.0"
4749

4850
[dev-dependencies]
4951
tokio-util = { version = "0.7.0", features = ["full"] }

‎src/agent.rs

Lines changed: 42 additions & 26 deletions
Original file line numberDiff line numberDiff line change
@@ -1,51 +1,67 @@
11
/* ###################################################### System Architecture #######################################################
22
3-
+-----------------------------+ +----------------------------+
4-
| Primary Network | | Secondary Network |
5-
| | | |
6-
| +--------------------+ | | +--------------------+ |
7-
| | RPC Node | | | | RPC Node | |
8-
| +--------------------+ | | +--------------------+ |
9-
| | ^ | | ^ | |
10-
| | | | | | | |
11-
| v | | | | v |
12-
| +----------+ +----------+ | | +----------+ +----------+ |
13-
| | Oracle | | Exporter | | | | Exporter | | Oracle | |
14-
| +----------+ +----------+ | | +----------+ +----------+ | +---------------------------------+
15-
| | ^ | | ^ | | | |
16-
+-------|--------------|------+ +------|--------------|------+ | Pythd Websocket API | +--------+
17-
| | | | | | | |
18-
| +--------------------------------+ | | +---------+ +----------+ | | |
19-
| | Local Store |<----------------------------| |<---| |<----------| |
20-
| +--------------------------------+ | | | | | JRPC | | | |
21-
v v | | Adapter | | WS | | | User |
22-
+------------------------------------------------------------+ | | | | Server | | | |
23-
| Global Store |------------->| |--->| |---------->| |
24-
+------------------------------------------------------------+ | +---------+ +----------+ | | |
25-
| | | |
26-
+---------------------------------+ +--------+
3+
+--------------------------------+ +--------------------------------+
4+
| RPC Node, e.g. Pythnet | | RPC Node, e.g. Solana Mainnet |
5+
+--------------------------------+ +--------------------------------+
6+
| ^ ^ |
7+
+---|---------------------|------+ +---|---------------------|------+
8+
| | Primary Network | | | Secondary Network | |
9+
| v | | | | v |
10+
| +--------+ +----------+ | | +----------+ +--------+ |
11+
| | Oracle | | Exporter | | | | Exporter | | Oracle | |
12+
| +--------+ +----------+ | | +----------+ +--------+ |
13+
| | ^ ^ | | ^ ^ | |
14+
+------|--------------|--|-------+ +-----|--|-------------|---------+
15+
| | | | | | +------------------------+
16+
| +--------|---------------|---------+ | | Pythd Websocket API |
17+
| | Local Store | |<---------------+-------+ +------+ |
18+
| +--------|---------------|---------+ | | | |<--|JRPC | |
19+
v | | | v | |Adapter| | WS | |
20+
+--------------------|---------------|--------|-----------+ | | |-->|Server| |
21+
| | Global Store | | |---->+-------+ +------+ |
22+
+--------------------|---------------|--------|-----------+ | ^ | |
23+
| | | | +---------------|----|---+
24+
| | v v | |
25+
+---------------------+ +--------------+ | |
26+
|Remote Keypair Loader| |Metrics Server| | |
27+
+---------------------+ +--------------+ | |
28+
^ | | |
29+
| v | v
30+
+-------------------------------------------------------------------+
31+
| User |
32+
+-------------------------------------------------------------------+
33+
Generated with textik.com on 2023-04-03
2734
2835
The arrows on the diagram above represent the direction of data flow.
2936
30-
Write path:
37+
Publisher data write path:
3138
- The user submits fresh price data to the system using the Pythd JRPC Websocket API.
3239
- The Adapter then transforms this into the Pyth SDK data structures and sends it to the Local Store.
3340
- The Local Store holds the latest price data the user has submitted for each price feed.
3441
- The Exporters periodically query the Local Store for the latest user-submitted data,
3542
and send it to the RPC node.
3643
37-
Read path:
44+
Publisher data read path:
3845
- The Oracles continually fetch data from the RPC node, and pass this to the Global Store.
3946
- The Global Store holds a unified view of the latest observed data from both networks, in the Pyth SDK data structures.
4047
- When a user queries for this data using the Pythd JRPC Websocket API, the Adapter fetches
4148
the latest data from the Global Store. It transforms this from the Pyth SDK data structures into the
4249
Pythd JRPC Websocket API data structures.
4350
- The Pythd JRPC Websocket API then sends this data to the user.
4451
52+
Remote Keypair Loading:
53+
- If no keypair is found at startup, Exporters poll the Remote Keypair Loader for a signing keypair
54+
- When the keypair is uploaded, it is given to the Exporters
55+
56+
Metrics Server:
57+
- Every update in global and local store is reflected in the metrics
58+
- Metrics are served using Prometheus
59+
4560
Note that there is an Oracle and Exporter for each network, but only one Local Store and Global Store.
4661
4762
################################################################################################################################## */
4863

64+
pub mod dashboard;
4965
pub mod metrics;
5066
pub mod pythd;
5167
pub mod remote_keypair_loader;

0 commit comments

Comments
 (0)