Implement self-monitoring for BanyanDB via OAP Server and UI (#13527)

Author: firebor

.github/workflows/skywalking.yaml

@@ -366,6 +366,8 @@ jobs:
             config: test/e2e-v2/cases/storage/banyandb/e2e.yaml
           - name: BanyanDB TLS
             config: test/e2e-v2/cases/storage/banyandb/tls/e2e.yaml
+          - name: BanyanDB monitoring
+            config: test/e2e-v2/cases/banyandb/e2e.yaml
           - name: Storage MySQL
             config: test/e2e-v2/cases/storage/mysql/e2e.yaml
           - name: Storage PostgreSQL

docs/en/banyandb/dashboards-banyandb.md

@@ -0,0 +1,49 @@
+# BanyanDB self observability dashboard
+
+[BanyanDB](https://skywalking.apache.org/docs/skywalking-banyandb/next/readme/), as an observability database, aims to ingest, analyze and store Metrics, Tracing, and Logging data. It's designed to handle observability data generated by **Apache SkyWalking**，it also provides a dashboard to visualize the self-observability metrics.
+
+## Data flow
+1. [BanyanDB](https://skywalking.apache.org/docs/skywalking-banyandb/next/readme/) collects metrics data internally and exposes a Prometheus http endpoint to retrieve the metrics.
+2. OpenTelemetry Collector fetches metrics from BanyanDB and pushes metrics to SkyWalking OAP Server via OpenTelemetry gRPC exporter.
+3. The SkyWalking OAP Server parses the expression with [MAL](../concepts-and-designs/mal.md) to filter/calculate/aggregate and store the results.
+
+## Set up
+1. Start [BanyanDB](https://skywalking.apache.org/docs/skywalking-banyandb/next/readme/),supporting both [Standalone Mode](https://skywalking.apache.org/docs/skywalking-banyandb/next/installation/standalone/) and [Cluster Mode](https://skywalking.apache.org/docs/skywalking-banyandb/next/installation/cluster/).
+2. Set up [OpenTelemetry Collector ](https://opentelemetry.io/docs/collector/getting-started/#docker). For details on Prometheus Receiver in OpenTelemetry Collector, refer to [here](../../../test/e2e-v2/cases/banyandb/otel-collector-config.yaml).
+3. Config SkyWalking [OpenTelemetry receiver](https://skywalking.apache.org/docs/main/next/en/setup/backend/opentelemetry-receiver/).
+
+## BanyanDB monitoring
+Self observability monitoring provides monitoring of the status and resources of the [BanyanDB](https://skywalking.apache.org/docs/skywalking-banyandb/next/readme/) server itself. `banyandb-server` is a `Service` in BanyanDB, and land on the `Layer: BANYANDB`.
+
+### Self observability metrics
+
+| Unit | Metric Name                                       | Description | Data Source |
+|------|---------------------------------------------------|-------------|-------------|
+| o/s | meter_banyandb_write_rate                        | Write Rate (Operations per Second) | BanyanDB |
+| GiB | meter_banyandb_total_memory                      | Total Memory | BanyanDB |
+| GiB | meter_banyandb_disk_usage                        | Disk Usage | BanyanDB |
+| r/s | meter_banyandb_query_rate                        | Query Rate (Requests per Second) | BanyanDB |
+| Count | meter_banyandb_total_cpu                        | Total CPU Cores | BanyanDB |
+| c/m | meter_banyandb_write_and_query_errors_rate      | Write and Query Errors Rate（Counts per Minute） | BanyanDB |
+| c/s | meter_banyandb_etcd_operation_rate               | Etcd Operation Rate（Counts per Second） | BanyanDB |
+| Count | meter_banyandb_active_instance                  | Active Instances | BanyanDB |
+| % | meter_banyandb_cpu_usage                        | CPU Usage Percentage | BanyanDB |
+| % | meter_banyandb_rss_memory_usage                 | RSS Memory Usage Percentage | BanyanDB |
+| % | meter_banyandb_disk_usage_all                   | Disk Usage Percentage | BanyanDB |
+| KiB/s | meter_banyandb_network_usage_recv               | Network Receive Rate | BanyanDB |
+| KiB/s | meter_banyandb_network_usage_sent               | Network Send Rate | BanyanDB |
+| o/s | meter_banyandb_storage_write_rate               | Storage Write Rate (Operations per Second) | BanyanDB |
+| s | meter_banyandb_query_latency                    | Query Latency (s) | BanyanDB |
+| Count | meter_banyandb_total_data                      | Total Data Elements | BanyanDB |
+| r/m | meter_banyandb_merge_file_data                 | Merge File Data Rate(Revolutions per Minute) | BanyanDB |
+| s | meter_banyandb_merge_file_latency              | Merge File Latency(s) | BanyanDB |
+| Count | meter_banyandb_merge_file_partitions          | Merge File Partitions | BanyanDB |
+| o/s | meter_banyandb_series_write_rate               | Series Write Rate (Operations per Second) | BanyanDB |
+| o/s | meter_banyandb_series_term_search_rate         | Series Term Search Rate (Operations per Second) | BanyanDB |
+| Count | meter_banyandb_total_series                   | Total Series Count | BanyanDB |
+| ops | meter_banyandb_stream_write_rate              | Stream Write Rate (Operations per Second) | BanyanDB |
+| ops | meter_banyandb_term_search_rate                | Term Search Rate (Operations per Second) | BanyanDB |
+| Count | meter_banyandb_total_document                 | Total Document Count | BanyanDB |
+
+## Customizations
+You can customize your own metrics/expression/dashboard panel.The metrics definition and expression rules are found in `/config/otel-rules/banyandb`.The [BanyanDB](https://skywalking.apache.org/docs/skywalking-banyandb/next/readme/) dashboard panel configurations are found in `/config/ui-initialized-templates/banyandb`.

docs/en/changes/changes.md

@@ -10,6 +10,7 @@
 
 #### OAP Server
 
+* Implement self-monitoring for BanyanDB via OAP Server.
 * BanyanDB: Support `hot/warm/cold` stages configuration.
 * Fix query continues profiling policies error when the policy is already in the cache.
 * Support `hot/warm/cold` stages TTL query in the status API and graphQL API.
@@ -116,6 +117,7 @@
 
 #### UI
 
+* Implement self-monitoring for BanyanDB via UI.
 * Enhance the trace `List/Tree/Table` graph to support displaying multiple refs of spans and distinguishing different parents.
 * Fix: correct the same labels for metrics.
 * Refactor: use the Fetch API to instead of Axios.