forked from apache/skywalking
-
Notifications
You must be signed in to change notification settings - Fork 0
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Support the telegraf receiver plugin module (apache#9620)
* The telegraf receiver plugin module development. * Refactored code of converting telegraf data and fixed some errors. * Support Telegraf receiver plugin module. * Add telegraf-receiver.md, update changes.md and vm-monitoring.md. * Rename YAML file and change code style. * Change receiver-telegraf of application.yml. * The receiver-telegraf e2e test. * The telegraf receiver e2e test. * Fix an issue and delete redundant configs. * Add Unit Test about converting Telegraf metrics. * Adjust Unit Test about converting Telegraf metrics. * Add License Header to Unit Test and change binary.xml. * Change module provider's config initialization mechanism. * Exclude the telegraf-rules in server-starter pom.xml. * Fix telegraf e2e test issues. * Change telegraf e2e test. * Fix issues * Fix issues. * Add Sample convert Unit Test. * Change vm.yaml, related documents and fix some issues. * Change backend-vm-monitoring.md. * Fix SampleConvertTest checkstyle issue. * Change vm.yaml swap MAL. * Update menu.yml and binary.xml. * Update Telegraf Unit test. * Delete telegraf config package, use meter.analyzer.prometheus package to load config file. * Reorder telegraf metrics in menu.yml. * Change e2e, vm, config, linux-service and vm.md. * Update telegraf.conf file. * Change url of telegraf.conf file. * Update e2e.yaml, conf file, menu.yml and delete useless code of provider. * Update grouping sampleFamily by timestamp and name, and add new UTs. * Update vm-monitoring.md and .asf.yaml. Co-authored-by: Superskyyy (ONLINE) <[email protected]> Co-authored-by: 吴晟 Wu Sheng <[email protected]>
- Loading branch information
3 people
authored
Nov 5, 2022
1 parent
d32a318
commit 42f3396
Showing
26 changed files
with
1,561 additions
and
16 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,45 @@ | ||
| # Telegraf receiver | ||
|
|
||
| The Telegraf receiver supports receiving InfluxDB Telegraf's metrics by meter-system. | ||
| The OAP can load the configuration at bootstrap. The files are located at `$CLASSPATH/telegraf-rules`. | ||
| If the new configuration is not well-formed, the OAP may fail to start up. | ||
|
|
||
| This is the [InfluxDB Telegraf](https://docs.influxdata.com/telegraf/v1.24/) Document, | ||
| the Telegraf receiver can handle Telegraf's [CPU Input Plugin](https://github.com/influxdata/telegraf/blob/release-1.24/plugins/inputs/cpu/README.md), | ||
| [Memory Input Plugin](https://github.com/influxdata/telegraf/blob/release-1.24/plugins/inputs/mem/README.md). | ||
|
|
||
| There are many other telegraf input plugins, users can customize different input plugins' rule files. | ||
| The rule file should be in YAML format, defined by the scheme described in [MAL](../../concepts-and-designs/mal.md). | ||
| Please see the [telegraf plugin directory](https://docs.influxdata.com/telegraf/v1.24/plugins/) for more input plugins information. | ||
|
|
||
| **Notice:** | ||
| * The Telegraf receiver module uses `HTTP` to receive telegraf's metrics, | ||
| so the outputs method should be set `[[outputs.http]]` in telegraf.conf file. | ||
| Please see the [http outputs](https://github.com/influxdata/telegraf/blob/release-1.24/plugins/outputs/http/README.md) | ||
| for more details. | ||
|
|
||
| * The Telegraf receiver module **only** process telegraf's `JSON` metrics format, | ||
| the data format should be set `data_format = "json"` in telegraf.conf file. | ||
| Please see the [JSON data format](https://docs.influxdata.com/telegraf/v1.24/data_formats/output/json/) | ||
| for more details. | ||
|
|
||
| * The default `json_timestamp_units` is second in JSON output, | ||
| and the Telegraf receiver module **only** process `second` timestamp unit. | ||
| If users configure `json_timestamp_units` in telegraf.conf file, `json_timestamp_units = "1s"` is feasible. | ||
| Please see the [JSON data format](https://docs.influxdata.com/telegraf/v1.24/data_formats/output/json/) | ||
| for more details. | ||
|
|
||
| The following is the default telegraf receiver YAML rule file in the `application.yml`, | ||
| Set `SW_RECEIVER_TELEGRAF:default` through system environment or change `SW_RECEIVER_TELEGRAF_ACTIVE_FILES:vm` | ||
| to activate the OpenTelemetry receiver with `vm.yml` in telegraf-rules. | ||
| ```yaml | ||
| receiver-telegraf: | ||
| selector: ${SW_RECEIVER_TELEGRAF:default} | ||
| default: | ||
| activeFiles: ${SW_RECEIVER_TELEGRAF_ACTIVE_FILES:vm} | ||
| ``` | ||
| | Rule Name | Description | Configuration File | Data Source | | ||
| |-----------|----------------|------------------------|-------------------------------------------------------------------------| | ||
| | vm | Metrics of VMs | telegraf-rules/vm.yaml | Telegraf inputs plugins --> Telegraf Receiver --> SkyWalking OAP Server | | ||
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
44 changes: 44 additions & 0 deletions
44
oap-server/server-receiver-plugin/skywalking-telegraf-receiver-plugin/pom.xml
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,44 @@ | ||
| <?xml version="1.0" encoding="UTF-8"?> | ||
| <!-- | ||
| ~ Licensed to the Apache Software Foundation (ASF) under one or more | ||
| ~ contributor license agreements. See the NOTICE file distributed with | ||
| ~ this work for additional information regarding copyright ownership. | ||
| ~ The ASF licenses this file to You under the Apache License, Version 2.0 | ||
| ~ (the "License"); you may not use this file except in compliance with | ||
| ~ the License. You may obtain a copy of the License at | ||
| ~ | ||
| ~ http://www.apache.org/licenses/LICENSE-2.0 | ||
| ~ | ||
| ~ Unless required by applicable law or agreed to in writing, software | ||
| ~ distributed under the License is distributed on an "AS IS" BASIS, | ||
| ~ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. | ||
| ~ See the License for the specific language governing permissions and | ||
| ~ limitations under the License. | ||
| ~ | ||
| --> | ||
|
|
||
| <project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd"> | ||
| <parent> | ||
| <artifactId>server-receiver-plugin</artifactId> | ||
| <groupId>org.apache.skywalking</groupId> | ||
| <version>9.3.0-SNAPSHOT</version> | ||
| </parent> | ||
| <modelVersion>4.0.0</modelVersion> | ||
|
|
||
| <artifactId>skywalking-telegraf-receiver-plugin</artifactId> | ||
| <packaging>jar</packaging> | ||
|
|
||
| <dependencies> | ||
| <dependency> | ||
| <groupId>org.apache.skywalking</groupId> | ||
| <artifactId>agent-analyzer</artifactId> | ||
| <version>${project.version}</version> | ||
| </dependency> | ||
| <dependency> | ||
| <groupId>org.apache.skywalking</groupId> | ||
| <artifactId>skywalking-sharing-server-plugin</artifactId> | ||
| <version>${project.version}</version> | ||
| </dependency> | ||
| </dependencies> | ||
|
|
||
| </project> |
33 changes: 33 additions & 0 deletions
33
...ava/org/apache/skywalking/oap/server/receiver/telegraf/module/TelegrafReceiverModule.java
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,33 @@ | ||
| /* | ||
| * Licensed to the Apache Software Foundation (ASF) under one or more | ||
| * contributor license agreements. See the NOTICE file distributed with | ||
| * this work for additional information regarding copyright ownership. | ||
| * The ASF licenses this file to You under the Apache License, Version 2.0 | ||
| * (the "License"); you may not use this file except in compliance with | ||
| * the License. You may obtain a copy of the License at | ||
| * | ||
| * http://www.apache.org/licenses/LICENSE-2.0 | ||
| * | ||
| * Unless required by applicable law or agreed to in writing, software | ||
| * distributed under the License is distributed on an "AS IS" BASIS, | ||
| * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. | ||
| * See the License for the specific language governing permissions and | ||
| * limitations under the License. | ||
| * | ||
| */ | ||
|
|
||
| package org.apache.skywalking.oap.server.receiver.telegraf.module; | ||
|
|
||
| import org.apache.skywalking.oap.server.library.module.ModuleDefine; | ||
|
|
||
| public class TelegrafReceiverModule extends ModuleDefine { | ||
|
|
||
| public TelegrafReceiverModule() { | ||
| super("receiver-telegraf"); | ||
| } | ||
|
|
||
| @Override | ||
| public Class[] services() { | ||
| return new Class[0]; | ||
| } | ||
| } |
37 changes: 37 additions & 0 deletions
37
...ava/org/apache/skywalking/oap/server/receiver/telegraf/provider/TelegrafModuleConfig.java
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,37 @@ | ||
| /* | ||
| * Licensed to the Apache Software Foundation (ASF) under one or more | ||
| * contributor license agreements. See the NOTICE file distributed with | ||
| * this work for additional information regarding copyright ownership. | ||
| * The ASF licenses this file to You under the Apache License, Version 2.0 | ||
| * (the "License"); you may not use this file except in compliance with | ||
| * the License. You may obtain a copy of the License at | ||
| * | ||
| * http://www.apache.org/licenses/LICENSE-2.0 | ||
| * | ||
| * Unless required by applicable law or agreed to in writing, software | ||
| * distributed under the License is distributed on an "AS IS" BASIS, | ||
| * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. | ||
| * See the License for the specific language governing permissions and | ||
| * limitations under the License. | ||
| * | ||
| */ | ||
|
|
||
| package org.apache.skywalking.oap.server.receiver.telegraf.provider; | ||
|
|
||
| import lombok.Getter; | ||
| import lombok.Setter; | ||
| import org.apache.skywalking.oap.server.core.Const; | ||
| import org.apache.skywalking.oap.server.library.module.ModuleConfig; | ||
|
|
||
| @Setter | ||
| @Getter | ||
| public class TelegrafModuleConfig extends ModuleConfig { | ||
|
|
||
| public static final String CONFIG_PATH = "telegraf-rules"; | ||
|
|
||
| /** | ||
| * active receive configs, files split by "," | ||
| */ | ||
| private String activeFiles = Const.EMPTY_STRING; | ||
|
|
||
| } |
Oops, something went wrong.