in_snmp: an input plugin for collect metrics by SNMP request #7671
Conversation
k402xxxcenxxx
changed the title
k402xxxcenxxx
force-pushed
the
feature/in_snmp
branch
from
869e75c to
eb87261
Compare
k402xxxcenxxx
requested review from
edsiper,
fujimotos,
koleini,
leonardo-albertovich,
niedbalski and
patrick-stephens
as code owners
|
I have found a leak in the code, and it can be resolved by applying this patch: net-snmp/net-snmp@4bd0d9a. However, I am unsure about the process of incorporating this fix into the fluent-bit project. Any suggestions or guidance on this matter would be greatly appreciated! |
patrick-stephens
left a comment
patrick-stephens
left a comment
There was a problem hiding this comment.
I've triggered a full build of all targets to confirm it builds for them, we probably need to modify dependencies to build.
Can you confirm it works for non-Linux targets as well? It may build but that doesn't mean it works...
CMakeLists.txt
Outdated
| option(FLB_IN_PROC "Enable Process input plugin" Yes) | ||
| option(FLB_IN_SYSTEMD "Enable Systemd input plugin" Yes) | ||
| option(FLB_IN_DUMMY "Enable Dummy input plugin" Yes) | ||
| option(FLB_IN_SNMP "Enable SNMP input plugin" Yes) |
There was a problem hiding this comment.
To confirm - does this work on Windows and macOS as well? It will need disabling if not in their specific override cmake/ files.
| flex \ | ||
| bison \ | ||
| libyaml-dev \ | ||
| libsnmp-dev \ |
There was a problem hiding this comment.
We presumably will also need this in the various package builds under packaging/distros too.
|
@patrick-stephens thanks, I will try to build and run on Windows device |
k402xxxcenxxx
force-pushed
the
feature/in_snmp
branch
from
eb87261 to
02ae8c0
Compare
k402xxxcenxxx
force-pushed
the
feature/in_snmp
branch
from
02ae8c0 to
63597e4
Compare
k402xxxcenxxx
force-pushed
the
feature/in_snmp
branch
2 times, most recently
from
265dab6 to
1c78626
Compare
|
I still fail to build it on Windows, so I choose to disable it in https://github.com/fluent/fluent-bit/blob/master/cmake/windows-setup.cmake. |
patrick-stephens
left a comment
patrick-stephens
left a comment
There was a problem hiding this comment.
Please resolve the build failures and comments raised by Coderabbit
|
Hey @k402xxxcenxxx, I'll review this PR but on a first glance I noticed that it seems that you're not aware of the fluen-bit coding style which you can check here. Could you please make the required changes before to ensure that the review focuses on the code as much as possible? Thank you. |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 1
📜 Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (2)
CMakeLists.txt(2 hunks)cmake/macos-setup.cmake(1 hunks)
🚧 Files skipped from review as they are similar to previous changes (1)
- CMakeLists.txt
🧰 Additional context used
🧠 Learnings (2)
📚 Learning: 2025-08-31T12:46:11.940Z
Learnt from: ThomasDevoogdt
Repo: fluent/fluent-bit PR: 9277
File: .github/workflows/pr-compile-check.yaml:147-151
Timestamp: 2025-08-31T12:46:11.940Z
Learning: In fluent-bit CMakeLists.txt, the system library preference flags are defined as FLB_PREFER_SYSTEM_LIB_ZSTD and FLB_PREFER_SYSTEM_LIB_KAFKA with the FLB_ prefix.
Applied to files:
cmake/macos-setup.cmake
📚 Learning: 2025-08-31T12:46:11.940Z
Learnt from: ThomasDevoogdt
Repo: fluent/fluent-bit PR: 9277
File: .github/workflows/pr-compile-check.yaml:147-151
Timestamp: 2025-08-31T12:46:11.940Z
Learning: In fluent-bit, the correct CMake flag for using system librdkafka is `FLB_PREFER_SYSTEM_LIB_KAFKA=ON`.
Applied to files:
cmake/macos-setup.cmake
🔇 Additional comments (1)
cmake/macos-setup.cmake (1)
15-21: LGTM: macOS-specific SNMP plugin override.The conditional block correctly disables the SNMP input plugin when
FLB_MACOS_DEFAULTSis enabled, consistent with the OS-specific defaulting pattern used for Windows.
leonardo-albertovich
left a comment
leonardo-albertovich
left a comment
There was a problem hiding this comment.
@k402xxxcenxxx it seems that you've missed my request.
Please address the coding style violations before this code can be reviewed.
k402xxxcenxxx
force-pushed
the
feature/in_snmp
branch
from
812befc to
6a418d2
Compare
There was a problem hiding this comment.
Actionable comments posted: 1
♻️ Duplicate comments (2)
plugins/in_snmp/in_snmp.c (1)
306-311: Free response PDU in cleanup block.Line 307 sets
response = NULLwithout freeing the PDU. While normal and error paths freeresponse(lines 212, 221, 229, 245, 284-286), defensive cleanup should callsnmp_free_pdu(response)to handle any edge case where execution reaches cleanup with an unfree'd PDU.Apply this fix:
collect_cleanup: - if (response) { response = NULL; } + if (response) { snmp_free_pdu(response); response = NULL; } if (ss) { snmp_close(ss); ss = NULL; }tests/runtime/in_snmp.c (1)
41-67: Add map size validation before accessing map elements.Lines 58-60 and 61-63 access
obj->via.map.ptr[0]without verifying thatobj->via.map.size >= 1. If the map is empty, this causes undefined behavior.Apply this diff:
while (msgpack_unpack_next(&result, records->records[i].data, records->records[i].size, &off) == MSGPACK_UNPACK_SUCCESS) { flb_time_pop_from_msgpack(&ftm, &result, &obj); TEST_CHECK(obj->type == MSGPACK_OBJECT_MAP); + TEST_CHECK(obj->via.map.size >= 1); TEST_CHECK(strncmp("iso.3.6.1.2.1.1.3.0", obj->via.map.ptr[0].key.via.str.ptr, obj->via.map.ptr[0].key.via.str.size) == 0);
🧹 Nitpick comments (1)
tests/runtime/in_snmp.c (1)
17-39: Consider safer realloc pattern to prevent potential memory leak.At lines 27-29, if
flb_reallocfails, the oldctx->recordspointer is lost, causing a memory leak. While the test cleanup (lines 171-175) eventually frees records, a realloc failure would leave the old allocation unreachable.Apply this defensive pattern:
} else { - ctx->records = (struct callback_record *) - flb_realloc(ctx->records, - (ctx->num_records+1)*sizeof(struct callback_record)); + struct callback_record *tmp = (struct callback_record *) + flb_realloc(ctx->records, + (ctx->num_records+1)*sizeof(struct callback_record)); + if (tmp == NULL) { + return -1; + } + ctx->records = tmp; } - if (ctx->records == NULL) { - return -1; - }
📜 Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (19)
CMakeLists.txt(2 hunks)cmake/FindNetsnmp.cmake(1 hunks)cmake/macos-setup.cmake(1 hunks)cmake/plugins_options.cmake(1 hunks)cmake/windows-setup.cmake(1 hunks)dockerfiles/Dockerfile(1 hunks)dockerfiles/Dockerfile.centos7(1 hunks)packaging/distros/amazonlinux/Dockerfile(4 hunks)packaging/distros/centos/Dockerfile(8 hunks)packaging/distros/debian/Dockerfile(8 hunks)packaging/distros/raspbian/Dockerfile(3 hunks)packaging/distros/ubuntu/Dockerfile(9 hunks)plugins/CMakeLists.txt(1 hunks)plugins/in_snmp/CMakeLists.txt(1 hunks)plugins/in_snmp/in_snmp.c(1 hunks)plugins/in_snmp/in_snmp.h(1 hunks)run_code_analysis.sh(1 hunks)tests/runtime/CMakeLists.txt(1 hunks)tests/runtime/in_snmp.c(1 hunks)
🚧 Files skipped from review as they are similar to previous changes (13)
- plugins/in_snmp/CMakeLists.txt
- plugins/CMakeLists.txt
- packaging/distros/centos/Dockerfile
- tests/runtime/CMakeLists.txt
- CMakeLists.txt
- packaging/distros/amazonlinux/Dockerfile
- cmake/macos-setup.cmake
- cmake/plugins_options.cmake
- run_code_analysis.sh
- packaging/distros/ubuntu/Dockerfile
- dockerfiles/Dockerfile.centos7
- dockerfiles/Dockerfile
- packaging/distros/debian/Dockerfile
🧰 Additional context used
🧠 Learnings (2)
📚 Learning: 2025-08-31T12:46:11.940Z
Learnt from: ThomasDevoogdt
Repo: fluent/fluent-bit PR: 9277
File: .github/workflows/pr-compile-check.yaml:147-151
Timestamp: 2025-08-31T12:46:11.940Z
Learning: In fluent-bit CMakeLists.txt, the system library preference flags are defined as FLB_PREFER_SYSTEM_LIB_ZSTD and FLB_PREFER_SYSTEM_LIB_KAFKA with the FLB_ prefix.
Applied to files:
cmake/windows-setup.cmake
📚 Learning: 2025-08-31T12:46:11.940Z
Learnt from: ThomasDevoogdt
Repo: fluent/fluent-bit PR: 9277
File: .github/workflows/pr-compile-check.yaml:147-151
Timestamp: 2025-08-31T12:46:11.940Z
Learning: In fluent-bit, the correct CMake flag for using system librdkafka is `FLB_PREFER_SYSTEM_LIB_KAFKA=ON`.
Applied to files:
cmake/windows-setup.cmake
🧬 Code graph analysis (2)
tests/runtime/in_snmp.c (2)
include/fluent-bit/flb_mem.h (2)
flb_calloc(84-96)flb_free(126-128)src/flb_lib.c (9)
flb_create(143-225)flb_input(266-276)flb_input_set(305-335)flb_output(279-289)flb_output_set(520-551)flb_start(983-994)flb_stop(1011-1055)flb_lib_free(762-769)flb_destroy(228-263)
plugins/in_snmp/in_snmp.c (5)
include/fluent-bit/flb_mem.h (2)
flb_calloc(84-96)flb_free(126-128)src/flb_log_event_encoder.c (6)
flb_log_event_encoder_begin_record(246-254)flb_log_event_encoder_set_current_timestamp(289-292)flb_log_event_encoder_commit_record(256-274)flb_log_event_encoder_reset(31-40)flb_log_event_encoder_init(42-74)flb_log_event_encoder_destroy(99-116)src/flb_input_log.c (1)
flb_input_log_append(1413-1423)include/fluent-bit/flb_input.h (1)
flb_input_config_map_set(717-743)src/flb_input.c (4)
flb_input_set_context(1610-1613)flb_input_set_collector_time(1685-1704)flb_input_collector_pause(2035-2072)flb_input_collector_resume(2095-2151)
🔇 Additional comments (9)
packaging/distros/raspbian/Dockerfile (1)
21-21: LGTM!The
libsnmp-devpackage additions are correctly placed and consistent across all three Raspbian base images (buster, bullseye, bookworm), enabling the build of the new SNMP input plugin.Also applies to: 34-34, 47-47
tests/runtime/in_snmp.c (2)
69-117: LGTM!The bounds checking at lines 98-102 correctly limits iteration to prevent out-of-bounds access to the
expected_keysandexpected_valsarrays.
180-184: LGTM!The test entry point correctly exercises both
snmp_getandsnmp_walkscenarios with appropriate validators.cmake/FindNetsnmp.cmake (1)
11-79: LGTM!The module correctly discovers Net-SNMP components with proper include and library search paths for Linux, macOS Homebrew, and MacPorts. The plural
NETSNMP_INCLUDE_DIRSconvention is properly implemented, and all variables are marked as advanced.Note:
NETSNMPTRAPD_LIBRARYis intentionally excluded fromNETSNMP_LIBRARIES(line 58), which is appropriate since trap daemon functionality is typically not needed by client applications.cmake/windows-setup.cmake (1)
84-84: LGTM!Disabling the SNMP plugin by default on Windows is appropriate given the build failures mentioned in the PR summary. The placement among other input plugin defaults is consistent with the file's structure.
plugins/in_snmp/in_snmp.c (3)
24-116: LGTM!The mock function correctly mirrors
snmp_synch_responsesemantics by freeing the request PDU and includes proper NULL checks for environment variables and allocations. Error cleanup is defensive with NULL guards.
433-450: LGTM!The exit function properly cleans up all allocated resources:
- Line 438: destroys the log encoder
- Lines 440-442: frees the dynamically allocated peername
- Line 443: frees the context structure
- Lines 444-447: properly shuts down Net-SNMP and socket layers
453-495: LGTM!Configuration map entries are well-documented with clear descriptions. The
oid_typehelp text (line 487) correctly describes the accepted values ('get' and 'walk') and their meanings.plugins/in_snmp/in_snmp.h (1)
1-26: LGTM!The header properly defines the plugin context structure with all necessary fields. The
target_portfield (line 17) is correctly utilized in the init function to compose the peername, addressing previous concerns about unused configuration.
This commit introduces a new SNMP input plugin for Fluent Bit with the following key features: - Implement SNMP GET and WALK operations using NET-SNMP library - Add configuration options for timeout and retries - Include comprehensive unit tests for SNMP operations - Fix memory management and resource cleanup - Add NET-SNMP dependencies in various Dockerfiles - Configure CMake to properly find and link NET-SNMP - Improve error handling and validation The plugin allows monitoring of SNMP-enabled devices and supports both single OID queries and walking through SNMP trees. Signed-off-by: k402xxxcenxxx <[email protected]>
…S by default. Signed-off-by: k402xxxcenxxx <[email protected]>
in_snmp: minor fix Signed-off-by: k402xxxcenxxx <[email protected]>
k402xxxcenxxx
force-pushed
the
feature/in_snmp
branch
from
6a418d2 to
9f62cc4
Compare
There was a problem hiding this comment.
Actionable comments posted: 1
♻️ Duplicate comments (3)
cmake/macos-setup.cmake (1)
16-16: Fix typo: "setttings" should be "settings".The status message contains "setttings" with three t's instead of "settings" with two t's.
Apply this fix:
- message(STATUS "Overriding setttings with macos-setup.cmake") + message(STATUS "Overriding settings with macos-setup.cmake")tests/runtime/in_snmp.c (1)
136-140: Add NULL check after records allocation.Line 136 allocates
recordsbut lines 137-138 immediately dereference it without checking if the allocation succeeded. Ifflb_callocreturns NULL (e.g., under memory pressure or during fuzzing), this causes undefined behavior.Apply this fix:
records = flb_calloc(1, sizeof(struct callback_records)); +if (!records) { + flb_error("[test] Failed to allocate callback_records"); + return; +} records->num_records = 0; records->records = NULL;plugins/in_snmp/CMakeLists.txt (1)
1-8: Addtarget_include_directories()to configure Net-SNMP include paths for the plugin target.The
NETSNMP_INCLUDE_DIRSvariable is set but never applied to the target. TheFLB_PLUGINmacro only handles linking libraries, not include directories. Other plugins (e.g.,out_pgsql,in_exec_wasi,filter_wasm) explicitly calltarget_include_directories()afterFLB_PLUGIN()to configure includes.Add after line 7:
target_include_directories(flb-plugin-in_snmp PRIVATE ${NETSNMP_INCLUDE_DIRS})
🧹 Nitpick comments (2)
cmake/FindNetsnmp.cmake (2)
72-79: *Consider marking the _NAMES variables as advanced.The individual library path variables are correctly marked as advanced, but the intermediate
*_NAMESvariables (NETSNMP_NAMES, NETSNMPAGENT_NAMES, etc.) are implementation details that should also be marked to keep CMake GUIs clean.Apply this diff to mark the NAMES variables as advanced:
MARK_AS_ADVANCED( NETSNMP_LIBRARY NETSNMPAGENT_LIBRARY NETSNMPHELPERS_LIBRARY NETSNMPMIBS_LIBRARY NETSNMPTRAPD_LIBRARY NETSNMP_INCLUDE_DIR + NETSNMP_NAMES + NETSNMPAGENT_NAMES + NETSNMPHELPERS_NAMES + NETSNMPMIBS_NAMES + NETSNMPTRAPD_NAMES )
1-79: Consider adding pkg-config support for improved robustness.Many modern CMake find modules use
pkg-configas a fallback when explicit path searches fail. Net-SNMP providesnetsnmp.pcfiles on most distributions, and leveraging pkg-config would help discover installations in non-standard prefixes and automatically handle transitive dependencies.Example pattern to add before the FIND_PATH/FIND_LIBRARY calls:
# Try pkg-config first FIND_PACKAGE(PkgConfig QUIET) IF(PKG_CONFIG_FOUND) PKG_CHECK_MODULES(PC_NETSNMP QUIET netsnmp) ENDIF() # Use pkg-config hints in FIND_PATH and FIND_LIBRARY FIND_PATH(NETSNMP_INCLUDE_DIR net-snmp/net-snmp-includes.h HINTS ${PC_NETSNMP_INCLUDE_DIRS} PATHS /usr/include /usr/local/include /opt/homebrew/include /opt/local/include )
📜 Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (19)
CMakeLists.txt(2 hunks)cmake/FindNetsnmp.cmake(1 hunks)cmake/macos-setup.cmake(1 hunks)cmake/plugins_options.cmake(1 hunks)cmake/windows-setup.cmake(1 hunks)dockerfiles/Dockerfile(1 hunks)dockerfiles/Dockerfile.centos7(1 hunks)packaging/distros/amazonlinux/Dockerfile(4 hunks)packaging/distros/centos/Dockerfile(8 hunks)packaging/distros/debian/Dockerfile(8 hunks)packaging/distros/raspbian/Dockerfile(3 hunks)packaging/distros/ubuntu/Dockerfile(9 hunks)plugins/CMakeLists.txt(1 hunks)plugins/in_snmp/CMakeLists.txt(1 hunks)plugins/in_snmp/in_snmp.c(1 hunks)plugins/in_snmp/in_snmp.h(1 hunks)run_code_analysis.sh(1 hunks)tests/runtime/CMakeLists.txt(1 hunks)tests/runtime/in_snmp.c(1 hunks)
🚧 Files skipped from review as they are similar to previous changes (10)
- dockerfiles/Dockerfile
- dockerfiles/Dockerfile.centos7
- plugins/CMakeLists.txt
- packaging/distros/debian/Dockerfile
- CMakeLists.txt
- run_code_analysis.sh
- packaging/distros/centos/Dockerfile
- cmake/plugins_options.cmake
- packaging/distros/ubuntu/Dockerfile
- plugins/in_snmp/in_snmp.c
🧰 Additional context used
🧠 Learnings (2)
📚 Learning: 2025-08-31T12:46:11.940Z
Learnt from: ThomasDevoogdt
Repo: fluent/fluent-bit PR: 9277
File: .github/workflows/pr-compile-check.yaml:147-151
Timestamp: 2025-08-31T12:46:11.940Z
Learning: In fluent-bit CMakeLists.txt, the system library preference flags are defined as FLB_PREFER_SYSTEM_LIB_ZSTD and FLB_PREFER_SYSTEM_LIB_KAFKA with the FLB_ prefix.
Applied to files:
cmake/windows-setup.cmakecmake/macos-setup.cmakeplugins/in_snmp/CMakeLists.txt
📚 Learning: 2025-08-31T12:46:11.940Z
Learnt from: ThomasDevoogdt
Repo: fluent/fluent-bit PR: 9277
File: .github/workflows/pr-compile-check.yaml:147-151
Timestamp: 2025-08-31T12:46:11.940Z
Learning: In fluent-bit, the correct CMake flag for using system librdkafka is `FLB_PREFER_SYSTEM_LIB_KAFKA=ON`.
Applied to files:
cmake/windows-setup.cmakecmake/macos-setup.cmake
🧬 Code graph analysis (1)
tests/runtime/in_snmp.c (2)
include/fluent-bit/flb_mem.h (2)
flb_calloc(84-96)flb_free(126-128)src/flb_lib.c (9)
flb_create(143-225)flb_input(266-276)flb_input_set(305-335)flb_output(279-289)flb_output_set(520-551)flb_start(983-994)flb_stop(1011-1055)flb_lib_free(762-769)flb_destroy(228-263)
🔇 Additional comments (15)
packaging/distros/amazonlinux/Dockerfile (1)
26-26: LGTM — Net-SNMP package correctly added for Amazon Linux.The
net-snmp-develpackage is the correct development headers package for Amazon Linux 2 and 2023. The addition is consistent across all four base image stages.Also applies to: 51-51, 76-76, 103-103
packaging/distros/raspbian/Dockerfile (1)
21-21: LGTM — Net-SNMP package correctly added for Raspbian.The
libsnmp-devpackage is the correct development headers package for Debian-based distributions like Raspbian. The addition is consistent across all three base image stages (buster, bullseye, bookworm).Also applies to: 34-34, 47-47
tests/runtime/in_snmp.c (3)
17-39: LGTM — Proper allocation and NULL handling.The callback correctly handles allocation failures and returns
-1whenflb_callocorflb_reallocfail.
94-113: LGTM — Proper bounds checking for map access.The function correctly validates
obj->via.map.size >= 2and limits the loop to avoid out-of-bounds access to the expected_keys and expected_vals arrays.
180-190: LGTM — Test registration is correct.The test wrapper and TEST_LIST registration follow the standard pattern for Fluent Bit runtime tests.
tests/runtime/CMakeLists.txt (1)
49-49: LGTM — Test registration follows standard pattern.The SNMP runtime test is correctly registered using the
FLB_RT_TESTmacro within the appropriate conditional block, consistent with other input plugin tests.cmake/windows-setup.cmake (1)
84-84: LGTM — Windows exclusion aligns with PR objectives.Disabling the SNMP plugin on Windows by default is consistent with the PR author's decision not to support Windows due to build failures. The macOS setup file similarly disables SNMP, ensuring consistent cross-platform defaults.
cmake/macos-setup.cmake (1)
15-21: LGTM — macOS SNMP exclusion is consistent with Windows.The FLB_MACOS_DEFAULTS block correctly disables the SNMP plugin on macOS, mirroring the Windows defaults and ensuring consistent cross-platform behavior for this optional plugin.
plugins/in_snmp/in_snmp.h (1)
1-26: LGTM — Header structure is well-defined.The
struct flb_snmpdefinition includes appropriate fields for SNMP session configuration and Fluent Bit integration. Include guards and headers are properly configured.Note: A past review mentioned that
ctx->portis not applied to the session, but that's an implementation issue inin_snmp.c, not a header definition issue.cmake/FindNetsnmp.cmake (6)
11-14: LGTM!Standard CMake cache optimization pattern is correctly implemented.
16-21: LGTM!The include path search correctly targets the header used by the plugin and covers standard installation locations across Linux, BSD, Homebrew, and MacPorts.
23-51: LGTM!Library search paths are consistent across all five libraries and correctly align with the include directory search paths.
61-62: LGTM!Correctly provides the plural form per CMake convention.
65-70: LGTM!Standard package handling is correctly configured with the plural include directory form.
53-59: No changes needed—exclusion is correct.Verification confirms the SNMP input plugin performs GET and WALK queries, not trap reception. The
netsnmptrapdlibrary is correctly excluded since the plugin has no trap-related functionality. The code is in proper working order; adding a clarifying comment remains optional for future maintainability.
…P plugin files Signed-off-by: k402xxxcenxxx <[email protected]>
Hi @leonardo-albertovich , |
…onfiguration in_snmp: add null check for callback_records allocation in SNMP tests Signed-off-by: k402xxxcenxxx <[email protected]>
|
Hello, And thanks for the work. ❤️ |
6 participants
Enter
[N/A]in the box, if an item is not applicable to your change.Testing
Before we can approve your change; please submit the following in a comment:
There is one leak and can be fixed after this patch is release: net-snmp/net-snmp@4bd0d9a
If this is a change to packaging of containers or native binaries then please confirm it works for all targets.
ok-package-testlabel to test for all targets (requires maintainer to do).Documentation
fluent/fluent-bit-docs#1162
Backporting
Fluent Bit is licensed under Apache 2.0, by submitting this pull request I understand that this code will be released under the terms of that license.
Summary by CodeRabbit
New Features
Build / Packaging
Tests