Skip to content

out_splunk: skip the flush if payload is empty #10314

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Draft
wants to merge 1 commit into
base: master
Choose a base branch
from

Conversation

lecaros
Copy link
Contributor

@lecaros lecaros commented May 8, 2025

Addresses #10313

Skips the flush if the payload is empty.
Empty payload can be the result of filtering the data. See #10313 for details.


Enter [N/A] in the box, if an item is not applicable to your change.

Testing
Before we can approve your change; please submit the following in a comment:

Relevant message:

[2025/05/08 11:44:10] [ warn] [output:splunk:splunk.1] Content length is 0, skipping flush
/fluent-bit -c drop.yml
Fluent Bit v4.0.2
* Copyright (C) 2015-2025 The Fluent Bit Authors
* Fluent Bit is a CNCF sub-project under the umbrella of Fluentd
* https://fluentbit.io

______ _                  _    ______ _ _             ___  _____ 
|  ___| |                | |   | ___ (_) |           /   ||  _  |
| |_  | |_   _  ___ _ __ | |_  | |_/ /_| |_  __   __/ /| || |/' |
|  _| | | | | |/ _ \ '_ \| __| | ___ \ | __| \ \ / / /_| ||  /| |
| |   | | |_| |  __/ | | | |_  | |_/ / | |_   \ V /\___  |\ |_/ /
\_|   |_|\__,_|\___|_| |_|\__| \____/|_|\__|   \_/     |_(_)___/ 


[2025/05/08 11:44:07] [ info] Configuration:
[2025/05/08 11:44:07] [ info]  flush time     | 1.000000 seconds
[2025/05/08 11:44:07] [ info]  grace          | 5 seconds
[2025/05/08 11:44:07] [ info]  daemon         | 0
[2025/05/08 11:44:07] [ info] ___________
[2025/05/08 11:44:07] [ info]  inputs:
[2025/05/08 11:44:07] [ info]      event_type
[2025/05/08 11:44:07] [ info] ___________
[2025/05/08 11:44:07] [ info]  filters:
[2025/05/08 11:44:07] [ info] ___________
[2025/05/08 11:44:07] [ info]  outputs:
[2025/05/08 11:44:07] [ info]      stdout.0
[2025/05/08 11:44:07] [ info]      splunk.1
[2025/05/08 11:44:07] [ info] ___________
[2025/05/08 11:44:07] [ info]  collectors:
[2025/05/08 11:44:07] [ info] [fluent bit] version=4.0.2, commit=5dd2b0b438, pid=21372
[2025/05/08 11:44:07] [debug] [engine] coroutine stack size: 36864 bytes (36.0K)
[2025/05/08 11:44:07] [ info] [storage] ver=1.1.6, type=memory+filesystem, sync=normal, checksum=off, max_chunks_up=128
[2025/05/08 11:44:07] [ info] [storage] backlog input plugin: storage_backlog.1
[2025/05/08 11:44:07] [ info] [simd    ] disabled
[2025/05/08 11:44:07] [ info] [cmetrics] version=1.0.0
[2025/05/08 11:44:07] [ info] [ctraces ] version=0.6.5
[2025/05/08 11:44:07] [ info] [input:event_type:event_type.0] initializing
[2025/05/08 11:44:07] [ info] [input:event_type:event_type.0] storage_strategy='filesystem' (memory + filesystem)
[2025/05/08 11:44:07] [debug] [input:event_type:event_type.0] [thread init] initialization OK
[2025/05/08 11:44:07] [ info] [input:event_type:event_type.0] thread instance initialized
[2025/05/08 11:44:07] [debug] [event_type:event_type.0] created event channels: read=38 write=39
[2025/05/08 11:44:07] [debug] [processor:metrics_selector:metrics_selector.0] action type EXCLUDE
[2025/05/08 11:44:07] [ info] [input:storage_backlog:storage_backlog.1] initializing
[2025/05/08 11:44:07] [ info] [input:storage_backlog:storage_backlog.1] storage_strategy='memory' (memory only)
[2025/05/08 11:44:07] [debug] [storage_backlog:storage_backlog.1] created event channels: read=42 write=43
[2025/05/08 11:44:07] [ info] [input:storage_backlog:storage_backlog.1] queue memory limit: 95.4M
[2025/05/08 11:44:07] [debug] [stdout:stdout.0] created event channels: read=44 write=45
[2025/05/08 11:44:07] [debug] [processor:metrics_selector:metrics_selector.0] action type EXCLUDE
[2025/05/08 11:44:07] [debug] [splunk:splunk.1] created event channels: read=54 write=55
[2025/05/08 11:44:07] [ info] [output:stdout:stdout.0] worker #0 started
[2025/05/08 11:44:07] [ info] [output:splunk:splunk.1] worker #0 started
[2025/05/08 11:44:07] [debug] [router] match rule event_type.0:stdout.0
[2025/05/08 11:44:07] [ info] [output:splunk:splunk.1] worker #1 started
[2025/05/08 11:44:07] [debug] [router] match rule event_type.0:splunk.1
[2025/05/08 11:44:07] [debug] [router] match rule storage_backlog.1:stdout.0
[2025/05/08 11:44:07] [debug] [router] match rule storage_backlog.1:splunk.1
[2025/05/08 11:44:07] [ info] [sp] stream processor started
[2025/05/08 11:44:09] [debug] [input:event_type:event_type.0] metrics, ret=0
[2025/05/08 11:44:09] [ info] [input:event_type:event_type.0] [OK] collector_time
[2025/05/08 11:44:09] [debug] [input coro] destroy coro_id=0
[2025/05/08 11:44:10] [debug] [task] created task=0x600002d70000 id=0 OK
[2025/05/08 11:44:10] [debug] [output:stdout:stdout.0] task_id=0 assigned to thread #0
[2025/05/08 11:44:10] [debug] [output:splunk:splunk.1] task_id=0 assigned to thread #0
[2025/05/08 11:44:10] [debug] [output:stdout:stdout.0] cmt decode msgpack returned : 1
[2025/05/08 11:44:10] [debug] [out flush] cb_destroy coro_id=0
[2025/05/08 11:44:10] [debug] [upstream] KA connection #93 to localhost:8090 is connected
[2025/05/08 11:44:10] [ warn] [output:splunk:splunk.1] Content length is 0, skipping flush
[2025/05/08 11:44:10] [debug] [upstream] KA connection #93 to localhost:8090 is now available
[2025/05/08 11:44:10] [debug] [out flush] cb_destroy coro_id=0
[2025/05/08 11:44:10] [debug] [task] destroy task=0x600002d70000 (task_id=0)
[2025/05/08 11:44:11] [debug] [input:event_type:event_type.0] metrics, ret=0
[2025/05/08 11:44:11] [ info] [input:event_type:event_type.0] [OK] collector_time
[2025/05/08 11:44:11] [debug] [input coro] destroy coro_id=1
^C[2025/05/08 11:44:12] [engine] caught signal (SIGINT)
[2025/05/08 11:44:12] [ info] [input] pausing storage_backlog.1
[2025/05/08 11:44:12] [ info] [output:stdout:stdout.0] thread worker #0 stopping...
[2025/05/08 11:44:12] [ info] [output:stdout:stdout.0] thread worker #0 stopped
[2025/05/08 11:44:12] [ info] [output:splunk:splunk.1] thread worker #0 stopping...
[2025/05/08 11:44:12] [ info] [output:splunk:splunk.1] thread worker #0 stopped
[2025/05/08 11:44:12] [ info] [output:splunk:splunk.1] thread worker #1 stopping...
[2025/05/08 11:44:12] [ info] [output:splunk:splunk.1] thread worker #1 stopped
[2025/05/08 11:44:12] [debug] [input:event_type:event_type.0] thread exit instance

Fluent Bit is licensed under Apache 2.0, by submitting this pull request I understand that this code will be released under the terms of that license.

@edsiper
Copy link
Member

edsiper commented May 8, 2025

as discussed internally, if a metrics processor can potentially ship a context without metric series, it should be dropped before being ingested into the pipeline.

@lecaros
Copy link
Contributor Author

lecaros commented May 9, 2025

moving this to draft while a better solution is designed and implemented

@lecaros lecaros marked this pull request as draft May 9, 2025 22:27
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants