[receiver/netflow] Netflow receiver implementation - PR 2 #36865
Conversation
This adds the implementation for netflow receiver Introduces: * The OtelLogsProducerWrapper * A parser function to convert proto to otel semantics * The listener implementation as well as the error handling function
|
|
||
| // Construct the actual log record based on the otel semantic conventions | ||
| // see https://opentelemetry.io/docs/specs/semconv/general/attributes/ | ||
| otelMessage := OtelNetworkMessage{ |
There was a problem hiding this comment.
Most of this looks like metadata on the log to me. Instead of creating this struct, could we create a plog.LogRecord and set these values as attributes using the corresponding semconv key for each value? We can probably use the network semantic conventions here. I'm having trouble linking to the Go registry copy of the docs, but you can use the go.opentelemetry.io/collector/semconv module to get constants for these, as seen here.
For the timestamps, we could just set those on the log record. I believe the body will be empty since the incoming messages don't seem to include any kind of equivalent arbitrary-text field.
There was a problem hiding this comment.
I dug into it some more and I see that the network semconv isn't necessarily a 1:1 fit here. I think we should do as much as we can, and we can create attributes for the pieces that don't fit so long as they're documented.
|
|
||
| case errors.Is(err, debug.PanicError): | ||
| var pErrMsg *debug.PanicErrorMessage | ||
| if errors.As(err, &pErrMsg) { |
There was a problem hiding this comment.
Could we remove a few of these (mostly the stacktrace), or if they're all useful, put a few of them at the "debug" log level? Ideally these should just inform users that a given message couldn't be parsed, and maybe give them some info to help figure out what in the message caused the issue.
Also, if possible, it would be nice to avoid to avoid using "panic" anywhere in here since from what I understand we're essentially handling malformed incoming messages, and I think "panic" makes this sound more alarming than it needs to be.
| } | ||
| nr.logger.Error("receiver panic", zap.Error(err)) |
There was a problem hiding this comment.
| } | |
| nr.logger.Error("receiver panic", zap.Error(err)) | |
| } else { | |
| nr.logger.Error("could not retrieve message parsing error from GoFlow2, this is an error in the Netflow receiver", zap.Error(err)) | |
| } |
I think we should try to make it a bit clearer that it's an issue with the underlying library or how we're reading from it. Realistically we should never hit this case. Also this should be in an else statement.
| # (Optional) One or more lines of additional information to render under the primary note. | ||
| # These lines will be padded with 2 spaces and then inserted directly into the document. | ||
| # Use pipe (|) for multiline entries. | ||
| subtext: Implement the UDP listener and the producer, as well as adding tests. |
There was a problem hiding this comment.
| subtext: Implement the UDP listener and the producer, as well as adding tests. | |
| subtext: The receiver now supports receiving NetFlow v5, NetFow v9, IPFIX, and sFlow v5 logs. |
Feel free to word this how you like, but I think we should just cover the user-facing changes here.
|
|
||
| var decodeFunc utils.DecoderFunc | ||
| var p utils.FlowPipe | ||
| switch nr.config.Scheme { |
There was a problem hiding this comment.
Could we explain in the readme what protocol support users can expect when configuring one of these three schemes? I had to dig into the GoFlow2 source to understand what is happening here.
Co-authored-by: Evan Bradley <[email protected]>
Co-authored-by: Evan Bradley <[email protected]>
Co-authored-by: Evan Bradley <[email protected]>
Co-authored-by: Evan Bradley <[email protected]>
Co-authored-by: Evan Bradley <[email protected]>
#34164 added the skeleton for the netflow receiver, this adds the implementation along with tests
Link to tracking Issue: #32732