Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[receiver/valkey]: Init new component #37005

Open
wants to merge 21 commits into
base: main
Choose a base branch
from

Conversation

rogercoll
Copy link
Contributor

@rogercoll rogercoll commented Jan 2, 2025

Description

Base receiver code for the new Valkey receiver (metadata, client interface, unit and integration tests). Just the valkey.client.connection.count metric will be available after this PR. Based on https://github.com/open-telemetry/semantic-conventions/blob/main/docs/database/database-metrics.md#metric-dbclientconnectioncount

Configuration options: Valkey endpoint + TLS options

Link to tracking issue

Fixes #33787

Testing

  • Unit and integration tests using testdata + contrib golden package.

Documentation

@github-actions github-actions bot added the cmd/otelcontribcol otelcontribcol command label Jan 2, 2025
@rogercoll rogercoll force-pushed the init_valkey_receiver branch from b991b59 to a82e488 Compare January 2, 2025 13:52
@rogercoll rogercoll force-pushed the init_valkey_receiver branch from a82e488 to 1644903 Compare January 2, 2025 14:03
@rogercoll rogercoll marked this pull request as ready for review January 2, 2025 14:30
@rogercoll rogercoll requested a review from a team as a code owner January 2, 2025 14:30
@rogercoll rogercoll requested a review from fatsheep9146 January 2, 2025 14:30
@rogercoll
Copy link
Contributor Author

cc @dmitryax

receiver/valkeyreceiver/client.go Outdated Show resolved Hide resolved
continue
}
pair := strings.Split(line, ":")
if len(pair) == 2 { // defensive, should always == 2
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If this expectation is broken, should we log a warning?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Agree, I went one step further and return an error instead. If the line cannot be split with the ":" delimiter, means we are parsing an unknown input. fc0f24b
Let me know what you think

if opts.TLSConfig, err = vs.cfg.TLS.LoadTLSConfig(context.Background()); err != nil {
return pmetric.Metrics{}, err
}
vs.client, err = newValkeyClient(opts)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If we're using the if ... err != nil pattern with the TLS load, we should use it again here.

Suggested change
vs.client, err = newValkeyClient(opts)
if vs.client, err = newValkeyClient(opts); err != nil {

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done fc0f24b

func (vs *valkeyScraper) recordConnectionMetrics(now pcommon.Timestamp, info map[string]string) {
recordConnection := func(infoKey string, attribute metadata.AttributeValkeyClientConnectionState) {
if val, ok := info[infoKey]; ok {
if i, err := strconv.ParseInt(val, 10, 64); err == nil {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Errors should be logged.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Makes sense fc0f24b

if len(line) == 0 || strings.HasPrefix(line, "#") {
continue
}
pair := strings.Split(line, ":")
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should use SplitN, as values (example from unit test listener0:name=tcp,bind=*,bind=-::*,port=6379) can have : characters.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not sure about this one, it looks like the keys available in the info output do not contain : characters. For the shared example, listener0 is the key, while name=tcp,bind=*,bind=-::*,port=6379 is value. This parsing function is only interested in capturing the keys and any value string (can contain :).

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Changed to Cut to only split on the first delimiter instance: 1a76c23

Copy link
Contributor

@dehaansa dehaansa left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Grouping keys by section is a minor concern, not important for this initial version, inability to recover if a client is closed is a larger concern that should be addressed.

}

result, err := vs.client.retrieveInfo(ctx)
if err != nil {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Are there errors that should be considered differently - IE errors that show the client is no longer valid and we need to construct a new one?

attrs := make(map[string]string)
lines := strings.Split(data, attrDelimiter)
for _, line := range lines {
if len(line) == 0 || strings.HasPrefix(line, "#") {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The lines with a # prefix appear to start a named section of key/value pairs. Is there a promise in the API that there will be no name conflicts across categories? If no, we may need to consider grouping the parsed pairs by category, not in one shared map.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

New component: Valkey receiver
3 participants