Skip to content

Send JSON encoded payloads without re-marshalling them? #127

Open
@grandstairs

Description

@grandstairs

I would like to use this library as the basis for multiple logging libraries. However, the only way I was able to figure was to create an io.WriteCloser that wraps this library to do the .Post(tag, data) with the actual log line.

type FluentWriter struct {
	Fluent *fluent.Fluent
	Tag    string
}

func (w *FluentWriter) Close() error {
	return w.Fluent.Close()
}

func (f *FluentWriter) Write(p []byte) (n int, err error) {
	// this is wasteful since the logger just spent time to json.Encode this very payload.
	msg := map[string]interface{}{}
	json.Unmarshal(p, &msg)
	err = f.Fluent.Post(f.Tag, msg)
	return len(p), err
}

The issue is that Post() and EncodeAndPostData() both expect an non-encoded map/struct/interface which isn't the b []byte that an io.Writer provides.

f, err := fluent.New(fluent.Config{});
logSink := &logger.FluentWriter{Fluent: f, Tag: "ApplicationLogs"}
logger := zerolog.New(logSink).With().Timestamp().Logger()

Is there either 1) a better way to wrap this library so I can avoid this encode/decode/encode process or 2) is there a way to send a raw b []byte payload via msgpack without fluent-logger-golang needing to mess with it?

As-is, there really isn't much performance savings going with zerolog or zap over using something like slog or logrus because there will be a ton of extra memory allocations running the JSON encoder/decoder over the payloads.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions