Skip to content

Conversation

@DengY11
Copy link
Contributor

@DengY11 DengY11 commented Sep 23, 2025

This PR introduces a performance optimization for the rich logger by implementing a caching mechanism that avoids repeated processFieldValue calls for the same field names. The solution adds a unique logger ID system and caches field processors to eliminate reflection overhead.

Changes Made

  1. Logger ID System
    Added atomic counter loggerIDCounter to generate unique IDs for each logger instance
    Modified rich logger methods (WithCallerSkip, WithContext, WithDuration, WithFields) to generate new unique IDs using atomic.AddUint64(&loggerIDCounter, 1)

  2. Caching Mechanism
    Cache Key: loggerID (e.g., "123")
    Cache Value: a slice of field processor functions that handle specific field types without reflection

  3. Performance Optimization
    Before: Every field value processed through processFieldValue with reflection overhead
    After: Field processors cached per field name, processing happens only once per field type
    Benefits: Eliminates reflection overhead for repeated field names, significant performance improvement for loggers with many fields

  4. Writer Interface Changes
    This is a break change

  • Updated output function signature to accept loggerID uint64 parameter
  • Modified all writer methods to pass the logger ID through the call chain
  • Maintained backward compatibility with existing logging patterns

close #5067

I also create benchmark with 10 fields and 1000 fields

BenchmarkWithCache10Fields-12                     353655              3351 ns/op            4103 B/op         49 allocs/op
BenchmarkWithoutCache10Fields-12                  328093              3471 ns/op            4376 B/op         61 allocs/op
BenchmarkWithCache1000Fields-12                     5157            229914 ns/op          196973 B/op       2029 allocs/op
BenchmarkWithoutCache1000Fields-12                  4833            246770 ns/op          221026 B/op       3031 allocs/op

The benchmark shows higher performance and less memory use and allocs

@codecov
Copy link

codecov bot commented Sep 23, 2025

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

@DengY11
Copy link
Contributor Author

DengY11 commented Sep 23, 2025

i have tried many other ways, such as adding a member to the struct richlogger to avoid using the map cache, but it requires a much bigger change ...

@DengY11 DengY11 changed the title feat: Enhance Rich Logger Performance with Field Processing Caching feat(richlogger): Enhance Rich Logger Performance with Field Processing Caching Sep 23, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Optimize logging for ContextWithFields.

1 participant