Prepare for structured logging

[webJose]

I like this logger and the fact that it does not provide writers, which gives me the liberty to create or choose writers.

Specifically, I would like to use this as I would use serilog in C#. Then I can create a writer that posts the data to a centralized location, like New Relic or Seq.

What is needed? I guess the concept of log properties, where each log entry carries a list of properties. Then a writer would have access to the collection of properties pertinent to the event. The sources of properties would be at least:

• The replaceable arguments in the logged message. For example, if I were to do log.notice('%s returned status code %i.', '/api/login', res.status);, then the event should carry the values of the route and the status codes as properties. What would be the property names? I guess the message format would require an upgrade: log.notice('{route:%s} returned status code {statusCode:%i}.', '/api/login', res.status);. Now the text I put inside { } would become the property name.
• Anything else, like an express middleware that collects data from the request or response object, thinking in terms of NodeJS being a web server, that is. For NodeJS projects that are not web servers, then maybe another way to add properties to events, or mechanisms that add properties to events automatically.

Anyway, if you want to know more, maybe check out Serilog's source code and documentation.

BTW, as a potential writer of log writers, I find the lack of documentation on how to do it a major problem. Will appreciate it if you could set up documentation for this.

[medikoo]

@webJose Great thanks for raising this. Indeed documentation leaves a lot to be desired. I'll try to prepare something solid in next days.

In a meantime, I think it's pretty easy to start with the following.

Every log written with this interface is emitted as an event, which can observed as follows:

const logEmitter = require('log/lib/emitter');

logEmitter.on('log', event => {
   const logger = event.logger; // logger used to write the log (level, namespace, and wether it's enabled (its logs should be printed) can be read from its properties))
   const messageTokens = event.messageTokens // raw log invocation arguments
});

There's also a dedicated log writer interface provided, which predefines some helper methods and allows to ensure that no two writers are registered at the same time. I'll try to find a moment in next days to document that

[webJose]

@medikoo thank you very much. I appreciate it.

In the meantime, could I pick your mind about messageTokens? It sounds like this is what one needs for structured logging. Just one step behind, I guess since they are the values only, right? Ideally, it should be dictionary of values with the names pre-set.

[medikoo]

@webJose messageTokens are simply an arguments as they were passed to log function:

const log = require('log');
const emitter = require('log/lib/emitter');

emitter.on('log', ({ messageTokens }) => {
  console.log("Log function arguments", ...messageTokens);
});


log.debug('%s returned status code %i.', '/api/login', 200); // Prints: "Log function arguments: "%s returned status code %i.", "/api/login", 200")

So how to process and handle those args is fully up to log writer

[webJose]

I have been thinking about this and I believe format specifiers are a no no for structured logging. After all, they specify formatting, as in a display preference. Structured logging does not concern about this. I guess format specifiers are only meaningful for text serializers of the log entry, like log-node that prints to the console.

If I were to create a log writer under the concept of structured logging that foregoes the use of formatters, what would be your saying about that?

[medikoo]

If I were to create a log writer under the concept of structured logging that foregoes the use of formatters, what would be your saying about that?

@webJose In my eyes, it's totally fine. This library doesn't enforce any particular handling. There are pre-prepared helpers (to e.g. format log messages with formatters) but it doesn't mean that log writers are under contract to rely on that.

Technically even if in your stack there are some low-level utilities (that you do not control), which rely on this library and use sprintf formatting (in e.g. debug messages), you may eventually handle (if you want to) log messages coming from those via different writer (as e.g. log-node).

Follow with whatever fits your use case best.