Logging Best Practices

Created on: September 12, 2023Modified on: September 16, 2023

Best practices = my opinion.

Use log levels correctly

fatal: use this when you want to be waken up at 3am.
error: for actual failures that should be investigated and resolved. Ideally these generate tickets.
warn: when we need to keep an eye on things but are not severe enough to be errors.
info: business related information
debug: for developers
trace: when you really need to be on top of what's going on step by step

Levels in production

Enable info and above all the time.

Ideally you have a mechanism in place that allows to activate debug and trace for a given file. When the error is repeatable and only happens in production you at least are able to turn debug on and gather more information.

What you should aim for is to capture debug entries in memory. When an error happens you write the error and all the debug messages.

Don't include PII

My knowledge here only goes as far as saying that the application should be responsible to not log any PII.

Structured Logging

Structured logging it's easier to read and to parse with tools. The tradeoff is more CPU usage and bigger log files.

Text

// 🔴 normal logging
2023-09-16T13:07:43.268Z Marcelos-MacBook-Air.local INFO [core:4] - Hello

// 🟢 structured logging
{"timestamp": "2023-09-16T13:07:43.268Z","host": "Marcelos-MacBook-Air.local","level": "INFO","file": "core.clj","line": 4,"msg": "Hello"}

Don't interpolate Strings

I'm not a fan of String interpolation with logs because they make it harder to parse. I prefer to use a structured approach:

Text

// 🔴 string interpolation
// (info (format "User %s is uploading video %s." (java.util.UUID/randomUUID) (java.util.UUID/randomUUID)))
User 2cf354df-f2bc-478b-9069-175a7d5895ee starting to upload video 2d978881-b095-4e0c-8b42-7a5059b5aab4.


// 🟢 clear separation between message and parameters
// (info "User is uploading video." {:user-id (java.util.UUID/randomUUID) :video-id (java.util.UUID/randomUUID)})
{"msg": "User starting to upload video", "msg-args": {"user-id": "2b4cc7eb-a23e-4866-b05f-ee25232cff42", "video-id": "0e5e0eb5-5372-4fa4-a64e-9ed8d8eb4d7d"}}

Use correlation IDs

TODO: write about correlation IDS https://www.w3.org/TR/trace-context/

Use different logging for development

During the development the needs are quite different than in production. I prefer to strip the information to a minimum and don't use structured logging. I also prefer to enable colors.

Default logger

Default timbre logger.

Dev logger

With colors, without the machine information, and with file extension. Dev timbre logger.

What information to include

Include:

Time
Level
Application
Server
Correlation ID
Message
Message arguments
File
Line

Might be useful to include

Git version
PID
Thread