How logging libraries help our teams support products

Efficient tech support is an important part of the True Engineering platform on which all our products are built. We’ve already spoken about our Production Ready criteria logging provision requirements that are also on this list. Basically, before a deployment, our developers have to ensure they can monitor the product and identify the issue sources quickly.

For all teams to have the same understanding of the logging requirements, we analyzed our products and developed a uniform standard for each of our products. It sets common requirements for the logs:

• Centralized storage in ELK or a similar stack
• Issue separation into business ones and unique cases
• Content:

1. Event time
2. Link to the user
3. App name
4. Trace
5. Server where the event was registered
6. Message body
7. Logging level and logger’s name

• Request execution time recording
• etc. — 13 points altogether.

Some of these requirements - composition of logs, data recording, and tracing data - do not change between products. We moved this "typical info" to the libraries to be reused in other projects.

Functional capabilities of our logging libraries

As a result, our teams obtained ready tools to record the following data:

1. All user’s actions inside the system.
2. All data exchange processes between microservices, and between microservices  + external systems.
3. All incoming requests and everything that’s sent outside as a response.
4. User request execution time on all stages.

A few weeks ago, we introduced another novelty — now our libraries can conceal private information in logs. Personal data, account numbers, unique tokens, payment amounts — all these are not necessary to monitor the system. All such data is displayed in logs marked with "Value is hidden by log policy". Soon, we will add expanded masking conditions — we will let the teams set the format, be it specific words, asterisks or something else they'll find more convenient to use.

We have already mapped out logging libraries for .Net and Java projects. Our immediate agenda is to develop a Python version. We’ll then embed the libraries into our microservice template.

Library application result

• Developers save time on launching common tools adhering to the Production Ready standard within products.
• Technical support has the means to analyze incidents or to localize a part of the system where the error occurred — and this is performed automatically.
• At the company level, we’ve become confident that every element of all the solutions works according to our standard.
• In the future, logging can be developed via the add-on method over the presently active functions, which is always easier than creating them from scratch.