When breaking down a monolith into microservices, we usually spend a lot of time thinking about business boundaries or the partitioning of our application logic, but we forget about the logs. From my own experience as a developer and software architect, I can say that developers do not usually pay much attention to logging. On the other hand, operation teams, which are responsible for application maintenance, are mainly dependent on logs. Regardless of one's area of expertise, it is indisputable that logging is something that all applications have to do, whether they have monolithic or microservices architecture.However, microservices force adding a whole new dimension to design and arrangement of application logs. There are many small, independent, horizontally scaled, intercommunicating services that are running on multiple machines. Requests are often processed by multiple services. We have to correlate these requests and store all the logs in...
One of the most important best practices for dealing with logging is to trace all the incoming requests and outgoing responses. Maybe it seems obvious to you, but I have seen a couple of applications that did not comply with that requirement. If you meet this demand, there is one consequence that occurs with microservices-based architecture. The overall number of logs in your system increases compared to monolithic applications, where there is no messaging. This, in turn, requires us to pay even more attention to logging than before. We should do our best to generate as little information as possible, even though this information can tell us much about the situation. How do we achieve this? First of all, it is good to have the same log message format across all the microservices. For example, let's consider how to print variables in the application logs. I suggest you use the JSON notation in view of the fact that, usually, messages exchanged between...
Spring Boot uses Apache Commons Logging for internal logging, but if you are including dependencies with starters, Logback will be used by default in your application. It doesn't inhibit the possibility of using other logging frameworks in any way. The default configurations are also provided for Java Util Logging, Log4J2, and SLF4J. Logging settings may be configured in the application.yml file with logging.* properties. The default log output contains the date and time in milliseconds, log level, process ID, thread name, the full name of the class that has emitted the entry, and the message. It may be overridden by using the logging.pattern.console and logging.pattern.file properties respectively for the console and file appenders.
By default, Spring Boot only logs on to a console. In order to allow the writing of log files in addition to a console output, you should set a logging.file or logging.path property. If you specify the logging.file property, the logs...
ELK is the acronym for three open source tools—Elasticsearch, Logstash, and Kibana. It is also called Elastic Stack. The heart of this system is Elasticsearch, a search engine based on another open source project written in Java, Apache Lucene. This library is especially suitable for applications that require full-text searches in cross-platform environments. The main reason for the popularity of Elasticsearch is its performance. Of course, it has some other advantages, such as scalability, flexibility, and easy integration by providing a RESTful, JSON-based API for searching stored data. It has a large community and many use cases, but the most interesting one for us is its ability to store and search logs generated by applications.Logging is the main reason for including Logstash in ELK Stack. This open source data-processing pipeline allows us to collect, process, and input data into Elasticsearch.
Logstash supports many inputs that pull events from external...
Spring Cloud Sleuth is a rather small, simple project, which nevertheless provides some useful features for logging and tracing. If you refer to the example discussed in theUsing LogstashTCPAppender section, you can easily see that there is no possibility to filter all the logs related to single request. In a microservices-based environment, it is also very important to correlate messages exchanged by the applications when handling requests that are coming into the system. This is the main motivation in creating the Spring Cloud Sleuth project.
If Spring Cloud Sleuth is enabled for the application, it adds some HTTP headers to the requests, which allows you to link requests with the responses and the messages exchanged by independent applications, for example, through RESTful API. It defines two basic units of work—span and trace. Each of these is identified by a unique 64 bit ID. The value of the trace ID is equal to the initial value of the span ID. Span refers to a...
Logging and tracing are usually not very important during development, but these are the key features that are used in the maintenance of the system. In this chapter, I have placed emphasis on the fields of development and operations. I have shown you how to integrate a Spring Boot microservice application with Logstash and Zipkin in several ways. I have also shown you some examples to illustrate how to enable Spring Cloud Sleuth features for an application in order to make it easier to monitor calls between many microservices. After reading this chapter, you should also be able to effectively use Kibana as a log aggregator tool and Zipkin as a tracing tool for discovering bottlenecks in communication inside your system.
Spring Cloud Sleuth, in conjunction with Elastic Stack and Zipkin, seems to be a very powerful ecosystem, which removes any doubts you might have about problems with monitoring systems that consist of many independent microservices.
© 2018 Packt Publishing Limited All Rights ReservedAuthor (1)
