Introduction to Distributed Tracing

Distributed Tracing

Distributed tracing is a technique to monitor and profile the applications, especially those built using microservice architecture. It is also known as distributed request tracing. Developers use distributed tracing to debug and optimize the code.

Distribute tracing provides a place where we can see that "what is happening with a specific request?" It is important because there are a variety of components that are involved in the microservices.

If we want to solve a problem or debug a problem, we need a centralized server. So the term distributed tracing comes into existence.

In this section, we will use Spring Cloud Sleuth with Zipkin. Spring Cloud Sleuth assigns a unique Id to each request that we have made. We can trace all the requests based on unique Ids across all the components.

Spring Cloud Sleuth

Spring Cloud Sleuth is a Spring Cloud library that provides the ability to track the progress of subsequent microservices by adding trace and span Ids on the appropriate HTTP request headers. The Sleuth library is based on the MDC (Mapped Diagnostic Context) concept, where we can easily extract values, put to context, and display them in the log.

Zipkin

Zipkin is an open-source, Java-based distributed tracing system. It has a management console that provides a mechanism for sending, receiving, storing, and visualizing traces details of the subsequent services.

With the help of the Zipkin server, we can put all the logs of all the components in the MQ (RabbitMQ). We send the logs to the Zipkin server where the logs consolidate. After doing this, we can monitor different requests. We can also find what is happening to a specific request?

Implementing distributed tracing Using Spring Cloud Sleuth

In this step, we will add Spring Cloud Sleuth for all the microservices. It adds a unique Id to all the requests. It is used to generate and attach the trace Id, span Id to the logs so that tools (Zipkin) can use these ids.

The Spring Cloud Sleuth token has the following components:

Application name: The name of the application that is defined in the properties file.
Trace Id: The Sleuth adds the Trace Id. It remains the same in all services for a given request.
Span Id: The Sleuth also adds the Span Id. It remains the same in a unit of work but different for different services for a given request.
Zipkin Export Flag: It indicates a boolean value. It can be either true or

The following figure shows the Spring Cloud Sleuth token.

Let's implement the Spring Cloud Sleuth in our projects.

Step 1: Select the project netflix-zuul-api-gateway-server.

Step 2: Open pom.xml and add the Sleuth dependency.

<dependency>
<groupId>org.springframework.cloud</groupId>
<artifactId>spring-cloud-starter-sleuth</artifactId>
</dependency>

Now we need to trace all the requests. If we want to trace all the requests, we would need to create ALWAYS_SAMPLE. We can create a Sampler by using a Bean.

Sampler

Distributed tracing may have a very high volume of data, so the sampling is important in distributed tracing. Spring Cloud Sleuth provides a Sampler strategy. With the help of Sampler, we can implement the sampling algorithm that provides control of the algorithm. By default, we get a procedure that continuously performs the tracing if a span (correlation: is an individual operation) is already active.

But the newly created spans are always marked as non-exportable. If all the applications are running with the Sampler, we can see trace (end-to-end latency graph that is composed of spans) in the log, not in any remote location. By default, Spring Cloud Sleuth sets all spans to non-exportable.

When we export span data to the Zipkin or Spring Cloud Stream, Spring Cloud Sleuth provides AlwaysSampler class that exports everything to the Zipkin. It also provides a PercentageBasedSampler class that samples a fixed fraction of span.

Remember: If you are using Spring 2.0.0 or above versions, use the following Sampler. We have used the same because we are using Spring version 2.2.1.

@Bean
public Sampler defaultSampler()
{
return Sampler.ALWAYS_SAMPLE;
}

Step 3: Open NetflixZuulApiGatewayServerApplication.java file and define a Bean.

NetflixZuulApiGatewayServerApplication.java

package com.javatpoint.microservices.netflixzuulapigatewayserver;
import org.springframework.boot.SpringApplication;
import org.springframework.boot.autoconfigure.SpringBootApplication;
import org.springframework.cloud.client.discovery.EnableDiscoveryClient;
import org.springframework.cloud.netflix.zuul.EnableZuulProxy;
import org.springframework.context.annotation.Bean;
import brave.sampler.Sampler;
@SpringBootApplication
@EnableDiscoveryClient
@EnableZuulProxy
public class NetflixZuulApiGatewayServerApplication 
{
public static void main(String[] args) 
{
SpringApplication.run(NetflixZuulApiGatewayServerApplication.class, args);
}
//creating a bean
@Bean
//creating a sampler called 
public Sampler defaultSampler()
{
return Sampler.ALWAYS_SAMPLE;
}
}

In the above code, we have added Spring Cloud Sleuth to the Zuul API Gateway server.

Now we have to define Bean in currency-exchange-service and currency-conversion-service also.

Step 4: Open pom.xml of currency-exchange-service and add the Sleuth dependency.

<dependency>
<groupId>org.springframework.cloud</groupId>
<artifactId>spring-cloud-starter-sleuth</artifactId>
</dependency>

Step 5: Open CurrencyExchangeServiceApplication.java file and define a Bean.

CurrencyExchangeServiceApplication.java

package com.javatpoint.microservices.currencyexchangeservice;
import org.springframework.boot.SpringApplication;
import org.springframework.boot.autoconfigure.SpringBootApplication;
import org.springframework.cloud.client.discovery.EnableDiscoveryClient;
import org.springframework.context.annotation.Bean;
import brave.sampler.Sampler;
@SpringBootApplication
@EnableDiscoveryClient
public class CurrencyExchangeServiceApplication 
{
public static void main(String[] args) 
{
SpringApplication.run(CurrencyExchangeServiceApplication.class, args);
}
@Bean
//creating a sampler called always sampler
public Sampler defaultSampler()
{
return Sampler.ALWAYS_SAMPLE;
}
}

Step 6: Similarly, Open the pom.xml of currency-conversion-service and add the Sleuth dependency.

<dependency>
<groupId>org.springframework.cloud</groupId>
<artifactId>spring-cloud-starter-sleuth</artifactId>
</dependency>

Step 7: Open CurrencyConversionServiceApplication.java file and define a Bean.

CurrencyConversionServiceApplication.java

package com.javatpoint.microservices.currencyconversionservice;
import org.springframework.boot.SpringApplication;
import org.springframework.boot.autoconfigure.SpringBootApplication;
import org.springframework.cloud.client.discovery.EnableDiscoveryClient;
import org.springframework.cloud.openfeign.EnableFeignClients;
import org.springframework.context.annotation.Bean;
import brave.sampler.Sampler;
@SpringBootApplication
@EnableFeignClients("com.javatpoint.microservices.currencyconversionservice")
@EnableDiscoveryClient
public class CurrencyConversionServiceApplication 
{
public static void main(String[] args) 
{
SpringApplication.run(CurrencyConversionServiceApplication.class, args);
}
@Bean
//creating a sampler called always sampler
public Sampler defaultSampler()
{
return Sampler.ALWAYS_SAMPLE;
}
}

Now we have three applications that are connect to Spring Cloud Sleuth.

Step 8: Launch the applications in the following order:

netflix-eureka-naming-server

Open the browser and invoke the URL http://localhost:8761. It returns the Eureka interface, as shown below.

currency-exchange-service (on port 8000)

currency-conversion-service

netflix-zuul-api-gateway-server

Remember: After launching each service, refresh the Eureka server.

It shows all the instances currently registered with the Eureka server.

Step 9: Open CurrencyExchangeController.java file and add a logger into it.

CurrencyExchangeController.java

package com.javatpoint.microservices.currencyexchangeservice;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.boot.autoconfigure.SpringBootApplication;
import org.springframework.core.env.Environment;
import org.springframework.web.bind.annotation.GetMapping;
import org.springframework.web.bind.annotation.PathVariable;
import org.springframework.web.bind.annotation.RestController;
@SpringBootApplication
@RestController 
public class CurrencyExchangeController 
{
private Logger logger=LoggerFactory.getLogger(this.getClass());
@Autowired
private Environment environment;
@Autowired
private ExchangeValueRepository repository;
@GetMapping("/currency-exchange/from/{from}/to/{to}")		//where {from} and {to} are path variable
public ExchangeValue retrieveExchangeValue(@PathVariable String from, @PathVariable String to)   //from map to USD and to map to INR
{		
ExchangeValue exchangeValue = repository.findByFromAndTo(from, to);
//setting the port
exchangeValue.setPort(Integer.parseInt(environment.getProperty("local.server.port")));
logger.info("{}", exchangeValue);
return exchangeValue;
}
}

Similarly, we will add logger into CurrencyConversionContoller.

Step 10: Open CurrencyConversionContoller.java file and add a logger into it.

CurrencyConversionContoller.java

package com.javatpoint.microservices.currencyconversionservice;
import java.math.BigDecimal;
import java.util.HashMap;
import java.util.Map;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.http.ResponseEntity;
import org.springframework.web.bind.annotation.GetMapping;
import org.springframework.web.bind.annotation.PathVariable;
import org.springframework.web.bind.annotation.RestController;
import org.springframework.web.client.RestTemplate;
@RestController
public class CurrencyConversionController 
{
private Logger logger=LoggerFactory.getLogger(this.getClass());
@Autowired
private CurrencyExchangeServiceProxy proxy;
@GetMapping("/currency-converter/from/{from}/to/{to}/quantity/{quantity}") //where {from} and {to} represents the column 
//returns a bean back
public CurrencyConversionBean convertCurrency(@PathVariable String from, @PathVariable String to, @PathVariable BigDecimal quantity)
{
//setting variables to currency exchange service
Map<String, String> uriVariables=new HashMap<>();
uriVariables.put("from", from);
uriVariables.put("to", to);
//calling the currency exchange service
ResponseEntity<CurrencyConversionBean> responseEntity=new RestTemplate().getForEntity("http://localhost:8000/currency-exchange/from/{from}/to/{to}", CurrencyConversionBean.class, uriVariables);
CurrencyConversionBean response=responseEntity.getBody();
//creating a new response bean and getting the response back and taking it into Bean
return new CurrencyConversionBean(response.getId(), from, to, response.getConversionMultiple(), quantity, quantity.multiply(response.getConversionMultiple()), response.getPort());
}
//mapping for currency-converter-feign service
@GetMapping("/currency-converter-feign/from/{from}/to/{to}/quantity/{quantity}") //where {from} and {to} represents the column 
//returns a bean 
public CurrencyConversionBean convertCurrencyFeign(@PathVariable String from, @PathVariable String to, @PathVariable BigDecimal quantity)
{
CurrencyConversionBean response=proxy.retrieveExchangeValue(from, to);
logger.info("{}", response);
//creating a new response bean
//getting the response back and taking it into Bean
return new CurrencyConversionBean(response.getId(), from, to, response.getConversionMultiple(), quantity, quantity.multiply(response.getConversionMultiple()), response.getPort());
}
}

Step 12: Execute the request http://localhost:8100/currency-converter-feign/from/EUR/to/INR/quantity/100. It returns the following response, as shown below.

Let's see the log of currency-conversion-service in the console. The currency-conversion-service shows the following log:

We can also see the log of currency-exchange-service. The currency-exchange-service shows the following log:

Similarly, we can see the log for netflix-zuul-api-gateway-server.

Let's have a close look at the above three logs for different services. We find that all three services have the same trace Id (533f8d3966d8f4e7).

Spring Cloud Sleuth assigns a trace Id to the request. We can use this Id to trace the requests across multiple components. But there is a problem that this log is distributed in multiple places. We use Zipkin to remove this problem. With the help of Zipkin, we can centralize the logs in one place.

Distributed Tracing with Zipkin

We have installed Spring Cloud Sleuth dependency in currency-conversion-service, currency-exchange-service, and netflix-zuul-api-gateway-server. We have seen that a unique Id was being assigned to each request. We use these Id to trace the request across the logs of these multiple services.

However, we face a challenge in tracing. If we want to trace a request, we have to check the log of the individual application. The solution to this problem is called centralized logs.

We need to centralize all the logs from all the microservices. We can search through Id assigned by Spring Cloud Sleuth. At the centralized place, we will able to search and find out what is happening to a specific request.

There are the following solutions for centralize logging:

ELK Stack (Elastic Search)
Kibana
Zipkin

In this distributed tracing, we will use Zipkin distribute tracing server. It gives us a consolidated view of all the microservices. We get all the logs messages form the individual microservices. The Zipkin server collects the log messages. All the microservices puts the log messages on the queue called RabbitMQ, and the Zipkin picks these log messages from the RabbitMQ. The Zipkin tracing server is connected with the database.

In our case, we use the in-memory database. We will pull log messages from the database. In the next step, we will install RabbitMQ.

Next TopicInstalling RabbitMQ Server

← prev next →