Add ZipKin to your Spring application

If your application contains multiple services interacting with each other the need for distributed tracing is increasing. You have a call towards one application that also calls another application, in certain cases the application to be accessed next might be a different one. You need to trace the request end to end and identify what happened to the call.
Zipkin is a Distributed Tracing system. Essentially by using Zipkin on our system we can track how a call spans across various Microservices.

ZipKin comes with Various database options. In our case we shall use Elasticsearch.

We will setup out ZipKin server using Compose

Let’s start with our Compose file:

services:
  redis:
    image: redis
    ports:
      - 6379:6379
  elasticsearch:
    image: elasticsearch:7.17.7
    ports:
      - 9200:9200
      - 9300:9300
    healthcheck:
      test: ["CMD", "curl", "-f", "http://localhost:9200/_cat/health"]
      interval: 20s
      timeout: 10s
      retries: 5
      start_period: 5s
    environment:
      - discovery.type=single-node
    restart: always
  zipkin:
    image: openzipkin/zipkin-slim
    ports:
      - 9411:9411
    environment:
      - STORAGE_TYPE=elasticsearch
      - ES_HOSTS=http://elasticsearch:9200
      - JAVA_OPTS=-Xms1G -Xmx1G -XX:+ExitOnOutOfMemoryError
    depends_on:
      - elasticsearch
    restart: always

We can run the above using

docker compose up

You can find more on Compose on the Developers Essential Guide to Docker Compose.

Let’s build our applications, our applications will be servlet based

We shall use a service for locations, this service essentially will persist locations on a Redis database using the GeoHash data structure.
You can find the service implementation on a previous blog.

We would like to add some extra dependencies so that Zipkin integration is possible.

...
    <dependencyManagement>
        <dependencies>
            <dependency>
                <groupId>org.springframework.cloud</groupId>
                <artifactId>spring-cloud-dependencies</artifactId>
                <version>2021.0.5</version>
                <type>pom</type>
                <scope>import</scope>
            </dependency>
        </dependencies>
    </dependencyManagement>
...
        <dependency>
            <groupId>org.springframework.cloud</groupId>
            <artifactId>spring-cloud-starter-sleuth</artifactId>
        </dependency>
        <dependency>
            <groupId>org.springframework.cloud</groupId>
            <artifactId>spring-cloud-sleuth-zipkin</artifactId>
        </dependency>
...

Spring Sleuth provides distributed tracing to our Spring application. By using spring Sleuth tracing data are generated. In case of a servlet filter or a rest template tracing data will also be generated. Provided the Zipkin binary is included the data generated will be dispatched to the Zipkin collector specified using
spring.zipkin.baseUrl.

Let’s also make our entry point application. This application will execute requests towards the location service we implemented previously.
The dependencies will be the following.

<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0"
         xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
         xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
    <groupId>org.example</groupId>
    <version>1.0-SNAPSHOT</version>
    <modelVersion>4.0.0</modelVersion>

    <artifactId>european-venue</artifactId>

    <properties>
        <maven.compiler.source>11</maven.compiler.source>
        <maven.compiler.target>11</maven.compiler.target>
        <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
    </properties>

    <dependencyManagement>
        <dependencies>
            <dependency>
                <groupId>org.springframework.cloud</groupId>
                <artifactId>spring-cloud-dependencies</artifactId>
                <version>2021.0.5</version>
                <type>pom</type>
                <scope>import</scope>
            </dependency>
        </dependencies>
    </dependencyManagement>

    <dependencies>
        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-web</artifactId>
            <version>2.7.5</version>
        </dependency>
        <dependency>
            <groupId>io.projectreactor</groupId>
            <artifactId>reactor-core</artifactId>
            <version>3.5.0</version>
        </dependency>
        <dependency>
            <groupId>org.projectlombok</groupId>
            <artifactId>lombok</artifactId>
            <version>1.18.24</version>
            <scope>provided</scope>
        </dependency>
        <dependency>
            <groupId>org.springframework.cloud</groupId>
            <artifactId>spring-cloud-starter-sleuth</artifactId>
        </dependency>
        <dependency>
            <groupId>org.springframework.cloud</groupId>
            <artifactId>spring-cloud-sleuth-zipkin</artifactId>
        </dependency>
    </dependencies>

</project>

We shall create a service interacting with the location service.

The location model shall be the same:

package org.landing;

import lombok.Data;

@Data
public class Location {

    private String name;
    private Double lat;
    private Double lng;

}

And the service:

package org.landing;

import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.beans.factory.annotation.Value;
import org.springframework.stereotype.Service;
import org.springframework.web.client.RestTemplate;


@Service
public class LocationService {

    @Autowired
    private RestTemplate restTemplate;

    @Value("${location.endpoint}")
    private String locationEndpoint;

    public void checkIn(Location location) {
        restTemplate.postForLocation(locationEndpoint, location);
    }

}

Following we will add the controller:

package org.landing;

import org.springframework.http.ResponseEntity;
import org.springframework.web.bind.annotation.PostMapping;
import org.springframework.web.bind.annotation.RequestBody;
import org.springframework.web.bind.annotation.RestController;

import lombok.AllArgsConstructor;

@RestController
@AllArgsConstructor
public class CheckinController {

    private final LocationService locationService;

    @PostMapping("/checkIn")
    public ResponseEntity<String> checkIn(@RequestBody Location location) {
        locationService.checkIn(location);
        return ResponseEntity.ok("Success");
    }

}

We need also RestTemplate to be configured:

package org.landing;

import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
import org.springframework.web.client.RestTemplate;

@Configuration
public class RestTemplateConfiguration {

    @Bean
    public RestTemplate restTemplate() {
        return new RestTemplate();
    }

}

Last but not least the main method:

package org.location;

import org.springframework.boot.SpringApplication;
import org.springframework.boot.autoconfigure.SpringBootApplication;

@SpringBootApplication
public class LocationApplication {

    public static void main(String[] args) {
        SpringApplication.run(LocationApplication.class);
    }

}

Now that everything is up and running let’s put this into action

curl --location --request POST 'localhost:8081/checkIn/' \
--header 'Content-Type: application/json' \
--data-raw '{
	"name":"Liverpool Street",
	"lat": 51.517336,
	"lng": -0.082966
}'
> Success

Let’s navigate now to the Zipkin Dashboard and search traces for the european-venue.
By expanding the calls we shall end up to a call like this.

Essentially we have an end to end tracing for our applications. We can see the entry point which is the european-venue checkin endpoint.
Also because the location service is called we have data points for the call received. However we also see data points for the call towards the redis database.
Essentially by adding sleuth to our application, beans that are ingress and egress points are being wrapped so that trace data can be reported.

You can find the code on GitHub.

Use Redis GeoHash with Spring boot

One very handy Data Structure when it comes to Redis is the GeoHash Data structure. Essentially it is a sorted set that generates a score based on the longitude and latitude.

We will spin up a Redis database using Compose

services:
  redis:
    image: redis
    ports:
      - 6379:6379

Can be run like this

docker compose up

You can find more on Compose on the Developers Essential Guide to Docker Compose.

Let’s add our dependencies

<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0"
         xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
         xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
    <groupId>org.example</groupId>
    <version>1.0-SNAPSHOT</version>
    <modelVersion>4.0.0</modelVersion>

    <artifactId>location-service</artifactId>

    <properties>
        <maven.compiler.source>11</maven.compiler.source>
        <maven.compiler.target>11</maven.compiler.target>
        <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
    </properties>

    <dependencies>
        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-web</artifactId>
            <version>2.7.5</version>
        </dependency>
        <dependency>
            <groupId>org.projectlombok</groupId>
            <artifactId>lombok</artifactId>
            <version>1.18.24</version>
            <scope>provided</scope>
        </dependency>
        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-data-redis</artifactId>
            <version>2.7.5</version>
        </dependency>
    </dependencies>
</project>

We shall start with our Configuration. For convenience on injecting we shall create a GeoOperations<String,String> bean.

package org.location;

import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
import org.springframework.data.redis.core.GeoOperations;
import org.springframework.data.redis.core.RedisTemplate;

@Configuration
public class RedisConfiguration {

    @Bean
    public GeoOperations<String,String> geoOperations(RedisTemplate<String,String> template) {
        return template.opsForGeo();
    }

}

Our model would be this one

package org.location;

import lombok.Data;

@Data
public class Location {

    private String name;
    private Double lat;
    private Double lng;

}

This simple service will persist venue locations and also fetch venues nearby of a location.

package org.location;

import java.util.List;
import java.util.stream.Collectors;

import org.springframework.data.geo.Circle;
import org.springframework.data.geo.Distance;
import org.springframework.data.geo.GeoResult;
import org.springframework.data.geo.GeoResults;
import org.springframework.data.geo.Metrics;
import org.springframework.data.geo.Point;
import org.springframework.data.redis.connection.RedisGeoCommands;
import org.springframework.data.redis.core.GeoOperations;
import org.springframework.data.redis.domain.geo.GeoLocation;
import org.springframework.stereotype.Service;

@Service
public class GeoService {

    public static final String VENUS_VISITED = "venues_visited";
    private final GeoOperations<String, String> geoOperations;

    public GeoService(GeoOperations<String, String> geoOperations) {
        this.geoOperations = geoOperations;
    }

    public void add(Location location) {
        Point point = new Point(location.getLng(), location.getLat());
        geoOperations.add(VENUS_VISITED, point, location.getName());

    }

    public List<String> nearByVenues(Double lng, Double lat, Double kmDistance) {
        Circle circle = new Circle(new Point(lng, lat), new Distance(kmDistance, Metrics.KILOMETERS));
        GeoResults<RedisGeoCommands.GeoLocation<String>> res = geoOperations.radius(VENUS_VISITED, circle);
        return res.getContent().stream()
                  .map(GeoResult::getContent)
                  .map(GeoLocation::getName)
                  .collect(Collectors.toList());
    }

}

We shall also add a controller

package org.location;

import java.util.List;

import org.springframework.http.ResponseEntity;
import org.springframework.web.bind.annotation.GetMapping;
import org.springframework.web.bind.annotation.PostMapping;
import org.springframework.web.bind.annotation.RequestBody;
import org.springframework.web.bind.annotation.RestController;

@RestController
public class LocationController {

    private final GeoService geoService;

    public LocationController(GeoService geoService) {
        this.geoService = geoService;
    }

    @PostMapping("/location")
    public ResponseEntity<String> addLocation(@RequestBody Location location) {
        geoService.add(location);
        return ResponseEntity.ok("Success");
    }

    @GetMapping("/location/nearby")
    public ResponseEntity<List<String>> locations(Double lng, Double lat, Double km) {
        List<String> locations = geoService.nearByVenues(lng, lat, km);
        return ResponseEntity.ok(locations);
    }

}

Then let’s add an element.

curl --location --request POST 'localhost:8080/location' \
--header 'Content-Type: application/json' \
--data-raw '{
	"lng": 51.5187516,
	"lat":-0.0814374,
	"name": "liverpool-street"
}'

Let’s retrieve the element of the api

curl --location --request GET 'localhost:8080/location/nearby?lng=51.4595573&lat=0.24949&km=100'
> [
    "liverpool-street"
]

And also let’s check redis

ZRANGE venues_visited 0 -1 WITHSCORES
1) "liverpool-street"
2) "2770072452773375"

We did it, pretty convenient for our day to day distance use cases.

Integrate Redis to a Node.js project

On this article we are going to add caching to our Node.js application using Redis.

We will install the recommended client for node.js as mentioned on the official Redis page.

npm install redis --save

Next we shall create our client connection

var redis = require('redis')

var hostname = '127.0.0.1'
var port = '6379'
var password = 'yourpassword'

var client = redis.createClient(port,hostname,{no_ready_check: true})

client.auth(password)

client.on('connect', function() {
        console.log('Client was connected')
})

The password provided on auth will be stashed and used on every connect.

Let’s see some basic actions

SMEMBERS actions


client.sadd('aset', 2)
client.sadd('aset', 1)
client.sadd('aset', 5)

client.smembers('aset',function(err,reply) {
    console.log(reply)
})

Get and Set actions

client.set('akey', "This is the value")

client.get('akey',function(err,reply) {
    console.log(reply)
})

Hash value actions

client.hset('hashone', 'fieldone', 'some value');
client.hset('hashone', 'fieldtwo', 'another value');

var hash = 'hashone'

client.hkeys(hash, function (err, fields) {

    fields.forEach(function(field,i) {

        console.log('The field is '+field)

        client.hget(hash,field,function (err, value) {
            console.log('The content is '+value)
        })
    })

});

List actions

client.rpush(['mylist', 'firstItem', 'secondItem'], function(err, listsize) {
    console.log(listsize)
});

client.lrange('mylist',0,-1,function(err,values) {
    console.log(values)
})

client.lpop('mylist',function(err,value) {
    console.log('Got '+value)
})

 

Conclusion

The Redis client for Node.js is pretty straightforward and easy to get started.

Keep in mind that one connection is adequate. Redis is single threaded therefore there is no need in opening multiple connections.

You can refer to the github page for more examples and usage showcases.

Integrate Redis to your Spring project

This article shows how to integrate Redis cache to your spring project through annotation configuration.

We will begin with our Gradle configuration. We will use the jedis driver.

group 'com.gkatzioura.spring'
version '1.0-SNAPSHOT'

apply plugin: 'java'
apply plugin: 'eclipse'
apply plugin: 'idea'
apply plugin: 'spring-boot'

buildscript {
    repositories {
        mavenCentral()
    }
    dependencies {
        classpath("org.springframework.boot:spring-boot-gradle-plugin:1.2.5.RELEASE")
    }
}

jar {
    baseName = 'gs-serving-web-content'
    version =  '0.1.0'
}

sourceCompatibility = 1.8

repositories {
    mavenCentral()
}

dependencies {
    compile "org.springframework.boot:spring-boot-starter-thymeleaf"
    compile 'org.slf4j:slf4j-api:1.6.6'
    compile 'ch.qos.logback:logback-classic:1.0.13'
    compile 'redis.clients:jedis:2.7.0'
    compile 'org.springframework.data:spring-data-redis:1.5.0.RELEASE'
    testCompile group: 'junit', name: 'junit', version: '4.11'
}

task wrapper(type: Wrapper) {
    gradleVersion = '2.3'
}

Will proceed with the Redis configuration using spring annotations.

package com.gkatzioura.spring.config;

import org.springframework.cache.CacheManager;
import org.springframework.cache.annotation.CachingConfigurerSupport;
import org.springframework.cache.annotation.EnableCaching;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
import org.springframework.data.redis.cache.RedisCacheManager;
import org.springframework.data.redis.connection.RedisConnectionFactory;
import org.springframework.data.redis.connection.jedis.JedisConnectionFactory;
import org.springframework.data.redis.core.RedisTemplate;
import org.springframework.data.redis.serializer.RedisSerializer;
import org.springframework.data.redis.serializer.StringRedisSerializer;

@Configuration
@EnableCaching
public class RedisConfig extends CachingConfigurerSupport {

    @Bean
    public JedisConnectionFactory redisConnectionFactory() {

        JedisConnectionFactory jedisConnectionFactory = new JedisConnectionFactory();
        jedisConnectionFactory.setUsePool(true);
        return jedisConnectionFactory;
    }

    @Bean
    public RedisSerializer redisStringSerializer() {
        StringRedisSerializer stringRedisSerializer = new StringRedisSerializer();
        return stringRedisSerializer;
    }

    @Bean(name="redisTemplate")
    public RedisTemplate<String, String> redisTemplate(RedisConnectionFactory cf,RedisSerializer redisSerializer) {
        RedisTemplate<String, String> redisTemplate = new RedisTemplate<String, String>();
        redisTemplate.setConnectionFactory(cf);
        redisTemplate.setDefaultSerializer(redisSerializer);
        return redisTemplate;
    }

    @Bean
    public CacheManager cacheManager() {
        return new RedisCacheManager(redisTemplate(redisConnectionFactory(),redisStringSerializer()));
    }

}

Next step is to create our caching interface

package com.gkatzioura.spring.cache;

import java.util.Date;
import java.util.List;

public interface CacheService {

    public void addMessage(String user,String message);

    public List<String> listMessages(String user);

}

A user will add messages and he will be able to retrieve them .
However on our implementation, user related messages will have a time to live of one minute.

Our implementation CacheService using Redis follows.

package com.gkatzioura.spring.cache.impl;

import com.gkatzioura.spring.cache.CacheService;
import org.springframework.data.redis.core.ListOperations;
import org.springframework.data.redis.core.RedisOperations;
import org.springframework.data.redis.core.SetOperations;
import org.springframework.stereotype.Service;

import javax.annotation.Resource;
import java.time.ZonedDateTime;
import java.time.temporal.ChronoUnit;
import java.util.Date;
import java.util.List;

@Service("cacheService")
public class RedisService implements CacheService {

    @Resource(name = "redisTemplate")
    private ListOperations<String, String> messageList;

    @Resource(name = "redisTemplate")
    private RedisOperations<String,String> latestMessageExpiration;

    @Override
    public void addMessage(String user,String message) {

        messageList.leftPush(user,message);

        ZonedDateTime zonedDateTime = ZonedDateTime.now();
        Date date = Date.from(zonedDateTime.plus(1, ChronoUnit.MINUTES).toInstant());
        latestMessageExpiration.expireAt(user,date);
    }

    @Override
    public List<String> listMessages(String user) {
        return messageList.range(user,0,-1);
    }

}

Our cache mechanism will retain a list of messages sent by each user. To achieve so we will employee the ListOperations interface using the user as a key.
The RedisOperations interface gives us the ability to specify a time to live for a key. In our case it is used for the user key.

Next we create a controller with the cache service injected.

package com.gkatzioura.spring.controller;

import com.gkatzioura.spring.cache.CacheService;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.web.bind.annotation.*;

import java.util.List;

@RestController
public class MessageController {


    @Autowired
    private CacheService cacheService;

    @RequestMapping(value = "/message",method = RequestMethod.GET)
    @ResponseBody
    public List<String> greeting(String user) {

        List<String> messages = cacheService.listMessages(user);

        return messages;
    }

    @RequestMapping(value = "/message",method = RequestMethod.POST)
    @ResponseBody
    public String saveGreeting(String user,String message) {

        cacheService.addMessage(user,message);

        return "OK";

    }

}

Last but not least our Application class

package com.gkatzioura.spring;

import org.springframework.boot.SpringApplication;
import org.springframework.boot.autoconfigure.SpringBootApplication;

@SpringBootApplication
public class Application {

    public static void main(String[] args) {
        SpringApplication.run(Application.class, args);
    }

}

In order to run just issue

gradle bootRun