Spring boot and Apache Camel

As the world of software moves on, more complex systems are being developed, which have to integrate with each other. It started with SOA and it continues with microservices.

Camel is the number one integration tool that comes to my mind since nowadays spring boot with camel is a very strong combination.

 

Apache Camel

The first step is to include the camel dependencies to our spring project.

buildscript {
	ext {
		springBootVersion = '1.5.9.BUILD-SNAPSHOT'
	}
	repositories {
		mavenCentral()
		maven { url "https://repo.spring.io/snapshot" }
		maven { url "https://repo.spring.io/milestone" }
	}
	dependencies {
		classpath("org.springframework.boot:spring-boot-gradle-plugin:${springBootVersion}")
	}
}

apply plugin: 'java'
apply plugin: 'eclipse'
apply plugin: 'org.springframework.boot'

group = 'com.gkatzioura'
version = '0.0.1-SNAPSHOT'
sourceCompatibility = 1.8

repositories {
	mavenCentral()
	maven { url "https://repo.spring.io/snapshot" }
	maven { url "https://repo.spring.io/milestone" }
}


dependencies {
	compile('org.apache.camel:camel-spring-boot-starter:2.20.0')
	testCompile('org.springframework.boot:spring-boot-starter-test')
    testCompile('org.apache.camel:camel-test-spring:2.20.0')
}

In order to have a faster project setup from scratch you can always use the online spring initializer.

Now let’s add a simple route

package com.gkatzioura.springcamel.routes;

import org.apache.camel.builder.RouteBuilder;
import org.springframework.stereotype.Component;

@Component
public class TimerRoute extends RouteBuilder {

    public static final String ROUTE_NAME = "TIMER_ROUTE";

    @Override
    public void configure() throws Exception {
        from("timer:initial//start?period=10000")
                .routeId(ROUTE_NAME)
                .to("log:executed");
    }
}

We don’t have to worry about the camel context configuration since the Camel auto-configuration creates a SpringCamelContext for you and takes care of the proper initialization and shutdown of that context.

Also camel auto-configuration collects all the RouteBuilder instances from the Spring context and automatically injects them into the provided CamelContext. Thus we don’t have to register our routes to the CamelContext.

As you can see our route has a timer with a period of 10000 milliseconds which routes to a log endpoint. The log endpoint will print the executed string every 10000 milliseconds.

Keep in mind that if no routeId is specified, camel will assign a name on its own, therefore giving a name to our route definition is a good practice in case we want to retrieve the root definition.

In order for camel to stay up, we need to keep our main thread blocked. Thus we add this configuration to our application.yml file.

camel:
  springboot:
    main-run-controller: true

Instead of this we can include the spring-boot-starter-web dependency, but our application has as few dependencies as possible, and we need to keep it this way.

However the most difficult part in the integration with other systems is testing. Throughout the years there have been rapid advancements on testing and the tools that we use.
Camel also comes packaged with some great tools in order to unit test.

For example we will implement a test of the route specified previously.

@RunWith(CamelSpringBootRunner.class)
@SpringBootTest
public class SpringCamelApplicationTests {

    @EndpointInject(uri = MOCK_RESULT)
    private MockEndpoint resultEndpoint;

    @Autowired
    private CamelContext camelContext;

    @EndpointInject(uri = MOCK_TIMER)
    private ProducerTemplate producer;

    private static final String MOCK_RESULT = "mock:result";
    private static final String MOCK_TIMER = "direct:mock-timer";

    @Before
	public void setup() throws Exception {

	    camelContext.getRouteDefinition(TimerRoute.ROUTE_NAME)
                .autoStartup(true)
                .adviceWith(camelContext, new AdviceWithRouteBuilder() {
                    @Override
                    public void configure() throws Exception {
                        replaceFromWith(MOCK_TIMER);
                        interceptSendToEndpoint("log*")
                                .skipSendToOriginalEndpoint()
                                .to(MOCK_RESULT);
                    }
                });
    }

    @Test
    public void sendMessage() throws Exception {

        resultEndpoint.expectedMessageCount(1);
        producer.sendBody("A message");
        resultEndpoint.assertIsSatisfied();
    }

}

Let’s have a look on each part of the test.

Our JUnit runner of choice would be the CamelSpringBootRunner.class

@RunWith(CamelSpringBootRunner.class)

We inject a ProducerTemplate. The ProducerTemplate interface allows you to send message exchanges to endpoints in a variety of different ways to make it easy to work with Camel Endpoint instances from Java code.

Then we inject a MockEndpoint. The MockEndpoint will serve us by replacing the original endpoint. Then we will set the expected number of messages to be received. Once the processing is done we assert that the amount of received messages is satisfied.

On our setup method we will replace our original endpoint with the fake producer template endpoint. Thus our route will receive the events that we will issue from the ProducerTemplate.
Then we will also intercept the log endpoint and direct the message to the MockEndpoint previously specified.

So we ended up withe a camel application and a unit test for the route specified.
You can find the source code on github.

Spring and Threads: Transactions

In order to be able to use transactions with our thread we need to understand how transactions work with spring. Transaction information in spring is stored in ThreadLocal variables. Therefore these variables are specific for an ongoing transaction on a single thread.

icon-spring-framework

When it comes to an action run by a single thread the transaction gets propagated among the spring components called hierarchically.

Thus in case of a @Transactional  annotated service which spawns a thread, the transaction will not be propagated from the @Transactional service to the newly created thread. The result will be an error indicating that the transaction is missing.

Since the action that take place inside your thread, requires database access through jpa, a new transaction has to be created.

By looking at the @Transactional documentation  we can get more information on the transaction propagation types. The default propagation mode for @Transactional is REQUIRED.

Therefore by annotating a method with the @Transactional, a new transaction will be created and will be propagated to the other services called from our thread.

For example our async method can be annotated as Transactional


@Async
@Transactional
public void executeTransactionally() {
    System.out.println("Execute a transaction from the new thread");
}

The same applies for the method which will be invoked from the run function of a Runnable class. Although async is pretty simple to use, behind the scenes it wraps the call in a Runnable, which dispatched to an executor.

To sum up when it comes to work with threads and transaction in spring, it should be done with extra care. Also keep in mind that the transactions cannot be passed from thread to thread. Last but not least make sure that your @Async and @Transactional functions are public and go though the proxy that will make the necessary actions before being invoked.

Last but not least I’ve compiled a cheat sheet that lists some helpful spring & threads tips.
Sign up in the link to receive it.

Spring and Threads: Async

Previously we started working with spring and the TaskExecutor, thus we became more familiar on how to use threads on a spring application.

However using the task executor might be cumbersome especially when we need to execute a simple action.

Spring’s Asynchronous methods come to the rescue.

Instead of messing with runnables and the TaskExecutor, you trade the control of the executor for the simplicity of the async functions.
In order to execute your function in another thread all you have to do is to annotate your functions with the @Async annotation.

Asynchronous methods come with two modes.

A fire and forget mode: a method which returns a void type.

    @Async
    @Transactional
    public void printEmployees() {

        List<Employee> employees = entityManager.createQuery("SELECT e FROM Employee e").getResultList();
        employees.stream().forEach(e->System.out.println(e.getEmail()));
    }

A results retrieval mode: a method which returns a future type.

    @Async
    @Transactional
    public CompletableFuture<List<Employee>> fetchEmployess() {
        List<Employee> employees = entityManager.createQuery("SELECT e FROM Employee e").getResultList();
        return CompletableFuture.completedFuture(employees);
    }

Pay extra attention to the fact that @Async annotations do not work if they are invoked by ‘this’. @Async behaves just like the @Transactional annotation. Therefore you need to have your async functions as public. You can find more information on the aop proxies documentation.

However using only the @Async annotation is not enough. We need to enable Spring’s asynchronous method execution capability by using the @EnableAsync annotation in one of our configuration classes.

package com.gkatzioura.config;

import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
import org.springframework.core.task.TaskExecutor;
import org.springframework.scheduling.concurrent.ThreadPoolTaskExecutor;

import java.util.concurrent.Executor;

/**
 * Created by gkatzioura on 4/26/17.
 */
@Configuration
@EnableAsync
public class ThreadConfig {

    @Bean
    public TaskExecutor threadPoolTaskExecutor() {

        ThreadPoolTaskExecutor executor = new ThreadPoolTaskExecutor();
        executor.setCorePoolSize(4);
        executor.setMaxPoolSize(4);
        executor.setThreadNamePrefix("sgfgd");
        executor.initialize();

        return executor;
    }

}

The next question is how we declare the resources and the threads pools that the async functions will use. We can get the answer from the documentation.

By default, Spring will be searching for an associated thread pool definition: either a unique TaskExecutor bean in the context, or an Executor bean named “taskExecutor” otherwise. If neither of the two is resolvable, a SimpleAsyncTaskExecutor will be used to process async method invocations.

However in some cases we don’t want the same thread pool to run all of application’s tasks. We might want separate threads pools with different configurations backing our functions.

To achieve so we pass to the @Async annotation the name of the executor we might want to use for each function.

For example  an executor with the name ‘specificTaskExecutor’ is configured.

@Configuration
@EnableAsync
public class ThreadConfig {

    @Bean(name = "specificTaskExecutor")
    public TaskExecutor specificTaskExecutor() {

        ThreadPoolTaskExecutor executor = new ThreadPoolTaskExecutor();
        executor.initialize();
        return executor;
    }

}

Then our function should set the qualifier value to determine the target executor of a specific Executor or TaskExecutor.

@Async("specificTaskExecutor")
public void runFromAnotherThreadPool() {
    System.out.println("You function code here");
}

On the next article we will talk about transactions on threads.

You can find the sourcecode on github.

Last but not least I’ve compiled a cheat sheet that lists some helpful spring & threads tips.
Sign up in the link to receive it.

Spring and Threads: TaskExecutor

Using threads in a web application is not unusual especially when you have to develop long running tasks.

Considering spring we must pay extra attention and use the tools it already provides, instead of spawning our own threads.
We want our threads to be managed by spring and thus be able to use the other components of our application without any repercussions, and shutdown our application gracefully without any work being in progress.

Spring provides the TaskExecutor as an abstraction for dealing with executors.
The Spring’s TaskExecutor interface is identical to the java.util.concurrent.Executor interface.
There are a number of pre-built implementations of TaskExecutor included with the Spring distribution, you can find more about them from the official documentation.
By providing to your spring environment a TaskExecutor implementation you will be able to inject the TaskExecutor to your beans and have access to managed threads.

package com.gkatzioura.service;

import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.context.ApplicationContext;
import org.springframework.core.task.TaskExecutor;
import org.springframework.stereotype.Service;
import java.util.List;

/**
 * Created by gkatzioura on 4/26/17.
 */
@Service
public class AsynchronousService {

    @Autowired
    private ApplicationContext applicationContext;

    @Autowired
    private TaskExecutor taskExecutor;

    public void executeAsynchronously() {

        taskExecutor.execute(new Runnable() {
            @Override
            public void run() {
                //TODO add long running task
            }
        });
    }
}

First step is to add the TaskExecutor configuration to our spring application.

package com.gkatzioura.config;

import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
import org.springframework.core.task.TaskExecutor;
import org.springframework.scheduling.concurrent.ThreadPoolTaskExecutor;

import java.util.concurrent.Executor;

/**
 * Created by gkatzioura on 4/26/17.
 */
@Configuration
public class ThreadConfig {

    @Bean
    public TaskExecutor threadPoolTaskExecutor() {

        ThreadPoolTaskExecutor executor = new ThreadPoolTaskExecutor();
        executor.setCorePoolSize(4);
        executor.setMaxPoolSize(4);
        executor.setThreadNamePrefix("default_task_executor_thread");
        executor.initialize();

        return executor;
    }

}

Once we have our executor setup the process is simple. We inject the executor to a spring component and then we submit Runnable classes containing the tasks to be executed.

Since our asynchronous code might as well need to interact with other components of our application and have them injected, a nice approach is to create prototype scoped runnable instances.

package com.gkatzioura;

import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.springframework.context.annotation.Scope;
import org.springframework.stereotype.Component;

/**
 * Created by gkatzioura on 10/18/17.
 */
@Component
@Scope("prototype")
public class MyThread implements Runnable {

    private static final Logger LOGGER = LoggerFactory.getLogger(MyThread.class);

    @Override
    public void run() {
        
        LOGGER.info("Called from thread");
    }
}

Then we are ready to inject the executor to our services and use it to execute runnable instances.

package com.gkatzioura.service;

import com.gkatzioura.MyThread;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.context.ApplicationContext;
import org.springframework.core.task.TaskExecutor;
import org.springframework.stereotype.Service;

import java.util.List;

/**
 * Created by gkatzioura on 4/26/17.
 */
@Service
public class AsynchronousService {

    @Autowired
    private TaskExecutor taskExecutor;

    @Autowired
    private ApplicationContext applicationContext;

    public void executeAsynchronously() {

        MyThread myThread = applicationContext.getBean(MyThread.class);
        taskExecutor.execute(myThread);
    }

}

On the next article we will bring our multit-hreaded codebase to a new level by using spring’s asynchronous functions.

You can find the sourcecode on github.

Last but not least I’ve compiled a cheat sheet that lists some helpful spring & threads tips.
Sign up in the link to receive it.

Spring Data with JPA and @NamedQueries

If you use Spring Data and @NamedQuery annotations at your JPA entity you can easily use them in a more convenient way using the spring data repository.

On a previous blog we created a spring data project using spring boot and docker. We will use the pretty same project and enhance our repository’s functionality.

We will implement a named query that will fetch employees only if their Last Name has as many characters as the ones specified.

package com.gkatzioura.springdata.jpa.persistence.entity;

import javax.persistence.*;

/**
 * Created by gkatzioura on 6/2/16.
 */
@Entity
@Table(name = "employee", schema="spring_data_jpa_example")
@NamedQuery(name = "Employee.fetchByLastNameLength",
        query = "SELECT e FROM Employee e WHERE CHAR_LENGTH(e.lastname) =:length "
)
public class Employee {

    @Id
    @Column(name = "id")
    @GeneratedValue(strategy = GenerationType.SEQUENCE)
    private Long id;

    @Column(name = "firstname")
    private String firstName;

    @Column(name = "lastname")
    private String lastname;

    @Column(name = "email")
    private String email;

    @Column(name = "age")
    private Integer age;

    @Column(name = "salary")
    private Integer salary;

    public Long getId() {
        return id;
    }

    public void setId(Long id) {
        this.id = id;
    }

    public String getFirstName() {
        return firstName;
    }

    public void setFirstName(String firstName) {
        this.firstName = firstName;
    }

    public String getLastname() {
        return lastname;
    }

    public void setLastname(String lastname) {
        this.lastname = lastname;
    }

    public String getEmail() {
        return email;
    }

    public void setEmail(String email) {
        this.email = email;
    }

    public Integer getAge() {
        return age;
    }

    public void setAge(Integer age) {
        this.age = age;
    }

    public Integer getSalary() {
        return salary;
    }

    public void setSalary(Integer salary) {
        this.salary = salary;
    }
}

Pay extra attention to the query name and the convention we follow @{EntityName}.{queryName}.
Then we will add the method to our spring data repository.

package com.gkatzioura.springdata.jpa.persistence.repository;

import com.gkatzioura.springdata.jpa.persistence.entity.Employee;
import org.springframework.data.jpa.repository.JpaRepository;
import org.springframework.data.repository.query.Param;
import org.springframework.stereotype.Repository;

import java.util.List;

/**
 * Created by gkatzioura on 6/2/16.
 */
@Repository
public interface EmployeeRepository extends JpaRepository<Employee,Long>, EmployeeRepositoryCustom {

    List<Employee> fetchByLastNameLength(@Param("length") Long length);
}

And last but not least add some functionality to our controller.

package com.gkatzioura.springdata.jpa.controller;

import com.gkatzioura.springdata.jpa.persistence.entity.Employee;
import com.gkatzioura.springdata.jpa.persistence.repository.EmployeeRepository;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.web.bind.annotation.RequestMapping;
import org.springframework.web.bind.annotation.RequestParam;
import org.springframework.web.bind.annotation.RestController;

import java.util.List;

/**
 * Created by gkatzioura on 6/2/16.
 */
@RestController
public class TestController {

    @Autowired
    private EmployeeRepository employeeRepository;

    @RequestMapping("/employee")
    public List<Employee> getTest() {

        return employeeRepository.findAll();
    }

    @RequestMapping("/employee/filter")
    public List<Employee> getFiltered(String firstName,@RequestParam(defaultValue = "0") Double bonusAmount) {

        return employeeRepository.getFirstNamesLikeAndBonusBigger(firstName,bonusAmount);
    }

    @RequestMapping("/employee/lastnameLength")
    public List<Employee> fetchByLength(Long length) {
        return employeeRepository.fetchByLastNameLength(length);
    }

}

You can find the source code on github.

Hibernate Caching with HazelCast: Basic configuration

Previously we went through an introduction on JPA caching, the mechanisms and what hibernate offers.

What comes next is a hibernate project using Hazelcast as a second level cache.

We will use a basic spring boot project for this purpose with JPA. Spring boot uses hibernate as the default JPA provider.
Our setup will be pretty close to the one of a previous post.
We will use postgresql with docker for our sql database.

group 'com.gkatzioura'
version '1.0-SNAPSHOT'

buildscript {
    repositories {
        mavenCentral()
    }
    dependencies {
        classpath("org.springframework.boot:spring-boot-gradle-plugin:1.5.1.RELEASE")
    }
}

apply plugin: 'java'
apply plugin: 'idea'
apply plugin: 'org.springframework.boot'


repositories {
    mavenCentral()
}

dependencies {
    compile("org.springframework.boot:spring-boot-starter-web")
    compile group: 'org.springframework.boot', name: 'spring-boot-starter-data-jpa'
    compile group: 'org.postgresql', name:'postgresql', version:'9.4-1206-jdbc42'
    compile group: 'org.springframework', name: 'spring-jdbc'
    compile group: 'com.zaxxer', name: 'HikariCP', version: '2.6.0'
    compile group: 'com.hazelcast', name: 'hazelcast-hibernate5', version: '1.2'
    compile group: 'com.hazelcast', name: 'hazelcast', version: '3.7.5'
    testCompile group: 'junit', name: 'junit', version: '4.11'
}

By examining the dependencies carefully we see the hikari pool, the postgresql driver, spring data jpa and of course hazelcast.

Instead of creating the database manually we will automate it by utilizing the database initialization feature of Spring boot.

We shall create a file called schema.sql under the resources folder.

create schema spring_data_jpa_example;
 
create table spring_data_jpa_example.employee(
    id  SERIAL PRIMARY KEY,
    firstname   TEXT    NOT NULL,
    lastname    TEXT    NOT NULL,   
    email       TEXT    not null,
    age         INT     NOT NULL,
    salary         real,
    unique(email)
);
 
insert into spring_data_jpa_example.employee (firstname,lastname,email,age,salary) 
values ('Test','Me','test@me.com',18,3000.23);

To keep it simple and avoid any further configurations we shall put the configurations for datasource, jpa and caching inside the application.yml file.

spring:
  datasource:
    continue-on-error: true
    type: com.zaxxer.hikari.HikariDataSource
    url: jdbc:postgresql://172.17.0.2:5432/postgres
    driver-class-name: org.postgresql.Driver
    username: postgres
    password: postgres
    hikari:
      idle-timeout: 10000
  jpa:
    properties:
      hibernate:
        cache:
          use_second_level_cache: true
          use_query_cache: true
          region:
            factory_class: com.hazelcast.hibernate.HazelcastCacheRegionFactory
    show-sql: true

The configuration spring.datasource.continue-on-error is crucial since once the application relaunches, there should be a second attempt to create the database and thus a crash is inevitable.

Any hibernate specific properties reside at the spring.jpa.properties path. We enabled the second level cache and the query cache.

Also we set show-sql to true. This means that once a query hits the database it shall be logged through the console.

Then create our employee entity.

package com.gkatzioura.hibernate.enitites;

import javax.persistence.*;

/**
 * Created by gkatzioura on 2/6/17.
 */
@Entity
@Table(name = "employee", schema="spring_data_jpa_example")
public class Employee {

    @Id
    @Column(name = "id")
    @GeneratedValue(strategy = GenerationType.SEQUENCE)
    private Long id;

    @Column(name = "firstname")
    private String firstName;

    @Column(name = "lastname")
    private String lastname;

    @Column(name = "email")
    private String email;

    @Column(name = "age")
    private Integer age;

    @Column(name = "salary")
    private Integer salary;

    public Long getId() {
        return id;
    }

    public void setId(Long id) {
        this.id = id;
    }

    public String getFirstName() {
        return firstName;
    }

    public void setFirstName(String firstName) {
        this.firstName = firstName;
    }

    public String getLastname() {
        return lastname;
    }

    public void setLastname(String lastname) {
        this.lastname = lastname;
    }

    public String getEmail() {
        return email;
    }

    public void setEmail(String email) {
        this.email = email;
    }

    public Integer getAge() {
        return age;
    }

    public void setAge(Integer age) {
        this.age = age;
    }

    public Integer getSalary() {
        return salary;
    }

    public void setSalary(Integer salary) {
        this.salary = salary;
    }
}

Everything is setup. Spring boot will detect the entity and create an EntityManagerFactory on its own.
What comes next is the repository class for employee.

package com.gkatzioura.hibernate.repository;

import com.gkatzioura.hibernate.enitites.Employee;
import org.springframework.data.jpa.repository.JpaRepository;
import org.springframework.data.repository.CrudRepository;

/**
 * Created by gkatzioura on 2/11/17.
 */
public interface EmployeeRepository extends JpaRepository<Employee,Long> {
}

And the last one is the controller

package com.gkatzioura.hibernate.controller;

import com.gkatzioura.hibernate.enitites.Employee;
import com.gkatzioura.hibernate.repository.EmployeeRepository;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.web.bind.annotation.PathVariable;
import org.springframework.web.bind.annotation.RequestMapping;
import org.springframework.web.bind.annotation.RequestParam;
import org.springframework.web.bind.annotation.RestController;

import java.util.List;

/**
 * Created by gkatzioura on 2/6/17.
 */
@RestController
public class EmployeeController {

    @Autowired
    private EmployeeRepository employeeRepository;

    @RequestMapping("/employee")
    public List<Employee> testIt() {

        return employeeRepository.findAll();
    }

    @RequestMapping("/employee/{employeeId}")
    public Employee getEmployee(@PathVariable Long employeeId) {

        return employeeRepository.findOne(employeeId);
    }

}

Once we issue a request at
http://localhost:8080/employee/1

Console will display the query issued at the database

Hibernate: select employee0_.id as id1_0_0_, employee0_.age as age2_0_0_, employee0_.email as email3_0_0_, employee0_.firstname as firstnam4_0_0_, employee0_.lastname as lastname5_0_0_, employee0_.salary as salary6_0_0_ from spring_data_jpa_example.employee employee0_ where employee0_.id=?

The second time we issue the request, since we have the second cache enabled there won’t be a query issued upon the database. Instead the entity shall be fetched from the second level cache.

You can download the project from github.

Push Spring Boot Docker images on ECR

On a previous blog we integrated a spring boot application with EC2.
It is one of the most raw forms of deployment that you can have on Amazon Web Services.

On this tutorial we will create a docker image with our application which will be stored to the Amazon EC2 container registry.

You need to have the aws cli tool installed.

We will get as simple as we can with our spring application therefore we will use an example from the official spring source page. The only changes applied will be on the packaging and the application name.

Our application shall be named ecs-deployment

rootProject.name = 'ecs-deployment'

Then we build and run our application

gradle build
gradle bootRun

Now let’s dockerize our application.
First we shall create a Dockerfile that will reside on src/main/docker.

FROM frolvlad/alpine-oraclejdk8
VOLUME /tmp
ADD ecs-deployment-1.0-SNAPSHOT.jar app.jar
RUN sh -c 'touch /app.jar'
ENV JAVA_OPTS=""
ENTRYPOINT [ "sh", "-c", "java $JAVA_OPTS -Djava.security.egd=file:/dev/./urandom -jar /app.jar" ]

Then we should edit our gradle file in order to add the docker dependency, the docker plugin and an extra gradle task in order to create our docker image.

buildscript {
    ...
    dependencies {
        ...
        classpath('se.transmode.gradle:gradle-docker:1.2')
    }
}

...
apply plugin: 'docker'


task buildDocker(type: Docker, dependsOn: build) {
    push = false
    applicationName = jar.baseName
    dockerfile = file('src/main/docker/Dockerfile')
}

And we are ready to build our docker image.

./gradlew build buildDocker

You can also run your docker application from the newly created image.

docker run -p 8080:8080 -t com.gkatzioura.deployment/ecs-deployment:1.0-SNAPSHOT

First step is too create our ecr repository

aws ecr create-repository  --repository-name ecs-deployment

Then let us proceed with our docker registry authentication.

aws ecr get-login

Then run the command given in the output. The login attempt will succeed and your are ready to proceed to push your image.

First tag the image in order to specify the repository that we previously created and then do a docker push.

docker tag {imageid} {aws account id}.dkr.ecr.{aws region}.amazonaws.com/ecs-deployment:1.0-SNAPSHOT
docker push {aws account id}.dkr.ecr.{aws region}.amazonaws.com/ecs-deployment:1.0-SNAPSHOT

And we are done! Our spring boot docker image is deployed on the Amazon EC2 container registry.

You can find the source code on github.

Spring-Boot and Cache Abstraction with HazelCast

Previously we got started with Spring Cache abstraction using the default Cache Manager that spring provides.

Although this approach might suit our needs for simple applications, in case of complex problems we need to use different tools with more capabilities. Hazelcast is one of them. Hazelcast is hands down a great caching tool when it comes to a JVM based application. By using hazelcast as a cache, data is evenly distributed among the nodes of a computer cluster, allowing for horizontal scaling of available storage.

We will run our codebase using spring profiles thus ‘hazelcast-cache’ will be our profile name.

group 'com.gkatzioura'
version '1.0-SNAPSHOT'


buildscript {
    repositories {
        mavenCentral()
    }
    dependencies {
        classpath("org.springframework.boot:spring-boot-gradle-plugin:1.4.2.RELEASE")
    }
}

apply plugin: 'java'
apply plugin: 'idea'
apply plugin: 'org.springframework.boot'

repositories {
    mavenCentral()
}


sourceCompatibility = 1.8
targetCompatibility = 1.8

dependencies {
    compile("org.springframework.boot:spring-boot-starter-web")
    compile("org.springframework.boot:spring-boot-starter-cache")
    compile("org.springframework.boot:spring-boot-starter")
    compile("com.hazelcast:hazelcast:3.7.4")
    compile("com.hazelcast:hazelcast-spring:3.7.4")

    testCompile("junit:junit")
}

bootRun {
    systemProperty "spring.profiles.active", "hazelcast-cache"
}

As you can see we updated the gradle file from the previous example and we added two extra dependencies hazelcast and hazelcast-spring. Also we changed the profile that our application will run by default.

Our next step is to configure the hazelcast cache manager.

package com.gkatzioura.caching.config;

import com.hazelcast.config.Config;
import com.hazelcast.config.EvictionPolicy;
import com.hazelcast.config.MapConfig;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
import org.springframework.context.annotation.Profile;

/**
 * Created by gkatzioura on 1/10/17.
 */
@Configuration
@Profile("hazelcast-cache")
public class HazelcastCacheConfig {

    @Bean
    public Config hazelCastConfig() {

        Config config = new Config();
        config.setInstanceName("hazelcast-cache");

        MapConfig allUsersCache = new MapConfig();
        allUsersCache.setTimeToLiveSeconds(20);
        allUsersCache.setEvictionPolicy(EvictionPolicy.LFU);
        config.getMapConfigs().put("alluserscache",allUsersCache);

        MapConfig usercache = new MapConfig();
        usercache.setTimeToLiveSeconds(20);
        usercache.setEvictionPolicy(EvictionPolicy.LFU);
        config.getMapConfigs().put("usercache",usercache);

        return config;
    }

}

We just created two maps with a ttl policy of 20 seconds. Therefore 20 seconds since the map gets populated a cache eviction will occur. For more hazelcast configurations please refer to the official hazelcast documentation.

Another change that we have to implement is to change UserPayload into a serializable Java object, since objects stored in hazelcast must be Serializable.

package com.gkatzioura.caching.model;

import java.io.Serializable;

/**
 * Created by gkatzioura on 1/5/17.
 */
public class UserPayload implements Serializable {

    private String userName;
    private String firstName;
    private String lastName;

    public String getUserName() {
        return userName;
    }

    public void setUserName(String userName) {
        this.userName = userName;
    }

    public String getFirstName() {
        return firstName;
    }

    public void setFirstName(String firstName) {
        this.firstName = firstName;
    }

    public String getLastName() {
        return lastName;
    }

    public void setLastName(String lastName) {
        this.lastName = lastName;
    }
}

Last but not least we add another repository bound to the hazelcast-cache profile.

The result is our previous spring-boot application integrated with hazelcast instead of the default cache, configured with a ttl policy.

You can find the sourcecode on github.

Spring boot and Cache Abstraction

Caching is a major ingredient of most applications, and as long as we try to avoid disk access it will stay strong.
Spring has great support for caching with a wide range of configurations. You can start as simple as you want and progress to something much more customizable.

This would be an example with the simplest form of caching that spring provides.
Spring comes by default with an in memory cache which is pretty easy to setup.

Let us start with our gradle file.

group 'com.gkatzioura'
version '1.0-SNAPSHOT'


buildscript {
    repositories {
        mavenCentral()
    }
    dependencies {
        classpath("org.springframework.boot:spring-boot-gradle-plugin:1.4.2.RELEASE")
    }
}

apply plugin: 'java'
apply plugin: 'idea'
apply plugin: 'org.springframework.boot'

repositories {
    mavenCentral()
}


sourceCompatibility = 1.8
targetCompatibility = 1.8

dependencies {
    compile("org.springframework.boot:spring-boot-starter-web")
    compile("org.springframework.boot:spring-boot-starter-cache")
    compile("org.springframework.boot:spring-boot-starter")
    testCompile("junit:junit")
}

bootRun {
    systemProperty "spring.profiles.active", "simple-cache"
}

Since the same project will be used for different cache providers there are gonna be multiple spring profiles. The spring profile for this tutorial would be the simple-cache since we are going to use the ConcurrentMap-based Cache which happens to be the default.

We will implement an application which will fetch user information from our local file system.
The information shall reside on the users.json file

[
  {"userName":"user1","firstName":"User1","lastName":"First"},
  {"userName":"user2","firstName":"User2","lastName":"Second"},
  {"userName":"user3","firstName":"User3","lastName":"Third"},
  {"userName":"user4","firstName":"User4","lastName":"Fourth"}
]

Also we will specify a simple model for the data to be retrieved.

package com.gkatzioura.caching.model;

/**
 * Created by gkatzioura on 1/5/17.
 */
public class UserPayload {

    private String userName;
    private String firstName;
    private String lastName;

    public String getUserName() {
        return userName;
    }

    public void setUserName(String userName) {
        this.userName = userName;
    }

    public String getFirstName() {
        return firstName;
    }

    public void setFirstName(String firstName) {
        this.firstName = firstName;
    }

    public String getLastName() {
        return lastName;
    }

    public void setLastName(String lastName) {
        this.lastName = lastName;
    }
}

Then we will add a bean that will read the information.

package com.gkatzioura.caching.config;

import com.fasterxml.jackson.databind.ObjectMapper;
import com.gkatzioura.caching.model.UserPayload;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.beans.factory.annotation.Value;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
import org.springframework.context.annotation.Profile;
import org.springframework.core.io.Resource;

import java.io.IOException;
import java.io.InputStream;
import java.util.Arrays;
import java.util.Collections;
import java.util.List;

/**
 * Created by gkatzioura on 1/5/17.
 */
@Configuration
@Profile("simple-cache")
public class SimpleDataConfig {

    @Autowired
    private ObjectMapper objectMapper;

    @Value("classpath:/users.json")
    private Resource usersJsonResource;

    @Bean
    public List<UserPayload> payloadUsers() throws IOException {

        try(InputStream inputStream = usersJsonResource.getInputStream()) {

            UserPayload[] payloadUsers = objectMapper.readValue(inputStream,UserPayload[].class);
            return Collections.unmodifiableList(Arrays.asList(payloadUsers));
        }
    }
}

Obviously in order to access the information we will use the bean instantiated containing all the user information.

Next step will be to create a repository interface to specify the methods that will be used.

package com.gkatzioura.caching.repository;

import com.gkatzioura.caching.model.UserPayload;

import java.util.List;

/**
 * Created by gkatzioura on 1/6/17.
 */
public interface UserRepository {

    List<UserPayload> fetchAllUsers();

    UserPayload firstUser();

    UserPayload userByFirstNameAndLastName(String firstName,String lastName);

}

Now let’s dive into the implementation which will contain the cache annotations needed.

package com.gkatzioura.caching.repository;

import com.gkatzioura.caching.model.UserPayload;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.cache.annotation.CacheEvict;
import org.springframework.cache.annotation.Cacheable;
import org.springframework.context.annotation.Profile;
import org.springframework.stereotype.Repository;

import java.util.List;
import java.util.Optional;

/**
 * Created by gkatzioura on 12/30/16.
 */
@Repository
@Profile("simple-cache")
public class UserRepositoryLocal implements UserRepository {

    @Autowired
    private List<UserPayload> payloadUsers;

    private static final Logger LOGGER = LoggerFactory.getLogger(UserRepositoryLocal.class);

    @Override
    @Cacheable("alluserscache")
    public List<UserPayload> fetchAllUsers() {

        LOGGER.info("Fetching all users");

        return payloadUsers;
    }

    @Override
    @Cacheable(cacheNames = "usercache",key = "#root.methodName")
    public UserPayload firstUser() {

        LOGGER.info("fetching firstUser");

        return payloadUsers.get(0);
    }

    @Override
    @Cacheable(cacheNames = "usercache",key = "{#firstName,#lastName}")
    public UserPayload userByFirstNameAndLastName(String firstName,String lastName) {

        LOGGER.info("fetching user by firstname and lastname");

        Optional<UserPayload> user = payloadUsers.stream().filter(
                p-> p.getFirstName().equals(firstName)
                &&p.getLastName().equals(lastName))
                .findFirst();

        if(user.isPresent()) {
            return user.get();
        } else {
            return null;
        }
    }

}

Methods that contain the @Cacheable will trigger cache population contrary to methods that contain @CacheEvict which trigger cache eviction.
By using @Cacheable instead of just specifying the cache map that our values will be stored, we can proceed into specifying also keys based on the method name or the method arguments. Thus we achieve method caching.
For example the method firstUser, uses as a key the method name whilst the method userByFirstNameAndLastName uses the method arguments in order to create a key.

Two methods with the @CacheEvict annotation will empty the caches specified.

LocalCacheEvict will be the component that will handler the eviction.

package com.gkatzioura.caching.repository;

import org.springframework.cache.annotation.CacheEvict;
import org.springframework.context.annotation.Profile;
import org.springframework.stereotype.Component;

/**
 * Created by gkatzioura on 1/7/17.
 */
@Component
@Profile("simple-cache")
public class LocalCacheEvict {

    @CacheEvict(cacheNames = "alluserscache",allEntries = true)
    public void evictAllUsersCache() {

    }

    @CacheEvict(cacheNames = "usercache",allEntries = true)
    public void evictUserCache() {

    }

}

Since we use a very simple form of cacheh ttl eviction is not supported. Therefore we will add a scheduler only for this particular case which will evict the cache after a certain period of time.

package com.gkatzioura.caching.scheduler;

import com.gkatzioura.caching.repository.LocalCacheEvict;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.context.annotation.Profile;
import org.springframework.scheduling.annotation.Scheduled;
import org.springframework.stereotype.Component;

/**
 * Created by gkatzioura on 1/7/17.
 */
@Component
@Profile("simple-cache")
public class EvictScheduler {

    @Autowired
    private LocalCacheEvict localCacheEvict;

    private static final Logger LOGGER = LoggerFactory.getLogger(EvictScheduler.class);

    @Scheduled(fixedDelay=10000)
    public void clearCaches() {

        LOGGER.info("Invalidating caches");

        localCacheEvict.evictUserCache();
        localCacheEvict.evictAllUsersCache();
    }


}

To wrap up we will use a controller to call the methods specified

package com.gkatzioura.caching.controller;

import com.gkatzioura.caching.model.UserPayload;
import com.gkatzioura.caching.repository.UserRepository;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.web.bind.annotation.RequestMapping;
import org.springframework.web.bind.annotation.RequestMethod;
import org.springframework.web.bind.annotation.RestController;

import java.util.List;

/**
 * Created by gkatzioura on 12/30/16.
 */
@RestController
public class UsersController {

    @Autowired
    private UserRepository userRepository;

    @RequestMapping(path = "/users/all",method = RequestMethod.GET)
    public List<UserPayload> fetchUsers() {

        return userRepository.fetchAllUsers();
    }

    @RequestMapping(path = "/users/first",method = RequestMethod.GET)
    public UserPayload fetchFirst() {
        return userRepository.firstUser();
    }

    @RequestMapping(path = "/users/",method = RequestMethod.GET)
    public UserPayload findByFirstNameLastName(String firstName,String lastName ) {

        return userRepository.userByFirstNameAndLastName(firstName,lastName);
    }

}

Last but not least our Application class should contain two extra annotations. @EnableScheduling is needed in order to enable schedulers and @EnableCaching in order to enable caching

package com.gkatzioura.caching;

import org.springframework.boot.SpringApplication;
import org.springframework.boot.autoconfigure.SpringBootApplication;
import org.springframework.cache.annotation.EnableCaching;
import org.springframework.scheduling.annotation.EnableScheduling;

/**
 * Created by gkatzioura on 12/30/16.
 */
@SpringBootApplication
@EnableScheduling
@EnableCaching
public class Application {

    public static void main(String[] args) {
        SpringApplication.run(Application.class,args);
    }

}

You can find the sourcecode on github.

Integrate Spring Boot and EC2 using Cloudformation

On a previous blog we integrated a spring boot application with elastic beanstalk.
The application was a servlet based application responding to requests.

On this tutorial we are going to deploy a spring boot application, which executes some scheduled tasks on an ec2 instance.
The application will be pretty much the same application taken from the official spring guide with some minor differences on packages.

The name of our application will be ec2-deployment

rootProject.name = 'ec2-deployment'

Then we will schedule a task to our spring boot application.

package com.gkatzioura.deployment.task;

import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.springframework.scheduling.annotation.Scheduled;
import org.springframework.stereotype.Component;

/**
 * Created by gkatzioura on 12/16/16.
 */
@Component
public class SimpleTask {

    private static final Logger LOGGER = LoggerFactory.getLogger(SimpleTask.class);

    @Scheduled(fixedRate = 5000)
    public void reportCurrentTime() {
        LOGGER.info("This is a simple task on ec2");
    }

}


Next step is to build the application and deploy it to our s3 bucket.

gradle build
aws s3 cp build/libs/ec2-deployment-1.0-SNAPSHOT.jar s3://{your bucket name}/ec2-deployment-1.0-SNAPSHOT.jar 

What comes next is a bootstrapping script in order to run our application once the server is up and running.

#!/usr/bin/env bash
aws s3 cp s3://{bucket with code}/ec2-deployment-1.0-SNAPSHOT.jar /home/ec2-user/ec2-deployment-1.0-SNAPSHOT.jar
sudo yum -y install java-1.8.0
sudo yum -y remove java-1.7.0-openjdk
cd /home/ec2-user/
sudo nohup java -jar ec2-deployment-1.0-SNAPSHOT.jar > ec2dep.log

This script is pretty much self explanatory. We download the application from the bucket we uploaded it previously, we install the java version needed and then we run the application (this script serves us for example purposes, there are certainly many ways to set up you java application running on linux).

Next step would be to proceed to our cloudformation script. Since we will download our application from s3 it is essential to have an IAM policy that will allow us to download items from the s3 bucket we used previously. Therefore we will create a role with the policy needed

"RootRole": {
      "Type": "AWS::IAM::Role",
      "Properties": {
        "AssumeRolePolicyDocument": {
          "Version" : "2012-10-17",
          "Statement": [ {
            "Effect": "Allow",
            "Principal": {
              "Service": [ "ec2.amazonaws.com" ]
            },
            "Action": [ "sts:AssumeRole" ]
          } ]
        },
        "Path": "/",
        "Policies": [ {
          "PolicyName": "root",
          "PolicyDocument": {
            "Version" : "2012-10-17",
            "Statement": [ {
              "Effect": "Allow",
              "Action": [
                "s3:Get*",
                "s3:List*"
              ],
              "Resource": {"Fn::Join" : [ "", [ "arn:aws:s3:::", {"Ref":"SourceCodeBucket"},"/*"] ] }
            } ]
          }
        } ]
      }
    }

Next step is to encode our bootstrapping script to Base64 in order to be able to pass it as user data.
Once the ec2 instance is up and running it will run the shell commands previously specified.

Last step is to create our instance profile and specify the ec2 instance to be launched

    "RootInstanceProfile": {
      "Type": "AWS::IAM::InstanceProfile",
      "Properties": {
        "Path": "/",
        "Roles": [ {
          "Ref": "RootRole"
        } ]
      }
    },
    "Ec2Instance":{
      "Type":"AWS::EC2::Instance",
      "Properties":{
        "ImageId":"ami-9398d3e0",
        "InstanceType":"t2.nano",
        "KeyName":"TestKey",
        "IamInstanceProfile": {"Ref":"RootInstanceProfile"},
"UserData":"IyEvdXNyL2Jpbi9lbnYgYmFzaA0KYXdzIHMzIGNwIHMzOi8ve2J1Y2tldCB3aXRoIGNvZGV9L2VjMi1kZXBsb3ltZW50LTEuMC1TTkFQU0hPVC5qYXIgL2hvbWUvZWMyLXVzZXIvZWMyLWRlcGxveW1lbnQtMS4wLVNOQVBTSE9ULmphcg0Kc3VkbyB5dW0gLXkgaW5zdGFsbCBqYXZhLTEuOC4wDQpzdWRvIHl1bSAteSByZW1vdmUgamF2YS0xLjcuMC1vcGVuamRrDQpjZCAvaG9tZS9lYzItdXNlci8NCnN1ZG8gbm9odXAgamF2YSAtamFyIGVjMi1kZXBsb3ltZW50LTEuMC1TTkFQU0hPVC5qYXIgPiBlYzJkZXAubG9n"
      }
    }

KeyName stands for the ssh key name, in case you want to login to the ec2 instance.

So we are good to go and create our cloudformation stack. You have to add the CAPABILITY_IAM flag.

aws s3 cp ec2spring.template s3://{bucket with templates}/ec2spring.template
aws cloudformation create-stack --stack-name SpringEc2 --parameters ParameterKey=SourceCodeBucket,ParameterValue={bucket with code} --template-url https://s3.amazonaws.com/{bucket with templates}/ec2spring.template --capabilities CAPABILITY_IAM

That’s it. Now you have your spring application up and running on top of an ec2 instance.
You can download the source code from GitHub.