Read replicas and Spring Data Part 4: Configuring the read repository

Previously we set up two EntityManagers in the same application. One for the reads and one for the writes. Now it’s time to create our read repository.

The read only repository will use the secondary read only EntityManager.

In order to make it a read only repository, it is essential not to have any save and persist actions.

package com.gkatzioura.springdatareadreplica.repository;

import java.util.List;

import org.springframework.data.repository.Repository;

import com.gkatzioura.springdatareadreplica.config.ReadOnlyRepository;
import com.gkatzioura.springdatareadreplica.entity.Employee;

/**
 * This is a read only repository
 */
public interface ReadEmployeeRepository extends Repository {

    List findAll();

}

Our next task would be to create this repository with the read database entity manager.
This means that all repositories shall be created using the default entity manager except from the read only repositories.

I would create an Annotation first. This annotation will declare my repository as Read only. Also I will use this annotation for the scanning operation so that the appropriate EntityManager will be used.

package com.gkatzioura.springdatareadreplica.config;

import java.lang.annotation.Documented;
import java.lang.annotation.ElementType;
import java.lang.annotation.Retention;
import java.lang.annotation.RetentionPolicy;
import java.lang.annotation.Target;

@Retention(RetentionPolicy.RUNTIME)
@Target({ElementType.TYPE})
@Documented
public @interface ReadOnlyRepository {
}

Now I know that spring boot removes the need for annotations and does repository creation in an automated way however our case is a peculiar one.

By making some adjustments our read only repository will look like this

package com.gkatzioura.springdatareadreplica.repository;

import java.util.List;

import org.springframework.data.repository.Repository;

import com.gkatzioura.springdatareadreplica.config.ReadOnlyRepository;
import com.gkatzioura.springdatareadreplica.entity.Employee;

/**
 * This is a read only repository
 */
@ReadOnlyRepository
public interface ReadEmployeeRepository extends Repository {

    List findAll();

}

And now it’s time to work with our repository scanning. All the repositories will be injected with the main EntityManager except from the ones annotated with the @ReadOnlyRepository annotation.

package com.gkatzioura.springdatareadreplica.config;

import javax.sql.DataSource;

import org.springframework.beans.factory.annotation.Qualifier;
import org.springframework.beans.factory.annotation.Value;
import org.springframework.boot.jdbc.DataSourceBuilder;
import org.springframework.boot.orm.jpa.EntityManagerFactoryBuilder;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.ComponentScan;
import org.springframework.context.annotation.Configuration;
import org.springframework.context.annotation.Primary;
import org.springframework.data.jpa.repository.config.EnableJpaRepositories;
import org.springframework.orm.jpa.LocalContainerEntityManagerFactoryBean;

@Configuration
@EnableJpaRepositories(
        basePackages = "com.gkatzioura",
        excludeFilters = @ComponentScan.Filter(ReadOnlyRepository.class),
        entityManagerFactoryRef = "entityManagerFactory"
)
public class PrimaryEntityManagerConfiguration {

    @Value("${spring.datasource.username}")
    private String username;

    @Value("${spring.datasource.password}")
    private String password;

    @Value("${spring.datasource.url}")
    private String url;

    @Bean
    @Primary
    public DataSource dataSource() throws Exception {
        return DataSourceBuilder.create()
                                .url(url)
                                .username(username)
                                .password(password)
                                .driverClassName("org.postgresql.Driver")
                                .build();
    }

    @Bean
    @Primary
    public LocalContainerEntityManagerFactoryBean entityManagerFactory(
            EntityManagerFactoryBuilder builder,
            @Qualifier("dataSource") DataSource dataSource) {
        return builder.dataSource(dataSource)
                      .packages("com.gkatzioura.springdatareadreplica")
                      .persistenceUnit("main")
                      .build();
    }

}

Also we will add the configuration for the read only repositories.

package com.gkatzioura.springdatareadreplica.config;

import javax.sql.DataSource;

import org.springframework.beans.factory.annotation.Qualifier;
import org.springframework.beans.factory.annotation.Value;
import org.springframework.boot.jdbc.DataSourceBuilder;
import org.springframework.boot.orm.jpa.EntityManagerFactoryBuilder;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.ComponentScan;
import org.springframework.context.annotation.Configuration;
import org.springframework.data.jpa.repository.config.EnableJpaRepositories;
import org.springframework.orm.jpa.LocalContainerEntityManagerFactoryBean;

@Configuration
@EnableJpaRepositories(
        basePackages = "com.gkatzioura",
        includeFilters= @ComponentScan.Filter(ReadOnlyRepository.class),
        entityManagerFactoryRef = "readEntityManagerFactory"
)
public class ReadOnlyEntityManagerConfiguration {

    @Value("${spring.datasource.username}")
    private String username;

    @Value("${spring.datasource.password}")
    private String password;

    @Value("${spring.datasource.readUrl}")
    private String readUrl;

    @Bean
    public DataSource readDataSource() throws Exception {
        return DataSourceBuilder.create()
                                .url(readUrl)
                                .username(username)
                                .password(password)
                                .driverClassName("org.postgresql.Driver")
                                .build();
    }

    @Bean
    public LocalContainerEntityManagerFactoryBean readEntityManagerFactory(
            EntityManagerFactoryBuilder builder,
            @Qualifier("readDataSource") DataSource dataSource) {
        return builder.dataSource(dataSource)
                      .packages("com.gkatzioura.springdatareadreplica")
                      .persistenceUnit("read")
                      .build();
    }

}

The secondary entity manager will be injected only to the repositories that only have the @ReadOnlyRepository annotation.

And to show this let’s make some changes to our controller.

package com.gkatzioura.springdatareadreplica.controller;

import java.util.List;

import org.springframework.http.HttpStatus;
import org.springframework.web.bind.annotation.GetMapping;
import org.springframework.web.bind.annotation.PostMapping;
import org.springframework.web.bind.annotation.RequestBody;
import org.springframework.web.bind.annotation.ResponseStatus;
import org.springframework.web.bind.annotation.RestController;

import com.gkatzioura.springdatareadreplica.entity.Employee;
import com.gkatzioura.springdatareadreplica.repository.EmployeeRepository;
import com.gkatzioura.springdatareadreplica.repository.ReadEmployeeRepository;

@RestController
public class EmployeeContoller {

    private final EmployeeRepository employeeRepository;
    private final ReadEmployeeRepository readEmployeeRepository;

    public EmployeeContoller(EmployeeRepository employeeRepository,
                             ReadEmployeeRepository readEmployeeRepository) {
        this.employeeRepository = employeeRepository;
        this.readEmployeeRepository = readEmployeeRepository;
    }

    @GetMapping("/employee")
    public List getEmployees() {
        return employeeRepository.findAll();
    }

    @GetMapping("/employee/read")
    public List getEmployeesRead() {
        return readEmployeeRepository.findAll();
    }

    @PostMapping("/employee")
    @ResponseStatus(HttpStatus.CREATED)
    public void addEmployee(@RequestBody Employee employee) {
        employeeRepository.save(employee);
    }

}

As you add employees to the system the read only repository will keep fetching the old employees while the main repository will fetch all of them including the recently persisted.

Read replicas and Spring Data Part 3: Configuring two entity managers

Our previous setup works as expected. What we shall do now is to get one step further and configure two separate entity managers without affecting the functionality we achieved previously.

The first step would be to set the default entity manager configuration to a primary one.
This is the first step

package com.gkatzioura.springdatareadreplica.config;

import javax.sql.DataSource;

import org.springframework.beans.factory.annotation.Qualifier;
import org.springframework.beans.factory.annotation.Value;
import org.springframework.boot.jdbc.DataSourceBuilder;
import org.springframework.boot.orm.jpa.EntityManagerFactoryBuilder;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.ComponentScan;
import org.springframework.context.annotation.Configuration;
import org.springframework.context.annotation.Primary;
import org.springframework.data.jpa.repository.config.EnableJpaRepositories;
import org.springframework.orm.jpa.LocalContainerEntityManagerFactoryBean;

@Configuration
public class PrimaryEntityManagerConfiguration {

    @Value("${spring.datasource.username}")
    private String username;

    @Value("${spring.datasource.password}")
    private String password;

    @Value("${spring.datasource.url}")
    private String url;

    @Bean
    @Primary
    public DataSource dataSource() throws Exception {
        return DataSourceBuilder.create()
                                .url(url)
                                .username(username)
                                .password(password)
                                .driverClassName("org.postgresql.Driver")
                                .build();
    }

    @Bean
    @Primary
    public LocalContainerEntityManagerFactoryBean entityManagerFactory(
            EntityManagerFactoryBuilder builder,
            @Qualifier("dataSource") DataSource dataSource) {
        return builder.dataSource(dataSource)
                      .packages("com.gkatzioura.springdatareadreplica")
                      .persistenceUnit("main")
                      .build();
    }

}

If you run your application with this configuration it will run just like our application previously.
Now it is time to configure the read only entity manager.

package com.gkatzioura.springdatareadreplica.config;

import javax.sql.DataSource;

import org.springframework.beans.factory.annotation.Qualifier;
import org.springframework.beans.factory.annotation.Value;
import org.springframework.boot.jdbc.DataSourceBuilder;
import org.springframework.boot.orm.jpa.EntityManagerFactoryBuilder;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Primary;
import org.springframework.orm.jpa.LocalContainerEntityManagerFactoryBean;

@Configuration
public class ReadOnlyEntityManagerConfiguration {

    @Value("${spring.datasource.username}")
    private String username;

    @Value("${spring.datasource.password}")
    private String password;

    @Value("${spring.datasource.readUrl}")
    private String readUrl;

    @Bean
    public DataSource readDataSource() throws Exception {
        return DataSourceBuilder.create()
                                .url(readUrl)
                                .username(username)
                                .password(password)
                                .driverClassName("org.postgresql.Driver")
                                .build();
    }

    @Bean
    public LocalContainerEntityManagerFactoryBean readEntityManagerFactory(
            EntityManagerFactoryBuilder builder,
            @Qualifier("readDataSource") DataSource dataSource) {
        return builder.dataSource(dataSource)
                      .packages("com.gkatzioura.springdatareadreplica")
                      .persistenceUnit("read")
                      .build();
    }

}

Also I will add a method to a controller in order to save the models.

package com.gkatzioura.springdatareadreplica.controller;

import java.util.List;

import org.springframework.http.HttpStatus;
import org.springframework.web.bind.annotation.GetMapping;
import org.springframework.web.bind.annotation.PostMapping;
import org.springframework.web.bind.annotation.RequestBody;
import org.springframework.web.bind.annotation.ResponseStatus;
import org.springframework.web.bind.annotation.RestController;

import com.gkatzioura.springdatareadreplica.entity.Employee;
import com.gkatzioura.springdatareadreplica.repository.EmployeeRepository;

@RestController
public class EmployeeContoller {

    private final EmployeeRepository employeeRepository;

    public EmployeeContoller(EmployeeRepository employeeRepository) {
        this.employeeRepository = employeeRepository;
    }

    @GetMapping("/employee")
    public List<Employee> getEmployees() {
        return employeeRepository.findAll();
    }

    @PostMapping("/employee")
    @ResponseStatus(HttpStatus.CREATED)
    public void addEmployee(@RequestBody Employee employee) {
        employeeRepository.save(employee);
    }

}

If you do try to add the an employee using the controller and then query the read database you shall see that no entry is being added at all.

So we have our primary entity manager up and running and we also have a secondary one. The secondary one is not used yet. The next blog focuses on putting the secondary read only entity manager in use.

Read replicas and Spring Data Part 2: Configuring the base project

In our previous post we set up multiple PostgreSQL instances with the same data.
Our next step would be to configure our spring project by using the both servers.

As stated previously we shall use some of the code taken from the Spring Boot JPA post, since we use exactly the same database.

This shall be our gradle build file

plugins {
	id 'org.springframework.boot' version '2.1.9.RELEASE'
	id 'io.spring.dependency-management' version '1.0.8.RELEASE'
	id 'java'
}

group = 'com.gkatzioura'
version = '0.0.1-SNAPSHOT'
sourceCompatibility = '1.8'

repositories {
	mavenCentral()
}

dependencies {
	implementation 'org.springframework.boot:spring-boot-starter-data-jpa'
	implementation 'org.springframework.boot:spring-boot-starter-web'
	implementation "org.postgresql:postgresql:42.2.8"
	testImplementation 'org.springframework.boot:spring-boot-starter-test'
}

Now let’s proceed on creating the model based on the table created on the previous blog.

package com.gkatzioura.springdatareadreplica.entity;

import javax.persistence.Column;
import javax.persistence.Entity;
import javax.persistence.GeneratedValue;
import javax.persistence.GenerationType;
import javax.persistence.Id;
import javax.persistence.Table;

@Entity
@Table(name = "employee", catalog="spring_data_jpa_example")
public class Employee {

    @Id
    @Column(name = "id")
    @GeneratedValue(strategy = GenerationType.IDENTITY)
    private Long id;

    @Column(name = "firstname")
    private String firstName;

    @Column(name = "lastname")
    private String lastname;

    @Column(name = "email")
    private String email;

    @Column(name = "age")
    private Integer age;

    @Column(name = "salary")
    private Integer salary;

    public Long getId() {
        return id;
    }

    public void setId(Long id) {
        this.id = id;
    }

    public String getFirstName() {
        return firstName;
    }

    public void setFirstName(String firstName) {
        this.firstName = firstName;
    }

    public String getLastname() {
        return lastname;
    }

    public void setLastname(String lastname) {
        this.lastname = lastname;
    }

    public String getEmail() {
        return email;
    }

    public void setEmail(String email) {
        this.email = email;
    }

    public Integer getAge() {
        return age;
    }

    public void setAge(Integer age) {
        this.age = age;
    }

    public Integer getSalary() {
        return salary;
    }

    public void setSalary(Integer salary) {
        this.salary = salary;
    }

}

And the next step is to create a spring data repository.

package com.gkatzioura.springdatareadreplica.repository;

import org.springframework.data.jpa.repository.JpaRepository;
import com.gkatzioura.springdatareadreplica.entity.Employee;

public interface EmployeeRepository extends JpaRepository<Employee,Long> {
}

Also we are going to add a controller.

package com.gkatzioura.springdatareadreplica.controller;

import java.util.List;

import org.springframework.web.bind.annotation.RequestMapping;
import org.springframework.web.bind.annotation.RestController;

import com.gkatzioura.springdatareadreplica.entity.Employee;
import com.gkatzioura.springdatareadreplica.repository.EmployeeRepository;

@RestController
public class EmployeeContoller {

    private final EmployeeRepository employeeRepository;

    public EmployeeContoller(EmployeeRepository employeeRepository) {
        this.employeeRepository = employeeRepository;
    }

    @RequestMapping("/employee")
    public List<Employee> getEmployees() {
        return employeeRepository.findAll();
    }

}

All that it takes is to just add the right properties in you application.yaml

spring:
  datasource:
    platform: postgres
    driverClassName: org.postgresql.Driver
    username: db-user
    password: your-password
    url: jdbc:postgresql://127.0.0.2:5432/postgres

Spring boot has made it possible nowadays not to bother with any JPA configurations.

This is all you need in order to run the application. Once your application is running just try to fetch the employees.

curl http://localhost:8080/employee

As you have seen we did not do any JPA configuration. Since Spring Boot 2 specifying the database url is sufficient for the auto configuration to kick in and do all this configuration for you.

However in our case we want to have multiple datasource and entity manager configurations. In the next post we shall configure the entity managers for our application.

Read replicas and Spring Data Part 1: Configuring the Databases

This is a series of blog posts on our quest to increase our application’s performance by utilizing read replicas.

For this project our goal is to set up our spring data application and use read repositories for writes and
repositories based on read replicas for reads.

In order to simulate this environment we shall use PostgreSQL instances through Docker.

The motives are simple. Your Spring application has become increasingly popular and you want it to handle more requests. Most of the applications out there have a higher demand for read operations rather than write operations. Thus I assume that your application falls into the same category.
Although SQL databases are not horizontally scalable on their own, you can work you way with them by using read replicas.

Our goal is not to make an actual Read replication in PostgreSQL

thereforeinstead of configuring any replication

we will just copy some data from both databases

This is the script we shall use to populate the databases.

#!/bin/bash
set -e

psql -v ON_ERROR_STOP=1 --username "$POSTGRES_USER" --dbname "$POSTGRES_DB" &amp;lt;&amp;lt;-EOSQL
    create schema spring_data_jpa_example;

    create table spring_data_jpa_example.employee(
        id  SERIAL PRIMARY KEY,
        firstname   TEXT    NOT NULL,
        lastname    TEXT    NOT NULL,
        email       TEXT    not null,
        age         INT     NOT NULL,
        salary         real,
        unique(email)
    );

    insert into spring_data_jpa_example.employee (firstname,lastname,email,age,salary)
    values ('John','Doe 1','john1@doe.com',18,1234.23);
    insert into spring_data_jpa_example.employee (firstname,lastname,email,age,salary)
    values ('John','Doe 2','john2@doe.com',19,2234.23);
    insert into spring_data_jpa_example.employee (firstname,lastname,email,age,salary)
    values ('John','Doe 3','john3@doe.com',20,3234.23);
    insert into spring_data_jpa_example.employee (firstname,lastname,email,age,salary)
    values ('John','Doe 4','john4@doe.com',21,4234.23);
    insert into spring_data_jpa_example.employee (firstname,lastname,email,age,salary)
    values ('John','Doe 5','john5@doe.com',22,5234.23);
EOSQL

Since we shall use and Docker and Docker Compose the script above shall be used in order to initialize the database.
Now on to create our Docker Compose stack.

version: '3.5'

services:
  write-db:
    image: postgres
    restart: always
    environment:
      POSTGRES_USER: db-user
      POSTGRES_PASSWORD: your-password
      POSTGRES_DB: postgres
    networks:
      - postgresql-network
    ports:
      - "127.0.0.2:5432:5432"
    volumes:
      - $PWD/init-db-script.sh:/docker-entrypoint-initdb.d/init-db-script.sh
  read-db-1:
    image: postgres
    restart: always
    environment:
      POSTGRES_USER: db-user
      POSTGRES_PASSWORD: your-password
      POSTGRES_DB: postgres
    networks:
      - postgresql-network
    ports:
      - "127.0.0.3:5432:5432"
    volumes:
      - $PWD/init-db-script.sh:/docker-entrypoint-initdb.d/init-db-script.sh
networks:
  postgresql-network:
    name: postgresql-network

As you see our configuration is pretty simple. If you are careful enough you would see that I gave the number one to the read-db. This is because in the future we will add more replicas to it.

What I also did is bounding the machines to different local ips.

If you have problem binding addresses like 127.0.0.*:5432
You should try

sudo ifconfig lo0 alias 127.0.0.2 up
sudo ifconfig lo0 alias 127.0.0.3 up

If you are unsuccessful then just change the ports and it will work. It might not be as convenient but it’s still ok.

So let’s get up and running our Docker Compose stack.

docker-compose -f ./postgresql-stack.yaml up

We must be able to query data in both postgresql instances.

docker exec -it deploy_read-db-1_1 /bin/bash
root@07c502968cb3:/# psql -v --username "$POSTGRES_USER" --dbname "$POSTGRES_DB"
db-user=# select*from spring_data_jpa_example.employee;
 id | firstname | lastname |     email     | age | salary
----+-----------+----------+---------------+-----+---------
  1 | John      | Doe 1    | john1@doe.com |  18 | 1234.23
  2 | John      | Doe 2    | john2@doe.com |  19 | 2234.23
  3 | John      | Doe 3    | john3@doe.com |  20 | 3234.23
  4 | John      | Doe 4    | john4@doe.com |  21 | 4234.23
  5 | John      | Doe 5    | john5@doe.com |  22 | 5234.23
(5 rows)

We pretty much set up for our next step. We have some databases up and running and we are going to spin up a spring application running upon them. The next blog focuses on implementing an application running upon our primary database.

Spring Security with Spring Boot 2.0: Password Encoder

On a previous post we used the user details service in order to provide a way to load our data from a function based on a username given.

The implementation of the user details might be backed by an in-memory mechanism, a sql/no-sql database etc.
The options are unlimited.

What we have to pay attention when it comes to password storage is the password hashing.
For security reasons we want to store passwords in a hashed form.
Supposing someone gets unauthorised access to the table storing our user data. By storing the passwords clear text that person can retrieve the password of every user in the system.

So we want a way to hash our passwords before storing them to database.
Always be aware that your hashing has to be robust and up to date.
For example MD5 was very popular in the past but nowadays leads to poor security. Actually it is possible to crack MD5 passwords fairly easy if you use a gpu.

Spring Security provides us with out of the box functionality when it comes to encoding passwords.
Password encoder is an interface which is used through the authorisation process.


package org.springframework.security.crypto.password;


public interface PasswordEncoder {

	String encode(CharSequence rawPassword);

	boolean matches(CharSequence rawPassword, String encodedPassword);

}

The encode function shall be used to encode your password and the matches function will check if your raw password matches the encoded password. Once your user details service fetches the user information from the database then the password given to authorise shall be validated with the one fetched from the database. In this case spring will use the matches function.

Now spring provides us with various implementations of a password encoder.
Let’s try to create a password encoder bean.

package com.gkatzioura.security.passwordencoder.security;

import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
import org.springframework.security.crypto.password.PasswordEncoder;

@Configuration
public class PasswordEncoderConfig {

    @Bean
    public PasswordEncoder passwordEncoder() {
        return new PasswordEncoder() {
            @Override
            public String encode(CharSequence rawPassword) {
                return rawPassword.toString();
            }

            @Override
            public boolean matches(CharSequence rawPassword, String encodedPassword) {
                return rawPassword.toString().equals(encodedPassword);
            }
        };
    }
}

This bean is no different that the NoOpPasswordEncoder which comes with spring boot.
No we are going to do a small experiment and add a custom password encoder.
Our password encoder will compare the clear text password submitted by the user hash it and the compare it with an already hashed password from the equivalent user in our database.

To do the hashing we will user bcrypt.

    @Bean
    public PasswordEncoder customPasswordEncoder() {

        return new PasswordEncoder() {

            @Override
            public String encode(CharSequence rawPassword) {

                return BCrypt.hashpw(rawPassword.toString(), BCrypt.gensalt(4));
            }

            @Override
            public boolean matches(CharSequence rawPassword, String encodedPassword) {

                return BCrypt.checkpw(rawPassword.toString(), encodedPassword);
            }
        };
    }

To test this we will set up our security by using the environmental variables as we’ve seen on a previous post.

First we need to have our password encoded. Our system will not have the password stored in any clear text form.

System.out.println(BCrypt.hashpw("user-password",BCrypt.gensalt(4)));
$2a$04$i4UWtMw6surai4dQMhoKSeLddi1XlAh2sSyG58K3ZvBHqVkhz8Y3y

So what we are gonna do next is to set our environmental variables before running our spring boot application.

SPRING_SECURITY_USER_NAME=test-user
SPRING_SECURITY_USER_PASSWORD=$2a$04$i4UWtMw6surai4dQMhoKSeLddi1XlAh2sSyG58K3ZvBHqVkhz8Y3y

Next step is to go to your login screen and give the credentials user-name and user-password.
As you can see you have just been authenticated.
Behind the scenes spring hashed the password you submitted and compared to the one existing through the environmental varialbles.

Spring Security with Spring Boot 2.0: UserDetailsService

As we have seen on a previous post the username and password for our spring application was configured through environmental variables. This is ok for prototype purposes however in real life scenarios we have to provide another way to make the users eligible to login to the application.
To do so we use the UserDetailsService Interface.

The user details service comes with the loadUserByUsername function. The loadUserByUsername locates the user based on the username. The result of the search if existing then validates the credentials given through the login form with the user information retrieved through the UserDetailsService.

So let’s start with a very simple custom user details service.

@Service
public class UserDetailsServiceImpl implements UserDetailsService {

    @Override
    public UserDetails loadUserByUsername(String username) throws UsernameNotFoundException {

        if(username.equals("test")) {

            return User.withDefaultPasswordEncoder()
                       .username("test")
                       .password("test")
                       .roles("test")
                       .build();
        } else {
            throw new UsernameNotFoundException();
        }
    }
}

As you can see the only user who is able to login is the one with the username test. Also spring provides us with a builder when it comes to user details. As a password encoder we have specified the default password encoder which is actually an encoder that does no password hashing at all since we provide the password clear-text.

Although the password encoder will be covered in another tutorial it is always good to remind that you should always hash the password stored in a database for security reasons.

Now do you need to add any extra information? Well no. Just having a bean that implements the UserDetailsService, in you spring context, is enough. Spring security will pick the UserDetailsService implementation you provided and this will be used to authenticate.

For example you can even provide the UserDetailsService by using the @Bean Configuration.

@Configuration
public class SecurityConfig {

    @Bean
    public UserDetailsService createUserDetailsService() {
        return new UserDetailsServiceImpl();
    }
    
}

By this way regardless where your store your user information whether it is on an sql database, a nosql-database or even a csv file the only thing that you have to do is in your loadUserByUsername to load the user and pass him back by creating a UserDetails object.

Spring Security with Spring Boot 2.0: Simple authentication using the Servlet Stack

Spring security is a great framework saving lots of time and effort from the developers. Also It is flexible enough to customize and bring it down to your needs. As spring evolves spring security involves too making it easier and more bootstrapping to setup up security in you project.

Spring Boot 2.0 is out there and we will take advantage of it for our security projects.

On this Project we aim at creating an as simple security backed project as possible. To get started we shall create a simple spring boot 2.0 project.

We can use the spring SPRING INITIALIZR application.

The end result of the project would be to have a spring boot 2 project with gradle.

buildscript {
	ext {
		springBootVersion = '2.0.1.RELEASE'
	}
	repositories {
		mavenCentral()
	}
	dependencies {
		classpath("org.springframework.boot:spring-boot-gradle-plugin:${springBootVersion}")
	}
}

apply plugin: 'java'
apply plugin: 'eclipse'
apply plugin: 'org.springframework.boot'
apply plugin: 'io.spring.dependency-management'

group = 'com.gkatzioura.security'
version = '0.0.1-SNAPSHOT'
sourceCompatibility = 1.8

repositories {
	mavenCentral()
}


dependencies {
	compile('org.springframework.boot:spring-boot-starter-security')
        compile('org.springframework.boot:spring-boot-starter-web')
	testCompile('org.springframework.boot:spring-boot-starter-test')
	testCompile('org.springframework.security:spring-security-test')
}

Now be aware that with Spring Boot 2 there are two stacks to go. Either the Servlet stack or the WebFlux reactive stack. On this tutorial we shall use the servlet stack. We will cover WebFlux on another tutorial.

Let’s go and add our first controller.

package com.gkatzioura.security.simple.controller;

import org.springframework.http.HttpStatus;
import org.springframework.http.ResponseEntity;
import org.springframework.web.bind.annotation.GetMapping;
import org.springframework.web.bind.annotation.RestController;

@RestController
public class HelloWorldController {

    @GetMapping("/hello")
    public ResponseEntity<String> hello(String name) {

        return new ResponseEntity<>("Hello "+name, HttpStatus.OK);
    }

}

If we try to access the endpoint http://localhost:8080/hello?name=john we will be presented with a login screen.
Thus including the security dependency in our project auto secures our endpoints and configures a user with a password.
In order to retrieve the password you can check at the login screen.
The username would be ‘user’ and the password will be the one that spring autogenerates.

Of course using an autogenerated password is not sufficient, thus we are going to provide the username and the password of our choice.

One of the ways to set your username and password on the application.yaml file

spring:
  security:
    user:
      name: test-user
      password: test-password

Now putting you passwords in the file system especially when not encrypted is not a good practice, let alone being uploaded in you version control since application.yaml is a source file. Also anyone with access to the binary can retrieve the username and password

Therefore instead of putting these sensitive information in the application.yaml file you can set them by using environmental variables.

So your environmental variables would be

SPRING_SECURITY_USER_NAME=test-user
SPRING_SECURITY_USER_PASSWORD=test-password

To sum up this was the easiest and fastest way to add security to your project.
On the next blog we will do the same but using the WebFlux reactive stack.

Spring boot and Apache Camel

As the world of software moves on, more complex systems are being developed, which have to integrate with each other. It started with SOA and it continues with microservices.

Camel is the number one integration tool that comes to my mind since nowadays spring boot with camel is a very strong combination.

 

Apache Camel

The first step is to include the camel dependencies to our spring project.

buildscript {
	ext {
		springBootVersion = '1.5.9.BUILD-SNAPSHOT'
	}
	repositories {
		mavenCentral()
		maven { url "https://repo.spring.io/snapshot" }
		maven { url "https://repo.spring.io/milestone" }
	}
	dependencies {
		classpath("org.springframework.boot:spring-boot-gradle-plugin:${springBootVersion}")
	}
}

apply plugin: 'java'
apply plugin: 'eclipse'
apply plugin: 'org.springframework.boot'

group = 'com.gkatzioura'
version = '0.0.1-SNAPSHOT'
sourceCompatibility = 1.8

repositories {
	mavenCentral()
	maven { url "https://repo.spring.io/snapshot" }
	maven { url "https://repo.spring.io/milestone" }
}


dependencies {
	compile('org.apache.camel:camel-spring-boot-starter:2.20.0')
	testCompile('org.springframework.boot:spring-boot-starter-test')
    testCompile('org.apache.camel:camel-test-spring:2.20.0')
}

In order to have a faster project setup from scratch you can always use the online spring initializer.

Now let’s add a simple route

package com.gkatzioura.springcamel.routes;

import org.apache.camel.builder.RouteBuilder;
import org.springframework.stereotype.Component;

@Component
public class TimerRoute extends RouteBuilder {

    public static final String ROUTE_NAME = "TIMER_ROUTE";

    @Override
    public void configure() throws Exception {
        from("timer:initial//start?period=10000")
                .routeId(ROUTE_NAME)
                .to("log:executed");
    }
}

We don’t have to worry about the camel context configuration since the Camel auto-configuration creates a SpringCamelContext for you and takes care of the proper initialization and shutdown of that context.

Also camel auto-configuration collects all the RouteBuilder instances from the Spring context and automatically injects them into the provided CamelContext. Thus we don’t have to register our routes to the CamelContext.

As you can see our route has a timer with a period of 10000 milliseconds which routes to a log endpoint. The log endpoint will print the executed string every 10000 milliseconds.

Keep in mind that if no routeId is specified, camel will assign a name on its own, therefore giving a name to our route definition is a good practice in case we want to retrieve the root definition.

In order for camel to stay up, we need to keep our main thread blocked. Thus we add this configuration to our application.yml file.

camel:
  springboot:
    main-run-controller: true

Instead of this we can include the spring-boot-starter-web dependency, but our application has as few dependencies as possible, and we need to keep it this way.

However the most difficult part in the integration with other systems is testing. Throughout the years there have been rapid advancements on testing and the tools that we use.
Camel also comes packaged with some great tools in order to unit test.

For example we will implement a test of the route specified previously.

@RunWith(CamelSpringBootRunner.class)
@SpringBootTest
public class SpringCamelApplicationTests {

    @EndpointInject(uri = MOCK_RESULT)
    private MockEndpoint resultEndpoint;

    @Autowired
    private CamelContext camelContext;

    @EndpointInject(uri = MOCK_TIMER)
    private ProducerTemplate producer;

    private static final String MOCK_RESULT = "mock:result";
    private static final String MOCK_TIMER = "direct:mock-timer";

    @Before
	public void setup() throws Exception {

	    camelContext.getRouteDefinition(TimerRoute.ROUTE_NAME)
                .autoStartup(true)
                .adviceWith(camelContext, new AdviceWithRouteBuilder() {
                    @Override
                    public void configure() throws Exception {
                        replaceFromWith(MOCK_TIMER);
                        interceptSendToEndpoint("log*")
                                .skipSendToOriginalEndpoint()
                                .to(MOCK_RESULT);
                    }
                });
    }

    @Test
    public void sendMessage() throws Exception {

        resultEndpoint.expectedMessageCount(1);
        producer.sendBody("A message");
        resultEndpoint.assertIsSatisfied();
    }

}

Let’s have a look on each part of the test.

Our JUnit runner of choice would be the CamelSpringBootRunner.class

@RunWith(CamelSpringBootRunner.class)

We inject a ProducerTemplate. The ProducerTemplate interface allows you to send message exchanges to endpoints in a variety of different ways to make it easy to work with Camel Endpoint instances from Java code.

Then we inject a MockEndpoint. The MockEndpoint will serve us by replacing the original endpoint. Then we will set the expected number of messages to be received. Once the processing is done we assert that the amount of received messages is satisfied.

On our setup method we will replace our original endpoint with the fake producer template endpoint. Thus our route will receive the events that we will issue from the ProducerTemplate.
Then we will also intercept the log endpoint and direct the message to the MockEndpoint previously specified.

So we ended up withe a camel application and a unit test for the route specified.
You can find the source code on github.

Spring and Threads: Transactions

In order to be able to use transactions with our thread we need to understand how transactions work with spring. Transaction information in spring is stored in ThreadLocal variables. Therefore these variables are specific for an ongoing transaction on a single thread.

icon-spring-framework

When it comes to an action run by a single thread the transaction gets propagated among the spring components called hierarchically.

Thus in case of a @Transactional  annotated service which spawns a thread, the transaction will not be propagated from the @Transactional service to the newly created thread. The result will be an error indicating that the transaction is missing.

Since the action that take place inside your thread, requires database access through jpa, a new transaction has to be created.

By looking at the @Transactional documentation  we can get more information on the transaction propagation types. The default propagation mode for @Transactional is REQUIRED.

Therefore by annotating a method with the @Transactional, a new transaction will be created and will be propagated to the other services called from our thread.

For example our async method can be annotated as Transactional


@Async
@Transactional
public void executeTransactionally() {
    System.out.println("Execute a transaction from the new thread");
}

The same applies for the method which will be invoked from the run function of a Runnable class. Although async is pretty simple to use, behind the scenes it wraps the call in a Runnable, which dispatched to an executor.

To sum up when it comes to work with threads and transaction in spring, it should be done with extra care. Also keep in mind that the transactions cannot be passed from thread to thread. Last but not least make sure that your @Async and @Transactional functions are public and go though the proxy that will make the necessary actions before being invoked.

Last but not least I’ve compiled a cheat sheet that lists some helpful spring & threads tips.
Sign up in the link to receive it.

Spring and Threads: Async

Previously we started working with spring and the TaskExecutor, thus we became more familiar on how to use threads on a spring application.

However using the task executor might be cumbersome especially when we need to execute a simple action.

Spring’s Asynchronous methods come to the rescue.

Instead of messing with runnables and the TaskExecutor, you trade the control of the executor for the simplicity of the async functions.
In order to execute your function in another thread all you have to do is to annotate your functions with the @Async annotation.

Asynchronous methods come with two modes.

A fire and forget mode: a method which returns a void type.

    @Async
    @Transactional
    public void printEmployees() {

        List<Employee> employees = entityManager.createQuery("SELECT e FROM Employee e").getResultList();
        employees.stream().forEach(e->System.out.println(e.getEmail()));
    }

A results retrieval mode: a method which returns a future type.

    @Async
    @Transactional
    public CompletableFuture<List<Employee>> fetchEmployess() {
        List<Employee> employees = entityManager.createQuery("SELECT e FROM Employee e").getResultList();
        return CompletableFuture.completedFuture(employees);
    }

Pay extra attention to the fact that @Async annotations do not work if they are invoked by ‘this’. @Async behaves just like the @Transactional annotation. Therefore you need to have your async functions as public. You can find more information on the aop proxies documentation.

However using only the @Async annotation is not enough. We need to enable Spring’s asynchronous method execution capability by using the @EnableAsync annotation in one of our configuration classes.

package com.gkatzioura.config;

import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
import org.springframework.core.task.TaskExecutor;
import org.springframework.scheduling.concurrent.ThreadPoolTaskExecutor;

import java.util.concurrent.Executor;

/**
 * Created by gkatzioura on 4/26/17.
 */
@Configuration
@EnableAsync
public class ThreadConfig {

    @Bean
    public TaskExecutor threadPoolTaskExecutor() {

        ThreadPoolTaskExecutor executor = new ThreadPoolTaskExecutor();
        executor.setCorePoolSize(4);
        executor.setMaxPoolSize(4);
        executor.setThreadNamePrefix("sgfgd");
        executor.initialize();

        return executor;
    }

}

The next question is how we declare the resources and the threads pools that the async functions will use. We can get the answer from the documentation.

By default, Spring will be searching for an associated thread pool definition: either a unique TaskExecutor bean in the context, or an Executor bean named “taskExecutor” otherwise. If neither of the two is resolvable, a SimpleAsyncTaskExecutor will be used to process async method invocations.

However in some cases we don’t want the same thread pool to run all of application’s tasks. We might want separate threads pools with different configurations backing our functions.

To achieve so we pass to the @Async annotation the name of the executor we might want to use for each function.

For example  an executor with the name ‘specificTaskExecutor’ is configured.

@Configuration
@EnableAsync
public class ThreadConfig {

    @Bean(name = "specificTaskExecutor")
    public TaskExecutor specificTaskExecutor() {

        ThreadPoolTaskExecutor executor = new ThreadPoolTaskExecutor();
        executor.initialize();
        return executor;
    }

}

Then our function should set the qualifier value to determine the target executor of a specific Executor or TaskExecutor.

@Async("specificTaskExecutor")
public void runFromAnotherThreadPool() {
    System.out.println("You function code here");
}

On the next article we will talk about transactions on threads.

You can find the sourcecode on github.

Last but not least I’ve compiled a cheat sheet that lists some helpful spring & threads tips.
Sign up in the link to receive it.