SQL Data Access in Play using Scala

Today’s modern application frameworks come with a promise of easy sql data access. There is no wonder why we have so many frameworks that make it easier to issue queries and handle transactions. SQL is the lingua franca of most applications when it comes to databases.

Play comes with the JDBC plugin. We encountered the JDBC plugin previously in order to modify our database schema.

The first step is to include the jdbc and the evolutions module.

libraryDependencies += evolutions
libraryDependencies += jdbc

Then we shall define the connection string needed. We will use a simple h2 database. The configuration is added at the application.conf.

db.default.driver=org.h2.Driver
db.default.url="jdbc:h2:/tmp/defaultdatabase"

Then we add a script that creates the users table.

# Users schema

# --- !Ups

CREATE TABLE users (
    id bigint(20) NOT NULL AUTO_INCREMENT,
    email varchar(255) NOT NULL,
    first_name varchar(255) NOT NULL,
    last_name varchar(255) NOT NULL,
    PRIMARY KEY (id),
    UNIQUE KEY (email)
);

# --- !Downs

DROP TABLE users;

Before creating our repository class let’s check what the jdbc plugin provides us with.

We have the plain getConnection method, responsible for returning a jdbc connection. This is similar to the DataSource.getConnection from Java. Thus pay extra attention since you must close the connection.

val connection = db.getConnection()
connection.close()

Next method is withConnection. By using withConnection you get Play to manage the connection for you. All you have to do is pass a block of code with jdbc actions.

  def fetchUsers(): List[User] = {

    db.withConnection { conn =>

      val stmt = conn.createStatement
      var rs = stmt.executeQuery("SELECT*FROM users");
      val listBuffer = ListBuffer[User]()

      while(rs.next()) {

        listBuffer.append(User(Option(rs.getLong("id")),rs.getString("email"),rs.getString("first_name"),rs.getString("last_name")))
      }

      listBuffer.toList
    }
  }

As you can see above, we’ve just returned back a list of our user entries.

And last but not least withTransaction. You’ve guessed right, what you receive back is a connection with autocommit set to false.

  def addUser(user:User): User = {

    db.withTransaction { conn =>
      val stmt = conn.createStatement

      val insertQuery = "INSERT INTO users ( email, first_name, last_name) VALUES( '"+user.email+"', '"+user.firstName+"','"+user.lastName+"') "
      stmt.executeUpdate(insertQuery,Statement.RETURN_GENERATED_KEYS)
      val resultSet = stmt.getGeneratedKeys;
      if(resultSet.next()) {
        val id = resultSet.getLong(1);
        new User(Option(id),user.email,user.firstName,user.lastName)
      } else {
        throw new Exception("User not persisted properly")
      }
    }
  }

In the above example a user is persisted. In case of failure we throw an exception and the transaction is rolled back.

To sum up we have just checked how to access a sql database using play. Also we have checked the extra functions that play api provides apart from the familiar jdbc api.
That’s all for now! Feel free to check the code on github.

Advertisement

Use JSON with Play and Scala

Once getting your hands into typing scala code using play, the first thing that comes to mind is JSON.
Without doubt JSON is one of the most basic components of web applications. Rest apis use json, your angular app has to consume json and the list goes on.

If you are lazy like me, you expect that it is sufficient to just pass back scala objects through your controller or specify a scala class as an argument to your controller. Somehow things don’t get far from that however some adjustments have to be done.

The first step is to specify the json module

libraryDependencies += json

The JSON library is pretty similar to the org.json library for java but with extra capabilities. The types we have out of the box are

JsString
JsNumber
JsBoolean
JsObject
JsArray
JsNull

However the key functionality comes from the Reads and Writes converters which can be used to marshal or unmarshal our data structures.

Suppose we have a class called User

case class User(id:Option[Long],email:String,firstName:String,lastName:String)

We want to use this class to pass data to our controllers or use it as a response, once our action has finished.

Thus we need to create a Reader and writer for the User object.

  implicit val userWrites = new Writes[User] {
    def writes(user: User) = Json.obj(
      "id" -> user.id,
      "email" -> user.email,
      "firstName" -> user.firstName,
      "lastName" -> user.lastName
    )
  }

  implicit val userReads: Reads[User] = (
    (__ \ "id").readNullable[Long] and
      (__ \ "email").read[String] and
      (__ \ "firstName").read[String] and
      (__ \ "lastName").read[String]
    )(User.apply _)

Most probably you’ve noticed that the id is optional. We do so in order to be able to either pass the id of the user or not.

Now let’s put them together in a controller.

package controllers

import javax.inject.Inject

import com.google.inject.Singleton
import play.api.libs.json._
import play.api.mvc.{Action, Controller}
import play.api.libs.functional.syntax._

/**
  * Created by gkatzioura on 4/26/17.
  */
case class User(id:Option[Long],email:String,firstName:String,lastName:String)

@Singleton
class UserController @Inject() extends Controller {

  def all = Action { implicit request =>
    val users = Seq(
      User(Option(1L),"gkazoura@example.com","Emmanouil","Gkatziouras"),
      User(Option(2L),"john@doe.com","John","Doe"),
      User(Option(3L),"john2@doe.com","John2","Doe2")
    )
    Ok(Json.toJson(users))
  }

  def greet = Action

  def add = Action { implicit request =>

    val user  = Json.fromJson[User](request.body.asJson.get).get
    val newUser = User(Option(4L),user.email,user.firstName,user.lastName)
    Ok(Json.toJson(newUser))
  }

  implicit val userWrites = new Writes[User] {
    def writes(user: User) = Json.obj(
      "id" -> user.id,
      "email" -> user.email,
      "firstName" -> user.firstName,
      "lastName" -> user.lastName
    )
  }

  implicit val userReads: Reads[User] = (
    (__ \ "id").readNullable[Long] and
      (__ \ "email").read[String] and
      (__ \ "firstName").read[String] and
      (__ \ "lastName").read[String]
    )(User.apply _)
}

And also the roots configuration

GET     /user/                   controllers.UserController.all
POST    /user/                   controllers.UserController.add

As we can see the all method returns a list of user objects in Json format while the add method is supposed to persist a user object and assign an id to it.

Let’s do a curl request and check our results

curl http://localhost:9000/user/

....

[{"id":1,"email":"gkazoura@example.com","firstName":"Emmanouil","lastName":"Gkatziouras"},{"id":2,"email":"john@doe.com","firstName":"John","lastName":"Doe"},{"id":3,"email":"john2@doe.com","firstName":"John2","lastName":"Doe2"}]

curl -H "Content-Type: application/json" -X POST -d '{"email":"emmanouil@egkatzioura.com","firstName":"Emmanouil","lastName":"Gkatziouras"}' http://localhost:9000/user/

...

{"id":4,"email":"emmanouil@egkatzioura.com","firstName":"Emmanouil","lastName":"Gkatziouras"}

So we didn’t get into any special json handling or reading instead we used only objects.
That’s it! Now your are ready for more JSON related action!

You can check the sourcecode on github.

A Java gist on zip and unzip

Compressing is one of the major actions that can be issued in our code when it comes to writing files. Thus I find a simple java snippet on zip and unzip essential and  has to be easily accessed.

This gist is in plain java and stores two files in a zip. Once done the produced zip is open and its contents are evaluated.


import org.apache.commons.io.IOUtils;
import org.junit.Assert;
import org.junit.Test;
import java.io.*;
import java.nio.file.Files;
import java.util.zip.ZipEntry;
import java.util.zip.ZipInputStream;
import java.util.zip.ZipOutputStream;
/**
* Created by gkatzioura on 4/12/17.
*/
public class ArhivingTest {
private static final String TEXT_ENTRY_1 = "text1.txt";
private static final String TEXT_ENTRY_2 = "text2.txt";
@Test
public void zipAndUnzip() throws IOException {
String text1 = "Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do ";
String text2 = "eiusmod tempor incididunt ut labore et dolore magna aliqua. ";
File tempZip = File.createTempFile("temp",".zip");
try(OutputStream outputStream = new FileOutputStream(tempZip);
ZipOutputStream zipOutputStream = new ZipOutputStream(outputStream)) {
zipOutputStream.putNextEntry(new ZipEntry(TEXT_ENTRY_1));
zipOutputStream.write(text1.getBytes());
zipOutputStream.closeEntry();
zipOutputStream.putNextEntry(new ZipEntry(TEXT_ENTRY_2));
zipOutputStream.write(text2.getBytes());
zipOutputStream.closeEntry();
}
try(InputStream inputStream = new FileInputStream(tempZip);
ZipInputStream zipInputStream = new ZipInputStream(inputStream)) {
ZipEntry entry = null;
while ((entry=zipInputStream.getNextEntry())!=null) {
if(entry.getName().equals(TEXT_ENTRY_1)) {
Assert.assertEquals(text1, IOUtils.toString(zipInputStream));
}
if(entry.getName().equals(TEXT_ENTRY_2)) {
Assert.assertEquals(text2,IOUtils.toString(zipInputStream));
}
zipInputStream.closeEntry();
}
} finally {
Files.deleteIfExists(tempZip.toPath());
}
}
}

view raw

gistfile1.txt

hosted with ❤ by GitHub

Database Initialization with play and Scala

Once starting your play prototype application one of the priorities is to initialize your database and also manage the database schema changes.

Play provides us with evolutions. By utilizing evolutions we are able to create our database and to manage any futures changes to the schema.

To get started we need  to add the jdbc dependency and the evolutions dependency.

libraryDependencies += evolutions
libraryDependencies += jdbc

Then we shall use a simple h2 database persisted on disk, as our play application’s default database.
We edit the conf/application.conf file and add the following lines.

db.default.driver=org.h2.Driver
db.default.url="jdbc:h2:/tmp/defaultdatabase"

Pay extra attention that our database location is at the tmp directory thus all change shall be deleted once we reboot our workstation.

Once we have configured our database we are ready to create our first sql statement.
Our scripts should be located at the conf/evolutions/{your database name} directory, thus in our case
/conf/evolutions/default.

Our first script ‘1.sql’, shall create the users table.

# Users schema

# --- !Ups

CREATE TABLE users (
    id bigint(20) NOT NULL AUTO_INCREMENT,
    email varchar(255) NOT NULL,
    first_name varchar(255) NOT NULL,
    last_name varchar(255) NOT NULL,
    PRIMARY KEY (id),
    UNIQUE KEY (email)
);

# --- !Downs

DROP TABLE users;

As we can see we got ups and downs. What do they stand for? As you have guessed ups describe the transformations while downs describe how to revert them.
So the next question would be, how this functionality comes in use?
Suppose you have two developers working on the 2.sql. Locally they have successfully migrated their database once they are done, however the merge result is far different than the file they executed on their database.
What evolutions do is detect if the file is different and reverts the old revision by applying downs and then applying the up to date revision.

Now we are all set to run our application.

sbt run

Once we navigate at localhost:9000 we shall be presented with a screen that forces us to run the evolutions detected.

Let us go one step further and see what has been done to our database schema. We can easily explore a h2 database using dbeaver or your ide .
By issuing show tables the results contain one extra table.


>SHOW TABLES;

TABLE_NAME,TABLE_SCHEMA
PLAY_EVOLUTIONS,PUBLIC
USERS,PUBLIC

The PLAY_EVOLUTIONS table keeps track of our changes

Id is the number of the evolution script that we created. The fields apply and revert are the ups and downs sql statements we created previously.
The field hash is used in order to detect changes to our file. In case of an evolution that has a different hash from the one applied the previous evolution is reverted and applies the new script.

For example let’s enhance our previous script and add one more field. The field username.

# Users schema

# --- !Ups

CREATE TABLE users (
    id bigint(20) NOT NULL AUTO_INCREMENT,
    email varchar(255) NOT NULL,
    username varchar(255) NOT NULL,
    first_name varchar(255) NOT NULL,
    last_name varchar(255) NOT NULL,
    PRIMARY KEY (id),
    UNIQUE KEY (email)
);

# --- !Downs

DROP TABLE users;

Once we start our application we will be presented with a screen that forces us to issue an evolution for our different revision. If we hit apply the users table shall contain the username field.

So the process of a new revision is pretty straight forward.
The hash from the new 1.sql file is extracted. Then a query checks if the 1.sql file has already been applied. If it has been applied a check is issued in case the hashes are the same. If they are not then the downs script from the current database entry is executed. Once finished the new script is applied.

Your first Web application with Play and Scala

Today we are going to develop a simple play application using Scala.

To do so we must have sbt installed to our system.

Once installed we issue the command

sbt new playframework/play-scala-seed.g8

Then we are presented with an interactive terminal in order to pass valuable information.

name [play-scala-seed]: PlayStarter
organization [com.example]: com.gkatzioura
scala_version [2.11.8]: 
scalatestplusplay_version [2.0.0]: 
play_version [2.5.13]: 

Then let us check what we have just created

cd playstarter
sbt run

Navigate to http://localhost:9000 and you have a basic Play hello world.

By looking to our project structure, as expected, we have a directory with our controllers.
Consider our request being handled as an action. We issue a request and we receive an html view.

  def index = Action { implicit request =>
    Ok(views.html.index())
  }

As you can see we the html that is rendered is located at the views directory. Play comes with Twirl as a template engine.

At conf/routes we can see how the route is configured to the index action

Let’s add a simple action to that controller that returns a text body.

  def greet(name: String) = Action {
    Ok("Hello " + name)
  }

We have to edit the routes file to specify the new route and the get parameter

GET     /greet                      controllers.HomeController.greet(name)

Then issue a request at http://localhost:9000/greet?john

On the next step we shall add a new route with a path param

Suppose we want to retrieve the total logins for a user.
We implement an action that send a fake number

  def loginCount(userId: String) = Action {
    Ok(14)
  }

And then we register the route

GET     /user/:userId/login/count          controllers.HomeController.loginCount(userId)

By issuing the request http://localhost:9000/user/18/login/count
we shall receive the number 14.

To sum up we just implemented our first Play application. We also implemented some basic actions to our controller and achieved to pass some path and request parameters.

A journey with Scala

To those who are regular visitors of this blog, it is well known that when it comes to developing code I am a Spring/Java guy. Also I use different technologies like node or python but this depends largely on the project’s needs.

Due to some recent projects and courses involving Spark, stumbling on Scala was inevitable. After some investigation I decided to adopt it, as one of my main tools and there are many reasons for that.

From a Java Developers perspective

  • It evolves faster
  • It has Flexible syntax
  • It is static typed
  • It is pretty recent thus exciting, but in a JVM flavor

I remember back then when I was anticipating the release of Java 7. Lambdas, streams and put some functional programming into action. All those features that Scala provided. Unfortunately lambdas were dropped from Java 7, and released as part of Java 8. Thus it took 3 years to get your hands on lambdas 😦

The Scala syntax is great and increases productivity. I really fancy the fact that you can skip a lot of boilerplate that your had to deal with java. Let alone the options like tuples, switches and parameter name specification on function calls. The list could go on and on.

I used both dynamic and static typed languages however by developing mainly on Java I am a bit biased. I believe that static typing is a good choice because you can detect errors early, build reliable code and increase maintainability. Also it makes collaboration with others much easier.

Every new technology is exciting and feels like a new toy. However adopting a new technology comes with the lack of libraries and frameworks (node anyone?) that were essential for your development process. Fortunately you can always bet on the JVM and use your Java libraries with your Scala source code.

From a ecosystem perspective

If you come from Java EE or Spring MVC, you have an already prooved and tested framework for your web application.  Luckily Play comes to the rescue. Is Play sufficient for all your needs? I doubt. But frameworks evolve and since we live in a  microservices-architecture era having some Java components does the work.

From  a project perspective.

The cloud and the overall technological burst has brought a variety of different types of applications. Developing applications has become more challenging and some of them involve big data processing, streaming and machine learning. Scala is first place on this type of applications, thus if your objective is to work on the big data world mastering Scala will definitely assist you.

It does the filtering for you. As mentioned previously Spring/Java can be used for various type of projects, batch Applications, CMS apps, Rest apis etc. Scala has a more specific identity and targets certain type of projects. By searching for opportunities and contracts that include Scala, you already have a sense of the project’s nature and the challenges ahead.

At last

Talk is cheap! There are many tutorials ahead to author and document my experience. Stay tuned for more Scala content.