Category: Python

Embed Jython to you java codebase.

Jython is a great tool for some quick java scripts using a pretty solid syntax. Actually it works wonderfully when it comes to implement some maintenance or monitoring scripts with jmx for you java apps.

In case you work with other teams with a python background, it makes absolute sense to integrate python to your java applications.

First let’s import the jython interpeter using the standalone version. 

group 'com.gkatzioura'
version '1.0-SNAPSHOT'

apply plugin: 'java'

sourceCompatibility = 1.5

repositories {
    mavenCentral()
}

dependencies {
    testCompile group: 'junit', name: 'junit', version: '4.11'
    compile group: 'org.python', name: 'jython-standalone', version: '2.7.0'
}

So the easiest thing to do is just to execute a python file in our class path. The file would be hello_world.py

print "Hello World"

And then pass the file as an inputstream to the interpeter

package com.gkatzioura;

import org.python.core.PyClass;
import org.python.core.PyInteger;
import org.python.core.PyObject;
import org.python.core.PyObjectDerived;
import org.python.util.PythonInterpreter;

import java.io.InputStream;

/**
 * Created by gkatzioura on 19/10/2016.
 */
public class JythonCaller {

    private PythonInterpreter pythonInterpreter;

    public JythonCaller() {
        pythonInterpreter = new PythonInterpreter();
    }

    public void invokeScript(InputStream inputStream) {

        pythonInterpreter.execfile(inputStream);
    }

}

    @Test
    public void testInvokeScript() {

        InputStream inputStream = this.getClass().getClassLoader().getResourceAsStream("hello_world.py");
        jythonCaller.invokeScript(inputStream);
    }

Next step is to create a python class file and and another python file that will import the class file and instantiate a class.

The class file would be divider.py.

class Divider:

    def divide(self,numerator,denominator):

        return numerator/denominator;

And the file importing the Divider class would be classcaller.py

from divider import Divider

divider = Divider()

print divider.divide(10,5);

So let us test it 

    @Test
    public void testInvokeClassCaller() {

        InputStream inputStream = this.getClass().getClassLoader().getResourceAsStream("classcaller.py");
        jythonCaller.invokeScript(inputStream);
    }

What we can understand from this example is that the interpreter imports successfully the files from the classpath.

Running files using the interpreter is ok, however we need to fully utilize classes and functions implemented in python.
Therefore next step is to create a python class and use its functions using java.

package com.gkatzioura;

import org.python.core.PyClass;
import org.python.core.PyInteger;
import org.python.core.PyObject;
import org.python.core.PyObjectDerived;
import org.python.util.PythonInterpreter;

import java.io.InputStream;

/**
 * Created by gkatzioura on 19/10/2016.
 */
public class JythonCaller {

    private PythonInterpreter pythonInterpreter;

    public JythonCaller() {
        pythonInterpreter = new PythonInterpreter();
    }

    public void invokeClass() {

        pythonInterpreter.exec("from divider import Divider");
        PyClass dividerDef = (PyClass) pythonInterpreter.get("Divider");
        PyObject divider = dividerDef.__call__();
        PyObject pyObject = divider.invoke("divide",new PyInteger(20),new PyInteger(4));

        System.out.println(pyObject.toString());
    }

}

You can find the sourcecode on github.

Implement a SciPy Stack Docker Image

SciPy is a powerful python library, but it has many dependencies including Fortran.
So Running your Scipy code in a docker container makes absolute sense.

We will use a private registry

docker run -d -p 5000:5000 --name registry registry:2

I will use a Centos image.
Centos is a very popular linux distribution based on RedHat which is a commercial Linux distribution. Oracle’s Linux and Amazon Linux is based on Red Hat Linux. 

docker pull centos
docker tag centos localhost:5000/centos
docker push localhost:5000/centos

Then we start a container 

docker run -i -t --name centoscontainer localhost:5000/centos /bin/bash

We install all binary dependencies

yum install -y epel-release
yum -y update
yum -y groupinstall "Development Tools"
yum -y install python-devel
yum -y install blas --enablerepo=epel
yum -y install lapack --enablerepo=epel
yum -y install Cython --enablerepo=epel
yum -y install python-pip

Then we install the scipy stack

pip install boto3
pip install numpy
pip install pandas
pip install scipy

And we are ready. Now we should proceed on committing the image.

docker commit -m 'Added scipy stack' -a "Emmanouil Gkatziouras" 4954f603d93b localhost:5000/scipy
docker push localhost:5000/scipy

Now we are ok to run our SciPy enabled container.

docker run -t -i localhost:5000/scipy /bin/bash

Last but not least we clear our registry.

docker stop registry && docker rm -v registry

Connecting to JMX through Jython

Jython is great when you want a dynamically typed language based on the JVM.

Also comes really in handy when you want to write small monitoring scripts based on JMX.This is an examble on how to call a function from a MBean through Jython using JMX.

jmxaction.py

from javax.management.remote import JMXConnector
from javax.management.remote import JMXConnectorFactory
from javax.management.remote import JMXServiceURL
from javax.management import MBeanServerConnection
from javax.management import MBeanInfo
from javax.management import ObjectName
from java.lang import String

from jarray import array
import sys   

if __name__=='__main__':
        
        if len(sys.argv)> 5:
                serverUrl = sys.argv[1]
                username = sys.argv[2]
                password = sys.argv[3] 
                beanName = sys.argv[4]
                action = sys.argv[5]
        else:
                sys.exit(-1)
        credentials = array([username,password],String)
        environment = {JMXConnector.CREDENTIALS:credentials}

        jmxServiceUrl = JMXServiceURL('service:jmx:rmi:///jndi/rmi://'+serverUrl+':9999/jmxrmi');
        jmxConnector = JMXConnectorFactory.connect(jmxServiceUrl,environment);
        mBeanServerConnection = jmxConnector.getMBeanServerConnection()
        objectName = ObjectName(beanName);
        mBeanServerConnection.invoke(objectName,action,None,None)
        jmxConnector.close()

By calling the script

jython jmxaction.py {ip} {jmx user} {jmx password} {mbean name} {action}

You can invoke the action of the mbean specified.

Pyftpdlib : An ftp server python library

In one of my projects there was the need to provide an ftp file transfer. So instead of setting up a ftp server I decided to make one of my own using the pyftpdlib library and add custom actions once a file was received or transferred. The other advantage was the fact that the ftp server would have been part of the existing project’s python software.

Here’s an example in case you want a custom handler and you don’t have the time to read the ftpserver.py source.

from pyftpdlib import ftpserver
from threading import Thread

class PythoFtpServer(Thread):
    
    def __init__(self): 
        Thread.__init__(self)
        self.daemon = True
        authorizer = ftpserver.DummyAuthorizer()
        authorizer.add_user("usr","pwd", "adir",perm='elradfmw')
        handler = CustomFtpHandler
        handler.authorizer = authorizer
        address = ('127.0.0.1',1024)
        self.server = ftpserver.FTPServer(address, handler)

    def run(self):
        Thread.run(self)
        self.server.serve_forever()

class CustomFtpHandler(ftpserver.FTPHandler):
    
    def on_file_sent(self, file):
        """Called every time a file has been succesfully sent.
        "file" is the absolute name of the file just being sent.
        """

    def on_file_received(self, file):
        """Called every time a file has been succesfully received.
        "file" is the absolute name of the file just being received.
        """

    def on_incomplete_file_sent(self, file):
        """Called every time a file has not been entirely sent.
        (e.g. ABOR during transfer or client disconnected).
        "file" is the absolute name of that file.
        """

    def on_incomplete_file_received(self, file):
        """Called every time a file has not been entirely received
        (e.g. ABOR during transfer or client disconnected).
        "file" is the absolute name of that file.
        """

    def on_login(self, username):
        """Called on user login."""

    def on_login_failed(self, username, password):
        """Called on failed user login.
        At this point client might have already been disconnected if it
        failed too many times.
        """

    def on_logout(self, username):
        """Called when user logs out due to QUIT or USER issued twice."""

Thanks to Giampaolo Rodola for this library 🙂

Pyftpdib @ Goole code