Application Profiling for Memory and Performance

Inspired by the experiences gathered during recent application profiling sessions at WSO2, we did a webinar on memory and performance tuning. Below are the slides.

Application Profiling for Memory and Performance from pradeepfn

Understanding Java NIO : My Notes

Why this blog post?

Recently I was working with a team to improve the mediation performance of Apache Synapse ESB. Java NIO was new to me when i took up the challenge. I spent few days learning I/O concepts and NIO particularly. Soon after my background reading period, I was pulled out to optimize Carbon kernel. Tough luck!. Then again I learnt few things and thought of compiling a blog post based on the initial research.

Basic steps in a request/response system.

In a typical request/response system eg: servers, messages undergo the following steps,

1. Input read/listening
2. Input Decode
3. Perform business Logic
4. output encode
5. output write/sending

The steps can overlap depending on the implementation details.

Blocking I/O

In a blocking I/O scenario, as the name implies, the processing thread is blocked on the message processing until the output is written back to the wire. In a typical scenario, each client connection get assigned to a separate thread. Since the threads are dependent on I/O performance, no matter how efficient you process the message, the system throughput is dependent on the I/O behaviour. If the network latency is very high, the system won’t scale with regard to number of requests it can serve even with a lower concurrency level.

Sample Code for Blocking I/O

        ServerSocket serverSocket = null;
        serverSocket = new ServerSocket(4444);
        Socket clientSocket = null;
        clientSocket = serverSocket.accept();
        PrintWriter out = new PrintWriter(clientSocket.getOutputStream(), true);
        BufferedReader in = new BufferedReader(
                new InputStreamReader(clientSocket.getInputStream()));
        String inputLine, outputLine;
        CustomProtocol customProtocol = new CustomProtocol();

        while ((inputLine = in.readLine()) != null) {
            outputLine = customProtocol.processInput(inputLine);
            out.println(outputLine);

Non Blocking I/O

In a non blocking I/O scenario, the partial read of the input is possible and the acceptor triggers an event when the input is available. Because of the eventing model, now the processing thread doesn’t have to blocking wait on the input.

Eg: A typical client worker thread blocks on I/O like below,

while (in.readLine()) != null)

Since input/output readiness is triggered by events, the worker threads can go back to the worker thread pool and handle another request/task which is in a ready state. Reactor acts as the event dispatcher. All the other components are handlers that registers themselves with the reactor for interested event.  For an example, Acceptor registers itself with the reactor for the event, ‘Operation_Accept’.

Some core semantics

Selectors are responsible for querying available events and keeping track of calling objects (Listeners). For an example non blocking socket channel registers itself with the selector for "OP_ACCEPT" event.

If a connection occurs (that is ACCEPT) the selector will pick it up and appropriate listener will get called.

        selector = Selector.open();
        serverSocketChannel = ServerSocketChannel.open();
        serverSocketChannel.socket().bind(new InetSocketAddress(port));
        serverSocketChannel.configureBlocking(false);
        SelectionKey selectionKey = serverSocketChannel.register(selector,SelectionKey.OP_ACCEPT);
        selectionKey.attach(new Acceptor(serverSocketChannel, selector));

During an event trig, the selector will return all the relevant selection keys, and we can retrieve the attached listener for the key and execute.

 
 while (!Thread.interrupted()) {
                selector.select();
                Set selected = selector.selectedKeys();
                Iterator it = selected.iterator();
                while (it.hasNext()) {
                    SelectionKey selectionKey = (SelectionKey)it.next();
                    // Our attached object implements runnable
                    Runnable runnable = (Runnable)selectionKey.attachment();
                    runnable.run();

Scaling
The above architecture diagram only depicts the one thread version of reactor pattern. It is possible to scale the above system by means of multiple reactors, thread pools for worker threads, etc. The configuration highly dependent on the target hardware platform.


Libraries support for NIO
Even though one can write a complete service platform using the basic NIO constructs found in Java, there are open-source libraries that abstract out some of the underlying complexities. For an example Apache Http-core project provides abstraction layer for HTTp based NIO usage. Apache synapse make use of Http-core for NIO transport.

Main resource
Scalable IO in Java (presentation slides) - by Doug Lea, State University of New York at Oswego

How to consume admin services in WSO2 Carbon with your own front end.

WSO2 Carbon is a OSGI based middle ware platform which powers the WSO2 products such as WSO2 Application server, WSO2 ESB, etc. In other words, Carbon provides core functionalities such as user management, feature management, registry services, etc which are essential part of any product. Product themselves add their unique behaviour via specific features (bundles).

The Carbon platform consists of a front-end and a back-end. In its most simplest deployment scenario you can deploy any carbon based WSO2 product  in a single web container such as Apache Tomacat. But its architecture allows you to deploy its front end as a separate web-app. This facility is immensely useful if you are working multiple instances of the product. Instead of having a separate front end for each and every instance you can use only one front-end to control all the deployed back-ends. The figure below gives a high level view of the carbon platform.

During a feature development for the WSO2 Carbon, I wanted to give some customized web services calls to the back end server. Since the existing management console does not have the functionality that I'm testing I had to write some code to give my own web service calls. Nothing new; just web services calls. Here I'm documenting my effort.

Running the WSO2 carbon in Standalone mode.

Download the binary distribution of Carbon from the project home, unzip it to your local file system. Root directory of the unzipped carbon directory is the CARBON_HOME. Open up a shell and within the $CARBON_HOME/bin , run the server as,

$> ./wso2server.sh

Log in to the system by giving the username and password (default being admin for both). You should see the bare bones carbon platform with the user manager and the feature manager.

Carbon admin services

The Carbon admin services are not visible via the Carbon management console. You can access the service description of the admin services if you know the admin service. Here I'm accessing the URL assuming that the default https port in carbon (9443).

> https://localhost:9443/services/AuthenticationAdmin?wsdl

  

Before accessing any other Admin services we have to get the credentials by authenticating ourselves via the Admin service. Then we can use the token to authenticate ourselves while accessing other admin services.

Generating code for the admin services

we have to generate the client stubs to access the back end web-services. You can write your own web services clients using axis2 client API, but the wsdl2java tool hides all the complexity from the user and present you with a proxy to the back end service.

The stub generation happens during the project build process within the maven pom files. It uses the maven ant run plug in to execute the wsdl2java tool.

Accessing the Admin service and authenticating ourselves

we should provide the back end server URL , client trust store details in order to log in to the back end server via a web services. We use the session cookie returned by the service to authenticate ourselves in the following service calls.

1 // setting the system properties for javax.net.ssl
2         AuthenticationAdminServiceClient.setSystemProperties(SampleConstants.CLIENT_TRUST_STORE_PATH,
3                 SampleConstants.KEY_STORE_TYPE, SampleConstants.KEY_STORE_PASSWORD );
4         AuthenticationAdminServiceClient.init(authenticationAdminURL);
5         System.out.println("retrieving the admin cookie from the logged in session....");
6         adminCookie = AuthenticationAdminServiceClient.login(SampleConstants.HOST_NAME,
7                 SampleConstants.USER_NAME, SampleConstants.PASSWORD);

Accessing the Provisioning service and getting the list of installed features.

Then during the provisioning stub invocation we have to give the retrieved admin cookie with the its service URL. View the ProvisioningAdminService service descriptor by pointing your browser to: 

> https://localhost:9443/services/ProvisioningAdminService?wsdl

1 // initializing the provisioning admin client  using the cookie we got from the authentication client.
2         ProvisioningAdminServiceClient.init(provisioningAdmingServiceURL, adminCookie);
3         FeatureWrapper[] featureWrapper = ProvisioningAdminServiceClient.getInstalledFeatures();
4         System.out.println(" the installed feature list...");
5         for(FeatureWrapper fw :featureWrapper){
6             // getting the names of installed features..
7             System.out.println(fw.getWrappedFeature().getFeatureName() + "- -" + 
8                     fw.getWrappedFeature().getFeatureDescription());
9         }

Things to note

• we have to provide our client with Java key store files found under resources folder, inside CARBON_HOME.
• I have saved WSDL in the local file system, for code generation.
• The path the client trust-store is hard coded in the SampleConstatns.java file

The code I used to test the behaviour is attached with this blog post. Since it uses maven it should not be hard to create a IDE specific project files from the maven pom.xml files. :)

Summary

The Carbon architecture allows you to control the server back end with your own front end. This can be a web-console, standalone java-app, etc. Carbon is the base platform for all the wso2 server products. Hence the demonstrated approach hold true for any carbon based wso2 product.

Download the code for the carbon simple front end.

so you want to learn Maven ???

Maven is the current trend among Java community. No surprises, as maven is a full project life-cycle management tool. The beauty of maven is, it standardizes all the project related activities. Where the sources should go, where the test resources should reside, etc. 

Once you learn how to use it, you will be able to work with any Java project that uses maven. The core part of maven only deals with parsing & manipulating XML s. The core of the maven is a platform which provides services to the plugins. The plugins takes care of the specific tasks. As an example the compiler plugin knows that it has to compile the sources under src/main/Java.

I myself did not pay much attention to maven at the beginning. I used to find a way around by referring to some one else's pom.xml . But then I understood that i cant get away with this tool.

So I  learnt it. Here are some valuable resources to learn maven. 

• The maven site : http://maven.apache.org

The site has a wealth of resources for learning maven. It is easy to get lost in the site though. The quick start guides are really good.The place is the premier place to learn about different maven plugins.

• The Maven by Example : book  

There is a book by Oreilly called Maven definitive guide. May be this book is bit outdated as well. The guys at Sonatype has divided this book in to two parts. The "Maven by Example" is a great place to learn maven(my personnel recommendation :)  ) It covers the basic design of maven platform & it is more example oriented. so you better start with that .....

maven by example : http://www.sonatype.com/books/mvnex-book/reference/public-book.html

• The maven complete Reference: book

If you are planning to go deep in to the maven world, this is the book. 

http://www.sonatype.com/books/mvnref-book/reference/public-book.html

happy coding!!!!