Debugging your OSGi Application using OSGi console commands

This blog post is mainly targeted at the WSO2 Carbon developers who make use of OSGi console to debug their bundles/components during the runtime. I'm here assuming that you have the basic shell which you would find in equinox environment.
You can start any of the WSO2 server products (that is based on Carbon) with OSGi console enabled, by passing the system property 'osgiConsole' during the startup. To do so, start the carbon server with,
./wso2server.sh -DosgiConsole
After full server startup pressing enter in the command shell will give you the OSGi shell command prompt, which looks like;
osgi>

After that you can valid OSGi command using this shell. Let us explore some of those,
The below commands will help you while debugging bundles in 'Module Layer':

Command	Use-Case
install file:/<filePath of bundle jar>	Install bundle jar in to runtime
ss	To simply list down the bundles along with the life-cycle state we can use this command.
start <bundleId found using ‘ss’ command>	starting the bundle given by bundle ID.
b <bundleId>	Prints all the details attached with bundle including  importing packages, exporting packages, etc.
diag <bundleId>	If the bundle is still in installed state we can use this command to find the missing constraints which are stopping bundle going in to next lifecycle stage
p <packageName>	List all the bundles that import/export the given package name within the runtime.
stop <bundleId>	stopping the bundle given by bundle ID
uninstall <bundleId>	uninstalling the bundle

To debug OSGi services(Service Components) layer you can use the following commands.

Command	Use-Case
ls	list down all the service components found in this runtime along with their component ID and life-cycle status
enable <componentID>	enabling service components
comp <componentID>	Use this command to find unsatisfied services of the given component, which prevent it from going it to satisfied state.
ls  -c <bundleID>	same as the above command, but you can retrieve the information on your service components using the bundle ID instead of component ID
services	list down all the services in the runtime including services provided by the system bundle. You can use simple filters to narrow down your list. eg: To find out the bundles/serviceComponents that expose the OSGi service ‘ org.foo.bar’ you can use the command: >services (objectclass=org.foo.bar)    or >services (objectclass=*foo.bar)
disable <componentID>	disabling component  given by the component ID

You don’t have to memorize these commands; all the commands along with their documentation can be found using the OSGi console command ‘help’. This is just a quick reference guide. You can extend the available set of commands by writing your own command provider. Hope this helps.

Revert not-commited local changes of a file in Git

git checkout  'file-name'

just a small note

POST variables get dropped while going through the WSO2 ESB proxy ... ?

This is more of a small note in case I forget what i did while trying things out. First of all I created a ESB proxy for a back-end service that accepts post request and does some manipulation on the received request.

Once you create a ESB proxy by giving the above REST service as the service endpoint , the InSequence element of the proxy.xml will look like this.

<inSequence>
         <send>
            <endpoint name="endpoint_urn_uuid_xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx">
               <address uri="http://localhost/restServiceEndPoint" format="rest" />
            </endpoint>
         </send>
      </inSequence>

The problem is the configuration drops the POST variables while building the SOAP envelope. Some debugging in to the Axis2 code revealed that the transport is attaching the incoming variable list to the Axis2MessageContext, but the attached data structure is MultipleEntryHashMap(It removes the value from the data-structure once you access it) which is a Axis2 implementation. However I couldnot access the the stored variables in the data structure within a Custom mediator, (the data has been already consumed at that point, may be by once of the dispatchers.)

So the other option was to use binary relay. It worked fine for me. Here are the steps I followed to get it working. First you have to enable the binary relay builder / Formatter for the relevant application-content type (In my case XForm-URL-encoded).

Then remove the format="rest"  attribute from the address element (InSequence of proxy.xml)

Thats it, the binary relay builder wraps your request as it is and expand it back when it sends back to to actual service.

Delete a set of elements in a xml file

Recently I wanted to delete a certain search pattern in an XML file. Within the XML file I had a XML snippet;

and i wanted to delete the commented out part. I was able to do it with this, search replace command in vim.

:%s/<!--moved to nexus\_.\{-}-->

I got this to working by looking at the mail[1]
[1] http://osdir.com/ml/editors.vim/2002-06/msg00468.html
its a handy command once you couple it with a vim macro.

Plotting graphs for scientific papers with gnuplot

I was writing our first IEEE grade paper few days back. It was first written using MS word as my text editor. Now that it got selected to be published on the IEEE explore, we thought of preparing a latex based paper for final publication.

One of the major proplem was the image quality of some of the figures. I tackled that using the popular open source vector grphics editor - > Inkscape. With Inkscape you can prepare some high quality images.

The next problem was with the graph quality, which had been drawn using MS excel. The solution was to use GNUPlot tool. It is a nice tool especially crafted for complex scientific drawings. I used it as a simple data plotter. But it is capable of plotting scientific functions and many more things.

So here is how to plot a simple graph if have the plotting data available with you. Installing the program is pretty trivial. ;)  

Preparing the data file : plot.dat

you have to prepare a data file, that to be used by the gnuplot. Data should be delimited by a common delimiter. The default delimiter is the 'space'. Here is what a sample data file would look like.

writing gnuplot script : plot.script

it is possible to use gnuplot tool in the interactive mode. But is is often useful to store your command as a script file so that you can call the script file over and over again.

set xlabel "number of nodes" font "Times-New Roman,16"

set ylabel "speedup ratio" font "Times-New Roman,16"

set xrange [0:24]

set yrange [0:25]

set xtics out  0,3,24

set mxtics 3

set ytics in  0,5,25

set mytics 5

set key 9,22

plot 'plot.dat' using 1:2 title 'actual speedup' with linespoints , \

'plot.dat' using 1:3 title 'ideal speedup' with lines

The most of the above commands are self explanatory, such as xlabel, xrange, etc.

set xtics instruct the program to set the labeling number after each 3 units. The set mxtics instructs to place subdivisions within the each unit.

Generating the graph

open up a terminal and type gnuplot. assuming you have already installed gnuplot in your machine, you should see the interactive gnuplot terminal. Give the following command to prepare the the graph using our script file. 

gnuplot> call './plot.script'   

This should spawn up a new window containing our graph. :)

                

Gnuplot uses the concept of drivers to output its generated graphs. Here I have used the output of default driver in the above figure. By changing the terminal driver we can print directly to a printer, to a command shell or to a postscript file. When you are preparing the the graphs for a latex document you got to specify your output terminal as the postscript (ps). The generated .ps file, can be later converted in to ps2pdf utility in UNIX. The converted .pdf image can be inserted in to your latex document and can be compiled with pdfLatex.

I have attached the plot.dat file and the plot.script file I used to generate the above plot.  

How to consume admin services in WSO2 Carbon with your own front end.

WSO2 Carbon is a OSGI based middle ware platform which powers the WSO2 products such as WSO2 Application server, WSO2 ESB, etc. In other words, Carbon provides core functionalities such as user management, feature management, registry services, etc which are essential part of any product. Product themselves add their unique behaviour via specific features (bundles).

The Carbon platform consists of a front-end and a back-end. In its most simplest deployment scenario you can deploy any carbon based WSO2 product  in a single web container such as Apache Tomacat. But its architecture allows you to deploy its front end as a separate web-app. This facility is immensely useful if you are working multiple instances of the product. Instead of having a separate front end for each and every instance you can use only one front-end to control all the deployed back-ends. The figure below gives a high level view of the carbon platform.

During a feature development for the WSO2 Carbon, I wanted to give some customized web services calls to the back end server. Since the existing management console does not have the functionality that I'm testing I had to write some code to give my own web service calls. Nothing new; just web services calls. Here I'm documenting my effort.

Running the WSO2 carbon in Standalone mode.

Download the binary distribution of Carbon from the project home, unzip it to your local file system. Root directory of the unzipped carbon directory is the CARBON_HOME. Open up a shell and within the $CARBON_HOME/bin , run the server as,

$> ./wso2server.sh

Log in to the system by giving the username and password (default being admin for both). You should see the bare bones carbon platform with the user manager and the feature manager.

Carbon admin services

The Carbon admin services are not visible via the Carbon management console. You can access the service description of the admin services if you know the admin service. Here I'm accessing the URL assuming that the default https port in carbon (9443).

> https://localhost:9443/services/AuthenticationAdmin?wsdl

  

Before accessing any other Admin services we have to get the credentials by authenticating ourselves via the Admin service. Then we can use the token to authenticate ourselves while accessing other admin services.

Generating code for the admin services

we have to generate the client stubs to access the back end web-services. You can write your own web services clients using axis2 client API, but the wsdl2java tool hides all the complexity from the user and present you with a proxy to the back end service.

The stub generation happens during the project build process within the maven pom files. It uses the maven ant run plug in to execute the wsdl2java tool.

Accessing the Admin service and authenticating ourselves

we should provide the back end server URL , client trust store details in order to log in to the back end server via a web services. We use the session cookie returned by the service to authenticate ourselves in the following service calls.

1 // setting the system properties for javax.net.ssl
2         AuthenticationAdminServiceClient.setSystemProperties(SampleConstants.CLIENT_TRUST_STORE_PATH,
3                 SampleConstants.KEY_STORE_TYPE, SampleConstants.KEY_STORE_PASSWORD );
4         AuthenticationAdminServiceClient.init(authenticationAdminURL);
5         System.out.println("retrieving the admin cookie from the logged in session....");
6         adminCookie = AuthenticationAdminServiceClient.login(SampleConstants.HOST_NAME,
7                 SampleConstants.USER_NAME, SampleConstants.PASSWORD);

Accessing the Provisioning service and getting the list of installed features.

Then during the provisioning stub invocation we have to give the retrieved admin cookie with the its service URL. View the ProvisioningAdminService service descriptor by pointing your browser to: 

> https://localhost:9443/services/ProvisioningAdminService?wsdl

1 // initializing the provisioning admin client  using the cookie we got from the authentication client.
2         ProvisioningAdminServiceClient.init(provisioningAdmingServiceURL, adminCookie);
3         FeatureWrapper[] featureWrapper = ProvisioningAdminServiceClient.getInstalledFeatures();
4         System.out.println(" the installed feature list...");
5         for(FeatureWrapper fw :featureWrapper){
6             // getting the names of installed features..
7             System.out.println(fw.getWrappedFeature().getFeatureName() + "- -" + 
8                     fw.getWrappedFeature().getFeatureDescription());
9         }

Things to note

• we have to provide our client with Java key store files found under resources folder, inside CARBON_HOME.
• I have saved WSDL in the local file system, for code generation.
• The path the client trust-store is hard coded in the SampleConstatns.java file

The code I used to test the behaviour is attached with this blog post. Since it uses maven it should not be hard to create a IDE specific project files from the maven pom.xml files. :)

Summary

The Carbon architecture allows you to control the server back end with your own front end. This can be a web-console, standalone java-app, etc. Carbon is the base platform for all the wso2 server products. Hence the demonstrated approach hold true for any carbon based wso2 product.

Download the code for the carbon simple front end.