Saturday, November 26, 2016

How to profile JVM running on a remote server docker container.

On a Ubuntu desktop install JProfiler GUI from here.

Download script and run it.
http://download-keycdn.ej-technologies.com/jprofiler/jprofiler_linux_9_2.sh

Provide key if you have , else use evaluation.

Now on the Ubuntu server where docker container is running, you need to stop containers first and modify Dockerfile to add jprofiler

wget http://download-keycdn.ej-technologies.com/jprofiler/jprofiler_linux_9_2.tar.gz && \
tar -xzf jprofiler_linux_9_1_1.tar.gz -C /usr/local

Also expose the port 8849 from container to host so that you can connect from your desktop.

If you use docker-compose.yml, map the port in the service , which you want to connect.
For example

version: "2"
services:
  some_service:
    build: .
    ports:
      - "8849:8849"
    depends_on:
      - "db"
    entrypoint:


This will download jprofiler 9.2 and unpack it in /usr/local when you run docker container next time and map the port on host server.

Once docker container is up , you can ssh to it
docker exec -it /bin/bash
And just start the jprofiler using following command.
/usr/local/jprofiler9/bin/jpenable
You will be asked for 2 options , 1) GUI connect 2. Using config.xml

Use option 1 GUI connect
Now from Desktop you can open JProfiler UI and connect to server-ip:8849
You can profile the JVM you want.


Sunday, August 21, 2016

Quick Emulator ( qemu ) for Hardware Virtualization

Have you ever met a requirement where you had to install a windows on top of linux ? Here is a cool solution. QEMU is a free and open-source hosted hypervisor that performs hardware virtualization. Lets see how to install a windows server on a Ubuntu server.

This tutorial assumes that your hardware supports virtualization.
You can check by running following command on your linux terminal.
egrep -c '(vmx|svm)' /proc/cpuinfo

If output is 0 , then not supported , any value above 0 means it is supported.

1. Install qemu
apt-get install qemu-kvm

Once QEMU has been installed, it should be ready to run a guest OS from a disk image.

Now we will test with a tiny example.Download a small image of linux from here.
This is a Small Linux disk image containing a 2.6.20 Linux kernel, X11 and various utilities to test QEMU.

Now unzip archive
bzip2 -d linux-0.2.img.bz2
Note, that this command will not preserve original archive file.
To preserve the original archive, add the -k option:
bzip2 -dk linux-0.2.img.bz2
Now the image is ready.

you can issue the following command.
sudo qemu-system-x86_64 -display vnc=0.0.0.0:1 -smp cpus=2 -m 250M -machine pc-1.0,accel=kvm -net user,hostfwd=tcp::80-:80,hostfwd=tcp::3389-:3389 -net nic linux-0.2.img

This command creates a virtualization with 2 cpu , 250MB memory with VNC server running on it.
You can connect through any vnc client ( realvnc or tightvnc ) 0.0.0.0:1



Now to install a windows OS , we first need to create an empty image

sudo qemu-img create -f raw win.img 40G

This creates an image with 40GB hardisk and raw format.
Now we can install windows os by using the iso image like below.

sudo qemu-system-x86_64 -display vnc=0.0.0.0:1 -cdrom /media/windows_server_2012_r2_with_update_x64_dvd_6052708.iso -smp cpus=2 -m 16G -machine pc-1.0,accel=kvm /var/spool/win.img

Once you run the above command , a vnc server is started and you can connect from a vnc client on display 1 or 5901, depending on client and install the windows.



Enable IIS.
Enable RDP.
Shutdown VM.
Run Windows VM with IIS via:

sudo qemu-system-x86_64 -display vnc=0.0.0.0:1 -smp cpus=2 -m 16G -machine pc-1.0,accel=kvm -net user,hostfwd=tcp::80-:80,hostfwd=tcp::3389-:3389 -net nic /var/spool/win.img

This will launch the machine with 16GB ram , 40G Hardisk . You can connect from RDP and work on it.

Monday, December 7, 2015

Docker permission denied error with Jenkins in Ubuntu 16.04

Are you trying to run docker commands from a Jenkins shell script and getting permission problems ?

Warning: failed to get default registry endpoint from daemon
(Got permission denied while trying to connect to the Docker daemon socket at unix:///var/run/docker.sock:
Get http://%2Fvar%2Frun%2Fdocker.sock/v1.26/info: dial unix /var/run/docker.sock: connect: permission denied). Using system default: https://index.docker.io/v1/
Got permission denied while trying to connect to the Docker daemon socket at
unix:///var/run/docker.sock: Post http://%2Fvar%2Frun%2Fdocker.sock/v1.26/images/create?fromImage=node&tag=6.3: dial unix /var/run/docker.sock: connect: permission denied

Jenkins runs with user name "jenkins" . You have to add this user to docker group.

sudo usermod -aG docker jenkins

Restart the jenkins fromcommand line ( not jenkins UI )

sudo service jenkins restart


Friday, February 22, 2013

Memory Profiling with JProfiler ( Basics )


Let’s see how we can use JProfiler to fix the memory leaks of a Java Application. JProfiler is a Java profiling tool targeted at Java EE and Java SE applications.

Profile means “drawing of the outline of anything” or summarize (  Read interesting story about the origin of this word  ) . In software engineering,  profiling is a form of dynamic program analysis that measures, for example, the space (memory) or time complexity of a program, the usage of particular instructions, or frequency and duration of function calls.

Let us take an example of a Java Application and try to solve a simple memory leak with JProfiler.

1.      Install JProfiler.
2.      Create a Simple Eclipse RCP Application with a View which has a memory leak.
3.      Run JProfiler and identify the leak.
4.      Fix the leak.
5.      Run JProfiler again to verify that leak is fixed.

Install JProfiler.


Download JProfiler evaluation version http://www.ej-technologies.com/download/jprofiler/files

During installation integrate JProfiler with you eclipse.



Create a Simple Eclipse RCP Application View with a memory leak.


1. Launch your eclipse IDE with a fresh workspace.
2. Go to File --> New --> Plugin Project.


3. Give a name to the Project and Press Next.

4. Select Yes for Rich Client Application and Press Next.

5. Select RCP Application with a view as shown below and press next.



In the next window, select Add Branding and say Finish.

This will take you to a Plugin development Perspective and your package explorer view should look something like below.


Now to introduce a memory leak we have to make some changes to our RCP. 

Open Perspective.java and  under createInitialLayout() method, delete the following line.

           layout.setFixed(true);

This is to make sure that we should be able to close the view. By default it is fixed and not closable.

Now open ApplicationActionBarAdvisor.java and override the following method.

@Override
protected void fillMenuBar(IMenuManager menuBar) {

IMenuManager mainMenu = getActionBarConfigurer().getMenuManager();
MenuManager windowMenu = new MenuManager("&Window",IWorkbenchActionConstants.M_WINDOW);
mainMenu.add(windowMenu);
windowMenu.add(ActionFactory.RESET_PERSPECTIVE.create(getActionBarConfigurer().getWindowConfigurer().getWindow()));

super.fillMenuBar(menuBar);

}

This is to make sure that after closing the view, we can bring it back by selecting the Reset Perspective Menu Action. This we need to test memory leak.

Now to introduce the memory leak, add a new singleton class and keep a HashMap in this singleton which stores our view as key by a strong reference.

package rcpwithmemoryleak;
import java.util.HashMap;
import org.eclipse.ui.part.ViewPart;

public class MemoryLeakerSingleton {

private static MemoryLeakerSingleton singleton;
HashMap<ViewPart, String> viewMap = new HashMap<ViewPart, String>();

private MemoryLeakerSingleton() {
}

public static MemoryLeakerSingleton getInstance() {
       if (singleton == null) {
              singleton = new MemoryLeakerSingleton();
       }
       return singleton;
}

public void add(ViewPart v) {
       viewMap.put(v, "Test");
}

}

The class should look something like above.  Now we add our view to this map whenever we create the view. This can be done by adding a line of code in createPartControl method of View.java

MemoryLeakerSingleton.getInstance().add(this);

Note that this kind of scenario we face in the real word when we use listeners or singletons which have the UI objects Hash Map.   Now run your RCP application. Right Click on the plugin and say RunAs à EclipseApplication.  You should see the application as below.


Run JProfiler and identify the leak.

Come back to your eclipse IDE.

You should have a toolbar item called Attach with JProfiler as shown below. Using this  tool you can attach Profiler with your running RCP application.


Click the tool and launch the Jprofiler. Open the Jprofiler in new window. Enter the evaluation license key if you have got one.

Select the Process Name you want to profile and press OK. In our case it is eclipse application.


By default JProfiler shows all the classes in memory. We can view according packages as well by changing the aggregation level.


OK. Now we are interested to see our RCP classes. It is very easy to do that. There is a View Filter below the window where you can type the fully qualified class name or package name to see all the classes in that package.  In our case package name is rcpwithmemoryleak. When you type this it shows all the classes.

Note that we have once instance of rcpwithmemoryleak.View.

Now you come to the RCP application and close the view by pressing X mark on the view and then come to JProfiler and press Run GC button on the tool bar. Now ideally after closing the view, the instance should be collected by Garbage Collector, but you still see once instance of View as it is being held by out Singleton Leaker.

You can reactivate the view by pressing Window à Reset Perspective. Close again and Reactivate again. Do it for multiple times and check in the JProfiler by Running the GC.

The second picture below shows the leak. I closed and open the view 6 times and old instances are not cleaned up.

 

Memory Leak for View Object

Now to check who is holding the view, you can take the heap dump and analyze.
Right click on the object of interest and say “Show Selection in Heap Walker”. This will take you to the Heap Walker. Select the Reference Tab from the below Tab Bar and select Incoming References from the combo box.


 

You can clearly see here which is holding your View in the memory.  I used Show Paths to GC Root option to see this tree. It directly takes you to the GC root.

Fix the leak.


Now you can go to the exact problematic point and fix your memory leak.
In our case we can fix this in two ways.

1.      Provide remove method in the MemoryLeakerSingleton like below

public void remove(ViewPart v) {
                     viewMap.remove(v);
       }

           Call this from the dispose() method of View as shown below.

@Override
              public void dispose() {
                     MemoryLeakerSingleton.getInstance().remove(this);
                     super.dispose();
       }


2.    You can make you HashMap as WeakHashMap , then GC will automatically collect these weak references.

Run JProfiler again to verify that leak is fixed.


Now you can run the RCP again after the code changes and rerun the profiler and do the steps described above. You should not see any instances if the view is closed. Your leak is fixed!
-------------------------------------------------------------------------------------------------------------------------------------------------------------------

One can use a word perfectly well under most circumstances without knowing its formal dictionary definition, but knowing that definition will certainly allow you to use it better. Knowing its etymology may bring out more nuances and connotations to the current semantics that you didn't realize are currently there. Here I've tried to give the etymology of word 'profile'.

Let’s now try to understand the etymology of word Profile.  The word Profile is made up of two words: Pro and File. Pro has been derived from Sanskrit Pra (प्र) means forth. 

What about file ? Here we have a big story about it. 

There is a root word (धातु) in Sanskrit called Shi (शि) which means Sharpen (निशाने). Based on this root, Sanskrit has derived a word - Shira (शिरा) which means a thin vein. In Sanskrit there is a rule which says “No difference between R and L” (रलयोरभेदः). Thus Shira became Shila (शिला). R got kicked by L.  

Now Sanskrit Shila (शिला) became žila in Slovak and Czech (žila and shila are very close in pronunciation) carrying same meaning (vein) and in Spanish – filón. Other few languages used this word for calling a thread or string (since thread/ string resembles a vein). Check the words used for thread/string in different languages below.

Croatian          žica
Latin                filum
Italian              fila, filo, filza
Catalan            fil
Galician           fío
French             fil

English used the word file. Earlier the word file was used for communicating “a string or wire on which documents are strung”. Now the word file means a document holder.  

In this way we can say that the meaning of word Profile is before getting into details drawing an outline of the subject. 

The journey of the words is really amazing! 

---------------------------------------------------------------------------------------------------------