Deploying a Java application accessing M7 native tables on Apache Tomcat (Linux)

Deploying a Java application accessing M7 native tables on Apache Tomcat(Linux)

In the tutorial below, we look at how to write a basic java program that reads data on an existing M7 table and then move on to deploy the same on Tomcat to display the results on a web page.

## Contents 1 Accessing the HBase table using APIs 1.1 Connect to the MapR cluster from Java 1.2 Basic elements of HBase access 1.3 Sample standalone program 1.4 Compiling and executing the code 2 Deploying on Apache Tomcat 2.1 Preparing a JSP page 2.2 Building the code 2.3 Deploy on Tomcat 3 Final Output * 3.1 Footnotes

Accessing the HBase table using APIs

Connect to the MapR cluster from Java

First lets define the mapr cluster we will be using to access the tables. Since I'm using only one cluster, we'll point the "default" setting rather than the clustername itself to point to the single CLDB node in my cluster. (See footnote 1)

 import com.mapr.fs.MapRFileSystem;
 MapRFileSystem mfs = new MapRFileSystem("default", new String[] {"10.10.80.91:7222"});

Basic elements of HBase access

Now lets go through the steps of accessing an HBase table from a java program.

  • Hadoop has a generic configuration object which we will use to construct the HBase configuration object. So first import the two Configuration classes and then create HBaseConfiguration object to tell your program (client) where to connect. The HBaseConfiguration reads hbase-site.xml on CLASSPATH and factors in hbase-default.xml if found (hbase-default.xml ships inside the hbase.X.X.X.jar).
 import org.apache.hadoop.conf.Configuration;
 import org.apache.hadoop.hbase.HBaseConfiguration;

  • Now create a conf object
 Configuration conf = HBaseConfiguration.create();

  • Create a table mapping between the namespace in MapRFS
 conf.set("hbase.table.namespace.mappings", "t12:/t12");

  • Create a byte class for the table name (HBase inputs are all in byte arrays)
 byte[] M7Table = Bytes.toBytes("t12");

  • Create an HTable object
 HTableInterface table1 = new HTable  (conf, M7Table);

  • Then retrieve the scan results using the HBase Scan object. This class is used to retrieve multiple rows and can be instantiated with start and stop rows or iterates over all rows if invoked without arguments. On the Scan instance, invoke the getScanner() method that return a ResultScanner instance.
 Scan scanobject = new Scan();
 ResultScanner rs = table1.getScanner(scanobject);

  • The ResultScanner is like a java iterator (iterate over Result instances) and is similar to a get operation. (In fact, a Get is implemented as a scan for a single row).The getScanner methods from HTable return ResultScanner instances where matching rows are returned on a row basis, wrapped in a Result object. So now display these rows accordingly using the Result Class. The Result instance wraps data retrieved for a single row from a get or a scan operation. For example, a string conversion of the Result object looks like this :
keyvalues={row1/cf1:col1/1300802024293/Put/vlen=7, row1/cf1:col2/1300802024325/Put/vlen=8}

Sample standalone program

Putting the above steps into a sample code, we now have a standalone java program that looks like this :

package M7Example;

import java.util.ArrayList;
import java.util.Arrays;
import java.io.IOException;

import org.apache.hadoop.conf.Configuration;
import org.apache.hadoop.hbase.HBaseConfiguration;
import org.apache.hadoop.hbase.HTableDescriptor;
import org.apache.hadoop.hbase.client.HTable;
import org.apache.hadoop.hbase.client.HTableInterface;
import org.apache.hadoop.hbase.client.Result;
import org.apache.hadoop.hbase.client.ResultScanner;
import org.apache.hadoop.hbase.client.Scan;
import org.apache.hadoop.hbase.util.Bytes;
import org.apache.hadoop.hbase.KeyValue;
import com.mapr.fs.MapRFileSystem;

public class M7ReadTomcat {

public static void main(String[] args){

try {

MapRFileSystem mfs = new MapRFileSystem("default", new String[] {"10.10.80.91:7222"});

Configuration conf = HBaseConfiguration.create();

byte[] M7Table = Bytes.toBytes("/t12");
HTableInterface table1 = new HTable  (conf, M7Table);

Scan scanobject = new Scan();
ResultScanner rs = table1.getScanner(scanobject);

try{    for (Result result: rs) {
                 for(KeyValue kv : result.raw()){
                System.out.print(new String(kv.getRow()) + " ");
                    System.out.print(new String(kv.getFamily()) + ":");
                        System.out.print(new String(kv.getQualifier()) + " ");
                         System.out.print(kv.getTimestamp() + " ");
                        System.out.println(new String(kv.getValue()));}
        }
   }finally {rs.close();}

System.out.println(Arrays.toString(args));
 table1.close();

} catch (IOException e) {
   e.printStackTrace();}

}

}

Compiling and executing the code

  • The code above would need the following libraries to compile successfully :
[admin@tomcat-server  src]$ ll ../WebContent/WEB-INF/lib/
total 28980
-rwxr-xr-x. 1 tomcat tomcat   575389 Mar  6 22:50 commons-collections-3.2.1.jar
-rwxr-xr-x. 1 tomcat tomcat   284220 Mar  6 22:47 commons-lang-2.6.jar
-rwxr-xr-x. 1 tomcat tomcat    38015 Mar  6 22:43 commons-logging-1.0.4.jar
-rwxr-xr-x. 1 tomcat tomcat  3925020 Mar  6 22:13 hadoop-0.20.2-dev-core.jar
-rw-r--r--. 1 tomcat tomcat  5435696 Mar  6 22:10 hbase-0.94.13-mapr-1401.jar
-rw-r--r--. 1 tomcat tomcat  1891110 Mar  6 22:24 guava-13.0.1.jar
-rwxr-xr-x. 1 tomcat tomcat  6245984 Mar  6 23:15 libprotodefs.jar
-rwxr-xr-x. 1 tomcat tomcat   391834 Mar  6 22:51 log4j-1.2.15.jar
-rwxr-xr-x. 1 tomcat tomcat 11218742 Mar  6 22:14 maprfs-1.0.3-mapr-3.0.2.jar
-rwxr-xr-x. 1 tomcat tomcat    60820 Mar  6 22:47 mapr-hbase-1.0.3-mapr-3.0.2.jar
-rwxr-xr-x. 1 tomcat tomcat   450280 Mar  6 23:15 protobuf-java-2.4.1.jar
-rwxr-xr-x. 1 tomcat tomcat  1031321 Mar  6 22:51 zookeeper-3.3.6.jar

  • So copy the above to any convenient location and compile the program as follows:
[admin@tomcat-server  src]$ javac -cp .:"lib/*" M7ReadTomcat.java
[admin@tomcat-server  src]$
[admin@tomcat-server  src]$ ll
total 16
-rw-rw-r--. 1 tomcat tomcat 3339 Mar 10 23:07 M7ReadTomcat.class
-rw-rw-r--. 1 tomcat tomcat 2499 Mar 10 14:56 M7ReadTomcat.java
[admin@tomcat-server  src]$

  • Now execute this and it will generate the following output on the console :
[admin@tomcat-server  src]$ java -cp .:"lib/*" M7ReadTomcat
14/03/10 23:07:44 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
14/03/10 23:07:44 INFO security.JniBasedUnixGroupsMappingWithFallback: Falling back to shell based
myrow-1 cf1:q1 1392056271991 value-1
myrow-2 cf1:q2 1392056272031 value-2
myrow-2 cf1:q3 1392056273245 value-3
myrow-3 cf1:q1 1392114845834 value-4
myrow-3 cf1:q2 1392114863733 value-5
myrow-4 cf1:q1 1392227075188 value-6
[]
[admin@tomcat-server  src]$

  • This corresponds correctly to the actual HBase table on M7.
hbase(main):002:0> scan "/t12"
ROW                                           COLUMN+CELL                                                                                                                       
 myrow-1                                      column=cf1:q1, timestamp=1392056271991, value=value-1                                                                             
 myrow-2                                      column=cf1:q2, timestamp=1392056272031, value=value-2                                                                             
 myrow-2                                      column=cf1:q3, timestamp=1392056273245, value=value-3                                                                             
 myrow-3                                      column=cf1:q1, timestamp=1392114845834, value=value-4                                                                             
 myrow-3                                      column=cf1:q2, timestamp=1392114863733, value=value-5                                                                             
 myrow-4                                      column=cf1:q1, timestamp=1392227075188, value=value-6                                                                             
4 row(s) in 0.0140 seconds

hbase(main):003:0>

Deploying on Apache Tomcat

Preparing a JSP page

  • Next, we need to make this a tomcat based java application. For this, we need to have the scan result returned as an arraylist rather than do a System.out.println. So I added another function that does the same but returns the scan results as an arraylist.

public ArrayList getDetails(){
    ArrayList<String> arrl = new ArrayList<String>();
try {

Configuration conf = HBaseConfiguration.create();
MapRFileSystem mfs = new MapRFileSystem("default", new String[] {"10.10.80.91:7222"});

HTableInterface table1 = new HTable  (conf, Bytes.toBytes("/t12"));
Scan scanobject = new Scan();
ResultScanner rs = table1.getScanner(scanobject);

try{    for (Result result: rs) {
          for(KeyValue kv : result.raw()){
          String str = new String(kv.getRow()) + " "+ new String(kv.getFamily()) + ":"+ new String(kv.getQualifier()) + " "+kv.getTimestamp() + " "+ new String(kv.getValue());
          arrl.add(str);
               }
        }
   }finally {rs.close();}

 table1.close();
return arrl;
} catch (IOException e) {
    arrl.add("Error occured "+e.toString());
   e.printStackTrace();}

return arrl;
}

  • Now retrieve this using a jsp page
[admin@tomcat-server  M7TomcatRead]$ cat WebContent/jsp/display.jsp
<%@ page import = "M7Example.M7ReadTomcat"%>
<%@ page import = "java.util.ArrayList" %>
<%@ page import = "java.util.Iterator" %>
<%@ page import = "org.apache.hadoop.hbase.HTableDescriptor" %>

<%
M7ReadTomcat ScanPage=null;
try {
ScanPage = new M7ReadTomcat();
ArrayList<String> arrl = ScanPage.getDetails();
 } catch(Exception e) {
out.println("<br><p>Phat gaya</p></br>");
}
%>

<html>
<title>M7 Scan Results from Tomcat</title>
<h1>Table Name :T12</h1>

<%
Iterator<String> itr = arrl.iterator();
        while(itr.hasNext()){
        %>
            <br><%=(String)itr.next();%></br>
<%            
        }
%>

</html>

Building the code

  • Next, we need to build a war file using the above to deploy on Tomcat. For this, we'll use a build file as follows:
[admin@tomcat-server  M7TomcatRead]$ cat build.xml
<?xml version="1.0" ?>
<project name="M7Example" default="war">
 <path id="compile.classpath">
   <fileset dir="WebContent/WEB-INF/lib">
     <include name="*.jar"/>
   </fileset>
 </path>
 <target name="init">
   <mkdir dir="build/classes"/>
   <mkdir dir="dist" />
 </target>
 <target name="compile" depends="init" >
   <javac destdir="build/classes" debug="true" srcdir="src">
       <classpath refid="compile.classpath"/>
   </javac>
 </target>
 <target name="war" depends="compile">
   <war destfile="dist/M7Example.war" webxml="WebContent/WEB-INF/web.xml">
     <fileset dir="WebContent"/>
     <lib dir="WebContent/WEB-INF/lib"/>
     <classes dir="build/classes"/>
   </war>
 </target>
 <target name="clean">
   <delete dir="dist" />
   <delete dir="build" />
 </target>
</project>

Create a directory structure as below :

[admin@tomcat-server  M7TomcatRead]$ tree
.
├── build.xml
├── src
│   ├── M7ReadTomcat.java
└── WebContent
    ├── jsp
    │   └── display.jsp
    └── WEB-INF
        ├── lib
        │   ├── commons-collections-3.2.1.jar
        │   ├── commons-lang-2.6.jar
        │   ├── commons-logging-1.0.4.jar
        │   ├── guava-13.0.1.jar
        │   ├── hadoop-0.20.2-dev-core.jar
        │   ├── hbase-0.94.13-mapr-1401.jar
        │   ├── libprotodefs.jar
        │   ├── log4j-1.2.15.jar
        │   ├── maprfs-1.0.3-mapr-3.0.2.jar
        │   ├── mapr-hbase-1.0.3-mapr-3.0.2.jar
        │   ├── protobuf-java-2.4.1.jar
        │   └── zookeeper-3.3.6.jar
        └── web.xml

9 directories, 18 files
[admin@tomcat-server  M7TomcatRead]$

  • Now build the application
[admin@tomcat-server  M7TomcatRead]$ ant
Buildfile: /home/admin/MyCode/M7TomcatRead/build.xml

init:

compile:
    [javac] /home/admin/MyCode/M7TomcatRead/build.xml:13: warning: 'includeantruntime' was not set, defaulting to build.sysclasspath=last; set to false for repeatable builds
    [javac] Compiling 1 source file to /home/admin/MyCode/M7TomcatRead/build/classes

war:
      [war] Building war: /home/admin/MyCode/M7TomcatRead/dist/M7Example.war

BUILD SUCCESSFUL
Total time: 3 seconds
[admin@tomcat-server  M7TomcatRead]$

  • You should now have the war file created inside dist. The tree looks like this now :
[admin@tomcat-server  M7TomcatRead]$ tree
.
├── build
│   └── classes
│       └── M7Example
│           └── M7ReadTomcat.class
├── build.xml
├── dist
│   └── M7Example.war
├── src
│   ├── M7ReadTomcat.java
│   └── M7ReadTomcat.java~
└── WebContent
    ├── jsp
    │   └── display.jsp
    └── WEB-INF
        ├── lib
        │   ├── commons-collections-3.2.1.jar
        │   ├── commons-lang-2.6.jar
        │   ├── commons-logging-1.0.4.jar
        │   ├── guava-13.0.1.jar
        │   ├── hadoop-0.20.2-dev-core.jar
        │   ├── hbase-0.94.13-mapr-1401.jar
        │   ├── libprotodefs.jar
        │   ├── log4j-1.2.15.jar
        │   ├── maprfs-1.0.3-mapr-3.0.2.jar
        │   ├── mapr-hbase-1.0.3-mapr-3.0.2.jar
        │   ├── protobuf-java-2.4.1.jar
        │   └── zookeeper-3.3.6.jar
        └── web.xml

9 directories, 18 files

Deploy on Tomcat

  • Drop the M7Example.war into the Tomcat webapps folder :
[admin@tomcat-server webapps]# pwd
/usr/share/tomcat/webapps
[admin@tomcat-server webapps]# ll
total 54980
drwxr-xr-x. 13 root   root       4096 Mar  5 18:57 docs
drwxr-xr-x.  8 tomcat tomcat     4096 Mar  5 18:57 examples
drwxr-xr-x.  5 root   tomcat     4096 Mar  5 18:57 host-manager
drwxr-xr-x.  5 tomcat tomcat     4096 Mar 10 18:00 M7Example
-rw-r--r--.  1 tomcat tomcat 56268685 Mar 10 18:00 M7Example.war
drwxr-xr-x.  5 root   tomcat     4096 Mar  5 18:57 manager
drwxr-xr-x.  3 tomcat tomcat     4096 Mar  5 18:57 ROOT
drwxr-xr-x.  5 tomcat tomcat     4096 Mar  5 18:57 sample
[admin@tomcat-server webapps]#

It should automatically create the folder M7Example and deploy your application.

  • Finally access it using the URL :
http://localhost:8080/M7Example/jsp/display.jsp

Final Output

  • The output should now show up as :

T12

myrow-1 cf1:q1 1392056271991 value-1

myrow-2 cf1:q2 1392056272031 value-2

myrow-2 cf1:q3 1392056273245 value-3

myrow-3 cf1:q1 1392114845834 value-4

myrow-3 cf1:q2 1392114863733 value-5

myrow-4 cf1:q1 1392227075188 value-6

Footnotes