Writing a Java Application

In your Java application, you will use a Configuration object to interface with MapR file system. When you instantiate a Configuration object, it is created with values from Hadoop configuration files.

When the program is build with JAR files from the MapR installation, the Hadoop 1 configuration files are in $MAPR_HOME/hadoop/hadoop-<version>/conf directory and the Hadoop 2 configuration files are in $HADOOP_HOME/etc/hadoop directory. This Hadoop configuration directory is in the hadoop classpath that you include when you compile and run the java program.

When the program is built through maven using MapR's maven artifacts, the default Hadoop configuration files are included in the maven artifacts.The user needs to programmatically update the Hadoop configuration to match the Hadoop configuration files on the MapR cluster.

Sample Code

The following sample code shows how to interface with MapR file system using Java. The example creates a directory, writes a file, then reads the contents of the file.

/* Copyright (c) 2009 & onwards. MapR Tech, Inc., All rights reserved */

//package com.mapr.fs;

import java.net.*;
import org.apache.hadoop.fs.*;
import org.apache.hadoop.conf.*;

/**
* Assumes mapr installed in /opt/mapr
*
* Compilation:
* javac -cp $(hadoop classpath) MapRTest.java
*
* Run:
* java -cp .:$(hadoop classpath) MapRTest /test
*/
public class MapRTest
{
 public static void main(String args[]) throws Exception {
        byte buf[] = new byte[ 65*1024];
        int ac = 0;
        if (args.length != 1) {
            System.out.println("usage: MapRTest pathname");
        return;
        }

        // maprfs:/// -> uses the first entry in /opt/mapr/conf/mapr-clusters.conf
        // maprfs:///mapr/my.cluster.com/
        // /mapr/my.cluster.com/

        // String uri = "maprfs:///";
        String dirname = args[ac++];

        Configuration conf = new Configuration();
        
        //FileSystem fs = FileSystem.get(URI.create(uri), conf); // if wanting to use a different cluster
        FileSystem fs = FileSystem.get(conf);
        
        Path dirpath = new Path( dirname + "/dir");
        Path wfilepath = new Path( dirname + "/file.w");
        //Path rfilepath = new Path( dirname + "/file.r");
        Path rfilepath = wfilepath;


        // try mkdir
        boolean res = fs.mkdirs( dirpath);
        if (!res) {
                System.out.println("mkdir failed, path: " + dirpath);
        return;
        }

        System.out.println( "mkdir( " + dirpath + ") went ok, now writing file");

        // create wfile
        FSDataOutputStream ostr = fs.create( wfilepath,
                true, // overwrite
                512, // buffersize
                (short) 1, // replication
                (long)(64*1024*1024) // chunksize
                );
        ostr.write(buf);
        ostr.close();

        System.out.println( "write( " + wfilepath + ") went ok");

        // read rfile
        System.out.println( "reading file: " + rfilepath);
        FSDataInputStream istr = fs.open( rfilepath);
        int bb = istr.readInt();
        istr.close();
        System.out.println( "Read ok");
        }
}