Partner App: Hadapt

Hadapt

Founded in 2010, Hadapt pioneered the SQL-on-Hadoop movement with it’s visionary product. Hadapt v3.0 completes that vision with a standard ANSI SQL interface and patent-pending flexible schema technology, allowing enterprise users to query data of all types like never before.

Application Description

Founded in 2010, Hadapt pioneered the SQL-on-Hadoop movement with it’s visionary product. Hadapt v3.0 completes that vision with a standard ANSI SQL interface and patent-pending flexible schema technology, allowing enterprise users to query data of all types like never before.

Customers can now use SQL to query JSON, XML, HL7, log files, and many more complex data types inside Hadoop without any arduous ETL required.


Component Version Connection Method
MapR Distribution 3.0+ HDFS API
Application Version: 2.0.1, 3.0.0

Download App

Installation instructions

= Install Hadapt =

Transfer Hadapt RPM to Hadoop Master node

Log into your Hadoop Master node as root

Install the Hadapt RPM package

rpm -i hadapt-mapr-3.0.0-1.x86_64.rpm

Place a default config file in /etc/hadapt/hadapt.cfg

[default]

[[hadapt]]

admin=hadapt

[[zookeeper]]

data_directory=/var/lib/zookeeper

port=2181

Configure Zookeeper, replace datanode_n with addresses for your Hadoop data nodes. If you have a pre-existing Zookeeper installation, please point this configuration to that node.

hadapt-admin configure zookeeper localhost:2181

Deploy Hadapt to the rest of your Hadoop data nodes

hadapt-admin deploy package /path/to/hadapt-mapr-3.0.0-1.x86_64.rpm –n datanode_1 datanode_2 datanode_3

Deploy configuration files to rest of nodes. Replace references to datanode_n with your Hadoop data nodes.

hadapt-admin deploy settings –p –z –n datanode_1 datanode_2 datanode_3

Assign Hadapt roles to your nodes. Replace references as appropriate. The –p passive node parameter is optional, and should be pointed to a node that is not the Master.

hadapt-admin configure suites –f hadapt_master –p hadapt_backup –b datanode_1 datanode_2 datanode_3

Initialize Hadapt services

hadapt-admin initialize all

Start Hadapt

hadapt-admin start all


Use Instructions

= Use Instructions Tutorial =

For this tutorial, please connect to the Hadapt CLI using the instructions in the Verification section. Once inside, we can run a set of queries that show some of Hadapt's functionality.

===== Download some sample JSON data and push it to HDFS =====

! hadoop fs -mkdir /hadapt_tutorial_data
! curl -s https://s3.amazonaws.com/hadapt-tutorial/data/sensor-20140501.json | hadoop
fs -put - /hadapt_tutorial_data/sensor-20140501.json

===== Peek at the data =====
! hadoop fs –text /hadapt_tutorial_data/sensor-20140501.json | head –n 20

===== Download our JSON interpreter and push it to HDFS =====

! hadoop fs -mkdir /user_jars
! curl -s https://s3.amazonaws.com/hadapt-tutorial/hadapt-json-interpreter-0.0.1-SNAPSHOT-jar-with-dependencies.jar | hadoop fs -put - /user_jars/hadapt-json-interpreter-0.0.1-SNAPSHOT-jar-with-dependencies.jar

===== Create a table with no columns and define the location of our data and our interpreter =====

create table hadapt_tutorial() 
<br>
WITH (storage_engine='HDFS_TEXT', path='maprfs:///hadapt_tutorial_data/',
interpreter_jars='maprfs:///user_jars/hadapt-json-interpreter-0.0.1-SNAPSHOT-jar-with-dependencies.jar',

interpreter_class='com.hadapt.interpreters.json.JsonInterpreter');

===== Describe the table =====

\dt hadapt_tutorial

===== Discover the schema of the table, using our interpreter =====

ANALYZE hadapt_tutorial;

===== Describe the table again =====

\dt hadapt_tutorial

===== Run a query =====

select count(1), avg(event_payload_glucose_mg_dl)
from hadapt_tutorial;

http://www.hadapt.com

Support Information

Email support@hadapt.com

Or go to hadapt.com/support