MapR 5.0 Documentation : MapR 2.0 Beta

Welcome to the MapR 2.0 Beta! This new release includes the following features:

  • MapR Metrics — Graphical display of job and task statistics
  • Central configuration — configure nodes from files stored on the cluster
  • Label-based scheduling — specify which jobs run on which nodes
  • SELinux support — keep your cluster secure
  • Central logging — easily diagnose the cluster

The easiest way to install the Beta is to follow the steps below. If you are installing on a moderate sized cluster (3 to 100 nodes), simply perform the following steps:

  1. PREPARATION — Make sure your nodes meet the Requirements for Installation
  2. ADDING THE REPOSITORY — Add the correct MapR repository for your operating system
  3. INSTALLATION
    • Install either the M3 or M5 version of MapR
    • Install MapR Metrics (requires M5)

Preparation

Before installing the MapR 2.0 Beta, make sure your nodes meet the following requirements:

  • Operating system:
    • 64-bit Red Hat 5.4 or greater, or 64-bit CentOS 5.4 or greater
    • 64-bit Ubuntu 9.04 or greater
  • RAM: 4 GB or more
  • At least one free unmounted drive or partition, 50 GB or more
  • At least 10 GB of free space on the operating system partition
  • Sun Java JDK version 1.6.0_24 (not JRE)
  • The root password, or sudo privileges
  • A Linux user chosen to have administrative privileges on the cluster
    • Make sure the user has a password (using sudo passwd <user> for example)

Each node must have a unique hostname, and keyless SSH set up to all other nodes.

This procedure assumes you have free, unmounted physical partitions or hard disks for use by MapR. If you are not sure, please read Setting Up Disks for MapR.

Adding the MapR Repository

The first step in deployment is to add the MapR repository. Follow the appropriate instructions for your operating system:

To add the MapR repository on Red Hat Enterprise Linux (RHEL) or CentOS:

  1. Change to the root user (or use sudo for the following commands).
  2. On all nodes, create a text file called maprtech.repo in the directory /etc/yum.repos.d/ with the following contents:
    [maprtech]
    name=MapR Technologies
    baseurl=http://package.mapr.com/releases/v2.0.0-beta/redhat/
    enabled=1
    gpgcheck=0
    protect=1

To add the MapR repository on Ubuntu:

  1. Change to the root user (or use sudo for the following commands).
  2. On all nodes, add the following line to /etc/apt/sources.list:
    deb http://package.mapr.com/releases/v2.0.0-beta/ubuntu/ mapr optional
  3. On all nodes, run the following command:
    apt-get update

Installation

For the steps that follow, make the following substitutions:

  • <user> - the chosen administrative username
  • <node 1>, <node 2>, <node 3>... - the IP addresses of nodes 1, 2, 3 ...

If you are installing a MapR cluster on nodes that are not connected to the Internet, contact MapR for assistance. If you are installing a cluster larger than 100 nodes, see the Advanced Installation Topics. In particular, CLDB nodes on large clusters should not run any other service (see Isolating CLDB Nodes).

M3 Installation

  1. Change to the root user (or use sudo for the following commands).
  2. On node 1, make sure the Ajaxterm package is installed:
    which Ajaxterm
    If it is not installed, install it:
    • RHEL/CentOS:
      yum install Ajaxterm
    • Ubuntu:
      apt-get install Ajaxterm
  3. On node 1, execute the following command:
    • RHEL/CentOS:
      yum install mapr-cldb mapr-fileserver mapr-jobtracker mapr-nfs mapr-tasktracker mapr-webserver mapr-zookeeper
    • Ubuntu:
      apt-get install mapr-cldb mapr-fileserver mapr-jobtracker mapr-nfs mapr-tasktracker mapr-webserver mapr-zookeeper
  4. On nodes 2 and 3, execute the following command:
    • RHEL/CentOS:
      yum install mapr-fileserver mapr-tasktracker mapr-zookeeper
    • Ubuntu:
      apt-get install mapr-fileserver mapr-tasktracker mapr-zookeeper
  5. On all other nodes (nodes 4...n), execute the following commands:
    • RHEL/CentOS:
      yum install mapr-fileserver mapr-tasktracker
    • Ubuntu:
      apt-get install mapr-fileserver mapr-tasktracker
  6. On all nodes, execute the following commands:
    /opt/mapr/server/configure.sh -C <node 1> -Z <node 1>,<node 2>,<node 3>
    /opt/mapr/server/disksetup -F /tmp/disks.txt
    
  7. On nodes 1, 2, and 3, execute the following command:
    service mapr-zookeeper start
  8. On node 1, execute the following command:
    service mapr-warden start
    Tips

    If you see "WARDEN running as process <process>. Stop it" it means the warden is already running. This can happen, for example, when you reboot the machine. Use service mapr-warden stop to stop it, then start it again.

  9. On node 1, give full permission to the chosen administrative user using the following command:
    /opt/mapr/bin/maprcli acl edit -type cluster -user <user>:fc
    Tips

    The Warden can take a few minutes to start. If you see the error "Couldn't connect to the CLDB service," wait a few minutes and try again.

  10. On a machine that is connected to the cluster and to the Internet, perform the following steps to install the license:
    • In a browser, view the MapR Control System by navigating to the node that is running the WebServer:
      https://:8443
      Your computer won't have an HTTPS certificate yet, so the browser will warn you that the connection is not trustworthy. You can ignore the warning this time.
    • The first time MapR starts, you must accept the agreement and choose whether to enable the MapR Dial Home service.
    • Log in to the MapR Control System as the administrative user you designated earlier.
    • In the navigation pane of the MapR Control System, expand the System Settings Views group and click MapR Licenses to display the MapR License Management dialog.
    • Click Add Licenses via Web.
    • If the cluster is already registered, the license is applied automatically. Otherwise, click OK to register the cluster on MapR.com and follow the instructions there.
      • If the cluster is not yet registered, the message "Cluster not found" appears and the browser is redirected to a registration page.
      • On the registration page, create an account and log in.
      • On the Register Cluster page, choose M3 and click Register.
      • When the message "Cluster Registered" appears, click Return to your MapR Cluster UI.
  11. On node 1, execute the following command:
    /opt/mapr/bin/maprcli node services -nodes <node 1> -nfs start
  12. On all other nodes (nodes 2...n), execute the following command:
    service mapr-warden start
  13. Log in to the MapR Control System.
  14. Under the Cluster group in the left pane, click Dashboard.
  15. Check the Services pane and make sure each service is running the correct number of instances:
    • Instances of the FileServer and TaskTracker on all nodes
    • 3 instances of ZooKeeper
    • 1 instance of the CLDB, JobTracker, NFS, and WebServer

M5 Installation

  1. Change to the root user (or use sudo for the following commands).
  2. On node 1, make sure the Ajaxterm package is installed:
    which Ajaxterm
    If it is not installed, install it:
    • RHEL/CentOS:
      yum install Ajaxterm
    • Ubuntu:
      apt-get install Ajaxterm
  3. On node 1, execute the following command:
    • RHEL/CentOS:
      yum install mapr-cldb mapr-jobtracker mapr-nfs mapr-zookeeper mapr-tasktracker mapr-webserver
    • Ubuntu:
      apt-get install mapr-cldb mapr-jobtracker mapr-nfs mapr-zookeeper mapr-tasktracker mapr-webserver
  4. On nodes 2 and 3, execute the following command:
    • RHEL/CentOS:
      yum install mapr-cldb mapr-jobtracker mapr-nfs mapr-zookeeper mapr-tasktracker
    • Ubuntu:
      apt-get install mapr-cldb mapr-jobtracker mapr-nfs mapr-zookeeper mapr-tasktracker
  5. On all other nodes (nodes 4...n), execute the following commands:
    • RHEL/CentOS:
      yum install mapr-fileserver mapr-nfs mapr-tasktracker
    • Ubuntu:
      apt-get install mapr-fileserver mapr-nfs mapr-tasktracker
  6. On all nodes, execute the following commands:
    /opt/mapr/server/configure.sh -C <node 1>,<node 2>,<node 3> -Z <node 1>,<node 2>,<node 3>
    /opt/mapr/server/disksetup -F /tmp/disks.txt
    
  7. On nodes 1, 2, and 3, execute the following command:
    service mapr-zookeeper start
  8. On node 1, execute the following command:
    service mapr-warden start
    Tips

    If you see "WARDEN running as process <process>. Stop it" it means the warden is already running. This can happen, for example, when you reboot the machine. Use service mapr-warden stop to stop it, then start it again.

  9. On node 1, give full permission to the chosen administrative user using the following command:
    /opt/mapr/bin/maprcli acl edit -type cluster -user <user>:fc
    Tips

    The Warden can take a few minutes to start. If you see the error "Couldn't connect to the CLDB service," wait a few minutes and try again.

  10. On a machine that is connected to the cluster and to the Internet, perform the following steps to install the license:
    • In a browser, view the MapR Control System by navigating to the node that is running the WebServer:
      https://:8443
      Your computer won't have an HTTPS certificate yet, so the browser will warn you that the connection is not trustworthy. You can ignore the warning this time.
    • The first time MapR starts, you must accept the agreement and choose whether to enable the MapR Dial Home service.
    • Log in to the MapR Control System as the administrative user you designated earlier.
    • In the navigation pane of the MapR Control System, expand the System Settings Views group and click MapR Licenses to display the MapR License Management dialog.
    • Click Add Licenses via Web.
    • If the cluster is already registered, the license is applied automatically. Otherwise, click OK to register the cluster on MapR.com and follow the instructions there.
      • If the cluster is not yet registered, the message "Cluster not found" appears and the browser is redirected to a registration page.
      • On the registration page, create an account and log in.
      • On the Register Cluster page, choose M5 Trial and click Register.
      • When the message "Cluster Registered" appears, click Return to your MapR Cluster UI.
  11. On node 1, execute the following command:
    /opt/mapr/bin/maprcli node services -nodes <node 1> -nfs start
  12. On all other nodes (nodes 2...n), execute the following command:
    service mapr-warden start
  13. Log in to the MapR Control System.
  14. Under the Cluster group in the left pane, click Dashboard.
  15. Check the Services pane and make sure each service is running the correct number of instances:
    • Instances of the FileServer, NFS, and TaskTracker on all nodes
    • 3 instances of the CLDB
    • 1 of 3 instances of the JobTracker
    • 1 instance of the WebServer

Installing MapR Metrics

MapR Metrics provides statistical information about jobs, tasks, and task attempts in easy-to-read graphical form.

Prerequisites

  • MySQL Server — MapR Metrics requires a MySQL server to store statistical data about jobs and tasks in the cluster. The MySQL server can be on a cluster node or on a separate machine.
  • EPEL Repository — Extra Packages for Enterprise Linux (EPEL) provides components that MapR Metrics needs (CentOS and Red Hat only).
  • M5 License — to get the most out of MapR Metrics, you'll need an M5 License. With an M3 license, you don't get the charts or histograms.

To enable the EPEL repository on CentOS or Red Hat 5.x:

  1. Download the EPEL repository:
    wget http://dl.fedoraproject.org/pub/epel/5/x86_64/epel-release-5-4.noarch.rpm
  2. Install the EPEL repository:
    rpm -Uvh epel-release-5*.rpm

To enable the EPEL repository on CentOS or Red Hat 6.x:

  1. Download the EPEL repository:
    wget http://download.fedoraproject.org/pub/epel/6/x86_64/epel-release-6-7.noarch.rpm
  2. Install the EPEL repository:
    rpm -Uvh epel-release-6*.rpm

Installation

To Install MapR Metrics:

  1. Start with a functioning MapR cluster. Note which nodes run the Jobtracker and Webserver.
  2. To get the most out of MapR metrics, apply an M5 license to the cluster.
  3. Install MySQL Server, either on a cluster node or on a separate system (or use an existing MySQL server).
  4. On all JobTracker nodes and Webserver nodes, install the mapr-metrics package, running the appropriate command as root or using sudo:
    • CentOS or Red Hat:
      yum install mapr-metrics
    • Ubuntu:
      apt-get install mapr-metrics
  5. Restart the warden:
    service mapr-warden restart
  6. Log on to the MapR Control System.
  7. In the Navigation pane, click System Settings > Metrics to display the Configure Metrics Database dialog.
    Image: Configure Metrics Database dialog

  8. In the URL field, enter the hostname and port of the machine running the MySQL server.
  9. In the Username and Password fields, enter the username and password of the MySQL user.
  10. Log on to one of the nodes on which mapr-metrics is installed, go to the mysql\> prompt, and source the script /opt/mapr/bin/setup.sql. Example:
    mysql> source /opt/mapr/bin/setup.sql;

Attachments:

Metrics-configure.png (image/png)