Select Services

In a typical cluster, most nodes are dedicated to data processing and storage, and a smaller number of nodes run services that provide cluster coordination and management. Some applications run on cluster nodes and others run on client nodes that can communicate with the cluster.

The services that you choose to run on each node will likely evolve over the life of the cluster. Services can be added and removed over time.

The following table shows some of the services that can be run on a node.

Service Category Service

Description

Management Warden

Warden runs on every node, coordinating the node's contribution to the cluster.

MapReduce TaskTracker

Hadoop TaskTracker starts and tracks MapReduce tasks on a node. The TaskTracker service receives task assignments from the JobTracker service and manages task execution.

MapReduce NodeManager Hadoop YARN NodeManager service. The NodeManager manages node resources and monitors the health of the node. It works with the ResourceManager to manage YARN containers that run on the node.
Storage FileServer

FileServer is the MapR service that manages disk storage for MapR-FS and MapR-DB on each node.

Storage CLDB

Maintains the container location database (CLDB) service. The CLDB service coordinates data storage services among MapR-FS FileServer nodes, MapR NFS gateways, and MapR clients.

Storage NFS

Provides read-write MapR Direct Access NFSā„¢ access to the cluster, with full support for concurrent read and write access.

Storage MapR HBase Client

Provides access to MapR-DB binary tables via HBase APIs. Required on all nodes that will access table data in MapR-FS, typically all TaskTracker nodes and edge nodes for accessing table data.

Management JobTracker

Hadoop JobTracker service. The JobTracker service coordinates the execution of MapReduce jobs by assigning tasks to TaskTracker nodes and monitoring task execution.

Management ResourceManager Hadoop YARN ResourceManager service. The ResourceManager manages cluster resources, and tracks resource usage and node health.
Management ZooKeeper

Enables high availability (HA) and fault tolerance for MapR clusters by providing coordination.

Management HistoryServer Archives MapReduce job metrics and metadata.
Management HBase Master

The HBase master service manages the region servers that make up HBase table storage.

Note: This service is only needed for Apache HBase. Your cluster supports MapR-DB without this service.
Management Web Server

Runs the MapR Control System.

Management Metrics

Provides optional real-time analytics data on cluster and job performance through the Analyzing Job Metrics interface. If used, the Metrics service is required on all JobTracker and Web Server nodes.

Application Hue Hue is Hadoop user interface that interacts with Apache Hadoop and its ecosystem components, such as Hive, Pig, and Oozie.
Application HBase Region Server

HBase region server is used with the HBase Master service and provides storage for an individual HBase region.

Note: This service is only needed for Apache HBase. Your cluster supports MapR-DB without this service.
Application Pig

Pig is a high-level data-flow language and execution framework.

Application Hive

Hive is a data warehouse that supports SQL-like ad hoc querying and data summarization.

Application Flume

Flume is a service for aggregating large amounts of log data

Application Oozie

Oozie is a workflow scheduler system for managing Hadoop jobs.

Application HCatalog

HCatalog aggregates HBase data.

Application Cascading

Cascading is an application framework for analyzing and managing big data.

Application Mahout

Mahout is a set of scalable machine-learning libraries that analyze user behavior.

Application Myriad Myriad is an application framework that enables YARN applications and Mesos frameworks to run side-by-side while dynamically sharing cluster resources.
Application Spark Spark is an processing engine for large datasets.
Application Sqoop

Sqoop is a tool for transferring bulk data between Hadoop and relational databases.

MapR is a complete Hadoop distribution, but not all services are required. Every Hadoop installation requires services to manage jobs and applications. JobTracker and TaskTracker manage MapReduce v1 jobs. ResourceManager and NodeManager manage MapReduce v2 and other applications that can run on YARN. In addition, MapR requires the ZooKeeper service to coordinate the cluster, and at least one node must run the CLDB service. The WebServer service is required if the browser-based MapR Control System will be used.

MapR Hadoop includes tested versions of the services listed here. MapR provides a more robust, read-write storage system based on volumes and containers. MapR data nodes typically run FileServer, TaskTracker, and NodeManager. Do not plan to use packages from other sources in place of the MapR distribution.

When using Myriad, the ResourceManager is deployed using Marathon and NodeManager is run as a Mesos task.