Service Layout Guidelines for Large Clusters

This section contains information about installing and segregating services on large clusters.

General Guidelines

The following are guidelines for installing services on large clusters:

  • JobTracker: Run the JobTracker services on dedicated nodes for clusters with over 250 nodes.
  • ResourceManager: Run the ResourceManager services on dedicated nodes for clusters with over 250 nodes.
  • Elasticsearch: Elasticsearch consumes significant CPU, disk, and memory resources. Review the following guidelines:
    • Whenever possible, Elasticsearch should have a dedicated disk for its index directory.
    • Depending on the number of indexed logs, you may want to run the Elasticsearch service on five or more dedicated nodes.
    • On production clusters, consider increasing Elasticsearch's memory allocation. After you install MapR Monitoring, see Configure the Elasticsearch Service Heap Size.
    • On clusters with high-density racks, run one or more Elasticsearch services on each rack. Also, configure Fluentd to write logs to Elasticsearch services that reside on the same rack as the Fluentd services. After you install MapR Monitoring, see Configure Fluentd Services to Write to Elasticsearch Nodes on the Same Rack.
  • OpenTSDB: Run the OpenTSDB service on five or more nodes for clusters over 100 nodes.

Services to Separate on Large Clusters

The following are guidelines about which services to separate on large clusters:

  • JobTracker and ResourceManager on ZooKeeper nodes: Avoid running the JobTracker and ResourceManager service on nodes that are running the ZooKeeper service. On large clusters, the JobTracker and ResourceManager services can consume significant resources.
  • MySQL on CLDB nodes: Avoid running the MySQL server that supports the MapR Metrics service on a CLDB node. Consider running the MySQL server on a machine external to the cluster to prevent the MySQL server’s resource needs from affecting services on the cluster.
  • TaskTracker on CLDB or ZooKeeper nodes: When the TaskTracker service is running on a node that is also running the CLDB or ZooKeeper services, consider reducing the number of task slots that this node's instance of the TaskTracker service provides.
  • Webserver on CLDB nodes: Avoid running the webserver on CLDB nodes. Queries to the MapR Metrics service can impose a bandwidth load that reduces CLDB performance.
  • Monitoring Services on CLDB Nodes: Avoid running the OpenTSDB, Elasticsearch, Kibana, or Grafana services on nodes that are running the CLDB service.