Multiple Drill Clusters within YARN

You can define multiple Drill clusters within a single YARN cluster. Each Drill cluster is a collection of Drillbits that work as an independent unit. For example, you might define one test Drill cluster that consists of a few machines on the same physical cluster that runs larger Drill clusters for development and marketing.

You must assign each Drill cluster a distinct ZooKeeper entry. Drill uses ZooKeeper to coordinate activities. Each Drill cluster also needs a distinct set of ports because YARN may launch Drillbits from different clusters on the same physical node.

The following steps summarize the process for defining multiple Drill clusters on a single YARN cluster:
  1. Create a new site directory.
  2. Configure Drill.
  3. Configure Drill-on-YARN.
  4. Start the cluster.

The following task provides instructions for each of the steps required to configure and run multiple Drill clusters under YARN: