MapR 5.0 Documentation : Persistent Configuration Storage

Drill stores persistent configuration data in a persistent configuration store (PStore). This data is encoded in JSON or Protobuf format. Drill can use the local file system, ZooKeeper, or MapR-DB to store this data. The data stored in a PStore includes state information for storage plugins, query profiles, and ALTER SYSTEM settings. The default type of PStore configured depends on the Drill installation mode. 

The following table provides the persistent storage mode for each of the Drill modes:

ModeDescription
EmbeddedDrill stores persistent data in the local file system.
You cannot modify the PStore location for Drill in embedded mode.
DistributedDrill stores persistent data in ZooKeeper, by default.
You can modify where ZooKeeper offloads data,
or you can change the persistent storage mode to MapR-DB.

ZooKeeper for Persistent Configuration Storage

To make Drill installation and configuration simple, Drill uses ZooKeeper to store persistent configuration data. The ZooKeeper PStore provider stores all of the persistent configuration data in ZooKeeper except for query profile data. 

The ZooKeeper PStore provider offloads query profile data to the ${DRILL_LOG_DIR:-/var/log/drill} directory on Drill nodes. If you want the query profile data stored in a specific location, you can configure where ZooKeeper offloads the data.

Configuring a ZooKeeper Offloading Directory

To modify where the ZooKeeper PStore provider offloads query profile data, configure the sys.store.provider.zk.blobroot property in the drill.exec block in <drill_installation_directory>/conf/drill-override.conf on each Drill node and then restart the Drillbit service.

Example
drill.exec: {
 cluster-id: "my_cluster_com-drillbits",
 zk.connect: "<zkhostname>:<port>",
 sys.store.provider.zk.blobroot: "maprfs://<directory to store pstore data>/"
}

Issue the following command to restart the Drillbit on all Drill nodes:

maprcli node services -name drill-bits -action restart -nodes <node IP addresses separated by a space>

MapR-DB for Persistent Configuration Storage

If you have MapR-DB in your cluster, you can use MapR-DB for persistent configuration storage. Using MapR-DB to store persistent configuration data can prevent memory strain on ZooKeeper in clusters running heavy workloads.

Configuring MapR-DB for Persistent Configuration Storage

To change the persistent storage mode to MapR-DB, add or modify the sys.store.provider block in <drill_installation_directory>/conf/drill-override.conf on each Drill node and then restart the Drillbit service.

Example
sys.store.provider: {
class: "org.apache.drill.exec.store.hbase.config.HBasePStoreProvider",
hbase: {
  table : "/tables/drill_store",
	}
},

Issue the following command to restart the Drillbit on all Drill nodes:

maprcli node services -name drill-bits -action restart -nodes <node IP addresses separated by a space>