Configuring Drill Memory

You can set the amount of memory allocated to the Drill or the Query Service in /opt/mapr/drill/drill-<version>/conf/drill-env.sh.

Edit the drill-env.sh file on each Drill node to configure the amount of memory allocated to Drill. If you configured multiple Drill clusters and registered a Drill cluster as a Query Service provider, you can also set the memory for the Query Service provider following the instructions listed here. Update the memory variables in the drill-env.sh file. If you run Drill under the Warden Service, you must also update the warden.drill-bits.conf file to reflect the total amount of memory set for Drill. Restart the Drillbits after you update the files. See Configuring Drill Memory below for instructions.

Default Memory Allocation for Drill and the Query Service

The amount of memory allocated to Drill and the Query Service differ. By default, when you install Drill, 13 G of memory is allocated to the Drillbit service running on a node, as follows:
  • 8 G direct
  • 4 G heap
  • 1 G core cache
The Query Service requires less memory than Drill. By default, the Query Service is allocated ~ 5 G of memory, as follows:
  • 1 G direct
  • 3 G heap
  • 512 M core cache

About the Drillbit Startup Script

The drill-env.sh file contains the following memory variables with default settings:
#export DRILL_MAX_DIRECT_MEMORY=${DRILL_MAX_DIRECT_MEMORY:-"8G"}
#export DRILL_HEAP=${DRILL_HEAP:-"4G"}
#export DRILLBIT_CODE_CACHE_SIZE=${DRILLBIT_CODE_CACHE_SIZE:-"1G"}

DRILL_MAX_HEAP is the maximum theoretical heap limit for the JVM per node. DRILL_MAX_DIRECT_MEMORY is the Java direct memory limit per node. DRILLBIT_CODE_CACHE_SIZE is the memory limit for the compiled code generated by the JVM JIT compiler.

To customize memory limits, uncomment a line and change the value.

If performance is an issue, add -Dbounds=false, as shown in the following example:
export DRILL_JAVA_OPTS="$DRILL_JAVA_OPTS -Dbounds=false"

Configuring Drill Memory

To configure Drill memory, complete the following steps:
  1. Change the value of the following variables in the /opt/mapr/drill/drill-<version>/conf/drill-env.sh file on each Drill node, as follows:

    export DRILL_MAX_DIRECT_MEMORY=${DRILL_MAX_DIRECT_MEMORY:-"<value>"}

    export DRILL_HEAP=${DRILL_HEAP:-"<value>”}

    export DRILLBIT_CODE_CACHE_SIZE=${DRILLBIT_CODE_CACHE_SIZE:-"<value>"}

    Note: Remove the # to uncomment each line.
  2. Change the value of the DRILLBIT_MAX_PROC_MEM variable, as well as the minimum and maximum heap sizes in the /opt/mapr/conf/conf.d/warden.drill-bits.conf file to reflect the total amount of memory assigned to Drill, as shown:
    service.env=DRILLBIT_MAX_PROC_MEM=<value>
    service.heapsize.min=<value>
    service.heapsize.max=<value>
    
    For example, for the total Drill memory requirement of 16 G (Heap=5G, MaxDirect=9G, CodeCache=2G), enter:
    service.env=DRILLBIT_MAX_PROC_MEM=16G
    service.heapsize.min=16384
    service.heapsize.max=16384
  3. Run the configuration script to update the node configuration, as shown:
    /opt/mapr/server/configure.sh -R
    Note: See configure.sh for more information about the script.
  4. Restart the Drillbit service on the nodes, as follows:
    $ maprcli node services -name drill-bits -action restart -nodes <node host names separated by a space>
    Note: Use the host name if possible. Using host names instead of IP addresses is a best practice.