Resource Allocation Properties

The following table lists resource allocation properties:

Property Description
yarn.scheduler.capacity. <queue-path>.capacity

Queue capacity in percentage (%) expressed as a float (for example, 12.5). The sum of capacities for all queues, at each level, must equal 100.

Applications in the queue may consume more resources than the queue's capacity if there are free resources, which provides elasticity.

yarn.scheduler.capacity. <queue-path>.maximum-capacity

Maximum queue capacity in percentage (%) expressed as a float.

This property limits the elasticity for applications in the queue. The default is -1 which disables it.

yarn.scheduler.capacity. <queue-path>.minimum-user-limit-percent

Sets the minimum value, expressed as an integer, on the percentage of resources allocated to a user, if there is a demand for resources.

A value of 100 implies no user limits are imposed. The default is 100.

The maximum value depends on the number of users who have submitted applications. For example, if this property is set to 25 and two users have submitted applications to a queue, the maximum percent of queue resources for each user is 50%. If a third user submits an application, no single user can use more than 33% of the queue resources. With four or more users, no user can use more than 25% of the queue resources.

yarn.scheduler.capacity. <queue-path>.user-limit-factor

The multiple of the queue capacity that can be configured to allow a single user to acquire more resources.

Value is specified as a float.

The default is 1, which ensures that a single user can never take more than the queue's configured capacity no matter how idle the cluster is.

yarn.scheduler.capacity. resource-calculator

Specifies the ResourceCalculator implementation to be used to compare resources in the scheduler. The default value is DiskBasedResourceCalculator, which uses memory, CPU and disk. Other values for this parameter include:

  • DefaultResourceCalculator, which uses memory only
  • DominantResourceCalculator, which uses dominant-resource to compare multi-dimensional resources such as memory and CPU
  • DiskBasedDominantResourceCalculator, which uses dominant-resource to compare multi-dimensional resources, such as memory, CPU and disk