Assigning Weights in an Allocation File

Defines an allocation file, and describes its entities.

An allocation file is an XML manifest that describes queues and their properties, as well as certain policy defaults. The file contains the following types of elements:

Queue Elements

Queue elements represent queues and can contain the following elements:
  • minResources
  • maxResources
  • maxRunningApps
  • weight
  • schedulingPolicy
  • aclSubmitApps
  • aclAdministerApps
  • minSharePreemptionTimeout
For more information on these elements, see Hadoop: Fair Scheduler.
Note: The queueMaxAppsDefault value is used for any parent queue that does not set a value for the maxRunningApps element.

User Elements

User elements represent settings that govern the behavior of individual users. They can contain a single property, maxRunningApps, which limits the number of running applications for a particular user. It contains the following elements:

  • userMaxAppsDefault
  • queueMaxAppsDefault
  • fairSharePreemptionTimeout
  • defaultQueueSchedulingPolicy
  • queuePlacementPolicy

For more information on these elements, see Hadoop: Fair Scheduler.

Note: If you set a value for queueMaxAppsDefault and do not set a value for maxRunningApps for the root queue, the value of queueMaxAppsDefault sets the application limit for all queues under the root queue hierarchy.

Example Allocation File

See example allocation file in Hadoop: Fair Scheduler.

Queue Access Control Lists

Queue Access Control Lists (Access Control List (ACL)s) allow administrators to control who may take actions on particular queues. They are configured with the aclSubmitApps and aclAdministerApps properties, which can be set per queue. Currently, the only supported administrative action is killing an application. Anyone who has permission to administer a queue may also submit applications to it. These properties take values in a format such as user1,user2 group1,group2 or group1,group2. An action on a queue is permitted if its user or group is in the ACL of that queue or in the ACL of any of that queue's ancestors. Therefore, if queue2 is inside queue1, and user1 is in queue1's ACL, and user2 is in queue2's ACL, then both users may submit to queue2.

For more information, see Hadoop: Fair Scheduler.

The yarn.admin.acl and yarn.acl.enable Properties

By default, on a secure cluster, users cannot kill jobs that do not belong to them.

On a secure cluster, you do not need to set the yarn.acl or the yarn.admin.acl properties. By default, they are set as follows. On unsecured clusters, these properties are not set by default.

<property>
  <name>yarn.acl.enable</name> > 
  <value>true</value> > 
</property>
<property>
  <name>yarn.admin.acl</name> > 
  <value> </value> > 
</property>

The yarn.admin.acl property is set by default to " ", meaning that an administrator is not specified on a cluster.

To allow users to kill jobs that do not belong to them, or to get access to their logs, you need to set the yarn.admin.acl property with the user or group name.
Fair and Capacity scheduler root queue admins

For both the Fair scheduler and Capacity scheduler, the default value of the administrators for the root queues is " " .