MapR 5.0 Documentation : Configuring JobTracker and YARN Servers with Kerberos

These sections explain the procedures that enable Kerberos on a node. The instructions in the first two sections apply to both MRv1 and MRv2.

Configuring Kerberos for MRv1

Configuring Kerberos for MRv2

 

For a more comprehensive discussion on Kerberos, see Configuring Kerberos User Authentication.

Creating a Kerberos Principal and a keytab File

In order to use JobTracker with Kerberos, you need to create a Kerberos principal and a keytab file. A Kerberos principal is a unique identity that represents a user or service in a Kerberos system. The user obtains a ticket for a principal name (through the kinit utility) and this ticket authenticates the user to the Kerberos server.

The keytab file contains principal names and their corresponding encrypted keys, or tickets.

Creating Kerberos Principals

Use the addprinc command at the kadmin console prompt to create two principals in the same realm as the MapR cluster:

  • a JobTracker user principal (used by SPNEGO webservers)

  • an HTTP user principal (for nodes that handle SPNEGO traffic)

For example, if the JobTracker service and the webserver service are running on a node called perfnode153.perf.lab and the realm is called dev-maprtech, the commands to add the JobTracker principal and the HTTP principal are:

kadmin: addprinc -randkey mapr/perfnode153.perf.lab@dev-maprtech
  addprinc -randkey HTTP/perfnode153.perf.lab@dev-maprtech

Creating a keytab File

Keytabs are created or appended to by extracting keys from the KDC database using the ktadd command inside the kadmin console prompt.

To create a keytab file for the JobTracker principal,  you use the same procedure that you use to create the keytab for the MapR-FS or mapred principal for a specific host.

  1. Create the keytab file for the JobTracker principal. Name this file mapr.keytab and put it in the directory /opt/mapr/conf on the machine running the JobTracker server, as shown:

    kadmin: ktadd -k /opt/mapr/conf/mapr.keytab mapr/perfnode153.perf.lab
  2. Set read-only permissions on the mapr.keytab file.

    $ sudo chmod 600 mapr:mapr /opt/mapr/conf/mapr.keytab
  3. Set the file's owner to the user running the JobTracker server (usually mapr).

    $ sudo chown mapr:mapr /opt/mapr/conf/mapr.keytab
  4. Copy the mapr.keytab file to all the nodes running Hue, Hive, httpfs, and Oozie services.

Modifying the env.sh File

The env.sh file contains a setting for MapR login options that defaults to the value maprsasl. Change this value to hybrid, which applies to Kerberos and other security protocols.

The new line (after the change) should look like this:

MAPR_LOGIN_OPTS="-Dhadoop.login=hybrid ${MAPR_JAAS_CONFIG_OPTS} ${MAPR_ZOOKEEPER_OPTS}"

Modifying the mapred-site.xml File

The mapred-site.xml file needs the following information for JobTracker:

Property NameDescriptionValue
mapreduce.jobtracker.kerberos.principalHostname and realmmapr/_HOST@<REALM>
mapreduce.jobtracker.keytab.filePath to the MapReduce keytab file/opt/mapr/conf/mapr.keytab

Add these properties to your mapred-site.xml file as shown in the following example. Note that this example uses dev-maprtech for the Kerberos realm.

<!-- JobTracker security configuration -->
<property>
  <name>mapreduce.jobtracker.kerberos.principal</name>
  <value>mapr/_HOST@dev-maprtech</value>
</property>
<property>
  <name>mapreduce.jobtracker.keytab.file</name>
  <value>/opt/mapr/conf/mapr.keytab</value> <!-- path to the MapReduce keytab -->
</property>

Restart JobTracker

JobTracker must be restarted in order for the configuration file changes to take effect. Enter the following command:

maprcli node services -jobtracker restart -nodes <jtnode>

 

Modifyinging the yarn-site.xml File

Make sure that the following tasks above are already completed before you begin this task:

Add the following properties to the yarn-site.xml file on every node in the cluster (/opt/mapr/hadoop/hadoop-2.7.0/etc/hadoop/yarn-site.xml).

Note that you need to use /opt/mapr/conf/mapr.keytab for the keytab property and mapr instead of yarn for the principal property.

<!-- ResourceManager security configs -->
<property>
 <name>yarn.resourcemanager.keytab</name>
 <value>/opt/mapr/conf/mapr.keytab</value>    <!-- path to the YARN keytab -->

</property>
<property>
 <name>yarn.resourcemanager.principal</name>
 <value>mapr/_HOST@YOUR-REALM.COM</value>
</property>

<!-- NodeManager security configs -->
<property>
 <name>yarn.nodemanager.keytab</name>
<value>/opt/mapr/conf/mapr.keytab</value>    <!-- path to the YARN keytab -->

</property>
<property>
 <name>yarn.nodemanager.principal</name>
 <value>mapr/_HOST@YOUR-REALM.COM</value>
</property>
<property>
 <name>yarn.nodemanager.container-executor.class</name>
 <value>org.apache.hadoop.yarn.server.nodemanager.LinuxContainerExecutor</value>
</property>
<property>
 <name>yarn.nodemanager.linux-container-executor.group</name>
 <value>mapr</value>
</property> 

Configure the mapred-site.xml File

Add the following properties to the mapred-site.xml file on every node in the cluster  (/opt/mapr/hadoop/hadoop-2.7.0/etc/hadoop/mapred-site.xml).

Note that you need to use /opt/mapr/conf/mapr.keytab for the keytab property and mapr instead of yarn for the principal property.

<!-- MapReduce Job History Server security configs -->
<property>
 <name>mapreduce.jobhistory.address</name>
 <value>host:port</value> <!-- Host and port of the MapReduce Job History Server; default port is 10020  -->
</property>
<property>
 <name>mapreduce.jobhistory.keytab</name>
 <value>/opt/mapr/conf/mapr.keytab</value>    <!-- path to the YARN keytab -->

</property>
<property>
 <name>mapreduce.jobhistory.principal</name>
 <value>mapr/_HOST@YOUR-REALM.COM</value>
</property> 

Restart ResourceManager, NodeManager, and JobHistoryServer

Restart the NodeManager, ResourceManager, and JobHistoryServer services, using either the maprcli node services command (with the name option) or the MCS.

After restarting the services, make sure you can run simple Hadoop jobs by running:

hadoop jar /opt/mapr/hadoop/hadoop-0.20.2/hadoop-0.20.2-dev-examples.jar pi