MapR 5.0 Documentation : ulimit

ulimit is a command that sets limits on the user's access to system-wide resources. Specifically, it provides control over the resources available to the shell and to processes started by it.

The mapr-warden script uses the ulimit command to set the maximum number of file descriptors (nofile) and processes (nproc) to 64000. Higher values are unlikely to result in an appreciable performance gain. Lower values, such as the default value of 1024, are likely to result in task failures.

MapR's recommended value is set automatically every time warden is started.

Depending on your environment, you might want to set limits manually rather than relying on Warden to set them automatically using ulimit. The following examples show how to do this, using the recommended value of 64000.

Setting resource limits on Centos/Redhat
  1. Edit /etc/security/limits.conf and add the following line:
    <MAPR_USER> - nofile 64000
  2. Edit /etc/security/limits.d/90-nproc.conf and add the following line:
    <MAPR_USER> - nproc 64000
  3. Check that the /etc/pam.d/system-auth file contains the following settings:
       auth            sufficient
       # Uncomment the following line to implicitly trust users in the "wheel" group.
       #auth           sufficient trust use_uid
       # Uncomment the following line to require a user to be in the "wheel" group.
       #auth           required use_uid
       auth            include         system-auth
       account         sufficient uid = 0 use_uid quiet
       account         include         system-auth
       password        include         system-auth
       session         include         system-auth
       session         required
       session         optional
Setting resource limits on Ubuntu
  1. Edit /etc/security/limits.conf and add the following lines:
    <MAPR_USER> - nofile 64000
    <MAPR_USER> - nproc 64000
  2. Edit /etc/pam.d/su and uncomment the following line:
    session    required

Use ulimit to verify settings:

  1. Reboot the system.
  2. Run the following command as the MapR user (not root) at a command line:
    ulimit -n

The command should report 64000.