Increasing Log Retention

To increase log retention for specific component, you can modify the configuration file and reset the value of the properties.

Increasing Log Retention for MapR Filesystem

In the /opt/mapr/conf/mfs.conf file, increase the value for the property. The value for this property is computed using the following formula:

maxSizePerLogFile = maxLogSize / MAX_NUM_OF_LOG_FILES
  • maxLogSize specifies the total amount of space that MapR Filesystem log files can consume.
  • MAX_NUM_OF_LOG_FILES specifies the total number of MapR Filesystem log files (5 hard coded).

For example, for a value of 10 GB, there will be 5 log files (mfs.log-3 to mfs.log-3.4) of 2GB each.

Increasing Log Retention for CLDB

In the /opt/mapr/conf/ file, modify the value for the following properties:

  • log4j.appender.R.MaxFileSize — specifies the size of each cldb.log before rolling over
  • log4j.appender.R.MaxBackupIndex — specifies the number of cldb.log* before the oldest one is deleted.

For example, suppose the following configuration:

  • log4j.appender.R.MaxFileSize = 1024MB
  • log4j.appender.R.MaxBackupIndex = 10

The CLDB log files will grow to 10GB before they are purged.

Note: This setting does not impact audit logging.

Increasing Log Retention for Hadoop Services

You can increase log retention for ResourceManager, NodeManager, HistoryServer, and TimelineServer by modifying the following properties in the /opt/mapr/hadoop/hadoop-<Version>/etc/hadoop/ file:

  • hadoop.log.maxfilesize — specifies the size of each <service-name>-hostname>.log before rolling over.
  • hadoop.log.maxbackupindex — specifies the number of <service-name>-<hostname>.log* before the oldest one is deleted.

For example, suppose the following configuration:

  • hadoop.log.maxfilesize = 1024Mb
  • hadoop.log.maxbackupindex = 10

There will be 10 GB of total service-specific logs (1GB per file) before the oldest file is purged.