MapR Monitoring Components - MEP 3.x.x Release Notes

The notes below relate specifically to the MapR Converged Data Platform.

Version The MEP 3.0.0 release contains the following monitoring component versions:
  • Collectd 5.7.1
  • Fluentd 0.14.00
  • Opentsdb 2.3.0
  • Elasticsearch 2.3.3
  • Grafana 4.1.2
  • Kibana 4.5.4
Release Date April 2017
MapR Version Interoperability MEP Components and OS Support
Package Names Package Names for MapR Expansion Packs (MEPs)

New in This Release

MapR Monitoring - MEP 3.0.0 release introduces the following enhancements:

  • New metrics are now available for MapR Event Store For Apache Kafka, disks, and topology.
  • You can now configure fluentd to forward logs to syslog servers.
  • Metrics are now tagged by default with ClusterName.
  • You can store custom collectd configuration files under the /opt/mapr/collectd/conf/ directory. Collectd will include configuration files from this directory at runtime and the files will not be altered during an upgrade.

Patches

None

Known Issues and Limitations

  • COLD-4: On nodes without support for the wordexp function, collectd does not read files from the custom configuration file directory (/opt/mapr/collectd/conf/).

    Workaround:

    1. Open the /opt/mapr/collectd/collectd-5.7.1/etc/collectd.conf file.
    2. Add the following lines to the end of the file:
      <Include "/opt/mapr/collectd/conf/">
       Filter "*.conf" 
      </Include>
    3. Restart the collectd service.
      maprcli node services -name collectd -nodes <space separated list of collectd nodes> -action restart 
  • KIB-1: On secure clusters, a syntax error in export_cert.sh causes in the following error message when you run configure.sh on a node where Kibana is installed:
     <TIMESTAMP>: ERROR: Failed to configure ssl for kibana
    This error appears in both the configure.sh console output and in the log file (/opt/mapr/logs/configure.log).

    Workaround:

    1. Open the /opt/mapr/kibana/kibana-4.5.4/bin/export_cert.sh file.
    2. Locate the following line:
      CLUSTERNAME=$(cat /opt/mapr/conf/mapr-clusters.conf | awk '{print $1} | head -n 1')
    3. Replace that line with the following line:
      CLUSTERNAME=$(cat /opt/mapr/conf/mapr-clusters.conf | awk '{print $1}' | head -n 1) 
      Notice that the location of the second backtick is different in this line.
    4. Save changes to the file and re-run configure.sh.
  • SPYG-838: On secure clusters, JMX ports can be accessed without authentication.
    Workaround: Complete the following steps to setup a password on each Node Manager and Resource Manager node:
    1. Under /opt/mapr/conf, create the following files:
      • jmxremote.access
      • jmxremote.password
    2. Set the owner and group to the mapr user:
      chown mapr:mapr jmxremote.access
      chown mapr:mapr jmxremote.password
    3. Set file permissions to read-only:
      chmod 400 jmxremote.access
      chmod 400 jmxremote.password
    4. Add the following entries to jmxremote.password:
      mapr mapr
      root mapr
      
    5. Add the following entries to jmxremote.access:
      mapr    readonly
    6. In the #Enable JMX for MaprMonitoring section of the yarn file (/opt/mapr/hadoop/hadoop-2.x.x/bin/yarn), update the JMX_OPTS parameter to the following:
       JMX_OPTS="-Dcom.sun.management.jmxremote -Dcom.sun.management.jmxremote.authenticate=true 
      -Dcom.sun.management.jmxremote.password.file=/opt/mapr/conf/jmxremote.password 
      -Dcom.sun.management.jmxremote.access.file=/opt/mapr/conf/jmxremote.access 
      -Dcom.sun.management.jmxremote.ssl=false 
      -Dcom.sun.management.jmxremote.port" 
    7. In collectd.conf file (/opt/mapr/collectd/collectd-<version>/etc/collectd.conf), add the following connection parameters:
      <Connection>
      user "mapr"
      password "mapr"
      ServiceURL "service:jmx:rmi:///jndi/rmi://mfs82.qa.lab:8025/jmxrmi"
      #IncludePortInHostname true
      Collect "QueueMetrics"
      ttl 120
      </Connection>
      <Connection>
      user "mapr"
      password "mapr"
      ServiceURL "service:jmx:rmi:///jndi/rmi://mfs82.qa.lab:8027/jmxrmi"
      #IncludePortInHostname true
      Collect "NodeManagerMetrics"
      ttl 120
      </Connection>
    8. Restart the Collectd, NodeManager, and Resource Manager services running on this node.

Resolved Issues

  • SPYG-757: CollectD no longer fails to retrieve Resource Manager metrics. CollectD determines which is the active Resource Manager before it collects ResourceManager metrics.
  • SPYG-806: After you install OpenTSDB, you no longer need to configure the clusterID, fqdnID, or VolumeID in the default dashboards.
  • SPYG-811: Grafana no longer expects OpenTSDB versions less than or equal to version 2.1. Dashboards now load as expected.
  • SPYG-609: Users no longer need to configure an index pattern before viewing logs in Kibana.