MapR Monitoring Tips and Troubleshooting

Monitoring a Secure Cluster

After regenerating the MapR user ticket, service failures occur for collectd and OpenTSDB
If you delete or regenerate the MapR user ticket, the running collectd and OpenTSDB services will fail. After updating the MapR user ticket, restart the collectd and OpenTSDB services.

Monitoring Logs

I notice a sudden increase in fluentd logs. What can I do?
A sudden increase in the log file for fluentd could mean that a feedback loop is occurring where fluentd logs an error in the log file for a fluentd issue and that log entry causes yet another error when fluentd tries to parse it. In this case, consider disabling the index of fluentd logs. See Configure Which Logs to Index.

Monitoring Metrics

Where should I store the Elasticsearch index?
Elasticsearch requires a lot of disk space. Also, when you upgrade Elasticsearch, the default index directory is removed along with the package update. Therefore, it is recommended to configure a separate file system for the index data. It is not recommended to store index data under the / or the /var file system.
Note: If you store the Elasticsearch index on a filesystem that is locally hosted, you will be able to access logs in the event that the MapR cluster is not available.
For more information about the Elasticsearch index and the default index directory, see Log Aggregation and Storage.
I see a "Bad Request" error message for my MapR Database metrics? What can I do?
If you have more than 1000 active tables in MapR Database and the MapR monitoring request size to OpenTSDB is more than 4 KB, you may see the following error message:
"Sorry but your request was rejected as being invalid.
The reason provided was: Chunked request not supported."  
You can increase the maximum request size of OpenTSDB to up to 64 KB by setting the following parameters in the opentsdb.conf file:
tsd.http.request.enable_chunked=true
tsd.http.request.max_chunk=65536
For more information, see the OpenTSDB configuration guide.

Installation and Configuration Errors

See Troubleshoot MapR Monitoring Installation Errors