MapR Installer Known Issues

This topic describes some MapR Installer known issues that you should be aware of while troubleshooting.

Issue Description
IN-1915 During an upgrade using the MapR Installer, refreshing the browser page can cause the Installer to forget upgrade parameters that were specified before the refresh.

Workaround Avoid refreshing the browser page during an upgrade operation. If you must refresh the page, go back to the first page of the upgrade operation and start over again to ensure that the Installer has the correct parameters before it begins the Verify phase of the upgrade.

IN-2035 During a version upgrade using the MapR Installer, if you select the Advanced Configuration button and then click Previous (one or more times) followed by Abort, the Installer can indicate that the upgrade completed even though the upgrade was aborted.
Workaround If this happens, you must reset the installer and reload the last known state. Follow these steps to reset the cluster state:
  1. Click Support > Reset Installer. A warning screen appears.
  2. Click OK.
  3. Click Support > Import State.
  4. Click Reset to recover the cluster to the last known state. It is safe to retry the upgrade at this point.
For more information about the Reset Installer and Import State commands, see Resetting the Installer Database and Importing or Exporting the Cluster State.
IN-2065 /mapr sometimes does not get mounted after you enable NFS (v3 or v4) using the MapR Installer Incremental Install function. The Incremental Install function is an online operation. Enabling NFS using an Incremental Install can create a race condition between when the mapr_fstab file gets created and NFS is started by Warden. If NFS is started by Warden before the mapr_fstab file is created, /mapr does not get mounted.

Workaround If /mapr is not mounted, check the time stamp of the /opt/mapr/conf/mapr_fstab file to see if it is older than the time stamp in the warden.log file for starting NFS. For example:

[root@atsqa4-61 logs]# ls -ld /opt/mapr/conf/fstab
rw-rr- 1 mapr mapr 39 Sep 26 11:31 /opt/mapr/conf/mapr_fstab

[root@atsqa4-61 logs]# fgrep starting warden.log | fgrep nfs
2018-09-26 11:29:33,407 INFO com.mapr.warden.service.baseservice.Service [Thread-34]: -------------Service is starting for: nfs4
If the time stamp of the mapr_fstab file is older than the Warden time stamp:
  1. Restart the NFS service:
    maprcli node services -nodes <node names> -nfs4 start
  2. Run the mount_local_fs.pl script to mount /mapr:
    /opt/mapr/bin/mount_local_fs.pl
INFO-420 The procedure for Configuring Storage on a Node using disksetup does not work for new installations of DARE-enabled MapR 6.1 clusters. With DARE enabled, disksetup fails on any node that is not a CLDB node because there is no local copy of the dare.master.key file. When you use disksetup, non-CLDB nodes try to contact the CLDB, which must be running when the nodes attempt contact.
Workaround After running configure.sh, you must:
  1. Format the disks on the CLDB nodes.
  2. Start ZooKeeper on the ZooKeeper nodes.
  3. Start Warden on the CLDB nodes.
  4. Format the remaining node disks using disksetup.
  5. Start Warden on the remaining nodes.
IN-2057 A fresh install of MapR 6.0.0 using the sample_advanced.yaml file for MapR Installer Stanzas (Installer version 1.9) can fail with the following error message:
ERROR: install command failed
Service mapr-data-access-gateway must be a member of a template group. Configured services require it: ['mapr-data-access-gateway']
The error is generated because the .yaml file is missing an entry for the mapr-data-access-gateway in the MASTER services section. The mapr-data-access-gateway service is needed for MapR-DB installations.

Workaround In the MASTER services section of the sample_advanced.yaml file, add mapr-data-access-gateway to at least one of the host groups, and retry the installation.

IN-1272 During an upgrade to MapR 6.0 or later (Drill 1.11), configure.sh sometimes fails to disable the storage plugin for HBase. The HBase server is not supported in MapR Core 6.0 or later, so the HBase storage plugin should be disabled before a cluster is upgraded to MapR 6.0 or later. Otherwise, Drill queries against HBase will hang.
Workaround Before upgrading to Drill 1.11 or later, manually disable the HBase storage plug-in. To manually disable the plug-in, you can use the Drill Web Console or Drill REST API commands. You can disable the HBase storage plugin on the Storage page of the Drill Web Console at http(s)://<drill-hostname>:8047. For more information, see this page:
https://drill.apache.org/docs/rest-api-introduction/#delete-/storage/{name}.json
IN-1747 If you use the MapR Installer 1.10 Uninstall button to uninstall MapR software and a node is unreachable, you will not be able to uninstall the node later when the node is reachable.

Workaround Uninstall the MapR software on the node manually. See Decommissioning a Node and Uninstalling MapR Software from the Command-line.

IN-2015 A fresh install of MapR 6.0.0 with MEP 4.1.1 using MapR Installer 1.9 can fail with the following error message:
file not found: /opt/mapr/elasticsearch/elasticsearch-5.4.1/etc/elasticsearch/sg/admin-usr-clientCombo.pem

Workaround Update the installer to version 1.10 or later, and retry the operation.

IN-2018 Logging on to Kibana results in an authentication failure. This can happen on a CentOS cluster if you use MapR Installer 1.10 to install MapR 6.0.1 MEP 5.0.0, and then upgrade to MapR 6.1.0 and MEP 6.0.0.

Workaround Try using Mapr Installer 1.9 to install the MapR 6.0.1 cluster and MapR Installer 1.10 to upgrade the cluster. See Updating the MapR Installer.

CORE-150 After using the Incremental Install function of the MapR Installer to apply security to an Azure-based MapR 6.1.0 cluster, the Hue and Spark-thrift server links are not accessible in the Installer interface. This issue can occur on an Azure-provisioned cluster whose internal DNS suffix starts with a number rather than a letter.

Workaround Re-create the cluster in Azure so that the internal DNS suffix starts with a letter and not a number.

IN-2025 The Extend Cluster operation can fail during the Verify Nodes phase with an error indicating Unscalable host groups found. This error can occur when the MASTER group is missing or a single-instance service (for example, Grafana) has been moved out of the MASTER group. The mapr-installer.log reveals which MapR services are supposed to be in the MASTER group.

Workaround Move any original MASTER services that caused the error back to the MASTER group. The mapr-installer.log indicates the services that need to be moved along with the Unscalable host groups found error.

IN-2006 On a cluster with mapr-drill installed, the probe command can return the wrong database type value.
Workaround After using the probe command, check to see if the resulting YAML file has the correct mapr_db setting. Possible settings are:
  • QS
  • DRILL
  • DRILLQS

If necessary, change the setting in the YAML file to match the value from the probed cluster.

IN-1955 If you install MapR software using the MapR Installer in a browser and then upgrade the MapR installer in the same browser tab and attempt an upgrade without starting a new browser, the stale browser cache can cause upgrade errors.

Workaround Clear your browser cache or open a new browser tab whenever you need to update the MapR Installer and perform a new installer operation.

IN-1983 After an upgrade from MapR 5.x to MapR 6.1 and MEP 6.0.0 using the MapR Installer, the kafka-connect service fails to start. This issue has been noticed on platforms that use systemd.

Workaround Stop the kafka-connect service manually, and restart the service.

IN-1972 During an upgrade from MapR 5.x to MapR 6.1, the MapR Installer prompts you for the MySQL user ID and password. If you enter a password that is different from the password you provided when you originally configured MySQL through the MapR Installer, the upgrade fails with this error: "Unable to connect to database.…"

Workaround When the MapR Installer prompts you for the MySQL user ID and password, enter the password that you specified when you first installed the cluster. If you did not specify a password for MySQL when you installed MapR 5.x, leave the password field blank.

IN-1904 If you initiate a system startup by clicking the Startup button on the MapR Installer web interface, the Authentication screen is displayed. If you subsequently click the Previous button, the following buttons are shown as active even though they are not usable during system startup:
  • Extend Cluster
  • Incremental Install
  • Maintenance Update
  • Shutdown
  • Uninstall

Workaround: Do not use the Previous button during startup.

IN-1657 After updating MapR Installer 1.7 or later, the Installer can lose awareness that a cluster was previously installed. For example, the MapR Installer might indicate the need for a fresh install.
Workaround: If this happens, do NOT proceed with installation or upgrade operations. Follow these steps to reset the cluster state:
  1. Click Support > Reset Installer. A warning screen appears.
  2. Click OK.
  3. Click Support > Import State.
  4. Click Reset to recover the cluster to the last known state. It is safe to use the MapR Installer at this point.
IN-1084 For MapR 6.0 or later clusters, enabling security by using the Incremental Install function can overwrite custom certificates in the ssl_truststore and ssl_keystore files. When you turn on security, the MapR Installer runs the configure.sh script on the CLDB master node to generate security keys and then distributes the keys to all the other CLDB nodes. The installer also distributes certificates to all the other nodes. This process can cause custom certificates to be overwritten. However, before enabling security, the MapR Installer makes a backup of the exisiting ssl_keystore and ssl_truststore files.
Workaround: After enabling security, locate the backup of the ssl_keystore and ssl_truststore files. The backup uses this format:
/opt/mapr/conf/ssl_keystore.sv.<timestamp>
Extract any custom certificates from the backup files, and manually merge or add them into the new ssl_keystore and ssl_truststore files.

To merge the files, you can use the /opt/mapr/server/manageSSLKeys.sh utility, as shown in Configuring Secure Clusters for Running Commands Remotely.

IN-997 When using MapR Installer 1.9 with Ubuntu distributions, an upgrade of the MapR Installer definitions requires a restart of the installer service. The restart is needed because the MapR Installer services version is not updated properly when you use either of the following commands:
  • mapr-setup.sh reload
  • apt-get install mapr-installer-definitions
Workaround: After installing or reloading the MapR Installer definitions, issue one of these commands to fix the services version:
  • service mapr-installer restart
  • systemctl restart mapr-installer
IN-1671 For MapR Installer 1.8 and earlier, installation on Ubuntu 16.04 can fail if Python 2 is not available or if the default is set to Python 3. The installer requires python and python-yaml to be installed on all nodes.
Workaround: To install the Python packages manually:
sudo apt-get install python python-yaml -y
IN-1336 The MapR Installer Retry function can be affected if the installer operation fails. Suppose you deselect a service during an Incremental Install operation. If the Incremental Install fails and you need to Retry, it's possible that the service will not be deselected (uninstalled).
Workaround: Manually remove (uninstall) the service by using one of these commands:
  • Red Hat / CentOS: yum remove
  • Ubuntu: apt-get remove
  • SUSE: zypper remove
IN-1392 During an Extend Cluster (add node) operation using the MapR Installer, if installation of the added node fails and you abort the operation, the installer can display the added node even though it did not get installed.

Workaround: When the MapR Installer indicates that a node did not get added correctly (typically during the Installation phase), select the node and click Remove Node. Then retry adding the node.

IN-1396 An installation using the MapR Installer fails with the following Ansible module error:
nValueError: need more than 1 value to unpack

Workaround: Check for syntax errors in the /etc/sysctl.conf file, which can cause an Ansible error when the MapR Installer is attempting to set various kernel parameters.

IN-1398 In the MapR Installer Verify Nodes page, if you click a host, the Disks Selected for MapR box in the right pane displays the disks that were specified for the host either manually or automatically. If you deselect a disk in the right pane and click Retry, the deselection is not always implemented.

Workaround: Click Previous to go back to the Node Configuration page, and re-specify the disks that you want. Then continue with the operation.

IN-1386 On a secure MapR cluster, YARN jobs can fail if you specify IP addresses rather than host names when you configure nodes using the MapR Installer.

Workaround: Do not use an IP address for node configuration with the MapR Installer. If you already used an IP address, change the IP address in the yarn-site.xml file on all nodes. In the following example, the 10.10.10.7 IP address must be changed to a host name, such as bld73.qa.lab:

<property>
<name>yarn.timeline-service.hostname</name>
<value>10.10.10.73</value>
</property>
IN-1333 On Ubuntu clusters, the mapr-setup.sh script fails to reload the MapR Installer definitions during an update of the MapR installer and definitions.
Workaround: After updating, restart the installer to load the definitions:
service mapr-installer restart
IN-907 The MapR Installer service fails if the mapr user or root user already exist and they are configured to use a shell other than bash. For more information about user requirements, see the MapR Installer Prerequisites and Guidelines.
IN-1079 Verification fails when the installed language pack is for a language other than English.
Workaround: Remove the non-English language pack and install the English language pack. In the following example, the non-English language pack is shown as German. Also, make sure your system locale is set to en_us, as described in Infrastructure.
sudo apt-get install language-pack-en language-pack-en-base manpages
sudo apt-get remove language-pack-de language-pack-de-base manpages-de
IN-804 Using the Incremental Install operation to add a third node to a CONTROL group generates an error: ERROR: configure_refresh.sh failed. This issue applies to MapR Installer versions 1.6 and earlier.

Workaround: Update the MapR Installer to version 1.7 or later, and retry the operation. See Updating the MapR Installer.

IN-1314 When you use the MapR Installer to install ecosystem components that require a MySQL component such as Hive, Oozie, or Hue, the passwords you provide to install the MySQL database are displayed in the mapr-installer.log and <nodename>.log files. Beginning with MapR Installer 1.7, the permissions for the mapr-installer.log and <nodename>.log files are changed so that these passwords are not world readable. However, the passwords are still present in log files created with earlier versions of the MapR Installer.

Workaround: For increased security, remove the earlier logs or change the user permissions for them.

IN-646 An upgrade using the MapR Installer can fail if ansible processes hang. Performing a service mapr-installer restart does not help. The installer-process.log indicates Lock file /opt/mapr/installer/data/mapr-installer.lock exists.
IN-1042 Installation of the 5.2.x mapr-metrics package on SUSE 12 SP2 fails because the libmysqulclient16 package is not present. This can happen when mapr-metrics is installed manually or using the MapR Installer. This issue was detected during installations of MapR 5.2.x with MEP 3.0.0.

Workaround: None.

IN-870 If your cluster uses 2-digit MEPs, and you use the MapR Installer Extend Cluster button to add a node, the node can be added with a patch version that is different from the patch version of other nodes in the cluster. See Understanding Two-Digit and Three-Digit MEPs.

A one-time change can prevent this issue. After updating the MapR Installer from version 1.5 to version 1.6 or later, but before performing any MapR Installer operations, use an Incremental Install to change your 2-digit MEP version to the equivalent 3-digit MEP version. See Updating the MapR Installer.

ES-27, IN‑1387 On a new installation of a secure cluster using the MapR Installer, Elasticsearch fails to start, and logs indicate that Elasticsearch key generation failed. When this happens, Kibana and Fluentd also do not start. The MapR Installer allows the installation to complete.

Workaround: Check the installer log for a message indicating that Elasticsearch could not be secured. Use the Incremental Install feature of the MapR Installer to retry installation of the MapR Monitoring logging components. Alternatively, you can configure security for the logging components manually. See Step 9: Install Log Monitoring.

IN-1332 On clusters with less than the recommended memory configuration (16 GB per node), services added during an Incremental Install operation might fail to start because Warden allocated available memory to the MapR file system. The MapR Installer might not indicate a problem with the newly added services. If this issue occurs, the MapR file system cannot relinquish memory without restarting Warden.
Note: This issue can also occur on clusters with more than 16 GB of memory per node if the installed services require more memory than is currently installed.

Workaround: Use MCS or the maprcli service list -node command to determine if the added services are running. If not, perform a rolling restart of the nodes to which the new services were added. The rolling restart will rebalance memory across the file system and services. One node at a time, restart Warden on each node following the group upgrade order prescribed in Manual Rolling Upgrade Description. Use the following steps:

  1. Change to the root user (or use sudo for the following commands).
  2. Stop Warden.
    sudo service mapr-warden stop
  3. Restart Warden.
    service mapr-warden start
IN-1339 Installation fails with the MapR Installer reporting an Unexpected failure during module execution, and the following entry is present in the Logs for the MapR Installer:
os.write(self.sshpass_pipe[1], to_bytes(self._play_context.password) + b'\n')
OSError: [Errno 9] Bad file descriptor

Workaround:

Change the ssh settings as described in known issue IN-405 later on this page, and retry the installation.
IN-553 New installations on Ubuntu 14.04 using MapR Installer 1.6 or 1.7 can fail because of a JDK 1.8 issue.

Workaround: If you are installing on Ubuntu 14.04, you must install Java JDK 1.8 before running the MapR Installer. For more information, see this website. If you are installing on RHEL/CentOS or SUSE, the MapR Installer installs Java JDK 1.8 for you.

IN-405 MapR installation or cluster import fails with the error message: "Failed to resolve remote temporary directory from ansible-tmp- ...."

Workaround: To proceed using the MapR Installer, disable SSH connection reuse by including this entry underneath the [ssh_connection] property of /opt/mapr/installer/etc/ansible.cfg:

ssh_args=-o ControlMaster=no -o ControlPath=none -o ControlPersist=no

This workaround can lead to longer install times. We recommend that you resolve any network connectivity issues in your environment.

IN-250 An upgrade to a new MapR core version and a new MEP using the MapR Installer can fail if the cluster being upgraded was initially installed with Hive Metastore but not with Hive. The Hive Metastore package has an installation dependency on Hive, but the Hive Metastore definitions do not enforce the dependency, resulting in inconsistencies in the installer database. This issue has been observed on Ubuntu platforms.

Workaround: Before upgrading, if you have Hive Metastore installed by itself, use the Incremental Install feature of the MapR Installer to install Hive. Then proceed with the upgrade.

Performing an upgrade without doing the Incremental Install of Hive will cause the upgrade to fail. In this scenario, you will have to reinstall or rebuild the database by using Stanza commands. You can use the reset command, followed by probe, and then edit the versions in the resulting YAML file. The last step is to import the edited YAML using the import command. See Using probe and import to Generate the Installer Database.

N/A The MapR Installer Web Interface can inadvertently deselect services that you have selected, preventing them from being installed. For example, if you select an auto-provisioning template on the Select Services page, and you also select additional services (for example, Streams Tools), and go to the next page, when you return to the Select Services page, Streams Tools will be deselected, and you will need to reselect it to ensure that it is installed.
MapR‑22652 The MapR Installer does not prevent the selection of Impala 2.2 when the cluster will be installed on Ubuntu nodes. Since Impala 2.2 is not supported on Ubuntu, the installation will not complete successfully.
MapR‑20606 The Configure Service Layout page may assign services to a group with the name "Unprovisioned Services."

Workaround: In the MapR Installer Web Interface, click Restore Default.

N/A You cannot use the MapR Installer after you upgrade the cluster using the command line.

After you use the command line to upgrade a cluster that you installed with the MapR Installer, the MapR Installer is not aware that the cluster runs the upgraded version. Therefore, MapR Installer does not install nodes and ecosystem components that apply to the upgraded version.

Workaround: Use the MapR Installer Stanzas probe and import commands to update the installer database. See Using probe and import to Generate the Installer Database.

MapR-19727 On nodes running RedHat/CentOS 6.5 or older, JDK 1.7 may cause an SSL connection error when you attempt to access the MapR Installer web interface.

Workaround:

  1. Install JDK 1.8.
  2. Use the alternatives system to set JDK 1.8 as the default Java. Or, uninstall JDK 1.7.
MapR-18668 Hue does not work on RedHat/CentOS 7 when it is configured to use a MySQL database.

When this issue occurs, the MapR Control System (MCS) displays the "Hue Down Alarm."

Workaround:
  1. Run the following commands to install MariaDB and the Redhat 6 compatibility library:
    yum install mariadb
    ver=$(rpm -qa mariadb|cut -d- -f2)
    rpm -ivh --nodeps http://yum.mariadb.org/$ver/rhel7-amd64/rpms/MariaDB-$ver-centos7-x86_64-compat.rpm 
  2. Run the following command to create a symlink for the Cyrus SASL library:
    ln -s /lib64/libsasl2.so.3.0.0 /lib64/libsasl2.so.2
  3. Run the following command to reconfigure Hue:
    bash -c "source /opt/mapr/hue/hue-<version>/build/env/bin/activate;
          /opt/mapr/hue/hue-<version>/build/env/bin/hue syncdb --noinput;
          /opt/mapr/hue/hue-<version>/build/env/bin/hue migrate"
  4. Run the following command to restart Hue:
    maprcli node services -name hue -action restart -nodes <space separated list of
          hostnames>
MapR-18574 Installation may fail on nodes where there is a package manager running or a stale lock file exists. When this type of failure occurs, the installation log for that node may contain an error message similar to the following:
2015-05-08 01:50:22.713: failed: [command /usr/bin/apt-get install -q -y {{mapr.node.core_packages|sort|join(' ') }}] [10.10.10.162] => {"changed": true,"cmd": ["/usr/bin/apt-get", "install", "-q", "-y", "mapr-cldb",
      "mapr-core","mapr-fileserver", "mapr-httpfs", "mapr-nfs", "mapr-nodemanager","mapr-resourcemanager", "mapr-webserver", "mapr-zookeeper"], "delta":"0:00:05.432067", "end": "2015-05-08 01:50:22.676193", "rc": 100,
      "start":"2015-05-08 01:50:17.244126", "warnings": ["Consider using apt-get modulerather than running apt-get"]}
Workaround: To clean up a running package manager process, run apt-get or yum on a shell and follow the instructions provided by the package manager.
Note: When you select a node on the “Installing MapR” page, the option to view the installation log for that node displays in the left panel.
MapR-18507 Metrics does not work on these operating systems: Ubuntu, RedHat 7, and SUSE/SLES 12.
MapR-18388 Known issues with spark-standalone and multiple spark masters:
  • After installation, Spark workers may not start due to the mechanism used to start services.
  • Spark worker does not start after the node that runs the worker is restarted.
Workaround:To start Spark workers, run the following command as the mapr user on each worker node:
/opt/mapr/spark/spark-$VERSION/sbin/start-slave.sh 1 spark://<comma-separated list of spark
      master hostname: port> 
Example:
/opt/mapr/spark/spark-$VERSION/sbin/start-slave.sh 1
     spark://host1:7077,host2:7077 

It is important to list all the spark master hostnames so that the worker nodes can connect to the active master.

You can run the following command to determine the nodes that run the spark master:
maprcli node list -columns service -filter '[service==spark-master]' -json | grep
      \"hostname\": | cut -d \" -f 4