New in This Release
rollingupgrade.sh upgrades a MapR cluster to a specified version of the MapR software, or to a specific set of MapR packages, either via SSH or node by node. This makes it easy to upgrade a MapR cluster with a minimum of downtime.
The MapR Client can now be installed on both 64-bit and 32-bit computers. See MapR Client.
Core File Removal
- Removing Nodes
- (Issue 4068) Upgrading Red Hat
- (Issue 3984) HBase Upgrade
- (Issue 3965) Volume Dump Restore Failure
- (Issue 3890) Sqoop Requires HBase
- (Issue 3560) Intermittent Scheduled Mirror Failure
- (Issue 2949) NFS Mounting Issue on Ubuntu
- (Issue 2815) File Cleanup is Slow
(Issue 4415) Select and Kill Controls in JobTracker UI
The Select and Kill controls in the JobTracker UI appear when the
webinterface.private.actions parameter in
mapred-site.xml is set to
true. In MapR clusters upgraded from the beta version of the software, the parameter must be added manually for the controls to appear.
To enable the Select and Kill controls in the JobTracker UI, copy the following lines from
(Issue 4307) Snapshot create fails with error EEXIST
EEXIST error indicates an attempt to create a new snapshot with the same name as an existing snapshot, but can occur in the following cases as well:
- If the node with the snapshot's name container fails during snapshot creation, the failed snapshot remains until it is removed by the CLDB after 30 minutes.
- If snapshot creation fails after reserving the name, then the name exists but the snapshot does not.
- If the response to a successful snapshot is delayed by a network glitch, and the snapshot operation is retried as a result,
EEXISTScorrectly indicates that the snapshot exists although it does not appear to.
In any of the above cases, either retry the snapshot with a different name, or delete the existing (or failed) snapshot and create it again.
(Issue 4269) Bulk Operations
The MapR Control System provides both a checkbox and a Select All link for selecting all alarms, nodes, snapshots, or volumes matching a filter, even if there are too many results to display on a single screen. However, the following operations can only be performed on individually selected results, or results selected using the Select Visible link at the bottom of the MapR Control System screen:
- Volumes - Edit Volumes
- Volumes - Remove Volumes
- Volumes - New Snapshot
- Volumes - Unmount
- Mirror Volumes - Edit Volumes
- Mirror Volumes - Remove Volumes
- Mirror Volumes - Unmount
- User Disk Usage - Edit
- Snapshots - Remove
- Snapshots - Preserve
- Node Alarms - Change Topology
- Nodes - Change Topology
- Volume Alarms - Edit
- Volume Alarms - Unmount
- Volume Alarms - Remove
- User/Group Alarms - Edit
In order to perform these operations on a large number of alarms, nodes, snapshots, or volumes, it is necessary to select each screenful of results using Select Visible and perform the operation before selecting the next screenful of results.
(Issue 4037) Starting Newly Added Services
After you install new services on a node, you can start them in two ways:
- Use the MapR Control System, the API, or the command-line interface to start the services individually
- Restart the warden to stop and start all services on the node
If you start the services individually, the node's memory will not be reconfigured to account for the newly installed services. This can cause memory paging, slowing or stopping the node. However, stopping and restarting the warden can take the node out of service.
For best results, choose a time when the cluster is not very busy if you need to install additional services on a node. If that is not possible, make sure to restart the warden as soon as it is practical to do so after installing new services.
(Issue 4024) Hadoop Copy Commands Do Not Handle Broken Symbolic Links
hadoop fs -copyToLocal and
hadoop fs -copyFromLocal commands attempt to resolve symbolic links in the source data set, to create physical copies of the files referred to by the links. If a broken symbolic link is encountered by either command, the copy operation fails at that point.
(Issue 4018)(HDFS-1768) fs -put crash that depends on source file name
Copying a file using the
hadoop fs command generates a warning or exception if a corresponding checksum file
.*.crc exists. If this error occurs, delete all local checksum files and try again. See http://firstname.lastname@example.org/msg15824.html
(Issue 3524) Apache Port 80 Open
The MapR UI runs on Apache. By default, installation does not close port 80 (even though the MapR Control System is available over HTTPS on port 8443). If this would present a security risk to your datacenter, you should close port 80 manually on any nodes running the MapR Control System.
(Issue 3488) Ubuntu IRQ Balancer Issue on Virtual Machines
In VM environments like EC2, VMWare, and Xen, when running Ubuntu 10.10, problems can occur due to an Ubuntu bug unless the IRQ balancer is turned off. On all nodes, edit the file
/etc/default/irqbalance and set
ENABLED=0 to turn off the IRQ balancer (requires reboot to take effect).
(Issue 3244) Volume Mirror Issue
If a volume dump restore command is interrupted before completion (killed by the user, node fails, etc.) then the volume remains in the "Mirroring in Progress" state. Before retrying the
volume dump restore operation, you must issue the
volume mirror stop command explicitly.
(Issue 3122) Mirroring with fsck-repaired volume
If a source or mirror volume is repaired with
fsck then the source and mirror volumes can go out of sync. It is necessary to perform a full mirror operation with
volume mirror start -full true to bring them back in sync. Similarly, when creating a dump file from a volume that has been repaired with
-full true on the
volume dump create command.
(Issue 3028) Changing the Time on a ZooKeeper Node
To avoid cluster downtime, use the following steps to set the time on any node running ZooKeeper:
- Use the MapR Dashboard to check that all configured ZooKeeper services on the cluster are running. Start any non-running ZooKeeper instances.
- Stop ZooKeeper on the node:
service mapr-zookeeper stop
- Change the time on the node or sync the time to NTP.
- Start ZooKeeper on the node:
service mapr-zookeeper start
(Issue 2809) NFS Dependencies
If you are installing the MapR NFS service on a node that cannot connect to the standard apt-get or yum repositories, you should install the following packages by hand:
- Red Hat:
When a node's IP number is reassigned to another node, remote procedure calls are misdirected to the wrong node. This can result in widespread errors affecting any system that use the RPC binding layer, including container resync operations that never finish or the creation of several superfluous containers. Restart the cluster to clear this condition.
If you use the latest version of any browser to view the MCS, the CLDB, HBase, JobTracker, and job log pages do not load when you click on the respective links. As a workaround, use the following instructions to disable mixed-content-blocking in your browser:
Scroll to the bottom of the screen, and click Show all content. The page refreshes and displays mixed content.
- At the top of the page, to the left of the address bar, click (the shield icon).
- In the pop-up window that appears, click the down arrow next to "Keep Blocking", and select Disable Protection on This Page. The page refreshes and displays mixed content. The shield icon is replaced with a warning symbol to indicate that the page is displaying both secure and non-secure items.
- Click (the shield icon) on the right side of the address bar.
- In the icon dialog box, click Load anyway. The page refreshes and displays mixed content. The URL in the address bar shows https crossed out to indicate that the page is displaying both secure and non-secure items.