MapR 5.0 Documentation : Rolling MapR Core Upgrade Using Manual Steps

The rolling upgrade is supported for upgrades from 4.0.x or 4.1 to 5.0  If you are upgrading from an earlier version, see Offline MapR Core Upgrade Using Manual Steps.  In a manual rolling upgrade, you upgrade the MapR software one node at a time so that the cluster as a whole remains operational throughout the process. The fileserver service on each node goes offline while packages are upgraded, but its absence is short enough that the cluster does not raise the data under-replication alarm. For an automated approach to rolling upgrades, see the Rolling MapR Core Upgrade Using a Script

This topic includes the following sections:

Manual Rolling Upgrade Process

The manual rolling upgrade steps reflect the behavior of the scripted rolling upgrade. When you upgrade each node, you also follow the same group upgrade order and the packages upgrade order as the scripted rolling upgrade. 

Group Upgrade Order

Upgrade cluster nodes in groups based on the services running on each node. Upgrade groups of nodes in the following order:

GroupNodes in this Group
1Each node only has Zookeeper. This establishes a stable ZooKeeper quorum on the new version, which will remain active through the rest of the upgrade process.
2

Each node only has Fileserver or it has Fileserver and Zookeeper.

3

Each node only has TaskTracker or it has TaskTracker, FileServer, and Zookeeper.

4Each node only has NodeManager or it has NodeManager,TaskTracker, FileServer, and Zookeeper
5Each node only has JobTracker or it has JobTracker,NodeManager, TaskTracker, FileServer, and Zookeeper. When you upgrade nodes in this groups, upgrade nodes with the standyby JobTrackers before you upgrade the node with the active JobTracker.
6Each node only has ResourceManager or it has ResourceManager, JobTracker, NodeManager, TaskTracker, FileServer, and Zookeeper. When you upgrade nodes in this groups, upgrade nodes with the standyby ResourceManagers before you upgrade the node with the active ResourceManager.
7

Each node only has HBase Region server or it has HBase Region server, ResourceManager, JobTracker,NodeManager,TaskTracker, FileServer, and Zookeeper.

8Each node only has HbaseMaster or it has HbaseMaster,Hbase Region server ,ResourceManager, JobTracker,NodeManager,TaskTracker, FileServer, and Zookeeper.
9

Each node only has CLDB server or it has CLDB server, HBaseMaster,HBase Region server,ResourceManager, JobTracker,NodeManager, TaskTracker, FileServer, and Zookeeper. When you upgrade nodes in this group, upgrade nodes with the slave CLDB before you upgrade the node with the master CLDB.

Package Upgrade Order

When you upgrade each node, upgrade existing packages in the following order: 

  • On all operating systems except SUSE, upgrade the mapr-core package first. Subsequent packages can be done in any order.
  • On SUSE, upgrade the mapr-compat-suse package first and the mapr-core package second. Subsequent packages can be done in any order.

The following is a list of the primary packages:

  • mapr-cldb
  • mapr-compat-suse (if upgrading on SUSE)
  • mapr-core-internal
  • mapr-core
  • mapr-fileserver
  • mapr-hadoop-core
  • mapr-historyserver
  • mapr-jobtracker
  • mapr-mapreduce1
  • mapr-mapreduce2
  • mapr-metrics
  • mapr-nfs
  • mapr-nodemanager
  • mapr-resourcemanager
  • mapr-tasktracker
  • mapr-webserver
  • mapr-zk-internal
  • mapr-zookeeper

Before you Upgrade

Perform the following steps before you begin the upgrade:

  • Based on the group upgrade order, determine the upgrade groups. To see a list of services on each node, you can run the following command: 

     maprcli node list -columns hostname,csvc
  • If the cluster is secure, the MapR Admin user must have a security ticket created before running the upgrade. Otherwise, some upgrade commands will not run.

Upgrade Steps

Complete the following upgrade steps for each node in each upgrade group. 

  1. Download the archive file.
  2. To extract the archive file, run the following command:   

    tar -xzvf <archive file>

    Note: When you upgrade each package, you will need to specify the full path to files in this local directory.  

  3. Determine if there is a patch installed.
    • On Red Hat / CentOS and SUSE, run the following command: 

      rpm -qa mapr-patch

      If no output displays, there is no patch installed.

    • On Ubuntu, run the following command: 

      dpkg l | grep mapr-patch
  4. To get the default mapreduce mode for the cluster, run the following command:

    maprcli cluster mapreduce get
  5. If the node runs TaskTracker and the mapreduce cluster setting is classic, run the following command as the MapR administrator to blacklist the node:

     hadoop job -blacklist-tracker <hostname>
  6. To set the node to maintenance mode, run the following command:

    sudo maprcli node maintenance -nodes <hostname> -timeoutminutes 30
  7. To notify the CLDB that the node is going to be upgraded, run the following command:

    sudo maprcli notifyupgrade start -node <hostname>
  8. To stop Warden, run the following command:

    sudo service mapr-warden stop
  9. If Zookeeper is installed on the node, run the following command:

    service mapr-zookeeper stop
  10. If patches are installed on the node, remove all patches.
    • On CentOS/Redhat or SUSE, run the following command:

      sudo rpm -e mapr-patch  
    • On Ubuntu, run the following command:

      sudo apt-get -y remove mapr-patch
  11. Upgrade each MapR package on the node based on the defined package upgrade order
    • On CentOS/RedHat or SUSE, run the following command for each package:

      sudo rpm --quiet --force --nosignature -U </FullPathToPackage/PackageName.rpm>
    • On Ubuntu, run the following command for each package:

      sudo dpkg --force-all -i </FullPathToPackage/PackageName.rpm>
  12. To configure the node, run the following command:

    sudo /opt/mapr/server/configure.sh -R
  13. If Zookeeper is installed on the node, run  the following command to start Zookeeper:

    service mapr-zookeeper start
  14. To start Warden, run the following command:

    sudo service mapr-warden start 
  15. Before you proceed to the next step, determine if the CLDB is running. Issue a maprcli command such as maprcli node list. If an output is displayed, the CLDB is running. 
  16. To unset maintenance mode on the node, run the following command:

    sudo maprcli node maintenance -nodes <hostname> -timeoutminutes 0
  17. To notify the CLDB about the upgraded version, run the following command: 

    sudo maprcli config save -values {mapr.targetversion:"`cat /opt/mapr/MapRBuildVersion`"}
  18. To notify the CLDB that the upgrade process is complete on the node, run the following command:

    sudo maprcli notifyupgrade finish -node <hostname>
  19. Wait for the containers to synchronize. 
    When the containers are synchonized, the following command displays no output: /opt/mapr/server/mrconfig info containers resync local

  20. If this node was previously blacklisted, run the following command as the MapR administrator to remove it from the blacklist:

    hadoop job -unblacklist-tracker <hostname>

After you Upgrade

After you complete the manual rolling upgrade steps, complete the following steps:

  1. Update the warden configuration files which are located in /opt/mapr/conf/ and /opt/mapr/conf/conf.d.

    • Manually merge new configuration settings from the /opt/mapr/conf.new/warden.conf file into the /opt/mapr/conf/warden.conf file.

    • Manually merge new configuration settings from the files in the /opt/mapr/conf/conf.d.new/ directory to the files in the /opt/mapr/conf/conf.d/ directory.

       

  2. For upgrades from 4.0.1, any script that points to files within the /opt/mapr/hadoop-2.4.1 directory must be updated to point to the new hadoop 2.x directory.

When you upgrade hadoop common, a new directory is created for the new hadoop common version and the configuration files in the existing /opt/mapr/hadoop/hadoop-2.x.x directory are automatically copied into the active directory associated with the new hadoop 2.x.x directory. For example, when you upgrade from 4.0.1 from 5.0, configuration files that were in the hadoop-2.4.1 directory are copied into the hadoop-2.7.1 directory.



-->Back to Upgrading Without the MapR Installer