MapR 4.0.x Documentation : Upgrading HBase

 

This page contains the following topics describing how to upgrade HBase in the MapR distribution for Apache Hadoop:

Before you upgrade, make sure that the version of the MapR core software on your cluster supports the version of HBase you want to upgrade to. See the HBase Release Notes and the Ecosystem Support Matrix

Update Repositories or Download Packages

MapR's rpm and deb repositories always contain the HBase version recommended for the latest release of the MapR core. The Version 4.x repositories are located at http://package.mapr.com/releases/ecosystem-4.x. You can also prepare a local repository with any version of HBase you need. For more details on setting up repositories, see Preparing Packages and Repositories.

If you don't want to install from a repository, you can download the package file for the specific release you want and install it manually. Individual package files are located at http://package.mapr.com/releases/ecosystem-all/.

To update the repository cache

If you plan to install from a repository, update the repository cache on each node where HBase is installed.

On RedHat and CentOS...

yum clean all

On Ubuntu...

apt-get update

 

Migrating Configuration Files

If you have changed configuration properties on your current installation of HBase, you probably want to apply those changes to the updated version. Configuration properties are located in /opt/mapr/hbase/hbase-<version>/conf/.

In general, you can migrate your configuration changes with the following procedure:

  1. Before upgrade, save configuration files on all nodes where HBase is installed.
  2. Upgrade HBase software.
  3. Migrate custom configuration settings into the new default files in the conf directory.

Planning for Upgrade

Upgrading an established deployment of HBase requires planning and consideration before beginning the upgrade process. Below are items to consider as you plan to upgrade:

  • The MapR-FS filesystem provides native storage for table data as of MapR version 3.0. MapR tables are API-compatible with the Apache HBase, and have higher performance, are more versatile for developers, and reduce administrative burden, compared to Apache HBase. Before upgrading HBase, consider whether migrating to MapR tables is appropriate for your needs.
  • The topic of upgrading HBase is discussed in depth in Apache literature. This page covers details of upgrading the HBase packages included in the MapR distribution for Apache Hadoop. However, administrators need to consider migration of data and maintenance of service for HBase clusters. For details, refer to the Apache HBase Reference Guide.
  • The data formats for ROOT and META tables change between minor release boundaries of HBase (such as 0.92.x to 0.94.x). HBase handles the data migration process so it is transparent to the administrator. However, after upgrading you cannot downgrade to a previous version without also restoring the pre-upgrade data.
  • Perform health checks and address any concerns before upgrading HBase. As a start, run hbck to check for any inconsistencies in HBase data. Refer to hbck in Depth in the Apache HBase Reference Guide for usage details.

    hbase hbck
  • If you also plan to upgrade the MapR core as part of upgrading your HBase cluster, upgrade the MapR core first. After successfully upgrading the MapR core and verifying cluster health, upgrade the HBase component.
  • While planning to upgrade, it is a good time to review your cluster service layout and determine if the right services are running on the right set of nodes. For example, as your cluster grows, you will tend to isolate cluster-management services from compute services on separate nodes. Review Planning the Cluster and Installing HBase for details on planning the service layout.
  • Because the upgrade process takes HBase services offline and requires careful planning, perform a test upgrade on a development cluster to make sure you understand the process. After you have experienced success on a dev cluster, proceed with your production cluster.

Version-Specific Considerations

Before you upgrade the software, note if there are any version-specific considerations that apply to you. The Apache HBase Reference Guide has a section dedicated to version-specific upgrade considerations.

Upgrading to MapR v4.x

If you are upgrading your cluster to MapR v4.x or later, you also need to upgrade HBase. 

If you are currently using an earlier version of HBase than 0.94.17-mapr-1403, you must upgrade to 0.94.21-mapr-1407 or later. You must specify the version explicitly; by default, if you install HBase on a 4.0.x or 4.1 cluster, the most recent version of Hbase that is compatible with the cluster version will be installed

Upgrading from 0.98.7 to 0.98.9

  1. Complete the steps in Upgrading the Software.
  2. Run configure.sh -R on all of the upgraded HBase nodes:
    $ /opt/mapr/server/configure.sh -R

Upgrading from 0.94.x to 0.98.x

A rolling upgrade from 0.94.x directly to 0.98.x does not work because of incompatibilities between these HBase versions. For more details about the upgrade to 0.98.x, see the instructions in the HBase upgrade documentation. (Note that the procedures for upgrading to 0.98.x are the same as the procedures for upgrading to 0.96.x, but you will not be upgrading to 0.96.x.)

If your upgrade plan includes both a new version of the MapR distribution and a new version of HBase, follow these steps before upgrading the MapR software.

Follow these upgrade steps: 

  1. Install the MapR HBase 0.98 package on one node in your existing cluster that is not running HBase 0.94 but has connectivity to ZooKeeper and MFS services on the cluster:
    yum install mapr-hbase

    or
    apt-get install mapr-hbase

    If you do not have a node available on your cluster where you can complete this step (and you cannot add a node to serve this purpose), prepare an HBase node first.

    Do not install the HBase Master and RegionServer packages.

    Warning: Do not install HBase 0.98 packages on nodes that are running HBase 0.94 services. This will remove the 0.94 files, and you will not be able to cleanly shut down HBase 0.94 services on those nodes.

  2. Run configure.sh on the node where you installed HBase 0.98. For example, if the node is already configured to access the cluster:
    $ /opt/mapr/server/configure.sh -R
    (If this node is a new node with no existing MapR packages and services, you need to run configure.sh with additional options.)
  3. Optional: Use the installed 0.98 package to check for incompatible files on the running 0.94.x cluster.
    $ /opt/mapr/hbase/hbase-0.98.x/bin/hbase upgrade -check
    Although it is optional, this step is recommended. If incompatible HFile format files are found, you must purge them by running a compaction.
  4. If necessary, run the compaction operation. Run this operation on an HBase 0.94 node for each table that has the incompatible format:

    $ echo "major_compact '<table_name>'" | hbase shell

    Wait for the compactions to complete, as indicated on the HBase Master UI.

    A major compaction operation can have a significant impact on the cluster and should be run with caution, preferably when cluster traffic is minimal.

     

  5. Run the hbase upgrade -check command again to confirm that no incompatible files remain.

  6. Shut down the HBase 0.94.x services on the cluster. Shut down the HBase Master services first, then the RegionServer services. You can use either the maprcli or the MCS to shut down the services.

  7. From the same node where you ran the upgrade check, execute the upgrade on the cluster:
    $ /opt/mapr/hbase/hbase-0.98.x/bin/hbase upgrade -execute
    Run this upgrade command once only for the entire cluster. You do not need to run it on each node. When this upgrade command is run successfully, the appropriate HBase packages can be upgraded to 0.98.x on each node.

  8. Install the HBase 0.98 packages on the appropriate nodes. See Upgrading the HBase Software.

  9. Run configure.sh on all of the upgraded HBase nodes:
    $ /opt/mapr/server/configure.sh -R

  10. Start the upgraded HBase services. You can use either the maprcli or the MCS to start the services.

Upgrading to MapR v3.x

Due to changes in the MapR HBase client for MapR tables which are available starting with MapR version 3.0, if you are upgrading your cluster to MapR v3.0 or later, you need to upgrade the HBase client packaged with the new version of the MapR distribution.

Upgrading from 0.92.x to 0.94.x

Refer to the Apache HBase Reference Guide for details.

Packaging changes between HBase releases 0.90.6 and 0.92.1

The following points apply when upgrading HBase from 0.90.6 (or earlier) to 0.92.1 (or later). MapR did not distribute any releases of HBase between these two versions.

  • MapR's file naming convention changed between these releases, which caused a special case when upgrading. Because of a reversal in the alphanumeric order of the filenames, package managers incorrectly perceive the newer version to be a downgrade. When upgrading the software, package management tools might require you to specify a particular version, rather than automatically upgrading to the latest version.
  • HBase release 0.92.1 (and onward) is stored in a separate repository than the MapR core software. This release corresponded to the release of MapR core v2.0. Prior to v2.0, HBase packages were located in the same repository with the MapR core. From MapR v2.0 onward, HBase packages are located in a separate repository, which requires some consideration in setting up repositories. See Installing MapR Software for details.

Upgrading from 0.90.x to 0.92.x

Refer to the Apache HBase Reference Guide for details.

Upgrading to HBase 0.90.x from 0.20.x or 0.89.x

Refer to the Apache HBase Reference Guide for details.

Supported HBase Versions

To see which versions of HBase are supported in each MapR release, go to the Ecosystem Support Matrix.

Upgrading the Software

Use one of the following methods to upgrade the HBase component:

To upgrade with a package manager

After configuring repositories so that the version you want to install is available, you can use a package manager to install from the repository. The upgrade process will remove all but the following directories in the current HBase directory: conf and logs.

On RedHat and CentOS

To upgrade an HBase region server node:

yum upgrade mapr-hbase mapr-hbase-regionserver

To upgrade an HBase master node:

yum upgrade mapr-hbase mapr-hbase-master

To upgrade an HBase client node:

yum upgrade mapr-hbase

If you have additional HBase services or libraries installed, you should also upgrade those packages to match the HBase version you are upgrading to.

On Ubuntu

To upgrade an HBase region server node:

apt-get install mapr-hbase mapr-hbase-regionserver

To upgrade an HBase master node:

apt-get install mapr-hbase mapr-hbase-master

To upgrade an HBase client node:

apt-get install mapr-hbase

If you have additional HBase services or libraries installed, you should also upgrade those packages to match the HBase version you are upgrading to.

If you are upgrading from HBase 0.90.6 or earlier, you might have to specify the particular version you want to upgrade to, because of #Version-Specific Considerations.

When the upgrade finishes, the package manager updates the file /opt/mapr/hbase/hbaseversion to contain the correct version, such as 0.98.9.

$ cat hbase/hbaseversion 
0.98.9
  • After the upgrade, verify that the file /opt/mapr/hbase/hbaseversion exists. If it does not, run the comand echo "<version>" > hbaseversion to re-create the file, substituting the new version. Example:

    echo "0.98.9" > hbaseversion

To upgrade by manually installing packages

For example, to install version 0.98.9 build 201503251553-1, perform the steps below.

On RedHat and CentOS
  1. Download the RPM package files mapr-hbase, mapr-hbase-master, and mapr-hbase-regionserver for version 0.98.9 from

    http://package.mapr.com/releases/ecosystem-all/

    .
  2. Install the package with rpm.
    To upgrade an HBase region server node:

    rpm -i --force mapr-hbase-0.98.9.201503251553-1.noarch.rpm mapr-hbase-regionserver-0.98.9.201503251553-1.noarch.rpm

    To upgrade an HBase master node:

    rpm -i --force mapr-hbase-0.98.9.201503251553-1.noarch.rpm mapr-hbase-master-0.98.9.201503251553-1.noarch.rpm
On Ubuntu
  1. Download the RPM package files mapr-hbase, mapr-hbase-master, and mapr-hbase-regionserver for version 0.98.9 from

    http://package.mapr.com/releases/ecosystem-all/

    .
  2. Install the package with dpkg.
    To upgrade an HBase region server node:

    dpkg -i mapr-hbase_0.98.9.201503251535_all.deb mapr-hbase-regionserver_0.98.9.201503251535_all.deb

    To upgrade an HBase master node:

    dpkg -i mapr-hbase_0.98.9.201503251535_all.deb mapr-hbase-master_0.98.9.201503251535_all.deb

Do not keep a prior version and install a newer version.
HBase installs into separate directories named after the version, such as /opt/mapr/hbase/hbase-<version>/, so the files for multiple versions can co-exist. However, HBase data cannot be shared between separate versions of the software, and the data format is not backward compatible. Furthermore, HBase master and region-server services are resource intensive. MapR does not recommend keeping multiple versions of HBase on a node.

Configure the Cluster for the New Version

After upgrading the HBase packages, run the configure.sh script to populate the new hbase-site.xml properties file with correct ZooKeeper information. Substitute <CLDBs> and <ZooKeepers> with a comma-separated list of the CLDB and ZooKeeper nodes.

/opt/mapr/server/configure.sh -C <CLDBs> -Z <ZooKeepers>