Set up LZO Compression for HBase

This topic provides details for setting up LZO compression on HBase.

This following links provide additional information for about setting up LZO compression.
  1. Make sure HBase is installed on the nodes where you plan to run it. See Planning the Cluster and Installing without the MapR Installer for more information.
  2. On each HBase node, ensure the native LZO base library is installed:
    On Ubuntu
    apt-get install liblzo2-dev liblzo2
    On Red Hat or CentOS
    yum install lzo-devel lzo
  3. Obtain the native connector library, set the compiler flags, and build the library.
    For more information, see the FAQ, "How do I configure Hadoop to use these classes?" at http://code.google.com/p/hadoop-gpl-compression/wiki/FAQ.
  4. Create a directory for the native libraries (use TAB completion to fill in the <version> placeholder):
    mkdir -p /opt/mapr/hbase/hbase-<version>/lib/native/Linux-amd64-64/
  5. Copy the build results into the appropriate HBase directories on every HBase node. Example:
    $ cp build/native/Linux-amd64-64/lib/libgplcompression.* /opt/mapr/hbase/hbase-<version>/lib/native/Linux-amd64-64/
  6. Download the hadoop-lzo compression library from https://github.com/twitter/hadoop-lzo.
  7. Create a symbolic link under /opt/mapr/hbase/hbase-<version>/lib/native/Linux-amd64-64/ to point to:
    On Ubuntu
    ln -s /usr/lib/x86_64-linux-gnu/liblzo2.so.2 /opt/mapr/hbase/hbase-<version>/lib/native/Linux-amd64-64/
    On Red Hat or CentOS
    ln -s /usr/lib64/liblzo2.so.2 /opt/mapr/hbase/hbase-<version>/lib/native/Linux-amd64-64/liblzo2.so.2
  8. Restart the RegionServer:
    maprcli node services -hbregionserver
    restart -nodes <hostname>
Once LZO is set up, you can specify it in the per-column family compression flag while creating tables in HBase shell. For example:
create 'mytable', {NAME=>'colfam:', COMPRESSION=>'lzo'}