Installing MapR Object Store with S3-Compatible API

This topic provides instructions for installing and configuring the MapR MapR Object Store with S3-Compatible API.

The MapR Object Store with S3-Compatible API is included in MEP repositories beginning with MEP 6.0.0. The MapR Object Store with S3-Compatible API is an object storage server that can be used with the Amazon S3 cloud storage service.

The MapR Object Store uses two packages:
mapr-objectstore-client The binary package.
mapr-objectstore-gateway The role package that includes Warden configuration information.
To install the MapR Object Store packages, you must use the steps on this page.
Note: The MapR Object Store cannot be installed using the MapR Installer.
You can install the MapR Object Store on an Edge node or on a server node (for example, a MapR cluster node) that is running MapR software. A POSIX client is required for both deployment modes.

Installing the MapR Object Store with S3-Compatible API on an Edge Node

Run the following commands as root or using sudo.
  1. Configure the repositories for MapR core and MEP components:

    On Ubuntu:

    sudo add-apt-repository 'deb <link-to-release-repository> binary trusty' -y
    sudo add-apt-repository 'deb <link-to-mep-repository> binary trusty' -y

    On RedHat/CentOS:

    See Adding the MapR repository on RedHat, CentOS, or Oracle Linux.

    On SUSE:

    See Adding the MapR Repository on SUSE.

  2. Update the repositories on the Edge node:

    On Ubuntu:

    sudo apt-get update

    On RedHat/CentOS:

    sudo yum clean all
    sudo yum update -y

    On SUSE:

    sudo zypper ref
  3. Install Java and the POSIX client. The following examples use the mapr-posix-client-basic license:

    On Ubuntu:

    sudo apt install openjdk-8-jdk-headless mapr-posix-client-basic

    On RedHat/CentOS:

    sudo yum install java-1.8.0-openjdk.x86_64 mapr-posix-client-basic

    On SUSE:

    sudo zypper install java-1.8.0-openjdk.x86_64 mapr-posix-client-basic
  4. Add the mapr group with a group ID (GID) of 5000:
    sudo groupadd mapr -g 5000
  5. Add the mapr user with a group ID (GID) and user ID (UID) of 5000:
    sudo useradd mapr -g 5000 -u 5000
  6. To satisfy preconditions for POSIX, perform the following steps:
    1. Create the /mapr directory with all allowed permissions:
      sudo mkdir -m 0777 /mapr
    2. Give "executable" permissions to all users of /opt/mapr/bin/fusermount:
      sudo chmod u+s /opt/mapr/bin/fusermount
  7. Run configure.sh to configure the Edge node:
    sudo /opt/mapr/server/configure.sh -N <clusterName> -C <clusterHostIp> -Z <clusterHostIp> -c
  8. Start the POSIX client service:
    sudo systemctl start mapr-posix-client-basic.service
  9. Check that the MapR file system was mounted at /mapr/*:
    ls /mapr/
    The command should return the <clusterName> directory.
  10. Install the MapR Object Store package:

    On Ubuntu:

    sudo apt install mapr-objectstore-client

    On RedHat/CentOS:

    sudo yum install mapr-objectstore-client

    On SUSE:

    sudo zypper install mapr-objectstore-client
  11. Run the /opt/mapr/objectstore-client/objectstore-client-x.x.x/bin/configure.sh script to configure the MapR Object Store:
    sudo /opt/mapr/objectstore-client/objectstore-client-1.0.0/bin/configure.sh -c -u <userName> -g <groupName> --path /path/to/filesystem/
    In this command:
    • -c is the client node with the objectstore-client.
    • -u(--user) is the user admin for working with the objectstore-client.
    • -g(--group) is the group.
    • -p(--path) is the path for mounting the file system.
  12. Start the MapR Object Store by running the script:
    sudo /opt/mapr/objectstore-client/objectstore-client-1.0.0/bin/objectstore.sh start
  13. Check that the minio process is running:
    ps aux | grep minio

Deleting the MapR Object Store with S3-Compatible API from an Edge Node

To delete the MapR Object Store with S3-Compatible API, you must stop the service and then delete the packages from the Edge node:
  1. Stop the MapR Object Store service:
    sudo /opt/mapr/objectstore-client/objectstore-client-1.0.0/bin/objectstore.sh stop
  2. Delete the packages from the system:

    On Ubuntu:

    sudo apt remove mapr-objectstore-client --purge

    On RedHat/CentOS:

    sudo yum remove mapr-objectstore-client 

    On SUSE:

    sudo zypper remove mapr-objectstore-client

Installing the MapR Object Store with S3-Compatible API on a Server

A "server" in this context refers to a node in a MapR cluster. Run the following commands as a user with admin privileges on the cluster:
  1. Ensure that the following preconditions are met:
    • MapR software is installed on the server, and the MEP 6.x repositories are configured. For more information about configuring repositories, see Step 2: Prepare Packages and Repositories.
    • The MapR POSIX client must be installed. This procedure includes a step to install the POSIX client.
    • For Ubuntu clusters, you must install syslinux-utils:
      sudo apt install syslinux-utils
    • To support impersonation in SUSE clusters, you must install setcap:
      sudo zypper install libcap-progs
  2. Update the repositories:

    On Ubuntu:

    sudo apt-get update

    On RedHat/CentOS:

    sudo yum clean all
    sudo yum update -y
    

    On SUSE:

    sudo zypper ref
  3. Install the mapr-objectstore-gateway package from the repository:

    On Ubuntu:

    sudo apt install mapr-objectstore-gateway

    On RedHat/CentOS:

    sudo yum install mapr-objectstore-gateway

    On SUSE:

    sudo zypper install mapr-objectstore-gateway
  4. Ensure that the MapR POSIX client is installed and configured. See Installing FUSE-based POSIX Client Packages.
  5. At first run time, run configure.sh to configure Warden:
    sudo /opt/mapr/server/configure.sh -R
  6. Check that the minio process is running:
    ps aux | grep minio

Configuring the MapR Object Store with S3-Compatible API on a Server

To start, restart, or stop the MapR Object Store:

maprcli node services -name objectstore -nodes <node_name> -action [ start | restart | stop ]
Note: After starting the service, wait at least 10 seconds before configuring the service.

To add, remove, or change tenants (MapR Object Store users), edit the tenants.json file:

/opt/mapr/objectstore-client/objectstore-client-1.0.0/conf/tenants.json
For an example of the tenants.json file, see tenants-sample.json:
{
    "tenants": [
        {
            "name": "tenant1",
            "uid": 5001,
            "gid": 5001
        }
    ],

    "credentials": [
        {
            "accessKey": "accessKey1",
            "secretKey": "secretKey1",
            "tenant": "tenant1"
        }
    ]
}

To change the deployment mode, edit the deploymentMode field in the minio.json file:

/opt/mapr/objectstore-client/objectstore-client-1.0.0/conf/minio.json
The MapR Object Store supports three deployment modes:
  • fs_only - File permissions protection. This mode supports setting the file-system UID and GID.
  • s3_only - Protection only by s3 bucket policy. In this mode, all the files in the file system are owned by the user that runs the MapR Object Store processes (typically the mapr user). Access is controlled by the secret access key and the key ID.
  • mixed - File and bucketPolicy mixed protection. This is the default mode. In this mode, the access key is ignored, and file-system permissions determine access.

For more information about deployment modes, see Using the MapR Object Store Authorization Model.

Using the Logs for the MapR Object Store with S3-Compatible API

Logs for the MapR Object Store with S3-Compatible API are available after the first run at this location:

/opt/mapr/objectstore-client/objectstore-client-1.0.0/logs/

The minio.log is refreshed daily. Logs from previous days are archived to minio.log-<date>.gz.

To change the level of detail (verbosity) displayed in the minio.log, you can edit the logLevel field in the minio.json file:

/opt/mapr/objectstore-client/objectstore-client-1.0.0/conf/minio.json
Specify a number (1 through 6) to set the log verbosity level:
  • 1 - Panic (less detail)
  • 2 - Fatal
  • 3 - Error
  • 4 - Warning
  • 5 - Info
  • 6 - Debug (more detail)

Deleting the MapR Object Store with S3-Compatible API from a Server

  1. Stop the objectstore service:
    maprcli node services -name objectstore -nodes node_name -action stop
  2. Delete the packages from the system:

    On Ubuntu:

    sudo apt remove mapr-objectstore-gateway --purge
    sudo apt remove mapr-objectstore-client --purge
    

    On RedHat/CentOS:

    sudo yum remove mapr-objectstore-gateway 
    sudo yum remove mapr-objectstore-client 

    On SUSE:

    sudo zypper remove mapr-objectstore-gateway 
    sudo zypper remove mapr-objectstore-client 

Installing the AWS CLI

You can use the AWS CLI to work with the MapR Object Store with S3-Compatible API.
To install the AWS CLI:
  • On Ubuntu:
    pip install awscli
  • On RedHat/CentOS:
    pip install awscli
  • On SUSE:
    pip install awscli
To configure the AWS CLI, use the aws configure command, and fill in these values:
For This Value Do This
AWS Access Key ID Insert the accessKey from the tenants.json file.
AWS Secret Access Key Insert the secretKey from the tenants.json file.
Default region name Specify 'us-west-1'
Default output format Specify json