• Try MapR
  • AWS
  • Manually Deploying

Manual AWS Deployment

This document assumes you are familiar with the AWS console and can navigate and create resources there.

The high level tasks are:

Create AWS Key-pair

See http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/ec2-key-pairs.html on creating key-pairs

Create Policies and Roles

If there are restriction in creating roles and policies, you can add the AWS API credentials on the installer node. See section on “Setup the Installer”

There is cloud-formation template that can create these for you: Launch Stack

Otherwise, follow these steps:

  1. Create Policy The policy statement would be:
             "Version": "2012-10-17",
             "Statement": [
                     "Effect": "Allow",
                     "Action": [
                  "Resource": "\*"
  2. Create Role with the policy created above.

Note: You as a user must have the IAM:PassRole permission

Create VPC, subnets, Internet Gateway, NAT Gateway, Security Groups

You can use an existing VPC if that satisfies the following requirements:

  • Instances can be created in a subnet that can talk to the Internet (to download MapR packages)
  • Instances can talk to each other within the VPC

Otherwise, follow these steps:

  1. Create VPC
    1. Make sure that "DNS Resolution" and "DNS Hostnames" are enabled for the VPC
  2. Create two subnets These will be configured such that one can be accessed from the Internet (public-subnet), another can talk to the Internet but cannot be accessed from outside (private-subnet)
  3. Create Internet Gateway
  4. Attach Internet Gateway to the VPC
  5. Create Elastic IP
  6. Create NAT Gateway - attach it to the public subnet
  7. Update default Route Table for VPC to allow Internet access via Internet Gateway
  8. Create a new Route
    1. Update the route to allow the private-subnet to talk to the Internet via NAT Gateway
    2. Associate private subnet to the new route
  9. Create Security groups
    1. SSH Access
      1. Ports: 22
    2. Web Access
      1. Ports: 9443, 8443, 8042, 8047, 8088, 8888, 18080, 19888

Create VPC

DNS Resolution and DNS Hostnames are enabled

Create two subnets in the VPC you created above (you need to do this step twice - once for each subnet). The subnet CIDR should be different and non-overlapping for the two subnets. So, if your VPC CIDR is, your public subnet CIDR could be and private subnet CIDR could be

Create Internet Gateway

Attach Internet Gateway to VPC

Create NAT Gateway - choose the public subnet

Update default route table for VPC to route all trafic to Internet Gateway

Create a new route table for private subnet

Update the route table to route all traffic to the NAT gateway

Associate route table with the private subnet

Create Security Groups

Add Inbound security rules

Create a Launch Configuration

From AWS console, create launch configuration:

  • Choose AMI
  • Choose machine type
  • Provide launch configuration name
    • Choose either the 2nd or 3rd option for IP Address Type under Advanced options
  • Add storage
    • Increase the root disk size to minimum of 128G
    • Add as many disks, along with their size, for data disks on the nodes
  • Select security groups
    • The default security group for the VPC that will allow the instances to talk to each other
    • The security group created to allow web access to the MapR services
  • Select the AWS Keypair to be used

You can go on to creating an auto scaling group with the launch configuration created from the final screen.

Create an Auto-scale Group

From AWS console, create auto scaling group:

  • Set auto-scale group properties
    • Give it a name
    • Select the VPC and the private-subnet
    • Choose 0 instances to start with (and let the MapR installer manage it)
  • Select to keep the group at initial size
  • After auto-scale group is created, edit the group to Suspend the following 4 services
    • Terminate, ReplaceUnhealthy, AZRebalance, HealthCheck

Suspend auto-scale management processes:

Create an EC2 instance for the installer

From AWS console, create an EC2 instance using an AMI with supported OS:

  1. Assign it to the VPC created and public-subnet. Assign a public IP address.
  2. Give role to be able to execute AWS commands without credentials
    1. Alternatively, create aws credentials files on the installer (see next section)
  3. Min disk size: 128G
  4. Assign the following security groups:
    1. Default security group to allow the node to talk to all nodes in the cluster
    2. The security group created to allow web access to the MapR services
    3. The security group created to allow SSH access to the nodes

Setup MapR installer

SSH to mapr installer and as root run:

Note: If the installer has not been setup with an IAM Role to allow it to execute AWS commands, follow the section "Create AWS credentials file" first.

Note: replace with the name of you wish to give your cluster

  1. wget http://package.mapr.com/releases/installer/mapr-setup.sh
  2. chmod +x mapr-setup.sh
  3. ./mapr-setup.sh -y
  4. cd /opt/mapr/installer
  5. source build/installer/bin/activate
  6. cd data
  7. ssh-keygen -b 2048 -t rsa -f -q -N '' -C "maprinstaller@maprcluster"
  8. aws ec2 import-key-pair --key-name --region us-east-2 --public-key-material "`cat.pub`"
  9. chmod 400
  10. touch config.yml
  11. chown mapr:mapr .pub config.yml
  12. vi config.yml

Copy and paste the content below. Update the values as per your environment.

     mapr_core_version: 5.2.2
     ssh_id: centos
     ssh_key_file: /opt/mapr/installer/data/<clustername>
     cluster_name: <clustername>
     mep_version: 3.0.1
       id: AWS
         aws_region: us-east-2
         auto_scaling_group: mapr-test-asg
         count: 3
         key_name: <clustername>
         disk_type: gp2
         disk_size: 100
         disk_count: 3
     hosts: []
  1. sudo -u mapr ../bin/mapr-installer-cli import -f -n --config -t config.yml
  2. passwd mapr

Next, launch the installer and complete the rest of the installation via the installer. The installer would be listening on: https://:9443

Create AWS credentials file

If you did not create an AWS role to be used by the MapR installer, you must create an AWS credentials file with AWS access key and secret. Information on how to obtain access key and secret can be found at http://docs.aws.amazon.com/IAM/latest/UserGuide/id_credentials_access-keys.html

  1. mkdir -p ~/home/mapr/.aws
  2. vi ~/home/mapr/.aws/credentials
    aws_access_key_id = <your aws access_key>
    aws_secret_access_key = <your aws secret key>