Offloading a Volume to a Tier

At the volume level, data can be offloaded automatically by creating and associating a schedule with the tiering enabled volume or manually by triggering the offload operation. See Data Offload and Purge for more information. The following sections describe how to set up an automatic offload of data and how to trigger a one-time manual offload data at the volume level using the MapR Control System, the CLI, and the REST API.

Note: For a tiered volume, there can be only one running job at any given time. For example, suppose another job is running on the tiered volume, if you trigger an offload operation, the offload operation will fail.
To offload volume data, you must have one of the following permissions:
  • Cluster level fc permissions
  • Volume level fc permissions
  • Volume modify permissions
Note: You can also offload individual files in a tiering-enabled volume to the associated tier. See Offloading a File to a Tier Using the CLI and REST API for more information.

Setting up Automatic Offload of Data Using the MapR Control System

  1. Create a tier.
  2. Create a storage policy to associate with the volume.
  3. Create an offload schedule.
  4. Create a tiering enabled volume and associate the tier, the storage policy, and schedule with the volume.

Triggering an Offload of all Data in a Volume to a Tier Using the MapR Control System

  1. Log in to MCS and click Data > Volumes.
  2. Select the tiered volumes to offload from the list of volumes in the Volumes pane.
    Selecting a volume makes the Actions drop-down menu available.
  3. Click Offload Data from the Actions drop-down menu to display the Offload Volume Data confirmation window.
  4. Review the list of volumes to offload and click Offload.
If the offload fails, CLDB retries the operation after some time. See Retrying Failed Operation for more information.

Setting up Automatic Offload of Data Using the CLI and REST API

To automatically offload data:
  1. Create a tier, a rule that contains the criteria for offloading data to the tier, and a schedule to automatically offload data to the tier.
    For example:
    /opt/mapr/bin/maprcli tier create -name <tier_name> -type ectier 
    /opt/mapr/bin/maprcli tier create -name <tier_name> -type cold -url <tier_url> -credential <credentials>.txt -json 
    /opt/mapr/bin/maprcli tier rule create -name <rule_name> -expr <expressions>
    /opt/mapr/bin/maprcli schedule create -schedule <JSON>
    curl -k -X POST 'https://abc.sj.us:8443/rest/tier/create?name=<tier_name>&type=ectier' --user mapr:mapr
    curl -k -X POST 'https://abc.sj.us:8443/rest/tier/create?name=<tier_name>&type=cold&url=<tier_url>&credential=<credential_str>' --user mapr:mapr
    curl -k -X POST 'https://abc.sj.us:8443/rest/tier/rule/create?name=<rule_name>&expr=<expressions>' --user mapr:mapr
    curl -k -X POST 'https://abc.sj.us:8443/rest/schedule/create?schedule=<JSON>' --user mapr:mapr
    For more information, see Managing Tiers and Managing Storage Policies.
  2. Create a tiering-enabled volume and associate the tier, rule, and schedule (that you created in step 1) with the volume.
    For example, to create a volume and enable it for:
    • Warm tier:
      /opt/mapr/bin/maprcli volume create -name <vol_name> -path <vol_mount_path> -tieringenable true -tiername <tier_name> -ecscheme <coding_scheme> -ectopology <ec_vol_topo> -tieringrule <rule_name> -offloadschedule <schedule_ID>
      curl -k -X POST 'https://abc.sj.us:8443/rest/volume/create?name=<vol_name>&path=<vol_mount_path>&tieringenable=true&tiername=<tier_name>&ecscheme=<coding_scheme>&ectopology=<ec_vol_topo>&tieringrule=<rule_name>&offloadschedule=<schedule_ID>' --user mapr:mapr
    • Cold tier:
      /opt/mapr/bin/maprcli volume create -name <vol_name> -path <vol_mount_path> -tieringenable true -tiername <tier_name> -tieringrule <rule_name> -offloadschedule <schedule_ID> -recallexpirytime 2 -json
      curl -k -X POST 'https://abc.sj.us:8443/rest/volume/create?name=<vol_name>&path=<vol_mount_path>&tieringenable=true&tiername=<tier_name>&tieringrule=<rule_name>&offloadschedule=<schedule_ID>&recallexpirytime=2' --user mapr:mapr
    You can also specify the maximum amount of data (in GB) to offload automatically for warm-tier volumes using the autooffloadthresholdgb parameter. For more information, see Working with Tiered Volumes.

Triggering an Offload of all Data in a Volume to a Tier Using the CLI and REST API

Run the following command to manually trigger an offload of all data in the volume:
maprcli volume offload -name <volume-name>

If you run the command with the ignorerule option value set to true, rules for the volume where the data resides is ignored and data is offloaded immediately. If the ignorerule option value is not specified or is set to false (default), data is offloaded based on the rules associated with the volume where the data resides.

Submit a request of type POST. For example:
# curl -k -X POST 'https://abc.sj.us:8443/rest/volume/offload?name=sampleVol' --user mapr:mapr
{"timestamp":1519947659597,"timeofday":"2018-03-01 03:40:59.597 GMT-0800 PM","status":"OK","total":0,"data":[],"messages":["Successfully started offload."]}
For more information, see volume offload.
If the offload fails, CLDB retries the operation after some time. See Retrying Failed Operation for more information.