table replica elasticsearch edit

Edits the mapping of a MapR-DB source binary table to an Elasticsearch type.

The user that runs the command must have the following permissions granted through access control expressions on the source table:

  • Permission to perform column-family and column reads, which is granted with the readperm access control expression.
  • Permission to perform replication, which is granted with the replperm access control expression.
Note: The mapr user is not treated as a superuser. MapR-DB does not allow the mapr user to run this command unless that user is given the relevant permission or permissions with access-control expressions.

Syntax

CLI
maprcli table replica elasticsearch edit
-path <source table path>
-target <Target cluster name>
-index <Elasticsearch index name>
-type <Elasticsearch type name>
[ -conversionclass <Conversion class name> ]
[ -conversionjar <Path to conversion class JAR file> ]
[ -columns <comma separated list of <family>[:<column>]> ]
[ -throttle <throttle replication ops> default: false ]
[ -networkencryption <enable on-wire encryption> default: false ]
[ -networkcompression <on-wire compression type: off|on|lzf|lz4|zlib> default: on ]
REST

http[s]://<host>:<port>/rest/table/replica/elasticsearch/edit?path=<path>&target=<Name of Elasticsearch cluster>&index=<Name of index>&type=<Name of type>...

Parameters

Parameter

Description

path

The path to the MapR-DB table.

  • For a path on the local cluster, start the path at the volume mount point. For example, for a table named test under a volume with a mount point at /volume1, specify the following path: /volume1/test
  • For a path on a remote cluster, you must also specify the cluster name in the path. For example, for a table named customer under volume1 in the sanfrancisco cluster, specify the following path: /mapr/sanfrancisco/volume1/customer

target

The name of the Elasticsearch cluster in which the index and type are located. This name is the name specified for the cluster when it was registered with the current MapR cluster.

index

The name of the Elasticsearch index in which the type is located.

type

The name of the Elasticsearch type that updates to the MapR-DB source table are being replicated to.

conversionclass

The name of the Java class that extends the MapR-provided interface MapRESConverter and specifies how to convert source data to Elasticsearch-supported data types that MapR-DB does not support. For more information about custom mapping of source data types to Elasticsearch-supported data types, see Converting Data to Elasticsearch Data Types That MapR-DB Does Not Support.

If use this parameter, you must also use the conversionjar parameter.

conversionjar

The path to the JAR file in which the class specified by the conversionclass parameter is located.

columns

A comma-separated list of column families and specific columns within column families. This list indicates which columns to replicate to the Elasticsearch type that is specified by the type parameter.

To replicate all columns within a column family, provide the name of the column family.

To replicate particular columns within a column family, provide the name of the column family and the name of the column. Separate the two with a colon.

For example, the following list causes the replication of all columns in the column family personal, all columns in the column family purchase, the number_of_stars column in the column family review, and the date column in the column family review:

-columns personal,purchase,review:number_of_stars,review:date

If you change the list by removing any columns or column families that have already been replicated to the Elasticsearch type, those columns and column families are not deleted from the type. MapR-DB replicates updates only; it does not replicate deletions or remove data from Elasticsearch types. If you do not want to keep the data in the type, follow these steps:

  1. On the source MapR cluster, remove the mapping of the source table to the type by running the command maprcli table replica elasticsearch remove . If the source table is being indexed in more than one type, and those types also contain the columns and column families that you are no longer indexing, run the command maprcli table replica elasticsearch list to list all of the types that are mapped to the source table. Then, run the command maprcli table replica elasticsearch remove for each type.
  2. In Elasticsearch, delete all of the types that were mapped to the source table.
  3. On the source MapR cluster, recreate each type and restart indexing by running the command maprcli table replica elasticsearch autosetup for each type.
Note: To copy the content of the source table into the type, the maprcli table replica elasticsearch autosetup command starts a MapReduce job. The length of the job depends on the size of the source table and the number of columns that you are indexing. Moreover, the volume of data and the speed at which the Elasticsearch type is populated could perceptibly slow the performance of other processes running at the same time on the Elasticsearch cluster. The less data there is to copy to the type, the faster the MapReduce job will end and the fewer resources the job will consume on the Elasticsearch cluster.

throttle

A Boolean value that specifies whether to throttle replication operations. Throttle the replication stream to minimize the impact of the replication process on incoming operations during periods of heavy load. The values are true or false. No throttle (false) is the default.

networkencryption

A Boolean value that specifies whether or not to enable on-wire encryption. The values are true or false. No encryption (false) is the default. If you set this to true, the local cluster and any other cluster that is part of the replication process must be enabled for security.

Note: Data is encrypted only until it reaches a MapR gateway. From the gateway to an Elasticsearch cluster, this parameter has no effect.

networkcompression

The type of on-wire compression.

The types are:

  • off
  • on (default)
  • lzf
  • lz4
  • zlib

lz4 is the default compression which it set by the parameter values on and lz4.

Note: Data is compressed only until it reaches a MapR gateway. From the gateway to an Elasticsearch cluster, this parameter has no effect.