If you installed Spark with the MapR Installer, use the latest version of the MapR Installer to perform the upgrade.
The following instructions explain how to upgrade an existing installation of Spark 1.x. Spark will be installed in a new subdirectory under /opt/mapr/spark.
MapR's rpm and deb repositories always contain the Spark version recommended for the release of the MapR core associated with that repository. You can connect to an internet repository or prepare a local repository with any version of Spark you need. You can also manually download packages.
If you plan to install from a repository, complete the following steps each node where Spark is installed:
Verify that the repository is configured correctly. See Preparing Packages and Repositories for information about setting up your ecosystem repository.
Update the repository cache.On RedHat and CentOS...
yum clean allOn Ubuntu...
- Back up any custom configuration files in your Spark environment. These cannot be upgraded automatically. For example, if Spark SQL is configured to work with Hive, copy the
/opt/mapr/spark/spark-<version>/conf/hive-site.xmlfile to a backup directory.
Shut down the spark-master and spark-historyserver services (if the spark-historyserver is running):
As the mapr user, stop the slaves:
Install the Spark packages.On Ubuntu...On RedHat / CentOS...
- Run the following commands to configure the slaves:
- Copy the
Add the hostnames of the Spark worker nodes. Put one worker node hostname on each line. For example:
- Copy the
Migrate Custom Configurations (optional).
Migrate any custom configuration settings into the new default files in the conf directory(/opt/mapr/spark/spark-<version>/conf).
Start spark-master services and spark-historyserver services (if installed):
Restart all the spark slaves as the mapr user: