By default, Spark on YARN uses a Spark JAR file that is installed locally. The Spark JAR file can also be added to a world-readable location on MapR-FS. When you add the JAR file to a world-readable location, YARN can cache it on nodes to avoid distributing it each time an application runs.
Complete the following steps to add the Spark JAR file to a world-readable location on MapR-FS:
Copy the following JAR file from the local file system to a world-readable location on MapR-FS:
Substitute your Spark version and specific JAR file name in the command.
spark.yarn.jarproperty in the spark-defaults.conf file to point to the world-readable location where you added the Spark JAR file. For example: