Configure SSL Encryption for Spark on YARN

As of Spark 1.5.2-1605 and Spark 1.6.1-1605, encryption between all the SparkMaster and SparkWorker nodes is disabled by default. In Spark 1.5.2-1603 and 1.6.1-1604, when the cluster is secure, encryption between all of the SparkMaster and SparkWorker nodes is enabled by default using the TLS v1.2 protocol. Complete the following step to manually configure encryption.

In the spark-defaults.conf on each spark node, configure the following properties:
spark.ssl.akka.enabled true
spark.ssl.fs.enabled true
spark.ssl.keyPassword mapr123
spark.ssl.keyStore /opt/mapr/conf/ssl_keystore
spark.ssl.keyStorePassword mapr123
spark.ssl.trustStore /opt/mapr/conf/ssl_truststore
spark.ssl.trustStorePassword mapr123
spark.ssl.protocol TLSv1.2
spark.ssl.enabledAlgorithms TLS_RSA_WITH_AES_128_CBC_SHA,TLS_RSA_WITH_AES_256_CBC_SHA

The spark-defaults.conf file is in the following location: /opt/mapr/spark/spark-<version>/conf/

Note: When you manually configure encryption between the SparkMaster and SparkWorker nodes, configure the same protocol and algorithms for each node. Otherwise, the connection between those components will fail.