Spark 1.5.2-1603 Release Notes

The notes below relate specifically to the MapR Converged Data Platform. You may also be interested in the open source Spark 1.5.2 Release Notes.
Spark Version 1.5.2
Release Date April 4, 2016
MapR Version Interoperability See the Ecosystem Support Matrix (Pre-5.2 releases) and Spark Support Matrix.
Source on GitHub https://github.com/mapr/spark/tree/1.5.2-mapr-1603
Package Names The following packages are associated with this release:
  • mapr-spark-1.5.2.201604040801-1.noarch.rpm
  • mapr-spark_1.5.2.201604040801_all.deb
  • mapr-spark-historyserver-1.5.2.201604040801-1.noarch.rpm
  • mapr-spark-historyserver_1.5.2.201604040801_all.deb
  • mapr-spark-master-1.5.2.201604040801-1.noarch.rpm
  • mapr-spark-master_1.5.2.201604040801_all.deb

New in This Release

This release of Apache Spark for MapR includes the following new features or behavior changes:
  • You can submit Spark jobs from a Windows client node.
  • On a secure cluster that runs Spark on YARN, SSL encryption between SparkWorker and SparkMaster nodes is automatically enabled.
  • On a secure cluster that runs Spark on YARN, authentication between SparkWorker and SparkMaster nodes is automatically enabled.
For details on the features available in the open source version of this component, see the Apache Spark documentation.

Patches

This release by MapR includes the following patches on the base Apache release. For complete details, refer to the commit log for this project in GitHub.

GitHub Commit Date (YYYY-MM-DD) Comment
03b314 2016-03-04 MAPR-21699: When you run Spark on YARN on a secure cluster, encryption and authentication now are enabled by default.
fbfaf1 2016-03-16 MAPR-18865: spark-env.cmd script is now included in the conf directory so that it can determine the hadoop classpath.
d9a0e3 2016-03-16 [MAPR-21699] When you run Spark on YARN on a secure cluster, SSL encryption is automatically enabled for Akka, broadcast, and MapR file system connections.

Known Issues

  • MAPR-17271: On secure clusters, the MapR Control System (MCS) does not display links for Spark-Master and Spark-HistoryServer.
  • MAPR-19761: On a secure cluster, MapR does not support the Spark SQL Thrift JDBC server. When the cluster is secure, the Spark Thrift server will not start.
  • Spark versions up to and including 2.3.0 have the following security vulnerability: CVE-2018-1334 Apache Spark local privilege escalation vulnerability