Spark 1.6.1-1605 Release Notes

The notes below relate specifically to the MapR Converged Data Platform. You may also be interested in the open source Spark 1.6.1 Release Notes

Spark Version 1.6.1
Release Date June 6, 2016
MapR Version Interoperability See the Ecosystem Support Matrix (Pre-5.2 releases) and Spark Support Matrix.
Source on GitHub https://github.com/mapr/spark/tree/1.6.1-mapr-1605
Package Names The following packages are associated with this release:
  • mapr-spark-1.6.1.201605311547-1.noarch.rpm
  • mapr-spark_1.6.1.201605311547_all.deb
  • mapr-spark-historyserver-1.6.1.201605311547-1.noarch.rpm
  • mapr-spark-historyserver_1.6.1.201605311547_all.deb
  • mapr-spark-master-1.6.1.201605311547-1.noarch.rpm
  • mapr-spark-master_1.6.1.201605311547_all.deb

New in this Release

This release of Apache Spark for MapR includes the following behavior changes:
  • On clusters that run Spark on YARN, SASL encryption between all SparkWorker and SparkMaster nodes is enabled by default.
  • SSL encryption between all SparkWorker and SparkMaster nodes is disabled by default.
For details on the features available in the open source version of this component, see the Apache Spark documentation.

Important Notes

This release of Apache Spark for MapR includes the following shared library dependencies:
  • Mesos version: 0.21.1
  • HBase version: 0.98.12-mapr-1506
  • Flume version 1.6.0
  • Zookeeper version: 3.4.5-mapr-1503
  • Hive version: 1.2.0-mapr-1603
  • Hive parquet version: 1.6.0
  • Kafka version: 0.9.0.0
  • Kafka clients version: 0.9.0.0-mapr-1602

Patches

This release by MapR includes the following patches on the base Apache release. For complete details, refer to the commit log for this project in GitHub.

GitHub Commit Date (YYYY-MM-DD) Comment
151ee01 2016-05-17 MAPR-23355: On clusters that run Spark on YARN, authentication between all SparkWorker and SparkMaster nodes is configured to use Spark SASL encryption by default.
e80140d 2016-05-18 MAPR-23243: datanucleus.schema.autoCreateAll is programatically set to true when Hive uses the embedded derby metastore.
c0741cb 2016-05-18 MAPR-23298: Spark SSL encryption is now disabled by default.

Known Issues