Spark 1.6.1-1604 Release Notes

The notes below relate specifically to the MapR Converged Data Platform. You may also be interested in the open source Spark 1.6.1 Release Notes

Spark Version 1.6.1
Release Date May 4, 2016
MapR Version Interoperability See the Ecosystem Support Matrix (Pre-5.2 releases) and Spark Support Matrix.
Source on GitHub https://github.com/mapr/spark/tree/1.6.1-mapr-1604
Package Names The following packages are associated with this release:
  • mapr-spark-1.6.1.201605031407-1.noarch.rpm
  • mapr-spark_1.6.1.201605031407_all.deb
  • mapr-spark-historyserver-1.6.1.201605031407-1.noarch.rpm
  • mapr-spark-historyserver_1.6.1.201605031407_all.deb
  • mapr-spark-master-1.6.1.201605031407-1.noarch.rpm
  • mapr-spark-master_1.6.1.201605031407_all.deb

New in this Release

This is the initial release of Spark Version 1.6.1 for MapR.

For details on the features available in the open source version of this component, see the Apache Spark documentation.

Patches

This release by MapR includes the following patches on the base Apache release. For complete details, refer to the commit log for this project in GitHub.

GitHub Commit Date (YYYY-MM-DD) Comment
d0b2386 2016-04-13 Backported SPARK-13599 to remove transitive groovy dependencies from spark-hive and spark-hiveserver.
9ded88e 2016-04-13 MAPR-23068: Spark now uses Py4J version 0.8.2.1.
a37d88b 2016-04-13 MAPR-23093: The Spark KafkaProducerExample no longer fails when it is submitted in yarn-client mode on a secure cluster that uses Kerberos authentication.
6f4ca35 2016-04-28 MAPR-23203: HiveFromSpark example no longer fails with a datanucleus.schema.autoCreateTables error when it is submitted in yarn-cluster mode.

Known Issues

  • MAPR-17271: On secure clusters, the MapR Control System (MCS) does not display links for Spark-Master and Spark-HistoryServer.
  • MAPR-19761: On a secure cluster, MapR does not support the Spark SQL Thrift JDBC server. When the cluster is secure, the Spark Thrift server will not start.
  • MAPR-22940/SPARK-11851: On clusters that use Kerberos authentication, Spark Thrift Server is unable to connect to beeline.
  • Spark versions up to and including 2.3.0 have the following security vulnerability: CVE-2018-1334 Apache Spark local privilege escalation vulnerability