Spark 1.5.2-1602 Release Notes

The notes below relate specifically to the MapR Distribution for Apache Hadoop. You may also be interested in the open source Spark 1.5.2 Release Notes.

Spark Version 1.5.2
Release Date February 29, 2016
MapR Version Interoperability See the Spark Support Matrix and the Ecosystem Support Matrix (Pre-5.2 releases).
Source on GitHub https://github.com/mapr/spark/tree/1.5.2-mapr-1602
Package Names The following packages are associated with this release:
  • mapr-spark-1.5.2.201602261506-1.noarch.rpm
  • mapr-spark_1.5.2.201602261506_all.deb
  • mapr-spark-historyserver-1.5.2.201602261506-1.noarch.rpm
  • mapr-spark-historyserver_1.5.2.201602261506_all.deb
  • mapr-spark-master-1.5.2.201602261506-1.noarch.rpm
  • mapr-spark-master_1.5.2.201602261506_all.deb

New in This Release

This release of Apache Spark for MapR includes the following feature:

  • Support for MapR Event Store For Apache Kafka through Kafka 0.9.0 API. This is a beta feature.

For details on the features available in the open source version of this component, see the Apache Spark documentation.

Patches

This release by MapR includes the following patches on the base Apache release. For complete details, refer to the commit log for this project in GitHub.

GitHub Commit Date (YYYY-MM-DD) Comment
784ad2e 2016-01-13 MAPR-22122: Spark now supports the Kafka 0.9.0 Streaming Consumer API.

Known Issues

  • MAPR-17271: On secure clusters, the MapR Control System (MCS) does not display links for Spark-Master and Spark-HistoryServer.
  • MAPR-19761: On a secure cluster, MapR does not support the Spark SQL Thrift JDBC server. When the cluster is secure, the Spark Thrift server will not start.
  • MAPR-22522: Spark jobs fail when Hive uses Kerberos authentication on a secure cluster that also uses Kerberos authentication.
    Workaround: Add "-Dhadoop.login=hybrid" to SPARK_SUBMIT_OPTS in spark-env.sh ($SPARK_HOME/conf/spark-env.sh). For example:
     if [ "$MAPR_SECURITY_STATUS" = "true" ]; then 
    SPARK_SUBMIT_OPTS="$SPARK_SUBMIT_OPTS -Dmapr_sec_enabled=true -Dhadoop.login=hybrid" 
    fi
  • Spark versions up to and including 2.3.0 have the following security vulnerability: CVE-2018-1334 Apache Spark local privilege escalation vulnerability