Spark 2.1.0-1904 (MEP 4.1.4) Release Notes

This section provides reference information, including new features, patches, and known issues for Spark 2.1.0.

The notes below relate specifically to the MapR Distribution for Apache Hadoop. This release of Spark has backward-compatibility changes, see the open-source Spark 2.1.0 Release Notes for more information.

These release notes contain only MapR-specific information and are not necessarily cumulative in nature. For information about how to use the release notes, see Ecosystem Component Release Notes.

Spark Version 2.1.0
Release Date April 2019
MapR Version Interoperability See MEP Components and OS Support.
Source on GitHub https://github.com/mapr/spark
GitHub Release Tag 2.1.0-mapr-1904
Maven Artifacts http://repository.mapr.com/maven/
Package Names Navigate to https://package.mapr.com/releases/MEP/ and select your MEP and OS to view the list of package names.
Important:
  • Spark 2.2 can connect to Hive Metastore 2.1. But, features of Hive added after Hive 1.2 are not supported by Spark.
  • Spark Yarn and Standalone modes are supported only on clusters in MRv2 (YARN) mode. They are not supported on clusters in MRv1 (classic) mode.

Hive Support

This version of Spark supports integration with Hive. However, note the following exceptions:

New in This Release

Patches

This MapR release includes the following new patches since the latest MapR Spark 2.3.1 release. For details, refer to the commit log for this project in GitHub.

GitHub Commit Date (YYYY-MM-DD) Comment
a51f07e 2019/03/28 MapR [SPARK-461] Stop graph after jobs completion to prevent "java.lang.IllegalStateException: No active subscriptions"

Known Issues

  • None.

Resolved Issues

  • None.