Spark 2.2.1-1904 (MEP 5.0.3) Release Notes

This section provides reference information, including new features, patches, and known issues for Spark 2.2.1.

The notes below relate specifically to the MapR Distribution for Apache Hadoop. This release of Spark has backward-compatibility changes, see the open-source Spark 2.2.1 Release Notes for more information.

These release notes contain only MapR-specific information and are not necessarily cumulative in nature. For information about how to use the release notes, see Ecosystem Component Release Notes.

Spark Version 2.3.3
Release Date April 2019
MapR Version Interoperability See MEP Components and OS Support.
Source on GitHub https://github.com/mapr/spark
GitHub Release Tag 2.2.1-mapr-1904
Maven Artifacts http://repository.mapr.com/maven/
Package Names Navigate to https://package.mapr.com/releases/MEP/ and select your MEP and OS to view the list of package names.
Important:
  • Starting with MEP 6.0.0, keyStore and trustStore passwords can be removed from the spark-defaults.conf file and can be set in the /opt/mapr/conf/ssl-client.xml file.
  • Starting with MEP 6.0.0, after an upgrade, configuration files of previous versions are saved in the /opt/mapr/spark directory.
  • The MapR 6.1 and MEP 6.0.0 release introduces "Simplified Security". If you are using these versions and enable security on your MapR cluster, MapR scripts automatically configure Spark security features.

Hive Support

This version of Spark supports integration with Hive. However, note the following exceptions:

New in This Release

Patches

This MapR release includes the following new patches since the latest MapR Spark 2.3.1 release. For details, refer to the commit log for this project in GitHub.

GitHub Commit Date (YYYY-MM-DD) Comment
41854fb 2019/01/23 MapR [SPARK-396] Interface change of sendToKafka
d6c12f5 2019/02/05 [SPARK-383] Avoid using 'SimpleChannelInboundHandler' to prevent 'TypeParameterMatcher' generation
26388c1 2019/02/07 [SPARK-10878][CORE] Fix race condition when multiple clients resolvesartifacts at the same time
a2f38c9 2019/02/15 [SPARK-449] Kafka offset commit issue fixed
ef8f5cd 2019/03/07 CVE-2018-1320 vulnerability in Apache Thrift
314e8e4 2019/03/12 MapR [SPARK-445] Messages loss
4a9322d 2019/03/15 MapR [SPARK-514] Recovery from checkpoint is broken
14618bb 2019/03/19 SPARK-463] MAPR_MAVEN_REPO variable for specifying mapR repository
b9e7086 2019/03/26 [SPARK-508] MapR-DB OJAI Connector for Spark isNull condition returns incorrect result
924aa51 2019/03/28 MapR [SPARK-510] nonmapr "admin" users not able to view other user logs in SHS
2ca0af1 2019/03/28 MapR [SPARK-461] Stop graph after jobs completion to prevent 'java.lang.IllegalStateException: No active subscriptions'
0607a96 2019/04/04 [SPARK-21642][CORE] Use FQDN for DRIVER_HOST_ADDRESS instead of ip address
ff48dbc 2019/04/11 MapR [SPARK-444] Fix of hive version for spark dev branches

Known Issues

  • None.

Resolved Issues

  • None.