Hive 2.3.6-1912 (MEP 6.3.0) Release Notes

This section provides reference information, including new features, fixes, known issues, and limitations for Hive 2.3.6-1912 for MEP 6.3.0.

These release notes contain only MapR-specific information and are not necessarily cumulative in nature. For information about how to use the release notes, see Ecosystem Component Release Notes.

Hive Version 2.3.6
Release Date December 2019
MapR Version Interoperability See Hive and HCatalog Support Matrix and Ecosystem Support Matrix and MEP Components and OS Support.
Source on GitHub https://github.com/mapr/hive
GitHub Release Tag

2.3.6-mapr-1912

Maven Artifacts See Maven Artifacts for MapR.
Package Names Navigate to https://package.mapr.com/releases/MEP/, and select your MEP and OS to view the list of package names.
ODBC / JDBC Drivers Hive 2.3.6 works with the following MapR Hive ODBC drivers:For additional driver information, see Connecting to HiveServer2.

Feature Support

  • MapR supports Hive-2.3.6 on Tez-0.9.1. For more information, see Tez 0.9.1-1912 (MEP 6.3.0) Release Notes.
  • MapR does not support Hive on Spark, so you cannot use Spark as an execution engine for Hive. However, you can run Hive and Spark on the same cluster. You can also use Spark SQL and Drill to query Hive tables.
  • MapR does not support HDFS encryption in Hive tables.
  • MapR supports HBase using Hive 2.3.6 starting with MEP 6.3.0 and MapR Core 6.1.0.
  • MapR does not support LLAP with Hive 2.3.6, as Apache Slider is not a MapR ecosystem component.
  • Starting from Hive 2.1, Hive needs to run the schematool command as an initialization step.

New Features

  • None.

Known Issues

  • HIVE-655 - Failed to launch Hive WebHCat job after update. See the workaround in Troubleshooting Hive and Tez.
  • HIVE-19502 Unable to insert values into table stored by JdbcStorageHandler
  • HIVE-19286 NPE in MERGE operator on MR mode
  • PIG-25 – Pig 0.16 failed to use ORC Storage. Pig - Hive dependency issue – MapR Pig does not support MapR Hive ORC integration.
  • HIVE-522 – "INSERT INTO" overwrites old data when the destination table is encapsulated by a backquote.

    After HQL harmonization with requirements of the SQL standard, queries with table names or database names that contain a dot (.) can fail. For detailed information, see Troubleshooting Hive and Tez.

Some select queries can be converted to single FETCH task minimizing latency. Currently, the query should be single sourced not having any subquery and should not have any aggregations or distincts (which incur RS), lateral views, and joins:
  • none: disable hive.fetch.task.conversion
  • minimal: select star, filter on partition columns, LIMIT only
  • more: SELECT, filter, LIMIT only (support TABLESAMPLE and virtual columns)
Important:

Hive-2.3.6 MEP-6.3.0 requires Pig-0.16 MEP-6.3.0 and will not support Pig-0.16 from old MEPs. For details, see the Pig 0.16.0-1912 Release Notes.

Changes in Security with Default Configuration

  • Added properties to default hive-site.xml configuration on secured cluster. See the following table:
    Table 1. Properties added to default hive-site.xml
    No. Property Value
    1 hive.server2.webui.jetty.response.headers.file /opt/mapr/hive/hive-2.3/conf/headers.xml
  • Added properties to default webhcat-site.xml configuration on secured cluster. See the following table.
    Table 2. Properties added to default webhcat-site.xml
    No. Property Value
    1 templeton.jetty.response.headers.file /opt/mapr/hive/hive-2.3/conf/headers.xml
    Note that the property above is added both in a Hive update (no version change) and a Hive upgrade (with version change).

Fixes

This release by MapR includes the following fixes on the base Apache release. For complete details, refer to the commit log for this project in GitHub.

Commit Date (YYYY-MM-DD) Comment
17fe3ee 2019-11-20 MAPR-HIVE-642: Joda Time Library update for Brazil to v2.10.3.jar
af42452 2019-11-19 MAPR-HIVE-641: CLONE - old jetty version at hive lib
3250b11 2019-07-01 MAPR-HIVE-376 : Jetty web server version for HS2 web UI should be updated
b923f16 2019-10-31 MAPR-HIVE-633: Update HBase dependency to 1.1.13 for MEP-6.3.0
3d3da90 2019-10-22 MAPR-HIVE-618 : Add security headers for WebHCat (Part II) (#845)
431b114 2019-10-18 MAPR-HIVE-617 : Add Junit tests for configure.sh script. Verification of hive-site.xml and webhcat-site.xml
ba53b45 2019-10-18 MAPR-HIVE-628 : Add utility method to represent xml properties as key-value map
74a7408 2019-10-18 MAPR-HIVE-627 : Remove unused option from configure.sh
4fcf1ea 2019-10-02 MAPR-HIVE-618 : Add security headers for WebHCat
df40ce4 2019-09-30 MAPR-HIVE-619 : Unnecessary excessive WebHCat security configuration on secure clusters after security refactoring
1a5bde7 2019-09-23 MAPR-HIVE-616 : Implement SBD behaviour in case of configuring Hive on KRB clusters after security refactoring
6ba42a4 2019-09-20 MAPR-HIVE-566: UDAF throws IllegalArgumentException for a complex input when column stats is not provided
5d464bb 2019-09-19 MAPR-HIVE-614 : Hive on MapR DB JSON table does not honour the column's comment
1e6abbf 2019-09-17 MAPR-HIVE-611: Hive - MapR-DB-json integration. NullPointerException in WHERE clause when no row found
ccdeb4a 2019-09-02 MAPR-HIVE-605 : Support of Avro 1.8
f255efd 2019-08-23 MAPR-HIVE-604 : Process kerberos authentication method in HiveConf object
ea4d923 2019-09-05 MAPR-HIVE-580: Hive on Mapr-db JSON table with nested schema showing wrong results
e3a107d 2019-08-02 MAPR-HIVE-593 : Spark thriftserver fails when work with hive-maprdb json table
0ef33d7 2019-07-31 MAPR-HIVE-586: hive.scratch.dir.permission doesn't work
d5455ce 2019-07-27 MAPR-HIVE-587 : Unable to retrieve Data through hcat/Pig to Hive
7417cb6 2019-07-12 MAPR-HIVE-573 : IndexOutOfBoundsException on parsing query
27463ef 2019-07-20 MAPR-HIVE-583 : Configure hive.metastore.authentication for KRB and MAPRSASL security cluster
fec9184 2019-07-17 MAPR-HIVE-559 : Hive configure.sh changes hive-site.xml permissions despite having .customSecure file presence
a48088c 2019-06-14 MAPR-HIVE-544 : Refactor Mapr SASL security implementation
5b5b8dd 2019-06-11 MAPR-HIVE-541 : Hive creates Kerberos-based thrift connection when there is no delegation token on MapR SASL cluster
65bc50a 2019-06-08 MAPR-HIVE-570 : ClassNotFoundException: HiveMapRDBJsonInputFormat in MaprDB-json integration
1e6a377 2019-06-24 MAPR-HIVE-556 : Error while starting Hive CLI on Tez: ERROR exec.Task: Failed to execute tez graph
1d86356 2019-06-17 MAPR-HIVE-520 : Use separate log file for each Hive service
2286789 2019-06-10 MAPR-HIVE-528 : UNION query with regular expressions works in Hive-2.1 and does not work in Hive-2.3
3d110189 2019-06-10 MAPR-HIVE-534 : Change hive-maprdb-json-common version to 2.3.5
dc6e806 2019-05-20 MAPR-HIVE-519 : Hive ORC read INT, BIGINT as NULL for Data created by spark
63b3829 2019-04-22 MAPR-HIVE-509 : MapRDBJsonSplit class should be available for Spark

This release by MapR also includes the following backported issues. For complete details, refer to the commit log for this project in GitHub.

Commit Date (YYYY-MM-DD) Comment
f6f1ebb 2019-11-12 HIVE-19085: FastHiveDecimal abs(0) sets sign to +ve (Gopal V, reviewed by Matt McCline)
d5dd4e4 2019-09-06 HIVE-17088: HS2 WebUI throws a NullPointerException when opened (Sergio Pena, reviewed by Aihua Xu)
739b41c 2019-06-27 HIVE-16049: upgrade to jetty 9 (Sean Busbey & Aihua Xu, reviewed by Sergio Peña)
fced649 2019-10-25 HIVE-22107: Correlated subquery producing wrong schema
8bdc8158 2019-10-22 HIVE-8472: Add ALTER DATABASE SET LOCATION (Mithun Radhakrishnan, reviewed by Alan Gates) (#851)
3abf19c 2019-10-22 HIVE-20409: Hive ACID: Update/delete/merge does not clean hdfs staging directory (Rajkumar Singh, reviewed by Vineet Garg) (#847)
626b50d 2019-10-14 HIVE-18553 : Support schema evolution in Parquet Vectorization reader (Ferdinand Xu, reviewed by Vihang Karajgaonkar)
94190e2 2019-10-14 HIVE-18323 : Vectorization: add the support of timestamp in VectorizedPrimitiveColumnReader for parquet (Vihang Karajgaonkar, reviewed by Aihua Xu and Ferdinand Xu)
cb2eb4d 2019-10-14 HIVE-17931: Implement Parquet vectorization reader for Array type
6c78cdf 2019-10-14 HIVE-17381: When we enable Parquet Writer Version V2, hive throws an exception: Unsupported encoding: DELTA_BYTE_ARRAY. (Colin Ma, reviewed by Ferdinand Xu)
622aab2 2019-10-11 HIVE-17085: ORC file merge/concatenation should do full schema check (Prasanth Jaychandran reviewed by Zoltan Haindrich)
fece5c9 2019-07-16 HIVE-21980: Parsing time can be high in case of deeply nested subqueries (Zoltan Haindrich reviewed by Vineet Garg)
d6bd1b6 2019-08-13 HIVE-22096 Backport HIVE-21584 to branch-2.3 (Yuming Wang via Alan Gates)
fd97a4e 2019-08-30 HIVE-21859 Backport HIVE-17466 (get_partition_values) to branch-2.3 (Piotr Findeisen via Alan Gates)
6752b7b 2019-08-30 HIVE-21786 : Update repo URLs in poms branch 2.3 version
a3860a7 2019-07-31 HIVE-4605 : Hive job fails while closing reducer output - Unable to rename
6a73e80 2019-07-26 HIVE-14557: Nullpointer When both SkewJoin and Mapjoin Enabled
ef3a1f5 2019-07-25 HIVE-21540: Query with join condition having date literal throws SemanticException (Sankar Hariappan, reviewed by Zoltan Haindrich)
3d11872 2019-07-20 HIVE-19990: Query with interval literal in join condition fails(Vineet Garg, reviewed by Zoltan Haindrich)
f26f063 2019-07-23 HIVE-18786 : NPE in Hive windowing functions (Dongwook Kwon via Ashutosh Chauhan)
45b037e 2019-06-24 HIVE-21104: PTF with nested structure throws ClassCastException (Rajesh Balamohan reviewed by Gopal
dee97c4 2019-06-22 HIVE-20318 : NullPointerException when union with lateral view
7e53093 2019-06-22 HIVE-16334: Query lock contains the query string, which can cause OOM on ZooKeeper (Peter Vary via Chaoyu Tang)
1616d4d 2019-06-19 HIVE-16907: "INSERT INTO" overwrite old data when destination table encapsulated by backquote
9c26a52 2019-06-19 HIVE-6590 : Hive does not work properly with boolean partition columns (wrong results and inserts to incorrect HDFS path)
1e4cb68 2019-06-19 HIVE-18907: Create utility to fix acid key index issue from HIVE-18817 (Jason Dere, reviewed by Prasanth Jayachandran)
0ae141a 2019-06-19 HIVE-18817 - ArrayIndexOutOfBounds exception during read of ACID table. (Eugene Koifman, Jason Dere, Prasanth Jayachandran, reviewed by Jason Dere)
c7f7fd6 2019-05-03 HIVE-21680 Backport HIVE-17644 to branch-2.3 (Yuming Wang via Alan Gates)
e822e4f 2019-04-25 Backport HIVE-17764 to branch-2.3 (Yuming Wang via Alan Gates)
20e7e54 2019-04-24 HIVE-21639 Spark test failed since HIVE-10632 (Yuming Wang via Alan Gates)
8396169 2019-06-19 HIVE-18624: Parsing time is extremely high (~10 min) for queries with complex select expressions (Zoltan Haindrich reviewed by Ashutosh Chauhan)

Resolved Issues

  • None.