Drill (MEP 6.1.0) Release Notes

This section provides reference information, including new features, improvements, resolved issues, known issues, and limitations for Drill

These release notes contain MapR-specific information and are not necessarily cumulative in nature. For information about how to use the release notes, see Ecosystem Component Release Notes.

The following release notes apply to the version of the Drill component:

Release Date February 2019
MapR Version Interoperability See Component Versions for Released MEPs.
Package Names Navigate to https://package.mapr.com/releases/MEP/, and select your MEP and OS to view the list of package names.
Drill works with the following MapR Drill drivers:

For additional driver information, see Drill Drivers.

New in this Release

Drill includes the following new features and improvements in the following areas:
Query optimization
Web UI
Multiple Web UI improvements that simplify use and administrative tasks, including:
  • The ability to cancel multiple queries at once via a check box next to executing queries.
  • The auto prompt in the Web UI SQL box now works on UDFs.
  • An enhanced Options page that includes detailed descriptions of each option.
  • Categorized options via Quick Filters on the Options page. (DRILL-5735)
  • Default button (DRILL-6668)
  • Web display options and updated option names (MD-4835, DRILL-6544, DRILL-6715)
  • Meta+Enter key combination to submit queries (DRILL-6611)
SQLLine (Drill shell)
System tables
  • System options table includes option descriptions. (DRILL-6684)
  • System functions table exposes the available SQL functions in Drill and also detects UDFs that have been dynamically loaded into Drill. (MD-4941, DRILL-3988)

For a list of additional features and improvements, see the Apache Drill 1.15 Release Notes.

For additional information, read the MapR Drill 1.15 blog post.

Default Configuration Changes

Starting in MEP 6.1.0, certain ecosystem components, including Drill, have four digit version numbers. For example, when you install Drill, the Drill version appears as

Resolved Issues

Drill includes the following resolved issues:
MapR Tracking Number Resolved Issue
MD-5253 The TO_TIMESTAMP function no longer loses precision for milliseconds.
MD-5226 DRILL-6918: Querying empty topics no longer fails with a "NumberFormatException."
MD-5208 Semi-join queries on sf1 data no longer return an UNSUPPORTED_OPERATION error.
MD-5149 DRILL-6869: Drill no longer allows views created outside of workspaces.
MD-5147 DRILL-6865: Queries no longer return the wrong results when filter pruning occurs.
MD-5142 DRILL-6857: Limit is pushed into the scan when selecting from a Parquet file with multiple row groups.
MD-5138 DRILL-6863: Drop table works when the path is within a workspace that starts with '/'.
MD-5105 DRILL-6818: Secondary index options have descriptions.
MD-5076 DRILL-6811: Type inference returns the correct data mode for boolean functions.
MD-5062 DRILL-6664: Drill returns the same results for queries on the same column of a Parquet table with the same predicate.
MD-5057 Queries against new files created on a volume that has reached the name container data threshold no longer fail with I/O exceptions.
MD-5023 DRILL-6810: Drill does not support nullable complex types (maps, arrays); therefore, UDFs cannot return null if @Output is of a complex type. Null handling of complex types is disabled by default. NullHandling.INTERNAL should be used instead of NullHandling.NULL_IF_NULL in UDFs. When using NullHandling.INTERNAL, the UDF must handle null input(s) on its own.
MD-5019 DRILL-6797: Drill aggregation queries that use custom UDF to split column data in a Parquet file no longer fail with a compilation error if the filter criteria results in no rows.
MD-5014 DRILL-6776: The Drill Web UI page source no longer has links to external sites.

Drill can leverage indexes for queries with ORDER BY DESC on indexed columns; for example, if a secondary index is defined on column a, Drill leverages the index for the query:


MD-5002 DRILL-6764: Queries no longer fail with IOB exception when UNNEST references a deep nested field, like t.c_orders.o_lineitems.

DRILL-6766 : Queries with LATERAL UNNEST no longer return the following system error when one batch of streaming aggregates is split across multiple output batches :

IllegalStateException: Unexpected case where rowId [] in right batch of lateral is smaller than rowId [] in left batch being processed.

MD-4968 DRILL-6762: Dynamic UDFs registered on one Drillbit are now visible on other Drillbits.
MD-4956 DRILL-6755: Hash-Join does not build hash tables when the probe side is empty.
MD-4938 The option planner.index.covering_selectivity_threshold takes effect in execution plan when set to values < 1.0 for complex data.
MD-4936 DRILL-6773: Queries, with column aliases, that returned inconsistent results after a Drill patch was applied no longer return inconsistent results.
MD-4867 DRILL-6726: Drill can query views created before DRILL-6492 when impersonation is enabled.
MD-4852 DRILL-6819: The invisible "back" link on the query results page in the Drill WebUI has been removed.
MD-4851 Drill queries on MapRDB JSON tables no longer fail on schema change when the only distinct values are null and text.
MD-4801 DRILL-6724: Operator context is dumped to logs when errors occur during query execution.
MD-4780 DRILL-6746: Queries no longer hang when one of the nodes encounters a connection error.
MD-4755 DRILL-6709: The batch statistics logging utility has been extended to mid-stream operators.
MD-4643 DRILL-6732: Disabled plugins no longer work as enabled.
MD-4617 "direct.used" metrics(jvm_direct_current) catches the direct memory usage.
MD-4585 The drillbit.log file includes query user information.
MD-4567 DRILL-6517 : Hash-Join no longer accesses batches after query cancellation.
MD-4499 DRILL-5365: File not found exception no longer results from reading a Parquet file.
MD-4362 Drill no longer fails to generate a non-covering index plan for queries on data with the reserved word 'date' as a column name.
MD-3879 DRILL-6039: Graceful shutdown waits for fragments to complete before stopping the Drillbit.
MD-2979 DRILL-6410: The Parquet reader no longer leaks memory during query cancellation.

Known Issues

Drill has the following known issues:
MapR Tracking Number Known Issue
MD-5390 Drill does not return an error message when impersonation prevents access to a directory.
MD-5376 DRILL-6991: The Kerberos ticket is being dumped in the log if the log level is "debug" for stdout.
MD-5348 TPCDS queries 56, 60, 83 are slower with plan change.
MD-5319 TPCDS query 95 slower with plan change.
MD-5250 TPCDS queries 56 and 60 have regressed.
MD-5232 TPCH query 4 has regressed for scale factor 1000.
MD-5231 DRILL-6896: TPCH 13 has "regressed".
MD-5204 DRILL-6845: TPCDS query 95 slower with semi-join enabled. You can disable semi-join through the planner.enable_semijoin option.
MD-5063 TPCH queries hit an IndexOutOfBoundsException when planner.enable_demux_exchange = true.