Major changes/feature additions since 0.1.0:

• Added support for Kafka 2.0.

• Table rebalancer now supports a minimum number of serving replicas during rebalance.

• Added support for UDF in filter predicates and selection.

• Added support to use hex string as the representation of byte array for queries (#4041)

• Added support for parquet reader (#3852)

• Introduced interface stability and audience annotations (#4063)

• Refactor HelixBrokerStarter to separate constructor and start() (#4100) - backwards incompatible

• Admin tool for listing segments with invalid intervals for offline tables

• Migrated to log4j2 (#4139)

• Added simple avro msg decoder

• Added support for passing headers in pinot client

• Table rebalancer now supports a minimum number of serving replicas during rebalance.

• Support transform functions with AVG aggregation function (#4557)

• Configurations additions/changes

  • Allow customized metrics prefix (#4392)
  • Controller.enable.batch.message.mode to false by default (#3928)
  • RetentionManager and OfflineSegmentIntervalChecker initial delays configurable (#3946)
  • Config to control kafka fetcher size and increase default (#3869)
  • Added a percent threshold to consider startup of services (#4011)
  • Make SingleConnectionBrokerRequestHandler as default (#4048)
  • Always enable default column feature, remove the configuration (#4074)
  • Remove redundant default broker configurations (#4106)
  • Removed some config keys in server (#4222)
  • Add config to disable HLC realtime segment (#4235)
  • Make RetentionManager and OfflineSegmentIntervalChecker initial delays configurable (#3946)
  • The following config variables are deprecated and will be removed in the next release:
    • pinot.broker.requestHandlerType will be removed, in favor of using the "singleConnection"
      broker request handler. If you have set this configuration, please remove it and use the default
      type ("singleConnection") for broker request handler.

Work in progress

• We are in the process of separating Helix and Pinot controllers, so that
  admninistrators can have the option of running independent Helix
  controllers and Pinot controllers.

• We are in the process of moving towards supporting SQL query format and results.

• We are in the process of seperating instance and segment assignment using instance
  pools to optimize the number of Helix state transitions in Pinot clusters with
  thousands of tables.

Other important notes

• Task management does not work correctly in this relese, due to bugs in Helix.
  We will upgrade to Helix 0.9.2 (or later) version to get this fixed.

• You must upgrade to this release before moving onto newer versions of Pinot
  release. The protocol between pinot-broker and pinot-server has been changed
  and this release has the code to retain compatibility moving forward.
  Skipping this release may (depending on your environment) cause query errors
  if brokers are upgraded and servers are in the process of being upgraded.

• As always, we recommend that you upgrade controllers first, and then brokers
  and lastly the servers in order to have zero downtime in production clusters.

• PR #4100 introduces a backwards incompatible change to pinot broker. If you
  use the Java constructor on HelixBrokerStarter class, then you will face a
  compilation error with this version. You will need to construct the object
  and call start() method in order to start the broker.

• PR #4139 introduces a backwards incompatible change for log4j configuration.
  If you used a custom log4j configuration (log4j.xml), you need to write a new
  log4j2 configuration (log4j2.xml). In addition, you may need to change the
  arguments on the command line to start pinot components.

  If you used pinot-admin command to start pinot components, you don't need any
  change. If you used your own commands to start pinot components, you will
  need to pass the new log4j2 config as a jvm parameter (i.e. substitute
  -Dlog4j.configuration or -Dlog4j.configurationFile argument with
  -Dlog4j2.configurationFile=log4j2.xml).