Pinot 0.1.0

This is the first official release of Apache Pinot.

Requirements

• Java 8 (It has been reported that Pinot cannot be compiled with Java 11 due to a missing package for sun.misc , #3625)

Highlights

Pluggable Storage

We have added a Pinot filesystem abstraction that provides users the option to plug-in their own storage backend. Currently, we support HDFS, NFS, and Azure Data Lake.

• Documentation (https://pinot.readthedocs.io/en/latest/pluggable_storage.html)

Pluggable Realtime Streams

We have decoupled Pinot from Kafka for realtime ingestion and added the abstraction for Streams. Users can now add their own plugins to read from any pub-sub systems.

• Documentation (https://pinot.readthedocs.io/en/latest/pluggable_streams.html)

Native Byte Array and TDigest Support

Pinot now can support byte[] type natively. Also, Pinot can accept byte serialized TDigest object (com.tdunning.math.stats.TDigest) and this can be queried to compute approximate percentiles as follows.

select percentileTDigest95(tDigestColumn) from myTable where ... group by... top N

Compatibility

Since this is the first official Apache release, these notes applies to people who have used Pinot in production by building from the source code.

Backward Incompatible Changes

• All the packages have been renamed from com.linkedin to org.apache.
• Method signatures has been changed in interface ControllerRestApi and SegmentNameGenerator.

Rollback Restrictions

• If you have used PartitionAwareRouting feature, we have changed the format of partition values from the segment metadata and zk segment metadata. (e.g. partitionRanges: [0 0], [1 1] -> partitions: 0, 1) The new code is backward compatible with the old format; however, old code will not be able to understand the new format in case of rollback.

Deprecation

• N/A

Credits

Thanks for everyone who have contributed to Pinot.

We would also like to express our gratitude to our Apache mentors @kishoreg, @felixcheung, @jimjag, @olamy and @rvs.